1

I'm an Oracle DBA and need suggestions to understand a weird situation that came up today.

One of our DBA job failing with saying 'Already its running' from Tivoli scheduler. However that is not the case. Running once but not sure why the UNIX creating 2 processes simultaneously and caused the job failure.

My analysis:

Ran the job as below from my UNIX id and it does create 2 processes. Not sure why its creating ( OS -AIX 6.1).

$sudo su - oracle -c /tmp/dba_audit_cleanup.sh

1) pandas5 20578520 9109824 0 12:08:03 pts/4 0:00 sudo su - oracle -c /tmp/dba_audit_cleanup.sh
2) oracle 36241502 20578520 0 12:08:03 pts/4 0:00 /bin/ksh /tmp/dba_audit_cleanup.sh

3) oracle 28180512 36241502 0 12:08:12 pts/4 0:00 grep dba_audit_cleanup.sh

CODE:

IsJobRunning()
{
if [[ $DEBUG -ne 0 ]]
then
set -x
fi

if [ `ps -ef | grep $PROGNAME | grep -v grep | wc -l` -ne 1 ]
then
return 1
else
return 0
fi
}

# MAIN
################################################################################

if IsJobRunning
then
continue
else
print " $PROGNAME is already running on this node "
exit 0
1

The behavior you're seeing is correct. The first process is the one owned by pandas5, running the sudo su - oracle part. The second process is the one running the actual dba_audit_cleanup.sh command as the oracle user. The problem lies in the check for running processes within the script itself - the way you've written it, it will always pick up both processes, and so will always fail this way. Your best bet is to write a PID file or some other form of flag file at the start of the script, check for it during the script and somehow make sure the process doing the checking is the one supposed to be running, and then remove it at the end of the script. Or, check for the pid file, exit if it's found, if it's not found, create it, then remove it at the end of the script.

7
  • The is our standardized audit clean script- running across all environments. However, failing only for one node as above. So, I would say not the script issue. The above is just a test to simulate the error in the same node with my id. Likely this job is running from TWS scheduler and having trouble only for this specific node. – Sid Mar 16 '16 at 18:40
  • The issue is definitely in the script logic. You've just gotten unreasonably lucky that it actually works elsewhere. – John Mar 16 '16 at 18:43
  • Same script running across all environments since years-except for this new node. So, this dose not make any sense for having script trouble. – Sid Mar 16 '16 at 19:36
  • Look, you said yourself you're a DBA. Which means you're not a systems admin. I am, and I'm telling you, the login in this script is a problem. If you don't want to fix it, fine, but it will continue to break, and it will do so on more and more of your environments, so if you choose to do nothing, on your head be it. – John Mar 16 '16 at 19:45
  • @Sid , I think John is right here. However we can easily prove what he is saying. Just modify the script and include following line before return 1 statement : ps -ef | grep $PROGNAME | grep -v grep > /tmp/debug.txt. When the script will fail examine the /tmp/debug.txt file – MAQ Mar 16 '16 at 23:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.