2

I have an unusual problem involving the output from the ps -ef command on AIX 7.1.

A shell script monitors processes by parsing this output. I've noticed on two occasions a process (a Perl program) was omitted from this list. Everything I've read on the subject says this is not possible. The program in question starts via crontab at 6am and runs until 11pm, when it self terminates. I checked the output of ps -ef immediately after being omitted by the monitor script, and it displays:

user  1249864  9569338   0 06:00:00   -  0:19 /usr/bin/perl -w /path/to/omittedProgram.pl

... which means it's the same process that was started at 6am. The program did not terminate, then restart.

What is causing it to be omitted from the ps -ef output?

This is the program that examines the output of ps -ef, which has been running successfully for about five years. I've only noticed this problem twice, but both have been in the last 2 months:

# set global variables
PROCESS_FILE=/tmp/processList.txt
TEMP_FILE=/tmp/greppedProcesses.tmp
BOX=`uname -n`
DATE=`date`
EMAIL_LIST="Support@email.address"

# Get list of running processes
ps -ef > $PROCESS_FILE

checkProcess() {
  PROCESS_NAME=$1
  PROCESS_ABBREVIATION=$2
  PROCESS_COUNT=$3
  UNIQUE_PROCESS_IDENTIFIER=$4

  GREPPED_LINES=$TEMP_FILE-$PROCESS_ABBREVIATION
  grep $UNIQUE_PROCESS_IDENTIFIER $PROCESS_FILE | grep -v grep > $GREPPED_LINES
  NUM=`cat $GREPPED_LINES | wc -l`
  if [[ $NUM -ne $PROCESS_COUNT ]]
     # Incorrect number of processes running!
     then MESSAGE=`printf "The \"$PROCESS_NAME\" process count is %1d, but it should be $PROCESS_COUNT!!!" $NUM`
          echo "Monitor - starting on $DATE\n\n$MESSAGE\n\n`cat $GREPPED_LINES`" | mail -s "Problem with $PROCESS_NAME on $BOX" $EMAIL_LIST
  fi
  # Delete the temp file
  rm $GREPPED_LINES
}

checkProcess "Full Name of Program" "Program Abbreviation" <expected number of processes running> "Unique string to identify program in ps output"
checkProcess ... (for other processes) ...

exit 0

Also, our install of AIX does not have pgrep.

  • just to be clear - the pid / process (e.g. 1249864) does not show up in /tmp/processList.txt? And what is the UNIQUE_PROCESS_IDENTIFIER (parameter 4 to checkProcess) that is supposed to find it? (it's elided in the post) – Jeff Schaller Sep 26 '16 at 17:38
  • also, is it possible that you're expecting a PROCESS_COUNT of 1, but the ps -ef output is catching a wrapper shell or cron process that also has the identifying information in it? – Jeff Schaller Sep 26 '16 at 17:39
  • I can't confirm for sure whether that PID was omitted from /tmp/processList.txt because it get overwritten on every iteration of the program. The UNIQUE_PROCESS_IDENTIFIER is a string that correlates specifically to that program, in this example it would be "/path/to/omittedProgram.pl". The PROCESS_COUNT in these cases has been zero. I confirmed this in the email I received from the shell script. – BC OnLine Tech Team Sep 26 '16 at 21:07
  • And although I cannot confirm that PID was listed in /tmp/processList.txt, I can confirm that it was there ten minutes before, as this program runs every ten minutes and did not generate any error emails at that time. – BC OnLine Tech Team Sep 26 '16 at 23:52
  • 1
    You said the process count was 0; and that the process was running. So the email says "the count is 1, but it should be 0!!!" ? Unclear to me if the ps failed, the grep failed, or the expected count was 0 when it should be 1 – Jeff Schaller Sep 26 '16 at 23:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.