2

We have a job that is set to run Monday thru Friday. Up until two weeks ago, it ran fine 5 days a week. For the last two weeks it has failed on Monday. I am unable to locate where to find the point of failure.

### Example Scripts
0 2 * * 1-5 /admin/scripts/example.exp 1>/dev/null 2>&1

We run AIX 7.1.

I've looked in the /var/log/ and there is no cron file there. Looking for advice to add to this so we can troubleshoot.

I found that the log is located in /var/adm/log. Further we are getting this error repeatedly ever since this date/time. How do I clear this max limit?

c queue max run limit reached Fri Nov 25 21:52:00 2016
! rescheduling a cron job Fri Nov 25 21:52:00 2016
  • Output of ps -ef | grep -i [c]ron added to your question, please. – roaima Dec 5 '16 at 15:26
  • 1
    What OS and version is this? Which cron implementation? – a CVn Dec 5 '16 at 16:08
  • I've updated the original submission as I found the log file (I believe) that explains my possible issue. These errors began around the time we started seeing this issue. – brodimaggio12 Dec 5 '16 at 16:27
  • 1
    We are on aix 7.1 – brodimaggio12 Dec 5 '16 at 16:35
5

The error message c queue max run limit reached means that you have reached the limit of concurrent cron jobs.

I believe that the default setting for cron on AIX is 50 concurrent jobs, so you really need to investigate why you have 50 jobs running at the same time. (Perhaps they are multiple instances of the same job overlapping each other.)

These two lines should give you the list of jobs running under cron, and from there you should be able to investigate the root cause of the issue:

p=$(ps -ef | awk '/[c]ron/{print $2}' | xargs | tr ' ' '|')
ps -ef | egrep "\<($p)\>"

If you really need to increase the number of concurrent jobs you can find the configuration setting in /var/adm/cron/queuedefs:

c.50j20n60w

where

  • c = The cron queue
  • Nj = The maximum number of jobs to be run simultaneously by cron
  • Nn = The nice value of the jobs to be run (default is 2)
  • Nw = The time a job has to wait until the next attempt to run it

See http://www-01.ibm.com/support/docview.wss?uid=isg3T1020382 for the AIX-specific source of this answer.

  • Thank you a ton for the response. I do see a lot of jobs running at once, and now that I see the jobs running I understand why there are so many. We are going to stop the jobs that are hanging up but should we kill the main PID to delete these? – brodimaggio12 Dec 5 '16 at 17:44
  • Yes, the article says "c", the queuedefs file says: # queues: # a - sh jobs d - sync event # b - batch jobs e - ksh jobs # c - cron event f - csh jobs # # here is an example of a low prority (nice 20), 50 entry batch queue # b.50j20n60w so, i am a bit confused by what is a "batch job" and what is a "cron event" – Michael Felt Dec 5 '16 at 19:13
  • @MichaelFelt I've known the queues defined like this: a is for at jobs. b is for at -b ("batch") jobs. c is for cron jobs, and IIRC d-z are generally for custom queues. – roaima Dec 5 '16 at 20:48
  • @roaima just not using cron enough I guess. In my mind a cron job is a recurring "batch" job, while an "at" job is a one-time "batch". It (still) amazes me that there is a special indicator for the "shell" used (e:ksh, f:csh - and I would have guessed as b:(b)sh as, until AIX 4.3 (or 4.2) the default. shell for cron on AIX was sh aka bsh. The other "amazing" label is c:"cron EVENT" (not cron job). The documentation is likely 20 years (or more) old and perhaps less accurate than when it was first written. – Michael Felt Dec 7 '16 at 16:03
2

The format of your crontab suggests it is a user crontab (as opposed to a system crontab which adds a "user" field after the time specification).

With your redirections, you are depriving yourself of informative log messages. Try this instead:

0 2 * * 1-5 /admin/scripts/example.exp >> $HOME/cron.log 2>&1

And you will get the result of the execution in a cron.log file in your home directory.

  • A failed command won't trigger an e-mail when all its output is thrown away (see the 1>/dev/null 2>&1 in the OP's crontab). – a CVn Dec 5 '16 at 16:09
1

On a default installation the cron jobs get logged to

/var/log/syslog

You can see just cron jobs in that logfile by running

grep CRON /var/log/syslog

If you haven't reconfigured anything,the entries will be in there.

  • Thanks for the response but nothing appears when running the grep command in syslog. – brodimaggio12 Dec 5 '16 at 15:12
  • 2
    On AIX the default location is /var/adm/cron/log – Michael Felt Dec 5 '16 at 19:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.