2

We have a job that is set to run Monday thru Friday. Up until two weeks ago, it ran fine 5 days a week. For the last two weeks it has failed on Monday. I am unable to locate where to find the point of failure.

### Example Scripts
0 2 * * 1-5 /admin/scripts/example.exp 1>/dev/null 2>&1

We run AIX 7.1.

I've looked in the /var/log/ and there is no cron file there. Looking for advice to add to this so we can troubleshoot.

I found that the log is located in /var/adm/log. Further we are getting this error repeatedly ever since this date/time. How do I clear this max limit?

c queue max run limit reached Fri Nov 25 21:52:00 2016
! rescheduling a cron job Fri Nov 25 21:52:00 2016
4
  • Output of ps -ef | grep -i [c]ron added to your question, please. Dec 5, 2016 at 15:26
  • 1
    What OS and version is this? Which cron implementation?
    – user
    Dec 5, 2016 at 16:08
  • I've updated the original submission as I found the log file (I believe) that explains my possible issue. These errors began around the time we started seeing this issue. Dec 5, 2016 at 16:27
  • 1
    We are on aix 7.1 Dec 5, 2016 at 16:35

3 Answers 3

5

The error message c queue max run limit reached means that you have reached the limit of concurrent cron jobs.

I believe that the default setting for cron on AIX is 50 concurrent jobs, so you really need to investigate why you have 50 jobs running at the same time. (Perhaps they are multiple instances of the same job overlapping each other.)

These two lines should give you the list of jobs running under cron, and from there you should be able to investigate the root cause of the issue:

p=$(ps -ef | awk '/[c]ron/{print $2}' | xargs | tr ' ' '|')
ps -ef | egrep "\<($p)\>"

If you really need to increase the number of concurrent jobs you can find the configuration setting in /var/adm/cron/queuedefs:

c.50j20n60w

where

  • c = The cron queue
  • Nj = The maximum number of jobs to be run simultaneously by cron
  • Nn = The nice value of the jobs to be run (default is 2)
  • Nw = The time a job has to wait until the next attempt to run it

See http://www-01.ibm.com/support/docview.wss?uid=isg3T1020382 for the AIX-specific source of this answer.

4
  • Thank you a ton for the response. I do see a lot of jobs running at once, and now that I see the jobs running I understand why there are so many. We are going to stop the jobs that are hanging up but should we kill the main PID to delete these? Dec 5, 2016 at 17:44
  • Yes, the article says "c", the queuedefs file says: # queues: # a - sh jobs d - sync event # b - batch jobs e - ksh jobs # c - cron event f - csh jobs # # here is an example of a low prority (nice 20), 50 entry batch queue # b.50j20n60w so, i am a bit confused by what is a "batch job" and what is a "cron event" Dec 5, 2016 at 19:13
  • @MichaelFelt I've known the queues defined like this: a is for at jobs. b is for at -b ("batch") jobs. c is for cron jobs, and IIRC d-z are generally for custom queues. Dec 5, 2016 at 20:48
  • @roaima just not using cron enough I guess. In my mind a cron job is a recurring "batch" job, while an "at" job is a one-time "batch". It (still) amazes me that there is a special indicator for the "shell" used (e:ksh, f:csh - and I would have guessed as b:(b)sh as, until AIX 4.3 (or 4.2) the default. shell for cron on AIX was sh aka bsh. The other "amazing" label is c:"cron EVENT" (not cron job). The documentation is likely 20 years (or more) old and perhaps less accurate than when it was first written. Dec 7, 2016 at 16:03
2

The format of your crontab suggests it is a user crontab (as opposed to a system crontab which adds a "user" field after the time specification).

With your redirections, you are depriving yourself of informative log messages. Try this instead:

0 2 * * 1-5 /admin/scripts/example.exp >> $HOME/cron.log 2>&1

And you will get the result of the execution in a cron.log file in your home directory.

1
  • A failed command won't trigger an e-mail when all its output is thrown away (see the 1>/dev/null 2>&1 in the OP's crontab).
    – user
    Dec 5, 2016 at 16:09
1

On a default installation the cron jobs get logged to

/var/log/syslog

You can see just cron jobs in that logfile by running

grep CRON /var/log/syslog

If you haven't reconfigured anything,the entries will be in there.

2
  • Thanks for the response but nothing appears when running the grep command in syslog. Dec 5, 2016 at 15:12
  • 2
    On AIX the default location is /var/adm/cron/log Dec 5, 2016 at 19:09

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .