Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I'm trying to chain some commands to periodically check/launch two processes via cron (I'm on a shared host, can't change things around). After a lot of googling around, all the things I've done don't work properly:

Trying to launch them separately in two cron jobs cascades and crushes the server (maybe because the grep command doesn't manage to catch the search words because they are parameters, I don't understand):

ps -fu $USER | grep celeryd >/dev/null || $HOME/python27/bin/python $HOME/utilities/manage.py celeryd -E -B --concurrency=1
ps -fu $USER | grep celerycam >/dev/null || $HOME/python27/bin/python $HOME/utilities/manage.py celerycam

And these other variants where I try to launch the processes together on the same cron, launch only the first process:

ps -u $USER | grep python >/dev/null || $HOME/python27/bin/python $HOME/utilities/manage.py celeryd -E -B --concurrency=1 & $HOME/python27/bin/python $HOME/utilities/manage.py celerycam &

ps -u $USER | grep python >/dev/null || ( $HOME/python27/bin/python $HOME/utilities/manage.py celeryd -E -B --concurrency=1 & $HOME/python27/bin/python $HOME/utilities/manage.py celerycam; )

ps -u $USER | grep python >/dev/null || ( $HOME/python27/bin/python $HOME/utilities/manage.py celeryd -E -B --concurrency=1; ) sleep 30s; ( $HOME/python27/bin/python $HOME/utilities/manage.py celerycam; )

I gave up trying to use grep and took advantage of the --pidfile option of the celery worker; celerycam doesn't allow multiple instances so I don't need any grep there to check for it:

$HOME/python27/bin/python $HOME/utilities/manage.py celery worker -E -B --concurrency=1 --pidfile=worker.pid
$HOME/python27/bin/python $HOME/utilities/manage.py celerycam
share|improve this question
Have you try to run your celery processes in screen/tmux sessions, that might be more easier to manage than cron jobs – number5 Aug 6 '13 at 23:51
@number5 yes screen was the solution I used before, but they either get automatically axed by my host after an amount of time or the server restarts too often (2,3 times during the day) – symbiotech Aug 7 '13 at 0:21
Yep I understand the pain to work in shared hosting env (Dream****), the ultimate solution might be to switch to some real VPS... (I know sometime it not an option either) – number5 Aug 7 '13 at 1:07
up vote 3 down vote accepted

$USER is not set in most crons. Luckily ps -u without a username defaults to the present user (which is the user from which cron is running for). However your grep has quite a high chance to match the grep itself as well as the celery processes. You can clean that issue up with a sneaky grep '[c]elery'. This will match processes with names/arguments that contain celery only (and not the grep which has argument [c]elery). The other issue I can see is that because you run the program with a giant path to manage.py you may be missing the "celery" part in your ps output. That can be remedied by adding more verbosity to the output of ps with the -w flag (the more of them there are the more verbose it becomes). So The suggested fix is:

ps fwwwu | grep '[c]eleryd' >/dev/null || $HOME/python27/bin/python $HOME/utilities/manage.py celeryd -E -B --concurrency=1
ps fwwwu  | grep '[c]elerycam' >/dev/null || $HOME/python27/bin/python $HOME/utilities/manage.py celerycam

EDIT: Removed the extraneous dash from the ps arguments above and added pgrep as a viable alternative.

Alternatively you can also make use of two other commands pidof and pgrep to get the process ids of running processes. I think in this instance (if your username is fred) you can replace the ps command with:

pgrep -fu fred celeryd

to achieve similar results.

EDIT 2: Add single quotes around '[c]elery' to stop shell globbing (the directory where the cron is ran from (the users $HOME) may contain a file that is called celery.

share|improve this answer
Thanks for your great explanation, it's much clear for me now what happens there. Unfortunately ps returns this error: "Warning: bad syntax, perhaps a bogus '-'?". Could be some version related issues? – symbiotech Aug 6 '13 at 22:01
Ah sorry that's a habbit I've never managed to shake, just use ps fwwwu (other 'flavors' of unix start ps arguments with a dash and I started on those - I just put the dash in by habbit :) – Drav Sloan Aug 6 '13 at 23:10
both "ps fwwwu" and "ps -fwwwu $USER" (with fwwu or fwwwu) work in the shell, but neither work with cron. I think cron might be limited in some way. Anyway thank you very much :) – symbiotech Aug 6 '13 at 23:40
Either that could be the $PATH does not include the directory where ps is or you could add the user to filter for with 'ps -fwwwwu fred' if the username is fred. I'll update the answer to give some alternative solutions. – Drav Sloan Aug 6 '13 at 23:43

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.