1

So I have some processes that run background jobs (resque) on servers and some of those jobs start a child process that occasionally gets stuck. I've found if I kill that child process everything goes back to normal. We have another script that can kill a binary process if it goes to long but it doesn't seem to work here. Usually I can do this:

killall --older-than 20m <process>

But these workers seem to be a little more tricky. Here is how they look in ps:

jbsmith@server:~$ ps aux | grep resque
www-data 17652  0.0  1.8 794148 145940 ?       Sl   11:03   0:05 resque-1.26.0: Processing email since 1494525823 [Email]
www-data  1026  0.0  1.9 787712 157080 ?       Sl   10:47   0:13 resque-1.26.0: Forked 17652 at 1494525823

The "Forked x at x" is the child process and the one I need to kill. Is there a one liner command I can use to find processes maybe matching "Forked" and older than 20 minutes and kill them?

  • can you run the jobs via timeout 20m yourjobhere ? – thrig May 11 '17 at 23:33
  • well resque launches the process I need to kill so I don't think so... the main parent process is a worker that is always running – Joel Smith May 11 '17 at 23:39
2

A "one-liner", split onto two lines for less scrolling:

ps aux | awk -v cutoff=$(( $(date +%s) - 20 * 60 ))  \
  '/resque.*Forked [0-9]+ at [0-9]+/ { if ($NF < cutoff) system("kill " $2) }'

It could be made tighter by using an output specifier for ps to limit the output to just the PID and COMM and ARG fields, then specifically matching COMM and ARGS, but the risks of a false-positive match seem low to me, here.

  • Note: if using this with a cron you have to escape the % so it would be: ps aux | awk -v cutoff=$(( $(date +\%s) - 20 * 60 )) \ '/resque.*Forked [0-9]+ at [0-9]+/ { if ($NF < cutoff) system("kill " $2) }' – Joel Smith May 22 '17 at 23:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.