5

Is there a shell tool which can probe if a subprocess is still producing output, and otherwise kill it after a specified timeout?

I'm using offlineimap for Gmail backup, but that's very very very flaky. Processing stops regularily due to IMAP conditions or whatever. The process needs to be constantly restarted to see any progress. Hencewhy I'm currently using:

(ulimit -t 300 ; offlineimap)

in a loop. Which restarts the tool every five minutes; no matter what.

That's somewhat unsatisfactory, as it either kills the process prematurely the few times when it's actually still performing some work, or still waits 4 minutes longer when it's already blocking. I would rather have something like:

offlineimap > output.log &
stillmakingoutput? --timeout 60 output.log || kill $!

That's probably doable with a custom exec/watching script, but isn't there some generic approach for such cases?


Improve this question
2
  • "Stops producing output" is a very vague criteria. The vagaries of OS scheduling mean that even a program that does nothing but write bytes out will occasionally "stop producing output" when other processes get scheduled. Any I/O buffering, in stdlib or something, will also produce "bursts" of output. You should find a different criteria for restarting.
    – user732
    Jul 25, 2011 at 12:45
  • I'm not sure if there is much buffering involved. Redirecting stdout to a log file does keep it updated in my case. And as criteria I deem it sufficient if that output does not grow for 60 seconds. It would be more workable anyway than the fixed ulimit timeout (which doesn't work reliably anyway, as the process does not hang in userspace).
    – mario
    Jul 25, 2011 at 13:00

3 Answers 3

Reset to default
2

So succeeded with a "little" timeout script that checks the log file size to determine if the process is still working. Not pretty, but did help:

#!/bin/sh
TIMEOUT=$1
FILE=$2
PID=$3
if [ -z "$PID" ]
then
   echo $0 timeout file pid
   echo "     e.g. 60 /tmp/log 16325"
   exit
fi
echo "stalekill: timeout=$TIMEOUT file=$FILE pid=$PID"    

sleep 1

SIZE=0
while true
do
   sleep 3

   NEWSIZE=$(stat -c%s "$FILE")
   if [ "$NEWSIZE" -eq "$SIZE" ]
   then
      echo "stalekill '$FILE' unchanged; $SIZE"

      sleep $TIMEOUT
      NEWSIZE=$(stat -c%s "$FILE")

      if [ "$NEWSIZE" -eq "$SIZE" ]
      then
         kill $PID || sleep 2 && kill -9 $PID
         exit
      fi
      echo "stalekill '$FILE' changed; $SIZE..$NEWSIZE"
   else
      echo "stalekill '$FILE' changed; $SIZE..$NEWSIZE"
   fi
   SIZE=$NEWSIZE 
done

Invoked as follows (in a loop):

./offlineimap.py >> ./log.txt 2>&1   &   stalekill 25 ./log.txt $!

The timing is not overly professional, and it would be even nicer if it also checked if the process already terminated itself; but basically did the feat.

But nevertheless, if anybody knows a different or more standardized solution to this task: accepted answer tick still available.

Improve this answer
3
  • I found your post with the same (exact) problem. Is this still your latest script / solution?
    – CPBL
    Jan 10, 2015 at 18:17
  • @CPBL Sorry, that's too long ago :| Switched to a Perl wrapper IIRC (don't have it around), but then a different IMAP fetching approach later. Meanwhile just got content with IMAP client polling.
    – mario
    Jan 10, 2015 at 19:10
  • Just a side note, producing output does still not mean that it is syncing correctly as the output may be only errors/warnings. While my setup is certainly also flawed I use a post sync hook to write the last timestamp of a sync and check on that.
    – user640916
    Dec 26, 2017 at 14:11
1

Give a try to http://pyropus.ca/software/getmail/. You can set the timeout for each IMAP account.

Improve this answer
2
  • Thanks. That's what I tried first. But both getmail and fetchmail do not recover from the connection loss. They constantly refetch all mails for IMAP4 and POP3.
    – mario
    Jul 25, 2011 at 19:52
  • The problem is GMail, they don't use standard servers. I had the same problem for POP3. You need remove the file "~/.getmail/oldmail-imap.gmail.com-993-you@gmail.com" (or similar) for not download again the same mails after of the connection problems.
    – Rufo El Magufo
    Jul 25, 2011 at 20:09
0

You might want to take a look at Monit which can probably do what you're looking for.

Improve this answer
1
  • That's more intended for monitoring system deamons, not ad-hoc processes. And while it has a IF SIZE CHANGED test, doesn't fit my case. Interesting nevertheless.
    – mario
    Jul 25, 2011 at 14:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.