0

I am testing out a Raspberry Pi with the aim to use it in a production system for logging manufacturing data. All is working well and I have been recording test data over the last month.

For redundancy and risk management I have a bunch of Python scripts that need to be scheduled.

In the past I have been working on Windows Server environments and the Task Scheduler did everything I needed. Logging issues, retrying tasks on failure, notifications and much more.

I have been playing with cron over the last few weeks and I have wasted too much time. The log files are not helpful and since I don't have an MTA installed the real errors get lost.

Any alternative that could make scheduling easier rather than harder?

Added detail on the issue:

  • I have tried the ">>" in the past with little luck. If that is the normal way to do it I will give it another go.
  • The Python scrips runs fine if I run it manually. But as soon as I put it in crontab it stops working and without a log I can't resolve it or at least understand the problem.
  • It seems to be an issue with writing files. The cron jobs that are working, are python scripts that connects to bluetooth devices and then logs the data in a MySQL db.
  • The Cron log is trying to report output from jobs but cause MTA is not setup it just reports the MTA issue and gives me zero extra info in the actual issue. (/var/log/cron.log this is the main log,right?)
  • I will look into MTA, internet connectivity is an issue and might not be available at all times.
  • My data logging jobs are in "crontab -e" and I have one job that is creating image files that is sitting in "sudo crontab -e". (two different crontab files I take it)
  • All of my cron jobs are similar to these : 00 6 * * * python /app/test.py
  • cron is normally not written in uppercase. I was a bit confused to figure out if you actually mean the job scheduler... – Manuel Faux Apr 7 '15 at 10:42
  • 1
    I believe this question is too general. Could you specify what your problem with cron is? We could then advise you how to amend them. Is it that your scripts are not run? Is it that the output is logged improperly? Or what? Please be more specific. – Erathiel Apr 7 '15 at 11:03
3

Actually, cron is production-ready. It's been battle-tested so many times it's hard to accuse it of malfunctioning. What you might be experiencing is issues resulting from simple errors. It would help a lot if you specified what your problems with cron are, exactly.

As already pointed out by gaueth, you can append >> /tmp/somefile 2>&1 to you command in crontab. What this does is:

  • >> /tmp/somefile means 'append the stdout stream of the command to /tmp/somefile'. This simply writes everything that would have otherwise been printed on your terminal into the file specified. Please note the use of >> (append) instead of > (overwrite).
  • 2>&1 means 'send stderr to stdout'. In human terms: write the output of any errors in the same place where you write standard output (explained above)

Using this you will have a full log of what happened after cron executed your script. Please note it may be a good idea to make the script print out current date (as well as some other things, perhaps) so that you have this data in the log file.

Another thing to bear in mind is the PATH variable. Cron runs in a slightly different environment than your interactive terminal session so it's usually a good idea to include the output of echo $PATH in the script itself or in the crontab line, like so: */20 * * * * export PATH=<paste output of echo $PATH here>; <command>.

  • Thanks for the help. I finally managed to sort it out. My problems was with the "2>$1" part. When my Python scripts failed they did not raise the appropriate error level. (This is due to exception hadling) For now I am using just the >> till things settle down. – Bertus Kruger Apr 10 '15 at 19:56
  • @BertusKruger Mind you, it's 2>&1, not 2>$1 as you have written. There's an ampersand there, not a dollar sign. I guess it's just a typo, but worth drawing attention to, anyway :) – Erathiel Apr 16 '15 at 8:21
1

You can use > /tmp/logfile 2>&1 after your crontab scheduled command to redirect output from the commands to a log file. Also check out gnome-schedule for a graphical UI to play around with crontab and at.

1

The production-ready scheduling facility on Unix systems is cron.

Logging issues

Cron logs what it does to the system logs. It sends the output of jobs, if any, over email.

retrying tasks on failure

Cron doesn't do that, for good reason. Retrying tasks on failure is business logic. How would the system know which failures should lead to retrying and when, and which ones are fatal errors? If you need to try again, either make your task try again as desired or schedule multiple jobs at different times.

notifications

See logging above. Normally successful jobs don't lead to a notification, only to a log entry. If you want a notification on job completion, add that to your task's script using your preferred notification mechanism. (Unlike Windows which has a quasi-mandatory GUI, Unix doesn't run batch jobs in a GUI environment, so there's no immanent GUI-based notification facility.)

much more.

What?

since I don't have an MTA installed the real errors get lost.

“Doctor, it hurts when I do this.”

Install an MTA. Although some distributions omit it from the base installations, an MTA capable of sending email out is an expected component of any Unix installation.

0

I have grown quite fond of Jenkins CI for all kinds of 'cron on steroids' tasks. I use it to make and restore backups, clean up stale files/dirs, custom routines involving multiple steps etc... Basically anything that can be automated. The logging is great and the features you seem to need are either in there by default or easily added with one of the many plugins that are available.

Give it a spin, you will wonder how you did without after getting to know the CI butler.

  • Thanks for the suggestion! We use at work and it did not "click" be usefull for that too.... – Bertus Kruger Apr 8 '15 at 1:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.