cron - production-ready alternative

Question

I am testing out a Raspberry Pi with the aim to use it in a production system for logging manufacturing data. All is working well and I have been recording test data over the last month.

For redundancy and risk management I have a bunch of Python scripts that need to be scheduled.

In the past I have been working on Windows Server environments and the Task Scheduler did everything I needed. Logging issues, retrying tasks on failure, notifications and much more.

I have been playing with cron over the last few weeks and I have wasted too much time. The log files are not helpful and since I don't have an MTA installed the real errors get lost.

Any alternative that could make scheduling easier rather than harder?

Added detail on the issue:

• I have tried the ">>" in the past with little luck. If that is the normal way to do it I will give it another go.
• The Python scrips runs fine if I run it manually. But as soon as I put it in crontab it stops working and without a log I can't resolve it or at least understand the problem.
• It seems to be an issue with writing files. The cron jobs that are working, are python scripts that connects to bluetooth devices and then logs the data in a MySQL db.
• The Cron log is trying to report output from jobs but cause MTA is not setup it just reports the MTA issue and gives me zero extra info in the actual issue. (/var/log/cron.log this is the main log,right?)
• I will look into MTA, internet connectivity is an issue and might not be available at all times.
• My data logging jobs are in "crontab -e" and I have one job that is creating image files that is sitting in "sudo crontab -e". (two different crontab files I take it)
• All of my cron jobs are similar to these : 00 6 * * * python /app/test.py

Answer 1

Actually, cron is production-ready. It's been battle-tested so many times it's hard to accuse it of malfunctioning. What you might be experiencing is issues resulting from simple errors. It would help a lot if you specified what your problems with cron are, exactly.

As already pointed out by gaueth, you can append >> /tmp/somefile 2>&1 to you command in crontab. What this does is:

• >> /tmp/somefile means 'append the stdout stream of the command to /tmp/somefile'. This simply writes everything that would have otherwise been printed on your terminal into the file specified. Please note the use of >> (append) instead of > (overwrite).
• 2>&1 means 'send stderr to stdout'. In human terms: write the output of any errors in the same place where you write standard output (explained above)

Using this you will have a full log of what happened after cron executed your script. Please note it may be a good idea to make the script print out current date (as well as some other things, perhaps) so that you have this data in the log file.

Another thing to bear in mind is the PATH variable. Cron runs in a slightly different environment than your interactive terminal session so it's usually a good idea to include the output of echo $PATH in the script itself or in the crontab line, like so: */20 * * * * export PATH=<paste output of echo $PATH here>; <command>.

Answer 2

You can use > /tmp/logfile 2>&1 after your crontab scheduled command to redirect output from the commands to a log file. Also check out gnome-schedule for a graphical UI to play around with crontab and at.

Answer 3

The production-ready scheduling facility on Unix systems is cron.

Logging issues

Cron logs what it does to the system logs. It sends the output of jobs, if any, over email.

retrying tasks on failure

Cron doesn't do that, for good reason. Retrying tasks on failure is business logic. How would the system know which failures should lead to retrying and when, and which ones are fatal errors? If you need to try again, either make your task try again as desired or schedule multiple jobs at different times.

notifications

See logging above. Normally successful jobs don't lead to a notification, only to a log entry. If you want a notification on job completion, add that to your task's script using your preferred notification mechanism. (Unlike Windows which has a quasi-mandatory GUI, Unix doesn't run batch jobs in a GUI environment, so there's no immanent GUI-based notification facility.)

much more.

What?

since I don't have an MTA installed the real errors get lost.

“Doctor, it hurts when I do this.”

Install an MTA. Although some distributions omit it from the base installations, an MTA capable of sending email out is an expected component of any Unix installation.

Answer 4

I have grown quite fond of Jenkins CI for all kinds of 'cron on steroids' tasks. I use it to make and restore backups, clean up stale files/dirs, custom routines involving multiple steps etc... Basically anything that can be automated. The logging is great and the features you seem to need are either in there by default or easily added with one of the many plugins that are available.

Give it a spin, you will wonder how you did without after getting to know the CI butler.