I need to run a python script using a specific anaconda environment, since I need to execute this script several times per day I would like to do it using the linux crontab file (/etc/crontab).
When I launch the script from the linux command line it works fine:
I added the process to the crontab file:
SHELL=/bin/sh PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin # m h dom mon dow user command 57 08 * * * user ~/anaconda2/envs/py36/bin/python ~/production/web_scraping.py | touch ~/test.txt
The line is correctly executed by cron, this is the cron log:
user@exmachina:~$ sudo cat /var/log/syslog | grep web_* Jan 2 08:57:02 localhost CRON: (user) CMD (~/anaconda2/envs/py36/bin/python ~/web_scraping.py | touch ~/test.txt)
| touch test.txt to the cron line to ensure the line is executed, and the test.txt file is generated.
The python script uses Selenium to extract data from a web page, by means of the chrome webdriver. Can this be the reason the script runs fine from the command line but not from the cron file?
Thanks for your help!