I have a Python / JS server combo that I need to keep running at all times. The Python server has a tendency to crash, as it's running on a server with 1GB of RAM.
I've written a script called "monitor.sh" which is set in crontab to run every minute, but it seems to fail whenever it tries to restart the python server.
I know that it is running, I can verify it is running by the timestamp it leaves every time it runs.
#!/bin/bash SHELL=/bin/sh PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/var/www/site.com echo "CHECKING" >> /var/www/site.com/monitor.log pidof python3 >/dev/null if [ $? -ne 0 ] ; then echo "Restarting Python App: $(date)" >> /var/www/site.com/monitor.log screen -dmS python_server python3 /var/www/site.com/app.py fi pidof nodejs >/dev/null if [ $? -ne 0 ] ; then echo "Restarting Inferno Server: $(date)" >> /var/www/site.com/monitor.log screen -dmS inferno_server nodejs /var/www/site.com/render_server.js fi
crontab -- (
crontab -e under the typical user)
* * * * * /var/www/site.com/monitor.sh
I will see "Restarting Python App" in the logs, and then nothing will happen. Attempting to log the output with
> out.txt 2> errors.txt produces nothing.
Things that have NOT worked so far:
- Using the system level crontab
- Running the server with sudo
chmod +xon all files
chmod -R 755on all files
os.chdir()from within the python script
Things that DO work:
- Running the app manually
- Running the script manually
- Running the script with any other Python file
- Monitoring and re-invoking the other JS server as necessary
The server is a bottle.py instance using Gunicorn.
Is there any way I can find out what's actually happening here? A log file I can check, or a way to actually get errors out of this command?