0

I have a crontab root file that looks like this: 

lab-1:/var/www/cdd# crontab -l
# do daily/weekly/monthly maintenance
# min   hour    day     month   weekday command
*       *       *       *       *       /etc/scripts/script1
*/15    *       *       *       *       /etc/scripts/script2
0       *       *       *       *       /etc/scripts/script3

I can see that all the jobs are triggered by running this command:

lab-1:/var/www/cdd# cat /var/log/messages | grep cron.info
Mar 15 13:00:00 lab-1 cron.info crond[7897]: USER root pid 26217 cmd /etc/scripts/script2
Mar 15 13:00:00 lab-1 cron.info crond[7897]: USER root pid 26219 cmd /etc/scripts/script3        
Mar 15 13:01:00 lab-1 cron.info crond[7897]: USER root pid 26293 cmd /etc/scripts/script1

The problem is that script3 (I've proven that script2 and script1 work) is not actually producing the expected output. It's supposed to create files in another folder. However, when I run it manually like so, it works just fine: 

 lab-1:/etc/scripts# bash script3

I'm not a real sys admin so not too sure what the best way is to go about troubleshooting this.

First thing that comes to mind is permissions. 

lab-1:/etc/scripts# ls -lah                            
total 24                                                                     
drwxr-xr-x    2 root     root        4.0K Mar 15 12:20 .                     
drwxr-xr-x   34 root     root        4.0K Mar 14 17:11 ..                    
-rwxr-xr-x    1 root     root        5.0K Mar 15 12:19 script3
-rwxr-xr-x    1 root     root        1.8K Mar 14 15:26 script1
-rwxr-xr-x    1 root     root        1.9K Mar 14 15:26 script2

Although... having said that, if it were a permissions problem would it even show up as being triggered / started in my /var/log/messages file?

How should I proceed?

EDIT 1

lab-1:/etc/scripts# ./script3 | head -n 4                                             
Working folder set to: /tmp/tmp.kOfhip                                                                        
*****Grab SQL Data from Remote Server: spp.mydomain.net *****                                             
COPY 344                                                                                                      
Warning: Permanently added 'spp.mydomain.net,10.1.1.1' (ECDSA) to the list of known hosts.           
Evaluate /tmp/tmp.kOfhip/spp.mydomain.net.db                                                          
lab-1:/etc/scripts#

EDIT 2

This is what my script looks like:

https://paste.pound-python.org/show/90vAlrOsAYP0CtYqNWfl/

As you can see, I'm creating a temporary folder and doing all my work in there.

EDIT 3

To prove to myself that it's not because of lines like line 9, I've commented out everything except lines 1 though 15. I added line 16 that does this:

 echo "done" >> /tmp/results.txt

ANd then I changed the schedule of the job to run every two minutes from one hour. i can see that it's run 3 times already. I guess I will continue with this approach until I find something that won't work / blows up. I don't quite understand the comment made below about using a PATH variable... but I guess I will google it.

EDIT 4

I changed the crontabs root file so it outputs the results of script3 to a file and this is what i see: 

Working folder set to: /tmp/tmp.GeNGDJ
*****Grab SQL Data from Remote Server: servername *****
COPY 344
Warning: Permanently added 'spp.mydomain.net,10.1.1.132' (ECDSA) to the list of known hosts.
Permission denied (publickey,keyboard-interactive).
Evaluate /tmp/tmp.GeNGDJ/spp.mydomain.net.db
cat: can't open '/tmp/tmp.GeNGDJ/spp.mydomain.net.db': No such file or directory

So it's dying while trying to scp the file. The remote SQL runs just fine and shows output. But as you can see I'm getting a permission denied But if i run the same command manually, it seems to work. Will have to keep poking around. Will try dumping the ENV like is suggested in the answer below.

|improve this question
  • please edit your question, and add the content of script3. It might be useful to compare it to script1 or script2 which seems to work. – Yaron Mar 15 '18 at 13:12
  • What's going on when you try to run script3 in this way: go to /etc/scripts directory and do ./script3. Could you provide output head -n 4 script3? – mariaczi Mar 15 '18 at 13:13
  • @mariaczi please see EDIT 1 in my original question. Thanks. – dot Mar 15 '18 at 13:16
  • @Yaron I actually haven't proven yet that script 1 or 2 works. I'm in the middle of doing that now... since I've discovered that script3 for sure doesn't work. will post an update once I know. – dot Mar 15 '18 at 13:17
  • 1
    The usual mistakes are a) use of relative instead of absolute path names, b) assume a working directory. When you tried bash script3 your CWD is /etc/scripts but when cron starts a script it's CWD is usualy / however it's not defined, so use absolute path names. To test this cd / ; bash /etc/scripts/script3 and see what your script does. – X Tian Mar 15 '18 at 13:19
2

A common mistake when writing scripts that will be executed later by cron, is that you assume the script will have exactly the same environment that you have when you are logged in and you are developing it. It hasn't!

Write a script4 which contains the following line

OFILE=/tmp/crons.environment
(/usr/bin/whoami
/usr/bin/env ) > $OFILE 2>&1

And get cron to run that

Now compare the output in /tmp/crons.environment to what you get when you just type env

Your script assumes for example that $PATH is setup correctly to find all the programs you execute, you are also querying a database, there could be further Environment variables required for those commands to run correctly.

To check the output from the cron job. Temporarily modify the command run by cron and redirect stdout and stderr to a known file, Like I did above.

0       *       *       *       *       /etc/scripts/script3 > /tmp/s3.out 2>&1
|improve this answer
  • i love the last tip... about directing the output! cool. i'll give that a try right now!! Thank you!! +1 just for suggesting that to me. Very useful as a troubleshooting tip ... which is really what I'm looking for. – dot Mar 15 '18 at 13:57
  • 1
    XTian, please see EDIT 4. Now that I have the output dumping successfully to a file, I'm going to check the env stuff as you suggest. Thanks. – dot Mar 15 '18 at 14:23
  • 1
    Marking this as the answer although i haven't found the fix. I think you've just taught me how to fish. Thanks! – dot Mar 15 '18 at 14:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.