0

I encountered a strange behavior (to me). I have written a larger script which is working well. When I try to start the script from a crontab-defined job, the following lines are causing issues:

down_ubuntu14=https://cloud-images.ubuntu.com/trusty/current/
ubuntu14=trusty-server-cloudimg-amd64-disk1.img
Ubuntu14_Date_Web=$(wget -qO- $down_ubuntu14 | grep $ubuntu14 | awk '{print $8 $9}' | sed -e "s/<.*>/ /g" | cut -d">" -f2 | awk '{print $2}')

In the last line, I strip down the download website to get the date of the specific download target.

soi       5076  5075  0 09:35 ?        00:00:00 wget -qO- https://cloud-images.ubuntu.com/trusty/current/
soi       5077  5075  0 09:35 ?        00:00:00 grep trusty-server-cloudimg-amd64-disk1.img
soi       5078  5075  0 09:35 ?        00:00:00 awk {print $8 $9}
soi       5079  5075  0 09:35 ?        00:00:00 sed -e s/<.*>/ /g
soi       5080  5075  0 09:35 ?        00:00:00 cut -d> -f2
soi       5081  5075  0 09:35 ?        00:00:00 awk {print $2}

It seems that the pipes are causing a line feed or start every part of the string into single processes.

Many thanks for your answers and recommendations: I try to make a more precise example.

If I setup a short script like this:

down_ubuntu14=https://cloud-images.ubuntu.com/trusty/current/
ubuntu14=trusty-server-cloudimg-amd64-disk1.img
Ubuntu14_Date_Web=$(wget -qO- $down_ubuntu14 | grep $ubuntu14 | awk '{print $8 $9}' | sed -e "s/<.*>/ /g" | cut -d">" -f2 | awk '{print $2}')
echo $Ubuntu14_Date_Web | tee /tmp/test
echo "Just another test line"  | tee -a /tmp/test

and start it from the ssh console I see the following in the test file:

cat /tmp/test
14-Jun-2018
Just another test line

In the crontab I enter the following line:

20 6 * * *      /home/soi/scripts/test.sh

Now the script is getting started by cron and the following I can see in the process list:

soi       6508  6507  0 06:20 ?        00:00:00 /bin/sh -c /home/soi/scripts/test.sh
soi       6509  6508  0 06:20 ?        00:00:00 wget -qO- https://cloud-images.ubuntu.com/trusty/current/
soi       6510  6508  0 06:20 ?        00:00:00 grep trusty-server-cloudimg-amd64-disk1.img
soi       6511  6508  0 06:20 ?        00:00:00 awk {print $8 $9}
soi       6512  6508  0 06:20 ?        00:00:00 sed -e s/<.*>/ /g
soi       6513  6508  0 06:20 ?        00:00:00 cut -d> -f2
soi       6514  6508  0 06:20 ?        00:00:00 awk {print $2}

... and only a space is written to the log file /tmp/test only the following is written to the log:

cat /tmp/test
Just another test line

I do not get the main cause for this issue. Hope someone can bring in some light here.

  • It's not unusual for each command in the pipeline to be it's own subprocess. What do you mean by "script fails in crontab" what is the error/unwanted output? – Jesse_b Jun 14 '18 at 14:52
  • You can also simplify your command to: wget -qO- "$down_ubuntu14" | grep "$ubuntu14" | awk '{print $9}' – Jesse_b Jun 14 '18 at 14:59
  • 1
    @Jesse_b ...or | awk -vp="$down_ubuntu14" '$0 ~ p { print $9 }' – ilkkachu Jun 14 '18 at 15:07
  • Hi Jesse_b, basically th script stops proceeding to the next loop. But at the moment i do not get the point, why everything is fine running from the script (e.g. called test.sh) but not from the crontab execution. – Franky Jun 14 '18 at 15:11
  • 2
    @Franky, what loop? The one internal to grep/sed/awk, or do you have one in the script itself? If yes, then edit the question to show the relevant parts. Actually, even better: edit the question to show a complete but minimal example of the script that produces the problem. – ilkkachu Jun 14 '18 at 15:26
2

Most implementations of cron do not start a shell with the process in it, but you a shell to parse out the pipe and run the two processes separately. If you add you commands to script.sh and call it from cron it will work.

  • Thanks, i will give it a try tomorrow. Actually i am not at the console. – Franky Jun 14 '18 at 15:08
  • Is there really some cron that doesn't run through a shell? I'm not saying there isn't, I just can't find one now. – ilkkachu Jun 14 '18 at 15:11
  • maybe anacron? en.wikipedia.org/wiki/Anacron Dunno just guessing – vfbsilva Jun 14 '18 at 17:53
  • @vfbsilva Anacron is not technically "a cron" (it has to be executed through cron). – Kusalananda Jun 15 '18 at 7:01
0

at least it seems to be something specific in the Linux installation. I have tried the script on five different boxes and Linux flavours and everywhere it works fine. So we decided to reinstall the box.

Many thanks for all your support and ideas. Very much appreciated.

Cheeers, Franky

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.