0

I have a problematic bash script and I really appreciate your help in identifying the problem. The bash script Script.sh is as follows:

#!/bin/bash
./command1 -i inFile.txt -o outFile1.bin > log1.txt
./command2 -i inFile.txt -o outFile2.txt > log2.txt
./command3 -i outFile2.txt -o outFile3.txt > log3.txt
...

Here, -i defines the input file and -o defines the output file. There are some dependencies, for example the output file of command 2 is used as the input file for command 3.

I call the script using

nohup ./Script.sh > scriptLog.txt 2> scriptErr.txt &

Here is the strange thing that happens: the script runs to completion without any errors (scriptErr.txt is empty). The output files (outFile1.txt, outFile2.txt and outFile3.txt) are produced and are correct as much as I can say. The jobs command shows that the run is complete (Done). The command

ps aux | grep ./Script.sh

returns nothing (only the grep itself). However, the command

ps aux | grep ./command1

shows that command1 is still running (that is not the case about command2 and command3). And here is what confuses me:

This script is supposed to be sequential. How is it possible that commands 2 and 3 are completed while command 1 is not completed yet?

If command 1 is not completed, then how is that the output file is correctly produced?

Thank you very much!

8
  • 1
    Could command1 be running from any other source? Did you maybe run nohup command1 & in test a run and it is still stuck? Try the following: after each step, echo the PID of the previous command to a file and compare that to your ps aux | grep command1. Do they match? (PID of previous command: echo $!). Alternatively echo a confirmation string after each step :comman1 <...> && echo "command1 finished"
    – FelixJN
    Commented Nov 11, 2019 at 18:48
  • 2
    Are you positive that the grep ./command1 did not find the grep command itself?
    – Jeff Schaller
    Commented Nov 11, 2019 at 19:08
  • If you replace grep ./command1 with grep ./c[o]mmand1 is it still the same?
    – rush
    Commented Nov 11, 2019 at 19:27
  • You will have to give more information on command1. We know nothing about it, some commands go to background in certain conditions, so the behaviour could be completely normal. What does command1 do and how? (single command, another script, many commands on a pipe ...) Commented Nov 11, 2019 at 20:47
  • Thank you very much @Fiximan. That is one possibility and I will check it in the future runs.
    – sanaz
    Commented Nov 13, 2019 at 0:18

2 Answers 2

0

The problem here comes from nohup + &.

You use those to send the process to background. & also forks you current process. So instead of out Script.sh you get two (or multiplied by number of times you use &). New Script.sh process are children of the process you started. commandX is started from forked Script.sh.

Once main script process is finished, the parent Script.sh is gone, all the children are moved to init process. Those are different processes, just the name match and might introduce some confusion.

3
  • Thank you very much @rush ! I am a little confused. You mean the parent process and the child process run simultaneously? Then, what happens if they are both writing to the same output file as is the case in my script? If that is the case, then what should I do if my script performs some critical writes and I want to run it in a remote server so that after I logout and login again it still continues running?
    – sanaz
    Commented Nov 16, 2019 at 17:39
  • @sanaz I meant there are multiple processes with the same name, but it doesn't mean they perform the same instructions and write to the same files (unless instructed so in code). In your case there will be multiple Script.sh processes, but those would control different children (command1, command2, command3, etc).
    – rush
    Commented Nov 16, 2019 at 18:24
  • Thank you very much @rush!
    – sanaz
    Commented Nov 17, 2019 at 0:46
0

It's easy to construct a script that stays running despite returning:

$ ./still_running 
I'm done
$ ps aux | grep '[s]till_running'
anthony  25666  0.0  0.0   7420  2092 pts/31   S    16:43   0:00 /bin/bash ./still_running

That script is fairly trivial:

#!/bin/bash

(
    sleep 5m
    echo 'done now'
) &

echo "I'm done"

It starts a subshell, which it runs in the background (due to the the ampersand), and it stays running even when the parent process exits.

It's impossible to say for sure what's happening with your script without seeing the script (ideally a minimal test case), but this or something similar is a possibility.

1
  • Thank you very much @derobert. Command1 runs an executable that converts a text file into a binary file (after processing it). It doesn't echo anything and it doesn't have any child processes. It is just a sequential C++ code. Also, the "Done" I mentioned was the status output by the command "jobs", it was not anything output by the command itself. Thank you again.
    – sanaz
    Commented Nov 13, 2019 at 0:31

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .