I have the following Perl scripts (though this applies to Python and other scripting languages): script1.pl, script2.pl, script3.pl

The way these scripts where written, users execute them with input flags, and the output is a file saved.

perl script1.pl --i input1.tsv   ## this outputs the file `outputs1`
perl script2.pl --i outputs1     ## this outputs the file `outputs2`
perl script3.pl --i outputs2     ## this outputs the file `final_output`

Now, I would like to create an executable bash script that allows users to simply work with input1 and get the output in return final_output.

Here's how I would do this with only one perl script, execute.sh:


source ~/.bash_profile


perl script1.pl $FLAG1

which could be run on the command line execute.sh input1.tsv

For my example with three scripts, how would I pipe the intermediate outputs into the intermediate scripts to created one execute.sh script, e.g. outputs1 into script2.pl, then outputs2 into scripts3.pl, etc.?

Isn't there a way for me to do this without rewriting the perl/python scripts?

  • Why can't you rewrite the scripts? Reading from standard in (or from files) is merely a while (readline) { ... in Perl, and outputting to standard out likewise trivial. – thrig Jan 31 '17 at 0:16

The 3 lines you wrote to demonstrate what you want are a valid bash script: a script is nothing more than a sequence of commands executed one after the other by a shell.

The only additional code you might want would be something to clean up the temporary files produced by script1.pl and script2.pl. Further, you might want to use absolute paths to specify where script{1,2,3}.pl live.

trap 'rm outputs1 outputs2' EXIT

perl script1.pl --i "$1"
perl script2.pl --i outputs1
perl script3.pl --i outputs2

$1 represents the value of the first argument passed to the script, so it would be called with

execute.sh input1.tsv

as desired.

You can't really set up pipes between the three scripts, because they appear to be hard-coded to produce specific output files.

| improve this answer | |
  • Well, I actually don't know the exact output file names---I do know the extensions though. "You can't really set up pipes between the three scripts, because they appear to be hard-coded to produce specific output files." This makes me think I'm asking for something which is not possible – ShanZhengYang Jan 31 '17 at 23:50
  • 1
    Yeah, if the scripts produce a file with an unknown name, you are in a bit of a jam. A well-designed script either writes to standard output or provides an option to specify the name of the output file (or at least construct the output file name from known pieces, like the name of the input file). – chepner Feb 1 '17 at 2:32

Assuming the output files are created locally, ie. in the directory where the script is located, you can try to run the scripts in a temporary working directory, to which you can copy the scripts before executing them, and in which no other processes will create files. Then you will be able to control and operate on the new files created by particular scripts.

You can create the temporary directory in /tmp or your active user directory. Add to the end of your script a command that will tear down the directory. If you want any output files to be saved elsewhere, you can copy them at the right time to the right place.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.