2

I have a command that needs to take multiple input files, which are originally from one directory and have certain file name patterns. For example:

In directory

/home/mydir/

I have files:

A.dat
B.dat
C.dat
readme.doc

I would like to learn how to pass all files with ending of ".dat" to this command, which then should look like:

command A.dat B.dat C.dat > /home/outputdir/output.dat

I could do it in python e.g. by storing the file names in a list, but how should I do it in shell please? Thanks a lot.

  • 1
    be careful that your output file name matches your input pattern (*.dat) - store it in a different directory or use a different filename/extension so that it's not accidentally included as an input file (it'll be created empty by the calling shell before the command is called) – Jeff Schaller Jun 20 '16 at 17:46
  • yes you are right. I edited the question also. – Helene Jun 20 '16 at 17:49
  • I'm puzzled why a simple shell glob such as ?.dat or possibly even *.dat wouldn't be acceptable. Helene, you do know that the shell itself expands a wildcard pattern and passes the matching set of files to the command, don't you...? – roaima Jun 20 '16 at 18:25
  • Thank you@roaima I didn't know that. I'm very new to shell script and thanks very much for pointing that out. – Helene Jun 20 '16 at 18:31
4

A standard UNIX shell will do something called globbing; this uses special characters to mean, for example, one character (?) or any number of characters (*). To use your example, you could run (where the initial $ represents your command prompt and not something you'd type):

$ command /home/mydir/*.dat > /home/outputdir/output.dat

Your shell will expand that to:

$ command /home/mydir/A.dat /home/mydir/B.dat /home/mydir/C.dat > /home/outputdir/output.dat

before actually calling the command. The * says "take any and all filenames in /home/mydir that end with ".dat".

For some variations of the command, given the same input files:

# all of the sample input files have a single letter before the ".dat"
$ command /home/mydir/?.dat > /home/outputdir/output.dat

# the square brackets say "any (one) of these characters"
$ command /home/mydir/[ABC].dat > /home/outputdir/output.dat
  • Happy to help; it' s important to realize when & what the shell does, so if bash is your shell of choice, have a look around the linked doc. – Jeff Schaller Jun 20 '16 at 18:33
2

The answer really depends on what tool you're trying to use. Some are designed to accept multiple files as parameters, which can be done with:

/path/to/some/tool file1 /path/to/file2 /path/to/lotsafiles/*

Others are designed to only accept one file as a parameter, and so will have to be repeatedly invoked with each file you want to address, which would be done thusly:

for file in file1 /path/to/file2 /path/to/lotsafiles/*; do
    /path/to/some/tool "$file"
done

To know whether or not you must only specify one file at a time, see the program's manual page (man tool).

  • Thanks for the reply. the tool could (in fact, have to) take multiple files, so I believe no loop is needed - but the problem here is that i need to a good way to locate these files. I don't know their name beforehand. – Helene Jun 20 '16 at 17:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.