How do I write a script to execute the files in multiple directories?

The problem is this: I have many directories, and each has a data file to be read and analyzed by a python script (say, a.py). I don't want to "cd" to each of the directories and type "a.py". Outputs are saved in each directory.


find will work magic for you.

find -name datafile.dat -execdir ~/a.py '{}' \;

The find searches recursively in all subdirectories for files that match a set of rules and performs an action on them.

The -name rule will let you find files with a name that matches what you give it. You can use globbing, for example, "*.dat" would find all the .dat files.

If necessary, you can use -regex instead of -name to match with a regex pattern instead of a glob pattern, so you could do ".*\.dat$" to match all the .dat files.

The -execdir will execute whatever command you give it from the directory of the found file, replacing "{}" with the found file.

  • Nice. I should really learn find, it has flags for every conceivable situation – Michael Mrozek Nov 6 '10 at 4:32
  • The man page for find isn't great, but the info pages are extensive. – Shawn J. Goff Nov 6 '10 at 5:47
  • 1
    @Michael: Zsh has similar features; they're more cryptic but less typing. @Shawn: -execdir is useful, but not portable. @all: In bash 4 or zsh, you can do for x in **/datafile.dat; do (cd "$(dirname "$x")" && ~/a.py "$(basename "$x")"); done – Gilles 'SO- stop being evil' Nov 6 '10 at 13:16

You can probably just use a for loop:

for i in first_dir second_dir third_dir; do
    pushd $i

It will run pushd $i; a.py; popd with $i set to first_dir, then again with $i as second_dir, and finally $i as third_dir. pushd switches to the given directory, and popd switches back to where you were

  • 3
    Instead of pushd and popd you can use a subshell - (cd $i && a.py). This construct also protects against $i not existing - you dont want to popd if pushd did not work. – camh Nov 5 '10 at 6:57
  • 3
    @Michael, @camh, @sinyeo: Always put double quotes around variable substitutions., e.g. cd -- "$i". Also, protect commands against files beginning with - by passing -- before file names. – Gilles 'SO- stop being evil' Nov 5 '10 at 8:30
  • 2
    Ok, as always the comments are right, but I can't remember the last time I posted a tiny shell snippet without somebody commenting on something like this -- do you guys seriously write one-off loops with all these redundancies built in? You're protecting against directories not existing or having spaces or starting with dashes as though it needs to accept arbitrary input; it's a one-time for loop that's iterating over the static array "first_dir second_dir third_dir" -- none of those things are a problem – Michael Mrozek Nov 5 '10 at 14:14
  • 2
    @Michael: I strive to include proper quoting and appropriate generalizations in my public posts. I can not know when someone random reader is going to copy some bit of an answer into a context that is (unbeknownst to the reader) just different enough from the original context. Using quoting and other generalization techniques can greatly expand the breadth of situations where any given code snippet can be correctly applied. I also like to use proper quoting, etc. in my interactive use since I find that such practice forms habits that benefit me when writing full shell programs. – Chris Johnsen Nov 6 '10 at 2:00
  • 1
    @Michael: I totally agree with Chris; I'm in the habit of quoting even in one-liners, and using && when possible, because you just never know when you'll forget there's a space in something, or make a typo causing a key command to fail, and depending on when it bites you, you could have a mess to unravel. – Cascabel Nov 6 '10 at 2:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.