3

How do I write a bash script that goes through each directory inside a parent_directory and executes a command on specific file.

The directory structure is as follows:

Parent_dir/
  dir1/
   acc.bam
  dir2/
   acc.bam
  dir3/
   acc.bam
... around 30 directories

This is the command I want to use :

java8 -jar /picard.jar CollectRnaSeqMetrics REF_FLAT=/refFlathuman.refflat STRAND_SPECIFICITY=NONE I=acc.bam O=output
10

The usual idiom is

for d in Parent_dir/*/
do
     (cd "$d" && $command)
done

The for loop executes once for each directory directly within Parent_dir. For each of those directories, a sub-shell is spawned; in the sub-shell, we attempt to change into that directory (which might fail, e.g. if we have insufficient permission), and if we succeed, then execute the command. Whether or not we succeeded, the cd has no effect on the parent shell, so we don't need to worry about being in the wrong place there.


If you want to make it more robust, you might

    (cd "$d" && test -r acc.bam &&  $command)

to ensure that acc.bam exists and is readable in that directory. You might also add a test -w . to avoid trying to run the command in directories that are not writable.


P.S. None of the above is specific to Bash; you can use /bin/sh for it quite portably.

2

Loop over the files in the parent directory with for FILE in *; do ...; done. If the file is a directory, cd into the directory, execute your command, then cd ...

cd $parent_directory
for FILE in *; do
    if [ -d "$FILE" ]; then
        cd "$FILE"
        java8 -jar /picard.jar CollectRnaSeqMetrics REF_FLAT=/refFlathuman.refflat STRAND_SPECIFICITY=NONE I=acc.bam O=output
        cd ..
    fi
done

You could also use the filename in your command to avoid changing directories such as I="$FILE/acc.bam".

7
  • Could you please explain it clearly. acc.bam is not a directory Apr 13 '16 at 18:22
  • The first line iterates over all files in the parent directory. The second line tests that each file is a directory (the -d argument). If it is, it switches to the directory (cd "$FILE"), runs your command, then goes back to the parent directory (cd ..).
    – asmodean
    Apr 13 '16 at 18:29
  • can I run this in one line? Apr 13 '16 at 18:30
  • You can run this on one line, but you'll have to add semi-colons in the appropriate places. cd $parent_directory; for FILE in *; do if [ -d "$FILE" ]; then cd "$FILE"; <YOUR COMMAND>; cd ..; fi; done;
    – asmodean
    Apr 13 '16 at 18:34
  • If this answers your question, could you mark it?
    – asmodean
    Apr 13 '16 at 19:37
2

First, be sure that it is, in fact, necessary to have each of those directories as the working directory for an invocation of the command. Well-written programs don't care where their working directory is, as long as they are given working paths for their arguments (and, if necessary, suitable $TMPDIR etc.).

If your command will work with actual pathnames, like

java8 -jar /picard.jar CollectRnaSeqMetrics \
    REF_FLAT=/refFlathuman.refflat STRAND_SPECIFICITY=NONE \
    I=subdir/acc.bam O=subdir/output

then you should be able to simply substitute the directory in each iteration of a loop:

const_args='CollectRnaSeqMetrics REF_FLAT=/refFlathuman.refflat STRAND_SPECIFICITY=NONE'
for d in Parent_dir/*/
do
     java8 -jar /picard.jar $const_args I="${d}acc.bam" O="${d}output"
done

(note, because $d includes a trailing /, I didn't add one where it's substituted into the command).

P.S. None of the above is specific to Bash; you can use /bin/sh for it quite portably.

2

All you need to do is to go in every directory in your parent_directory and run java command. To return back to the parent_directory use:

cd ..

To get the list of directories in parent_directory you can use:

find -mindepth 1 -maxdepth 2 -type d

It will list only directories and only for the one nesting level.

The loop looks like this:

cmd="java8 -jar /picard.jar CollectRnaSeqMetrics REF_FLAT=/refFlathuman.refflat STRAND_SPECIFICITY=NONE I=acc.bam O=output"
for path in `find -mindepth 1 -maxdepth 2 -type d` 
do
    cd "$path"
    $cmd
    cd ..
done

Java command was placed in a separate variable to make script more readable.

2
  • 2
    Could you please explain it clearly Apr 13 '16 at 18:24
  • You might want cd "$path" || continue so that we don't cd .. out when we hit an unreadable directory. Sep 11 '18 at 12:19
2

With find implementations that support -execdir (BSD, GNU, sfind at least):

find ParentDir -name acc.bam -execdir \
  java8 -jar /picard.jar CollectRnaSeqMetrics \
  REF_FLAT=/refFlathuman.refflat \
  STRAND_SPECIFICITY=NONE I=acc.bam O=output \;
2
  • Should I=acc.bam be I={} ?
    – iruvar
    Apr 14 '16 at 14:10
  • 2
    @1_CR, {} is always going to be acc.bam (or ./acc.bam with GNU find), and not all find implementations accept {} as part of an argument (sfind doesn't). Apr 14 '16 at 14:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.