1

Can anyone tell me how to traverse subfolders and identify fastq files using bash script:

/project/samples/data/intensities/r1.fastq
/project/samples/data/intensities/r2.fastq
/project/samples/sample1.txt

I will be passing only the base directory /project/samples/ to my bash script and I need to traverse through the sub folders to get the fastq files.

5

With find command:

find /project/samples/ -name "*.fastq"
2

In bash (not plain sh), you can use the **/ wildcard pattern to traverse directories recursively. You need to set the globstar option first. For example, to run mycommand on every .fastq file under /project/samples:

#!/bin/bash
shopt -s globstar
for x in /project/samples/**/*.fastq; do
  mycommand "$x"
done

This is only available since bash 4.0 (so not on the default bash on OSX, but the feature is also available in zsh). In bash versions before 4.3, beware that it also traverses symbolic links to directories.

If you need to run a single command on all the files, you can:

mycommand /project/samples/**/*.fastq

However you may run into a limit on the length of the command line. It helps to use a relative path:

cd /project/samples
mycommand **/*.fastq

If that still doesn't fit, you'll need to find another solution to do whatever you need to do, that doesn't involve passing all the file names as arguments to that command at once.

1
#!/bin/bash

if [ ! -d "$1" ]; then
    printf 'Not a directory: %s\n' "$1" >&2
    exit 1
fi

find "$1" -type f -name "*.fastq" -print

This is a small script that will simply list the fastq files in the given directory.

If you want to execute a command for each file, the simplest way would be to change the find to

find "$1" -type f -name "*.fastq" -exec utility {} ';'

Where utility is some program that takes the path of a fastq file on the command line ({} will be substituted for the path of the fastq file).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.