I would like to understand the for-loop below and perhaps simplify it. For example, I'd like to concatenate the rem files for each sample in the directory.




list=`for i in *rem*.1.fq; do echo $i | cut -f 1 -d \.; done`
for i in $list; do cat $i.rem.1.fq $i.rem.2.fq > $i.rem.b.fq; done

Can I do this without making a list? What does the cut -f 1 -d do? And why does cat $i.rem.1.fq work but not cat $i.1.fq if the the rem part of the file name is in between the two * on the list? Does that mean it captures everything before *rem* (e.g. file1)?

  • As a general comment, please note that the "backtick-notation" for command-substitutions is deprecated, and the $( ... ) notation should be used instead, (i.e. list="$(for i in ..... done)" in your example). – AdminBee Mar 6 '20 at 8:20


for i in *.rem.1.fq; do
    cat -- "$i" "${i%.1.fq}.2.fq" > "${i%.1.fq}.b.fq"

Maybe add a check for the existence of files:

for i in *.rem.1.fq; do
    if [ -e "${i%.1.fq}.2.fq" ] && [ ! -e "${i%.1.fq}.b.fq" ]; then
        cat -- "$i" "${i%.1.fq}.2.fq" > "${i%.1.fq}.b.fq"

The method presented in the question is error-prone – if a file contained a space, the second for loop might not function correctly.

cut -f 1 -d. cuts a string into fields (delimited in this case by .), and outputs the requested fields (in this case, just the first). If given the string file 1.whatever, it would output file 1. Again, this is error prone given the glob pattern *rem*.1.fq could return filenames with anyremthing.1.fq – the * wildcard matches anything (including nothing).

A better option is to do a single loop and use a parameter expansion, with some form of substitution inside the loop to match other files with relevant names.

  • Above, the glob pattern *.rem.1.fq is used – you may wish to narrow this down further – eg. file[0-9].rem.1.fq.
  • ${param%string} is used in the loop to remove the suffix .1.fq. Many shells also support other forms of parameter expansion substitutions – eg. ${param/string/repl}.

Also it's generally a good idea to quote all "$param" or "$(command)" substitutions – otherwise most shells will apply field splitting and filename generation and you might end up trying to cat file 1 instead of cat 'file 1', for example.

Also don't forget the -- to mark the end of options if you can't guarantee file names won't start with -.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.