I some directories that contain a similarly named file eg (*Sample_name*.base.coverage.txt). And I would like to paste all of the *base.coverage.txt files together. I have something written, but its not quite right as I don't think I am actually pasting them together.

cd /g/bo/vir/analysis
find -type d  |
while read dir; do
if [ -f $dir/*.base.coverage.txt ]; then
    paste  $dir/*.base.coverage.txt >> paste_all_l30.p90.base.cov.norm


I am also thinking of a different loop method, but I am unsure how to get the list of directories into an array.

for x in $dir; do cat /$x/*.base.coverage.txt;done > paste_all_l30.p90.base.cov.norm

Any help? thanks


try this:

find . -type f -name "*.base.coverage.txt" -exec sh -c 'paste {} paste_all_l30.p90.base.cov.norm > tmp ; mv tmp paste_all_l30.p90.base.cov.norm' \;
  • Hmm, snap! (almost) – ams Apr 11 '12 at 12:41
  • Thanks, but this also just yields a file with one column. – awall Apr 11 '12 at 12:43
  • Try to change cat to paste like I did in my answer. Is it what you're looking for? – rush Apr 11 '12 at 12:53
  • find . -type f -name "*.fastx.allbest.l30.p90.base.cov.norm" -exec paste {} > paste_all_l30.p90.base.cov.norm\ ; find: missing argument to `-exec' – awall Apr 11 '12 at 13:01
  • That's not going to work anyway because appending to the file is the wrong thing. – ams Apr 11 '12 at 13:04

Try this:

find . -type f -name '*.base.coverage.txt' \
  | xargs paste > paste_all_l30.p90.base.cov.norm

This will work as long as there are not too many files. If there are a lot of file then this:

touch paste_all_l30.p90.base.cov.norm
for file in `find . -type f -name '*.base.coverage.txt'`; do
  paste paste_all_l30.p90.base.cov.norm $file > tmp
  mv tmp paste_all_l30.p90.base.cov.norm
  • The OP wants to paste, not cat – Peter.O Apr 11 '12 at 12:48
  • Ok, I missed that. I've updated the answer – ams Apr 11 '12 at 13:01

The easiest thing to recurse through a number of directories would be to write a short python script and use the os.path.walk function to recurse through the nested directories. For appending, you could use shutil.copyfileobj within the loop, checking filenames with a regex.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.