I am trying to make a .txt file that contains only one line gathered from other 30 .log files. To extract only one line I used this:

$ sed -n '/Num mapped reads/p' /home/travc/seq_v2/AgamP4_v2/samples/ERS224561/qualimap/qualimap.log > /data/home/odkirling/Mali/Yeah1.txt

It works great but now I need to do it for other 29 files, how can I do that?

  • 1
    What are those other 29 files named? One simplistic solution would be to just repeat that command for the other 29 filenames. Is there a filename pattern that's common to those 30 files and only to those files? – Jeff Schaller Nov 14 '18 at 20:55
  • Can you provide a bit more information on the directory structure where the log files are located? – Jyotish P Nov 14 '18 at 20:57
  • This is the path for the file home/travc/seq_v2/AgamP4_v2/samples/ERS224561/qualimap/qualimap.log. The only thing that changes is the name of the file "ERS224561" – user117970 Nov 14 '18 at 20:58
  • Please do not add details in a comment. Add them in an edit to your question. – DopeGhoti Nov 14 '18 at 21:28
  • There's a tool that was designed specifically for this kind of job, it's called grep. – don_crissti Nov 14 '18 at 21:41

You can try something like

for filepath in $(ls -1 /home/travc/seq_v2/AgamP4_v2/samples/*/qualimap/qualimap.log);
  sed -n '/Num mapped reads/p' $filepath >> /data/home/odkirling/Mali/Yeah1.txt

This will need the path before and after * to be exactly the same.


If your files are in a sequence, say ERS224561 to ERS224591, you can use seq 61 91. In this case, the script will be

for filenum in $(seq -w 30 150);
  sed -n '/Num mapped reads/p' "/home/travc/seq_v2/AgamP4_v2/samples/ERS2245$filenum/qualimap/qualimap.log" >> /data/home/odkirling/Mali/Yeah1.txt

Or let's say you have a list of directories in a text file, in the following format


then you can do something like

for dirname in $(cat dir_names.txt);
  sed -n '/Num mapped reads/p' "/home/travc/seq_v2/AgamP4_v2/samples/$dirname/qualimap/qualimap.log" >> /data/home/odkirling/Mali/Yeah1.txt

dir_names.txt is the file containing your directory names. Similarly, you should be able to replace any part of the file path based on your requirements.

  • Jyotish, Thanks that works great. The only concern I have is that your code extracts the info from my whole database. What if I want to extract info from 30 out of 150 files? – user117970 Nov 14 '18 at 22:35
  • Updated the answer. Hope it helps. – Jyotish P Nov 15 '18 at 7:17
  • Thanks a lot, that is exactly what I needed and works perfectly! – user117970 Nov 17 '18 at 19:55

sed operates on an input stream that doesn't need to be limited to a single file; use multiple files instead, e.g. by supplying wild card characters in the files' paths like .../ERS*/.../*log

  • Thanks. So, that means that I have to write the name of each file li this /filename/ filename/ filename/...n/ – user117970 Nov 14 '18 at 22:24
  • Try to analyse, understand and apply my example. – RudiC Nov 14 '18 at 22:29
  • Thanks, i am trying to understand but i do not... – user117970 Nov 14 '18 at 22:33
  • @user117970 - well, this is the better answer but any absolute novice would have troubles understanding it... you need to do some serious reading up on globs/wildcards/filename patterns in order to understand it (at which point you will no longer ask this kind of question). Also, as I said above, the tool used here (sed) isn't the most suited for this task (though it does the job fine, no question). – don_crissti Nov 15 '18 at 12:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.