1

I have multiple text files, e.g.,

sample1.tsv

## index of replication (iRep) - thresholds: min cov. = 5, min wins. = 0.98, min r^2 = 0.9, max fragments/Mbp = 175, GC correction min r^2 = 0.0    
# genome    ./bam/18097D-02-01.sam.sorted.sam
18097D-02-01_bin.11.fa  1.295179372
18097D-02-01_bin.13.fa  1.284880274
18097D-02-01_bin.15.fa  1.339609918
#   
## un-filtered index of replication (iRep)  
# genome    ./bam/18097D-02-01.sam.sorted.sam
18097D-02-01_bin.11.fa  1.295179372
18097D-02-01_bin.13.fa  1.284880274
18097D-02-01_bin.15.fa  1.339609918
#   
## raw index of replication (no GC bias correction) 
# genome    ./bam/18097D-02-01.sam.sorted.sam
18097D-02-01_bin.11.fa  1.298934455
18097D-02-01_bin.13.fa  1.2885746
#

sample2.tsv

## index of replication (iRep) - thresholds: min cov. = 5, min wins. = 0.98, min r^2 = 0.9, max fragments/Mbp = 175, GC correction min r^2 = 0.0    
# genome    ./bam/18097D-02-02.sam.sorted.sam
18097D-02-02_bin.11.fa  1.59665286
18097D-02-02_bin.13.fa  1.332990306
18097D-02-02_bin.14.fa  1.499196606
18097D-02-02_bin.6.fa   1.323465715
18097D-02-02_bin.9.fa   1.583302299
#   
## un-filtered index of replication (iRep)  
# genome    ./bam/18097D-02-02.sam.sorted.sam
18097D-02-02_bin.11.fa  1.59665286
18097D-02-02_bin.13.fa  1.332990306
18097D-02-02_bin.14.fa  1.499196606
18097D-02-02_bin.6.fa   1.323465715
18097D-02-02_bin.9.fa   1.583302299
#   
## raw index of replication (no GC bias correction) 
# genome    ./bam/18097D-02-02.sam.sorted.sam
18097D-02-02_bin.11.fa  1.603339021
18097D-02-02_bin.13.fa  1.366124796
18097D-02-02_bin.14.fa  1.502052999
18097D-02-02_bin.6.fa   1.324573575
18097D-02-02_bin.9.fa   1.618136032
#   

I want to extract everything from all the files from first ## to second ##, that means I want output like-

## index of replication (iRep) - thresholds: min cov. = 5, min wins. = 0.98, min r^2 = 0.9, max fragments/Mbp = 175, GC correction min r^2 = 0.0    
# genome    ./bam/18097D-02-01.sam.sorted.sam
**18097D-02-01_bin.11.fa 1.295179372
18097D-02-01_bin.13.fa 1.284880274
18097D-02-01_bin.15.fa 1.339609918**
#   
## un-filtered index of replication (iRep)
## index of replication (iRep) - thresholds: min cov. = 5, min wins. = 0.98, min r^2 = 0.9, max fragments/Mbp = 175, GC correction min r^2 = 0.0    
# genome    ./bam/18097D-02-02.sam.sorted.sam
**18097D-02-02_bin.11.fa 1.59665286
18097D-02-02_bin.13.fa 1.332990306
18097D-02-02_bin.14.fa 1.499196606
18097D-02-02_bin.6.fa  1.323465715
18097D-02-02_bin.9.fa  1.583302299**
#   
## un-filtered index of replication (iRep)

I was trying to do this, but no ouptput- sed -n '/##=/{s/.##=//;s/\S=.*//;p}' *.tsv > ../test.tsv

I actually want-

18097D-02-01_bin.11.fa 1.295179372
18097D-02-01_bin.13.fa 1.284880274
18097D-02-01_bin.15.fa 1.339609918

18097D-02-02_bin.11.fa 1.59665286
18097D-02-02_bin.13.fa 1.332990306
18097D-02-02_bin.14.fa 1.499196606
18097D-02-02_bin.6.fa  1.323465715
18097D-02-02_bin.9.fa  1.583302299

Thanks

  • Are these ** in the file ?! or is it to show what you want extracted ?! – pLumo Oct 16 '19 at 13:05
1
#!/bin/bash
sample_dir="/C/Users/testuser/Desktop/sample" 
out_file="/C/Users/testuser/Desktop/sample/output.tsv"

for file in "$sample_dir"/*
do
   count=0
   while read line; do
        if [[ "$line" == "##"* ]] || [[ "$line" == "#"* ]]; then
           count=$((count+1))
           if [[ count -ge 4 ]]; then
              echo -e "" >> $out_file            
              continue 2
           fi
            continue 
        else
           echo -e "$line" >> $out_file
        fi
    done < $file
done
1

Use awk:

awk 'FNR==1{p=0}
     FNR==1 && NR>1 {print ""}
     $0 ~ /^##/ {p++}
     p==1 && $0 !~ /^#/
' sample*

Output:

18097D-02-01_bin.11.fa  1.295179372
18097D-02-01_bin.13.fa  1.284880274
18097D-02-01_bin.15.fa  1.339609918

18097D-02-02_bin.11.fa  1.59665286
18097D-02-02_bin.13.fa  1.332990306
18097D-02-02_bin.14.fa  1.499196606
18097D-02-02_bin.6.fa   1.323465715
18097D-02-02_bin.9.fa   1.583302299

Explanation:

  • FNR==1{p=0} set pointer to 0 for each new file (FNR is the line number of the current file)
  • FNR==1 && NR>1 {print ""} Print empty line for every file except the first
  • $0 ~ /^##/ {p++} When line begins with ##, increase the pointer
  • p==1 && $0 !~ /^#/ If pointer is 1 (which is the case from the first ## to the second ##, and the line does not begin with a #, print it.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.