3

I want to select the columns from a big file ,based on the rows from another file. This problem has been answered here but both the solutions does not work.

https://www.biostars.org/p/166527/

DATA.TXT

head0 head1 head2 head3 head4  
1 25 1364 22 13  
2 10 215 1 22  

LIST.TXT

head0  
head4 

Desired output:

head0 head4  
1 13  
2 22

The first command of the second solution works.

cat file1.txt | cut -f$(grep -wFf file2.txt TMP | cut -f1 | sed -e ':a' -e 'N' -e '$!ba' -e 's/\n/,/g') > OUTPUT cut: option requires an argument -- f

Any suggestions?

Thanks

Ron

  • 1
    You may receive some downvotes for using a URL instead of putting your code in the question. – glenn jackman Jul 18 '19 at 18:53
2

I had the same error on my Mac.

  1. you need to ensure that your files are tab-separated, and have no leading or trailing whitespace.
  2. that sed command can be replaced by paste -sd,
  3. leaving:

    cut -f"$(grep -wFf LIST.TXT TMP | cut -f1 | paste -sd,)" DATA.TXT 
    

But, use the awk solution, it's much tidier and doesn't use a tmp file, and is much more forgiving about whitespace:

awk -v OFS='\t' '
    NR == FNR {header[$1] = 1; next}
    FNR == 1 {for (i=1; i<=NF; i++) if ($i in header) column[i] = 1}
    {for (i in column) printf "%s" OFS, $i; print ""}
' headers file

After a bit of play, the pipeline that works (whiteout needing a tmp file) is

cut -f"$(head -1 DATA.TXT | tr '\t' '\n' | nl | grep -Fwf LIST.TXT | awk '{print $1}' | paste -sd,)" DATA.TXT
| improve this answer | |
0

Assuming you want the columns output in the order defined by list.txt:

$ cat tst.awk
NR==FNR {
    name2out[$1] = ++numOutFlds
    next
}
FNR == 1 {
    for (inFldNr=1; inFldNr<=NF; inFldNr++) {
        outFldNr = name2out[$inFldNr]
        out2in[outFldNr] = inFldNr
    }
}
{
    for (outFldNr=1; outFldNr<=numOutFlds; outFldNr++) {
        inFldNr = out2in[outFldNr]
        printf "%s%s", $inFldNr, (outFldNr < numOutFlds ? OFS : ORS)
    }
}

$ awk -f tst.awk list.txt data.txt
head0 head4
1 13
2 22
| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.