Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

Here is an example text file:

A B C D E F G
1 2 3 4 5 6 7
2 3 4 5 6 7 8
3 4 5 6 7 8 9

I would like to extract specific columns based on the values on the 3rd row, that is, 2 3 4 5 6 7 8. Let's say, I would like to extract all the columns with a value on the 3rd row larger than 5. That will be the final 3 columns. Hence, my goal is to select and generate the following:

E F G
5 6 7
6 7 8
7 8 9

Here is my code:

NR==3 {
    for (i=1; i<=NF; i++) {
        if ($i > 5)   x[j++] = i
    }
}
NR>= 1 {
  for (i=0 ;  i < j-1; i++ )
     printf("%s ",$x[i])
  printf("%s\n",$x[j-1])
}

However, this generates the following:

A B C D E F G
1 2 3 4 5 6 7
6 7 8
7 8 9

What did I miss?

share|improve this question
1  
your problem is that lines 1 and 2 are processed before line 3, and go through the NR>=1 code block, with no initialization for j and x –  Vincent Nivoliers Apr 4 '13 at 15:49
1  
I believe you have "rows" (lines of the file?) and "columns" somehow mixed up... and I see no way to use the contents of the third line without stashing away the first two somehow to process later. Or is the processing to begin with the fourth? –  vonbrand Apr 4 '13 at 15:56
    
This question is a good place to start –  Vincent Nivoliers Apr 4 '13 at 16:19
    
Thanks, guys. I was hoping to use awk only to accomplish the task. But since awk scans the file line by line, my alternative is to simplely arrange my the criteria line/row (3rd line in my example) to be the header for awk to scan it first. I have been hesitant to take that approach since my actual file is quite large. –  Aron Apr 4 '13 at 16:35
    
The other solution is a two pass filtering, as described in the question I mentioned above. –  Vincent Nivoliers Apr 4 '13 at 17:05
show 3 more comments

2 Answers

You could do:

code=$(
  awk '
    NR == 3 {
      for (i=1; i<=NF; i++)
        if ($i > 5) { printf "%s", sep "$" i; sep="," }
      exit sep == ""
    }' file
) &&
  awk "{print $code}" file

That is call awk twice on the same file. The first one reads the 3rd line to construct code for the second awk invocation. It exits after processing the 3rd line, so won't read the whole file fully. It outputs something like $5,$6,$7, so the next awk invocation becomes:

awk '{print $5,$6,$7}' file
share|improve this answer
    
This is clever ! Thank you so much. Really appreciated ! –  Aron Apr 5 '13 at 14:44
add comment

I got another awk solution to share :

cat > extract.columns.awk   
BEGIN {   
  infil=ARGV[1]  
    while (getline < infil > 0)  
      if (++n==3)  
      {  
        for(i=1;i<=NF;i++)  
            if ($(i) > 5) x[++j]=i  
    }  
close(infil)    
}  
{  
    for (i=1;i<j;i++)  
    printf("%s ",$x[i])  
    printf("%s\n",$x[j])  
}  

awk -f extract.columns.awk file

share|improve this answer
    
You may want to exit at the end of your BEGIN statement if j == 0 to avoid processing the file if there's nothing to display. –  Stephane Chazelas Apr 5 '13 at 15:05
    
Thank you, Stephane ! Do you mean adding " if (j==0) exit " at the end of the BEGIN block? Would you mind explaining a bit more? –  Aron Apr 5 '13 at 22:22
    
Yes, that's what I meant. –  Stephane Chazelas Apr 5 '13 at 22:24
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.