0

How to find the maximum value every 5 lines in a large file. I have file like for example:

186 5   24.0772 24.061  18.5996 2.14E+06  
123 4   25.838  29.4805 25.8075 1.25E+06  
47  4   24.0637 27.5751 18.7376 915260  
44  4   22.2291 25.915  18.7651 2.13E+06  
22  4   20.5204 31.1934 25.8222 958569  
2   4   18.6967 24.0007 24.0214 2.35E+06  
11  4   18.66   25.8085 22.2719 1.99E+06  
3   4   20.4814 22.3203 24.1144 2.02E+06  
4   4   20.5083 24.0258 22.2133 1.90E+06  
23  4   22.2675 20.584  24.0367 2.05E+06  
24  4   24.1038 18.7206 24.0185 2.33E+06  
25  4   24.0527 20.4908 22.3148 3.19E+06  
30  4   22.2784 24.118  20.574  3.06E+06  
31  4   22.2229 22.3165 22.3832 1.35E+06  
32  4   22.3877 23.9854 24.0684 2.45E+06  
33  4   24.0543 22.2423 24.0071 3.66E+06  
34  4   23.9802 24.1058 22.2199 2.71E+06  
185 5   24.0362 22.2425 20.4836 2.73E+06  
77  4   25.8136 20.4698 24.0521 2.94E+06  
189 5   25.8148 24.0693 24.0799 1.50E+06  

so I want to return the maximum value in every 5 lines from the 6th column at this file, so the result must be like:

2.14E+06  
2.35E+06  
....
....

any method can be done by awk or other methods in Linux.

  • If you replace the ...s in the expected output with the actual values you want to get then we'll have something we can easily test against. – Ed Morton Oct 7 at 20:22
1
$ awk '(NR-1) % 5 == 0 { m=$1; for (i=2; i<=NF; ++i) m=($i>m ? $i : m); print m }' file
2.14E+06
2.35E+06
2.33E+06
3.66E+06

or, with an explicit if statement:

$ awk '(NR-1) % 5 == 0 { m=$1; for (i=2; i<=NF; ++i) if ($i > m) m = $i; print m }' file
2.14E+06
2.35E+06
2.33E+06
3.66E+06

The awk program uses (NR-1) % 5 == 0 to trigger the block of code for lines 1, 11, 16, etc. The block finds the largest value across a line and prints it.

NR is the current record number (line number by default), and NF is the number of fields (columns) in the current record.

If you always want to get the last column, use the much quicker

$ awk '(NR-1) % 5 == 0 { print $NF }' file
2.14E+06
2.35E+06
2.33E+06
3.66E+06
  • thank you for your answer, but I couldn't get the correct one when I compare it manually, the real file I have is a large size file in GB, also my numbers are exponential and decimal so do this affect the awk use? – Jaguar Jom Oct 7 at 9:12
  • @JaguarJom It works on the example that you show in the question. Maybe you should add an example with actual data? – Kusalananda Oct 7 at 9:13
  • thank you, I have updated the question with the real data sample – Jaguar Jom Oct 7 at 9:21
  • @JaguarJom Looking at your data, it looks as if you just need the last column. Is that always true? If so, the awk code will simply be awk 'NR % 5 == 0 { print $NF }'. – Kusalananda Oct 7 at 9:44
  • @JaguarJom Also, you now seem to look at lines 1, 6, 11 etc. whereas you previously looked at lines 5, 10, 15. Can you confirm that this is what you want to do? I've changed my code to do the same. – Kusalananda Oct 7 at 9:47
1
$ awk '{max = ((max=="") || ($6 > max) ? $6 : max)} !(NR%5){print max; max=""}' file
2.14E+06
2.35E+06
3.19E+06
3.66E+06
1
$ perl -lane '
    $F[5] > ($m //= -InF) and $m = $F[5];
    $. % 5 or print, undef for $m;
' file 

2.14E+06
2.35E+06
3.19E+06
3.66E+06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.