In a file that looks like:

0.0000E+00 9.6988e-04 2.0580E-02
1.0000E+01 9.6988e-04 2.0580E-02
2.0000E+01 9.6988e-04 2.0580E-02
5.6000E+02 7.0997e-06 -3.7538E-04

Delete the last line if the difference between the number in first line and the first column of last line is greater than 10. So, in this case the last line will get deleted since 560 - 549.432086168 is greater than 10.

Any suggestions as to how can this be done efficiently?

  • One question. Will the value inside the last line be in scientific notation all the time? And how about the value in the first line? – user34720 Jan 13 '16 at 17:12
  • Is it only the last line you want to delete or any line that matches those criteria? – Stéphane Chazelas Jan 13 '16 at 17:20
  • Sorry, got offline for a bit. @nwildner yes, the values in last line is expected to be in scientific notations at all times. Stéphane, I need this only for the last line. – mkc Jan 13 '16 at 19:16

Typical job for awk:

awk 'NR == 1 {first = $1}; $1 - first <= 10' < file

Or to do it only for the last line:

awk 'NR == 1 {first = last = $0; next}
     {print last; last = $0}
     END {if (NR && last - first <= 10) print last}' < file

You can use perl:

perl -lne '
  $n = $_ if $. == 1;
  print unless eof;
  print if $_ - $n <= 10 and eof;
' <file
  • For the first line $. == 1, we set its value to variable $n
  • For next lines, print if it's not the last line
  • Print the last line if the different between its first column and $n less than or equal 10.

Here we used $_ - $n, forcing the last line in numeric context, so the first column value was used:

$ perl -le 'print "5.6000E+02 7.0997e-06 -3.7538E-04"-0'
  • Well, great script. Now i get the logic behind it, and i learned that Perl implicitly makes math even when a number is in exponetial notation. Now, im reading this - perldoc.perl.org/perlnumber.html – user34720 Jan 13 '16 at 17:43
  • I'm not familiar with perl, thanks for explaining the script, so I can learn it a bit. – Kira Jan 13 '16 at 19:03
  • @cuonglm This answer looks great. However, since the other stuff in my script is in either sed or awk, I am accepting the other answer from Stéphane. Thanks for your effort, I appreciate it. – mkc Jan 13 '16 at 19:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.