2

I have a file called file1 on my Linux machine with these contents:

ID        Time                    Energy
43 4.38665978376386365240533e-05 3.215e-02
43 4.38666558838421885677290e-05 4.788e-01
48 4.77674337753321466689890e-05 1.750e-01
48 4.77674676992522297732519e-05 3.360e-01
52 4.99184267458611271553633e-05 3.110e-01
52 4.99184755438830858337990e-05 2.000e-01
203 2.17111141925415134391192e-04 0.000e+00
203 2.17111189970080955017814e-04 1.685e-01
203 2.17111190317825032474949e-04 3.425e-01
206 2.17705422992319207490738e-04 3.197e-01
206 2.17705445825075834731174e-04 1.913e-01
245 2.54300470583329032894099e-04 2.130e-01
245 2.54300477743886715713273e-04 2.679e-02
245 2.54300499465897461830871e-04 4.887e-02
245 2.54300511632548666141052e-04 0.000e+00
245 2.54300542217183317417195e-04 0.000e+00
245 2.54300549393671508017351e-04 2.224e-01

and file2:

ID   Time                         Energy
43  4.38666558838421885677290e-05 4.788e-01
48  4.77674676992522297732519e-05 3.360e-01
52  4.99184267458611271553633e-05 3.110e-01
203 2.17111190317825032474949e-04 3.425e-01
206 2.17705422992319207490738e-04 3.197e-01
245 2.54300549393671508017351e-04 2.224e-01

I want to read file1 and, if the energy value of a row in file1 exists in file2, I should print the previous line.

The desired output from the example above is :

43 4.38665978376386365240533e-05 3.215e-02
48 4.77674337753321466689890e-05 1.750e-01
52 4.99184267458611271553633e-05 3.110e-01
203 2.17111189970080955017814e-04 1.685e-01
206 2.17705422992319207490738e-04 3.197e-01
245 2.54300542217183317417195e-04 0.000e+00

The question is: In same ID number, when the energy value of a row in file1 exists in file2, print the row before this one. can anyone kown how to do this by awk? I have tried code :

gawk '
NR==FNR {a[NR]=$0; next }
{
    split(a[FNR],flds,FS,seps)
    if(flds[1]=$1 && flds[2] <=$2)
  print flds}' file1 file2 > flie3

Then using sort | uniq to remove the repeated lines with same id.

Time value is incremental in all files, and once the line with same energy value as file2 is in the first line within same ID number, then only choose this line. like 52 4.99184267458611271553633e-05 3.110e-01

4
  • Can the IDs be repeated in file2 or will there always be one, unique value per id? – terdon Apr 30 '20 at 12:50
  • 1
    Welcome to the site. Could you edit your post to tell us what you already tried, and where you encountered problems? – AdminBee Apr 30 '20 at 12:50
  • Why is the line 52 4.99184267458611271553633e-05 3.110e-01 in your desired output? Don't you want to print the previous line? – terdon Apr 30 '20 at 13:25
  • Sorry I am not so skillful to use this tool, I re-edit my question just now. Within the same ID number, in file1, line with same energy value as file2 is just in the first line, then we just choose this first line as output.thank you. – galadriel_1996 Apr 30 '20 at 13:47
0
awk '
  NR==FNR{ if (FNR>1)a[$1]=$3; next } # file2: save ID,Energy value in array `a`
  $1 in a{                            # file1: if ID is present in array
     if (a[$1] != $3){                # if it is not the same Energy value...
       prev=$0                        # save previous line
     }
     else {                           # it is the same Energy value...
       print (prev=="") ? $0 : prev   # print previous line if saved or current line
       prev=""                        # reset previous line
     }
  }
' file2 file1 > file3

Output:

43 4.38665978376386365240533e-05 3.215e-02
48 4.77674337753321466689890e-05 1.750e-01
52 4.99184267458611271553633e-05 3.110e-01
203 2.17111189970080955017814e-04 1.685e-01
206 2.17705422992319207490738e-04 3.197e-01
245 2.54300542217183317417195e-04 0.000e+00
7
  • Thank you. The second column means time and is incremental from top to bottom. For example 52 4.99184267458611271553633e-05 3.110e-01 in file2 is located in the first row w.s.t the ID of 52, before it is rows of ID 48. So I just need to extract the first row: 52 4.99184267458611271553633e-05 3.110e-01 as output. In another word, it is not only printing previous line. Output needs to include all the ID value in file1, but only one row per ID. – galadriel_1996 May 1 '20 at 1:54
  • Ah, so it's take the previous line of file1 with the same ID if it exists. If not, then just take the line. And then you don't need my second command with sort, right? – Freddy May 1 '20 at 2:18
  • Yes,thank you very much. I should change if (FNR>1) to if(FNR>=1),otherwise, the first line of ID 43 will be lost. – galadriel_1996 May 1 '20 at 7:22
  • I added it to skip the header line in file2. You can remove the if (FNR>1) if you don't need it (removing it is the same as if (FNR>=1)). – Freddy May 1 '20 at 10:23
  • really thank you, it works well. And I wonder If I added extral column, for example, called location, which has the same usage as Energy. can I just use if (FNR>1)a[$1]=$3;b[$1]=$4; next } $1 in a{......}?like this format, saving energy and location in two different array? – galadriel_1996 May 1 '20 at 12:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.