3

I am trying to extract some data from a file that is constantly updating and I have figured out how to filter two strings with grep. The output is as follows:

!    total energy              =   -9744.24963670 Ry
     convergence has been achieved in 188 iterations
!    total energy              =   -9744.30001681 Ry
     convergence has been achieved in 140 iterations
!    total energy              =   -9744.33953891 Ry
     convergence has been achieved in 155 iterations
!    total energy              =   -9744.36584201 Ry
     convergence has been achieved in 164 iterations
!    total energy              =   -9744.37925372 Ry
     convergence has been achieved in 154 iterations
!    total energy              =   -9744.39185493 Ry
     convergence has been achieved in 153 iterations
!    total energy              =   -9744.39836617 Ry
     convergence has been achieved in 160 iterations

Now what I would like to do is to extract from these lines the numbers as follows: from the line starting with ! I want the number in column no 5 and from the next line in the grep output I want the number in column no 6. Next I would like these numbers written in a separate file as two separated columns as:

188 -9744.24963670
140 -9744.30001681
155 -9744.33953891
164 -9744.36584201

I was thinking that an approach with awk by looping through all these grep results and then looking at odd numbered lines and print column 5 and then for even lines print column 6. But I have no idea how to do this.

I have tried extracting individual results into variables separately:

var1=$(grep '!' input.file | awk '{print $5}')

and

var2=$(grep 'convergence has been achieved' input.file | awk '{print $6}')

and then I tried to write them to a file as:

echo $var1 $var2 > data.dat

However the result is not as expected:

188                                                                                                                                                                                             
140
155
164
154
153
160 -9744.24963670
-9744.30001681
-9744.33953891
-9744.36584201
-9744.37925372
-9744.39185493
-9744.39836617

I don't know how to write them in the form I mentioned above. Also since the file is constantly updated I imagine the piece of code to be combined with a while loop until and end condition (I know how to do this last part)

I hope I explained this clearly!

  • what is input file size ? how often is it changed ? hours ? seconds ? there is a tail -f command that will "follow" the file (print last line as they are appended), however this doesn't give initials values. – Archemar Nov 17 '17 at 10:58
  • the file size is not very big, under 1 mb by the time it is finished updating and the updating speed is also quite low. Maybe about every 10 minutes or so. I cannot use tail -f because i need to look at all the values in those lines, not just the last one that was updated. or maybe I don't know what you mean by using tail -f – lucian Nov 17 '17 at 11:41
3

awk solution:

awk 'v && NR==n{ print $6,v > "result.txt" }/^!/{ v=$5; n=NR+1 }' file
  • <condition1> { <statement> ... }<condition2>{ <statement> ... } - conditions with respective statements will be evaluated consecutively

  • /^!/{ v=$5; n=NR+1 } - on encountering line starting with ! - capture the 5th field value $5 and plan the next line number NR+1 (assigning to variable n)

  • v && NR==n - if we have the 1st crucial number v and the current record number NR is the needed "next line number" n - print the values into file result.txt


The result.txt file contents:

188 -9744.24963670
140 -9744.30001681
155 -9744.33953891
164 -9744.36584201
154 -9744.37925372
153 -9744.39185493
160 -9744.39836617
  • 2
    thank you so much! this works perfectly! do you mind explaining the solution a bit please? I like using awk whenever possible but I find it difficult to understand the sintax in cases like these! – lucian Nov 17 '17 at 11:32
  • @lucian, you're welcome. The explanation is ON. – RomanPerekhrest Nov 17 '17 at 12:41
1

With your own solution, you need to have paste command to print side by side the results.

paste <(echo "$var2") <(echo "$var1") #or better via 'printf'
paste <(printf '%s' "$var2") <(printf '%s' "$var1")

But with simple awk command, you would just do:

awk '/\!/{C5=$5;getline; print $6, C5 >"output.txt"}' infile

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.