5

I am trying to extract some data from a file that is constantly updating and I have figured out how to filter two strings with grep. The output is as follows:

!    total energy              =   -9744.24963670 Ry
     convergence has been achieved in 188 iterations
!    total energy              =   -9744.30001681 Ry
     convergence has been achieved in 140 iterations
!    total energy              =   -9744.33953891 Ry
     convergence has been achieved in 155 iterations
!    total energy              =   -9744.36584201 Ry
     convergence has been achieved in 164 iterations
!    total energy              =   -9744.37925372 Ry
     convergence has been achieved in 154 iterations
!    total energy              =   -9744.39185493 Ry
     convergence has been achieved in 153 iterations
!    total energy              =   -9744.39836617 Ry
     convergence has been achieved in 160 iterations

Now what I would like to do is to extract from these lines the numbers as follows: from the line starting with ! I want the number in column no 5 and from the next line in the grep output I want the number in column no 6. Next I would like these numbers written in a separate file as two separated columns as:

188 -9744.24963670
140 -9744.30001681
155 -9744.33953891
164 -9744.36584201

I was thinking that an approach with awk by looping through all these grep results and then looking at odd numbered lines and print column 5 and then for even lines print column 6. But I have no idea how to do this.

I have tried extracting individual results into variables separately:

var1=$(grep '!' input.file | awk '{print $5}')

and

var2=$(grep 'convergence has been achieved' input.file | awk '{print $6}')

and then I tried to write them to a file as:

echo $var1 $var2 > data.dat

However the result is not as expected:

188                                                                                                                                                                                             
140
155
164
154
153
160 -9744.24963670
-9744.30001681
-9744.33953891
-9744.36584201
-9744.37925372
-9744.39185493
-9744.39836617

I don't know how to write them in the form I mentioned above. Also since the file is constantly updated I imagine the piece of code to be combined with a while loop until and end condition (I know how to do this last part)

I hope I explained this clearly!

2
  • what is input file size ? how often is it changed ? hours ? seconds ? there is a tail -f command that will "follow" the file (print last line as they are appended), however this doesn't give initials values.
    – Archemar
    Commented Nov 17, 2017 at 10:58
  • the file size is not very big, under 1 mb by the time it is finished updating and the updating speed is also quite low. Maybe about every 10 minutes or so. I cannot use tail -f because i need to look at all the values in those lines, not just the last one that was updated. or maybe I don't know what you mean by using tail -f
    – lucian
    Commented Nov 17, 2017 at 11:41

2 Answers 2

5

awk solution:

awk 'v && NR==n{ print $6,v > "result.txt" }/^!/{ v=$5; n=NR+1 }' file
  • <condition1> { <statement> ... }<condition2>{ <statement> ... } - conditions with respective statements will be evaluated consecutively

  • /^!/{ v=$5; n=NR+1 } - on encountering line starting with ! - capture the 5th field value $5 and plan the next line number NR+1 (assigning to variable n)

  • v && NR==n - if we have the 1st crucial number v and the current record number NR is the needed "next line number" n - print the values into file result.txt


The result.txt file contents:

188 -9744.24963670
140 -9744.30001681
155 -9744.33953891
164 -9744.36584201
154 -9744.37925372
153 -9744.39185493
160 -9744.39836617
2
  • 2
    thank you so much! this works perfectly! do you mind explaining the solution a bit please? I like using awk whenever possible but I find it difficult to understand the sintax in cases like these!
    – lucian
    Commented Nov 17, 2017 at 11:32
  • 1
    @lucian, you're welcome. The explanation is ON. Commented Nov 17, 2017 at 12:41
1

With your own solution, you need to have paste command to print side by side the results.

paste <(echo "$var2") <(echo "$var1") #or better via 'printf'
paste <(printf '%s' "$var2") <(printf '%s' "$var1")

But with simple awk command, you would just do:

awk '/\!/{C5=$5;getline; print $6, C5 >"output.txt"}' infile

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .