4

I have this output.

[root@linux ~]# cat /tmp/file.txt
virt-top time  11:25:14 Host foo.example.com x86_64 32/32CPU 1200MHz 65501MB
   ID S RDRQ WRRQ RXBY TXBY %CPU %MEM   TIME    NAME
    1 R    0    0    0    0  0.0  0.0  96:02:53 instance-0000036f
    2 R    0    0    0    0  0.0  0.0  95:44:07 instance-00000372
virt-top time  11:25:17 Host foo.example.com x86_64 32/32CPU 1200MHz 65501MB
   ID S RDRQ WRRQ RXBY TXBY %CPU %MEM   TIME    NAME
    1 R    0    0    0    0  0.6 12.0  96:02:53 instance-0000036f
    2 R    0    0    0    0  0.2 12.0  95:44:08 instance-00000372

You can see it has two blocks and i want to extract last block (if you see first block it has all CPU zero which i don't care) inshort i want to extract following last lines (Notes: sometime i have more than two instance-*) otherwise i could use "tail -n 2"

1 R    0    0    0    0  0.6 12.0  96:02:53 instance-0000036f
2 R    0    0    0    0  0.2 12.0  95:44:08 instance-00000372

I have tried sed/awk/grep and all possible way but not get close to desire result.

  • It can help us help you if you provide some of your "all possible ways" you tried – Philippos Sep 10 at 15:39
  • I think i am closed sed '1,/instance/p' /tmp/foo.txt still its printing header line which is ID S RDRQ WRRQ RXBY TXBY %CPU %MEM TIME NAME i can trim it down with | grep -v ID but i believe there must be better way :) – Satish Sep 10 at 15:42
  • How about extracting all lines with non-zero CPU-Uage like (untested) grep -vE -e virt-top -e RDRQ -e "([^ ]* *){6}0.0"? – Philippos Sep 10 at 15:47
8

This feels a bit silly, but:

$ tac file.txt |sed -e '/^virt-top/q' |tac
virt-top time  11:25:17 Host foo.example.com x86_64 32/32CPU 1200MHz 65501MB
   ID S RDRQ WRRQ RXBY TXBY %CPU %MEM   TIME    NAME
    1 R    0    0    0    0  0.6 12.0  96:02:53 instance-0000036f
    2 R    0    0    0    0  0.2 12.0  95:44:08 instance-00000372

GNU tac reverses the file (many non-GNU systems have tail -r instead), the sed picks lines until the first that starts with virt-top. You can add sed 1,2d or tail -n +3 to remove the headers.

Or in awk:

$ awk '/^virt-top/ { a = "" } { a = a $0 ORS } END {printf "%s", a}' file.txt 
virt-top time  11:25:17 Host foo.example.com x86_64 32/32CPU 1200MHz 65501MB
   ID S RDRQ WRRQ RXBY TXBY %CPU %MEM   TIME    NAME
    1 R    0    0    0    0  0.6 12.0  96:02:53 instance-0000036f
    2 R    0    0    0    0  0.2 12.0  95:44:08 instance-00000372

It just collects all the lines to a variable, and clears that variable on a line starting with virt-top.

If the file is very large, the tac+sed solution is bound to be faster since it only needs to read the tail end of the file while the awk solution reads the full file from the top.

  • 1
    Only the tac solution is practical if file.txt is very large and the match is (as expected) towards its end. Also, it could be better written as tac file.txt | sed '/^virt-top/q' | tac – mosvy Sep 10 at 15:55
  • I really like tac first time i heard :D – Satish Sep 10 at 16:00
  • @mosvy, excellent points, thank you! – ilkkachu Sep 10 at 16:06
4

With ed you can regex-search upward using ?pattern? in place of the usual /pattern/ (which searches from above the current position). So for example:

$ printf '%s\n' '?ID?+1,$p' q | ed -s file.txt
    1 R    0    0    0    0  0.6 12.0  96:02:53 instance-0000036f
    2 R    0    0    0    0  0.2 12.0  95:44:08 instance-00000372
  • very simple and sweet, didn't thought about printf can run the show :) – Satish Sep 10 at 15:53
  • 2
    ed sucks with very large files. – mosvy Sep 10 at 15:58
  • ... because it reads the whole file into a buffer in memory before operating on it. – Ed Morton Sep 10 at 16:20
  • 2
    @Ed, I suppose it depends on the implementation, but at least GNU ed, while it reads the file fully, doesn't store it whole in memory (probably only records the offsets of each line within the file which explains why it's more efficient at dealing with longer lines). It does create a temporary copy of the file though. – Stéphane Chazelas Sep 10 at 16:58
3

If your input has a fixed number of blocks you could also do something like:

awk '/^virt-top/ && ++n == 2, 0' <your-file

To output the lines from the 2nd occurrence of virt-top to the end of the file (0 meaning false, means the end of that first,last range is never found).

1

Getting the last record from the line that starts with the string virt-top to the end using ed (suitable for dealing with files of the size comparable with what you show, not several megabytes big):

$ printf '%s\n' '?^virt-top?,$p' | ed -s file
virt-top time  11:25:17 Host foo.example.com x86_64 32/32CPU 1200MHz 65501MB
   ID S RDRQ WRRQ RXBY TXBY %CPU %MEM   TIME    NAME
    1 R    0    0    0    0  0.6 12.0  96:02:53 instance-0000036f
    2 R    0    0    0    0  0.2 12.0  95:44:08 instance-00000372

Or, just the lines after the last virt-top line that contain the substring instance:

$ printf '%s\n' '?^virt-top?,$g/instance/p' | ed -s file
    1 R    0    0    0    0  0.6 12.0  96:02:53 instance-0000036f
    2 R    0    0    0    0  0.2 12.0  95:44:08 instance-00000372

That last ed command, ?^virt-top?,$g/instance/p, first designates a range of lines from the last line that starts with virt-top (last, because ed starts you off on the last line of the buffer and ?re? searches backwards) to the end of the buffer ($) and applies the command g/instance/p to these lines. The g/re/p command prints all lines within the range that matches the given regular expression (that's where grep got its name from, by the way).


Using awk:

$ awk '/^virt-top/ { lines = "" } /instance/ { lines = (lines == "" ? $0 : lines ORS $0) } END { print lines }' file
    1 R    0    0    0    0  0.6 12.0  96:02:53 instance-0000036f
    2 R    0    0    0    0  0.2 12.0  95:44:08 instance-00000372

This saves lines of input in lines whenever there's a line that contains the substring instance. These lines are printed at the end. Whenever a line that starts with virt-top is found, the saved lines are discarded.

Virtually the same thing with sed, using the hold space to serve as the equivalent of the lines variable in the awk code:

$ sed -n '/^virt-top.*/{ s///; x; d; }; /instance/H; ${ x; s/\n//; p; }' file
    1 R    0    0    0    0  0.6 12.0  96:02:53 instance-0000036f
    2 R    0    0    0    0  0.2 12.0  95:44:08 instance-00000372
1

Here's yet another method to deal with it:

$ sed -e '
   /\n/q
   /virt-top/{h;d;}
   H;$!d;g
   s/\n//;D
' file.txt

Results

1 R    0    0    0    0  0.6 12.0  96:02:53 instance-0000036f
2 R    0    0    0    0  0.2 12.0  95:44:08 instance-00000372

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.