extract lines from bottom until regex match

Question

I have this output.

[root@linux ~]# cat /tmp/file.txt
virt-top time  11:25:14 Host foo.example.com x86_64 32/32CPU 1200MHz 65501MB
   ID S RDRQ WRRQ RXBY TXBY %CPU %MEM   TIME    NAME
    1 R    0    0    0    0  0.0  0.0  96:02:53 instance-0000036f
    2 R    0    0    0    0  0.0  0.0  95:44:07 instance-00000372
virt-top time  11:25:17 Host foo.example.com x86_64 32/32CPU 1200MHz 65501MB
   ID S RDRQ WRRQ RXBY TXBY %CPU %MEM   TIME    NAME
    1 R    0    0    0    0  0.6 12.0  96:02:53 instance-0000036f
    2 R    0    0    0    0  0.2 12.0  95:44:08 instance-00000372

You can see it has two blocks and i want to extract last block (if you see first block it has all CPU zero which i don't care) inshort i want to extract following last lines (Notes: sometime i have more than two instance-*) otherwise i could use "tail -n 2"

1 R    0    0    0    0  0.6 12.0  96:02:53 instance-0000036f
2 R    0    0    0    0  0.2 12.0  95:44:08 instance-00000372

I have tried sed/awk/grep and all possible way but not get close to desire result.

Answer 1

This feels a bit silly, but:

$ tac file.txt |sed -e '/^virt-top/q' |tac
virt-top time  11:25:17 Host foo.example.com x86_64 32/32CPU 1200MHz 65501MB
   ID S RDRQ WRRQ RXBY TXBY %CPU %MEM   TIME    NAME
    1 R    0    0    0    0  0.6 12.0  96:02:53 instance-0000036f
    2 R    0    0    0    0  0.2 12.0  95:44:08 instance-00000372

GNU tac reverses the file (many non-GNU systems have tail -r instead), the sed picks lines until the first that starts with virt-top. You can add sed 1,2d or tail -n +3 to remove the headers.

Or in awk:

$ awk '/^virt-top/ { a = "" } { a = a $0 ORS } END {printf "%s", a}' file.txt 
virt-top time  11:25:17 Host foo.example.com x86_64 32/32CPU 1200MHz 65501MB
   ID S RDRQ WRRQ RXBY TXBY %CPU %MEM   TIME    NAME
    1 R    0    0    0    0  0.6 12.0  96:02:53 instance-0000036f
    2 R    0    0    0    0  0.2 12.0  95:44:08 instance-00000372

It just collects all the lines to a variable, and clears that variable on a line starting with virt-top.

If the file is very large, the tac+sed solution is bound to be faster since it only needs to read the tail end of the file while the awk solution reads the full file from the top.

Answer 2

With ed you can regex-search upward using ?pattern? in place of the usual /pattern/ (which searches from above the current position). So for example:

$ printf '%s\n' '?ID?+1,$p' q | ed -s file.txt
    1 R    0    0    0    0  0.6 12.0  96:02:53 instance-0000036f
    2 R    0    0    0    0  0.2 12.0  95:44:08 instance-00000372

Answer 3

If your input has a fixed number of blocks you could also do something like:

awk '/^virt-top/ && ++n == 2, 0' <your-file

To output the lines from the 2^nd occurrence of virt-top to the end of the file (0 meaning false, means the end of that first,last range is never found).

Answer 4

Getting the last record from the line that starts with the string virt-top to the end using ed (suitable for dealing with files of the size comparable with what you show, not several megabytes big):

$ printf '%s\n' '?^virt-top?,$p' | ed -s file
virt-top time  11:25:17 Host foo.example.com x86_64 32/32CPU 1200MHz 65501MB
   ID S RDRQ WRRQ RXBY TXBY %CPU %MEM   TIME    NAME
    1 R    0    0    0    0  0.6 12.0  96:02:53 instance-0000036f
    2 R    0    0    0    0  0.2 12.0  95:44:08 instance-00000372

Or, just the lines after the last virt-top line that contain the substring instance:

$ printf '%s\n' '?^virt-top?,$g/instance/p' | ed -s file
    1 R    0    0    0    0  0.6 12.0  96:02:53 instance-0000036f
    2 R    0    0    0    0  0.2 12.0  95:44:08 instance-00000372

That last ed command, ?^virt-top?,$g/instance/p, first designates a range of lines from the last line that starts with virt-top (last, because ed starts you off on the last line of the buffer and ?re? searches backwards) to the end of the buffer ($) and applies the command g/instance/p to these lines. The g/re/p command prints all lines within the range that matches the given regular expression (that's where grep got its name from, by the way).

---

Using awk:

$ awk '/^virt-top/ { lines = "" } /instance/ { lines = (lines == "" ? $0 : lines ORS $0) } END { print lines }' file
    1 R    0    0    0    0  0.6 12.0  96:02:53 instance-0000036f
    2 R    0    0    0    0  0.2 12.0  95:44:08 instance-00000372

This saves lines of input in lines whenever there's a line that contains the substring instance. These lines are printed at the end. Whenever a line that starts with virt-top is found, the saved lines are discarded.

Virtually the same thing with sed, using the hold space to serve as the equivalent of the lines variable in the awk code:

$ sed -n '/^virt-top.*/{ s///; x; d; }; /instance/H; ${ x; s/\n//; p; }' file
    1 R    0    0    0    0  0.6 12.0  96:02:53 instance-0000036f
    2 R    0    0    0    0  0.2 12.0  95:44:08 instance-00000372

Answer 5

Here's yet another method to deal with it:

$ sed -e '
   /\n/q
   /virt-top/{h;d;}
   H;$!d;g
   s/\n//;D
' file.txt

Results

1 R    0    0    0    0  0.6 12.0  96:02:53 instance-0000036f
2 R    0    0    0    0  0.2 12.0  95:44:08 instance-00000372