0

I'd like to parse the btrfs scrub statistics that btrfs places in a file named /var/lib/btrfs/scrub.status.<uuid>. The file effectively consists of key/value pairs where key and value are separated by semicolons and tuples are separated by a pipe ("|") character:

scrub status:1
b5872f47-c87e-47ac-b036-4f2725cf6dc6:1|data_extents_scrubbed:4996799|tree_extents_scrubbed:1436139|data_bytes_scrubbed:156663988224|tree_bytes_scrubbed:23529701376|read_errors:0|csum_errors:0|verify_errors:0|no_csum:733729|csum_discards:0|super_errors:0|malloc_errors:0|uncorrectable_errors:0|corrected_errors:0|last_physical:202959224832|t_start:1597847400|t_resumed:0|duration:67|canceled:0|finished:1

In the end I'd like to emit each key/value pair on a separate line such as (this will serve as Prometheus metrics):

scrub status 1
data_extents_scrubbed 4996799
tree_extents_scrubbed 1436139
[...]

I have made various attempts to accomplish this nested parsing using sed/awk, but have failed so far.

Any hints how to accomplish this are much appreciated!

2
  • Can you please share your attempts? Why did you omit the first key-value from the expected output? – Quasímodo Aug 19 '20 at 21:58
  • Sure. What I tried last was: sed -r 's/^(([^:]*):([^|]*)[|]?)+$/\2 \3\n/g' scrub.status.<uid>. But this only emits the last key/value pair of a line. – Thilo-Alexander Ginkel Aug 19 '20 at 22:02
2
  • Any sed:

    sed 's/:/ /g;s/|/\
    /g' file
    
  • GNU sed:

    sed 's/:/ /g;s/|/\n/g' file
    

Here we change every : with a space and every | with a newline character.

With awk, you may set the record separator to the pipe character and then substitute : with space.

awk 'BEGIN{RS="|"}{gsub(":"," ");print}' file
1

With GNU awk for multi-char RS:

$ awk -v RS='[|]|\n$' -F':' '{$1=$1}1' file
scrub status 1
b5872f47-c87e-47ac-b036-4f2725cf6dc6 1
data_extents_scrubbed 4996799
tree_extents_scrubbed 1436139
data_bytes_scrubbed 156663988224
tree_bytes_scrubbed 23529701376
read_errors 0
csum_errors 0
verify_errors 0
no_csum 733729
csum_discards 0
super_errors 0
malloc_errors 0
uncorrectable_errors 0
corrected_errors 0
last_physical 202959224832
t_start 1597847400
t_resumed 0
duration 67
canceled 0
finished 1

Among other things, compared to an awk solution that just sets RS to |, the above won't print a spurious blank line at the end of its output.

0

Try:

$ awk -F'|' '{for (i=1;i<=NF;i++) {sub(/:/, " ", $i); print $i}}' file
scrub status 1
b5872f47-c87e-47ac-b036-4f2725cf6dc6 1
data_extents_scrubbed 4996799
tree_extents_scrubbed 1436139
data_bytes_scrubbed 156663988224
tree_bytes_scrubbed 23529701376

How it works:

  • -F'|' sets the field separator to |.

  • {for (i=1;i<=NF;i++) {sub(/:/, " ", $i); print $i}} loops through each field, substitutes a blank for : and prints the field.

0

sed

sed -e 'y/:|/ \n/' file

awk It's a matter of setting the record- and field-separators to the values shown and then have awk one more go at the field splitting operation. Finally, the records are unconditionally printed.

awk '
  BEGIN { RS = "|"; FS = ":" }
  {$1=$1}1
' file
0

Isn't a tr the fastest way to convert characters?

$ <file tr ':|' ' \n'

scrub status 1
b5872f47-c87e-47ac-b036-4f2725cf6dc6 1
data_extents_scrubbed 4996799
tree_extents_scrubbed 1436139
data_bytes_scrubbed 156663988224
tree_bytes_scrubbed 23529701376
read_errors 0
csum_errors 0
verify_errors 0
no_csum 733729
csum_discards 0
super_errors 0
malloc_errors 0
uncorrectable_errors 0
corrected_errors 0
last_physical 202959224832
t_start 1597847400
t_resumed 0
duration 67
canceled 0
finished 1

Of course, tr works only with one-byte characters. That in not a problem here for : and |.

An equivalent for sed (which may work with multi byte characters (GNU/ATT)) is:

<file sed 'y/:|/ \n/'

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.