How to process awk line matches?

Question

I have a file with some lines of data that I want to extract. I'm thinking awk is a good fit for this. In pseudocode I'm imagining doing something like:

1. Find match for line I want to process with awk
2. In the second part, inside the {} find a regex match for my data, maybe with grep -o?
3. Print the match, maybe at the end inside the {} or inside and END

I'm not sure how to go about doing this, I know I'm not using cut properly but I just showed it to give some idea of intent. Would love some help. Essentially I want to get my hands on the $0 and start extracting various regex matches so I can piece them together into my own CSV format.

This is what I've tried:

➜  tmp cat data.txt
Concurrency Level:      1
Time taken for tests:   0.004 seconds
Complete requests:      1
Failed requests:        0
Total transferred:      838 bytes
HTML transferred:       632 bytes
Requests per second:    257.40 [#/sec] (mean)
Time per request:       3.885 [ms] (mean)
Time per request:       3.885 [ms] (mean, across all concurrent requests)
Transfer rate:          210.65 [Kbytes/sec] received
➜  tmp cat data.txt | awk '/Complete requests:/ {print;}'
Complete requests:      1
➜  tmp cat data.txt | awk '/Complete requests:/ {print $0;}'
Complete requests:      1
➜  tmp cat data.txt | awk '/Complete requests:/ {print;}'|cut -d: -f2
      1

I'm going for something like

Time taken for tests, 0.0004, s 
Requests per second (mean), 257.40, #/sec
.
.
.
Transfer rate received, 210.65, Kbytes/sec

Because the data isn't very uniform that why I'm going to want to write a separate "post-regex-match" processing of $0 for each match /Concurrency Level:/, Time taken for tests:, etc

Answer 1

You can have as many awk rule-action pairs as you like. Since you appear to want to re-order some of the sub-fields, you probably want a global rule that splits everything after the : into an array that you can manipulate later on a per-rule basis.

For example (I've only shown the first two here):

awk '
BEGIN {FS=":[ \t]+"; OFS=", "}
{split($2, a, /[ \t]+/)}
#
# now the case-by-case rule-action pairs
/Time taken/ {print $1, a[1], a[2]}
/Requests per second/ {print $1" "a[3], a[1], substr(a[2], 2, length(a[2])-2)}
' data.txt
Time taken for tests, 0.004, seconds
Requests per second (mean), 257.40, #/sec

Answer 2

I too am not certain I understand what you need but I think I would do this in one go with something along the lines of this...

awk '/Time|Requests|Transfer/{print $0}' data.txt

In this way you can load up the patterns to match and use the "|". So /Time|Requests|Transfer/ means "Match Time or Requests or Transfer" and print out the line

Answer 3

Using TXR:

@(repeat)
@  (cases)
@param: @val [@units] (@rawdesc)
@  (or)
@param: @val [@units] @rawdesc
@  (or)
@param: @val [@units]
@    (bind rawdesc nil)
@  (or)
@param: @val @units (@rawdesc)
@  (or)
@param: @val @units @rawdesc
@  (or)
@param: @val @units
@    (bind rawdesc nil)
@  (or)
@param: @val
@    (bind units "")
@    (bind rawdesc nil)
@  (end)
@  (bind desc @(if rawdesc ` (@(regsub #/,/ "" rawdesc))` ""))
@  (output)
@param@desc, @val, @units
@  (end)
@(end)

Run:

$ txr tocsv.txr data.txt 
Concurrency Level, 1, 
Time taken for tests, 0.004, seconds
Complete requests, 1, 
Failed requests, 0, 
Total transferred, 838, bytes
HTML transferred, 632, bytes
Requests per second (mean), 257.40, #/sec
Time per request (mean), 3.885, ms
Time per request (mean across all concurrent requests), 3.885, ms
Transfer rate (received), 210.65, Kbytes/sec

Although it looks verbose due to the "ladder" of cases, these were easy to crank out with copy and paste. The code produced the desired results on first try and is transparent to understand: at one glance you get the picture of what it is looking for.

The order of the cases is significant. If the case starting with the match

@param: @val

were put first, val would consume the entire value after the colon and spaces.