Given a data file which contains various content, I would like to first ignore any line that does not start with an IP address, then output the IP address followed by any string found in the braces that does not contain an = sign. Any given line may have one or multiple strings.

For example:


junk34 [ joker penguin character=villian ] blah [ super_man bat_man character=hero ] blah [ lois_lane character=damsel ] blah  

OUTPUT: joker penguin super_man bat_man lois_lane  

There may be many ways of doing this, but in awk we can do something like

awk 'gsub(/^[0-9.]+ \[ /,$1 " ") { a=2; while (a<=NR && $a != "]") { if ($a !~ /=/) {print $1 " " $a } ; a++ } }'

Let's break this down into something more readable:

gsub(/^[0-9.]+ \[ /,$1 " ") { .... }

This will match lines that begin with numbers and dots, followed by a space and a [. So it will match [ but won't match junk23. It will replace it with the first field and then run the stuff inside the {...}.

So a line like [ joker penguin character=villian ] blah  

will enter that section looking like joker penguin character=villian ] blah  

Now the middle part, which might be easier to understand if we write it on multiple lines:

while (a<=NR && $a != "]")
  if ($a !~ /=/) {print $1 " " $a }

So we start at the 2nd field ("joker" in this case) and continue on until we run out of words or we see the ]. For each word we find, if it doesn't contain an = then we print the first field (the IP address) and the word.

The results: joker penguin super_man bat_man lois_lane

(There's probably better ways of doing this!)

  • To make this more robust, how could whitespace before and after the IP address be ignored? There will always be a new line before the IP address, however, there might be 0-5 spaces before as well. Same situation in between the IP address and the [. – Bernie Aug 17 '16 at 19:01
  • Modify the gsub() search to better match the pattern; eg /^ *[0-9.]+ +\[ / which allow zero or more spaces before the IP address and one or more spaces after it. – Stephen Harris Aug 17 '16 at 19:18
  • This worked great. I will give you the checkmark as soon as my rep is 15+. – Bernie Aug 18 '16 at 15:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.