0

Here is an example block of text from a file:


Now is the time for all blah:1; to come to the aid
Now is the time for all blah:1; to come to the aid
Now is the time for all blah:1; to come to the aid
Now is the time for all blah:10; to come to the aid
Go to your happy place  blah:100; to come to the aid
Go to your happy place  blah:4321; to come to the aid
Go to your happy place  blah:4321; to come to the aid
Now is the time for all blah:4321; to come to the aid
Now is the time for all blah:9876; to come to the aid
Now is the time for all blah:108636; to come to the aid
Now is the time for all blah:1194996; to come to the aid

Question: How would I extract all unique numbers from lines that have "is the" in them?

I've tried using grep -o -P -u '(?<=blah:).*(?=;)' but it doesn't like the semi colon

4
  • 1
    Ok a little clarification: the 'print $2' solution won't work as the text preceding "blah:" varies up to appx 30 characters and includes many special characters that get interpreted as separate fields
    – blake
    Sep 26 '17 at 18:16
  • I need to pull out only the numbers immediately following the "blah:" string because each line has multiple numbers followed by a colon
    – blake
    Sep 26 '17 at 18:29
  • 1
    You should put all your requirements in the question (you can edit your question) Sep 26 '17 at 19:47
  • If you're happy with one or several of the answers, upvote them. If one is solving your issue, accepting it would be the best way of saying "Thank You!" :-)
    – Kusalananda
    Sep 26 '17 at 19:51
5

You're looking for the \K directive to forget about the stuff you just matched.

grep -oP 'is the.*?blah:\K\d+'

Then sort -u

2
  • grumble grumble, hidden requirements, mutter, mutter Sep 26 '17 at 19:47
  • 1
    The story of every software developer...
    – Kusalananda
    Sep 26 '17 at 19:48
3

Using sed:

$ sed -n '/is the/s/^.*blah:\([0-9]*\);.*$/\1/p' file | sort -u
1
10
108636
1194996
4321
9876

The substitution replaces the contents of all lines containing the string is the with the number between blah: and ;. Lines not containing the string are ignored.

3
  • This works great however I ran into a problem where some lines have multiple numbers followed by a colon, so I need to pull out ONLY the numbers immediately following the "blah:" string
    – blake
    Sep 26 '17 at 18:27
  • @blake The sed solution would be easiest to modify: sed -n '/is the/s/^.*blah:\([0-9]*\);.*$/\1/p' file | sort -u
    – Kusalananda
    Sep 26 '17 at 18:33
  • Perfect! Thank you Kusalananda - that worked perfectly
    – blake
    Sep 26 '17 at 18:43
0
cat file | grep "is the" | awk -F':' '{print $2}'|awk -F';' '{print $1}'|sort -u
2
  • You can combine cat|grep|awk|awk with gawk -F '[:;]' '/is the/ {print $2}' file Sep 26 '17 at 19:14
  • And then you can get the unique ids by using an associative array: gawk -F '[:;]' '/is the/ {id[$2]} END {for (i in id) print i}' file Sep 26 '17 at 19:14
0

Try this:

grep "is the" file | sed 's/.*blah://;s/;.*//' | sort -u

Explanation:

  1. grep gets all lines with "is the" (in any part of the line)
  2. sed remove all before ":" and after ";" (you could use sed -e 's/.*blah://' -e 's/;.*//' instead for best understanding)
  3. sort sorts lines
2
  • Thanks Egor - I should have been more clear. in some of the lines there are multiple numbers followed by a colon. I need to pull out ONLY the numbers following the "blah:" string
    – blake
    Sep 26 '17 at 18:32
  • @blake i corrected the answer according your comments Sep 26 '17 at 18:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.