0

Here is an example block of text from a file:

Now is the time for all blah:1; to come to the aid
Now is the time for all blah:1; to come to the aid
Now is the time for all blah:1; to come to the aid
Now is the time for all blah:10; to come to the aid
Go to your happy place  blah:100; to come to the aid
Go to your happy place  blah:4321; to come to the aid
Go to your happy place  blah:4321; to come to the aid
Now is the time for all blah:4321; to come to the aid
Now is the time for all blah:9876; to come to the aid
Now is the time for all blah:108636; to come to the aid
Now is the time for all blah:1194996; to come to the aid

Question: How would I extract all unique numbers from lines that have "is the" in them?

I've tried using grep -o -P -u '(?<=blah:).*(?=;)' but it doesn't like the semi colon

Improve this question
4
  • 1
    Ok a little clarification: the 'print $2' solution won't work as the text preceding "blah:" varies up to appx 30 characters and includes many special characters that get interpreted as separate fields
    – blake
    Sep 26, 2017 at 18:16
  • I need to pull out only the numbers immediately following the "blah:" string because each line has multiple numbers followed by a colon
    – blake
    Sep 26, 2017 at 18:29
  • 1
    You should put all your requirements in the question (you can edit your question)
    – glenn jackman
    Sep 26, 2017 at 19:47
  • If you're happy with one or several of the answers, upvote them. If one is solving your issue, accepting it would be the best way of saying "Thank You!" :-)
    – Kusalananda ♦
    Sep 26, 2017 at 19:51

4 Answers 4

Reset to default
5

You're looking for the \K directive to forget about the stuff you just matched.

grep -oP 'is the.*?blah:\K\d+'

Then sort -u

Improve this answer
2
  • grumble grumble, hidden requirements, mutter, mutter
    – glenn jackman
    Sep 26, 2017 at 19:47
  • 1
    The story of every software developer...
    – Kusalananda ♦
    Sep 26, 2017 at 19:48
3

Using sed:

$ sed -n '/is the/s/^.*blah:\([0-9]*\);.*$/\1/p' file | sort -u
1
10
108636
1194996
4321
9876

The substitution replaces the contents of all lines containing the string is the with the number between blah: and ;. Lines not containing the string are ignored.

Improve this answer
3
  • This works great however I ran into a problem where some lines have multiple numbers followed by a colon, so I need to pull out ONLY the numbers immediately following the "blah:" string
    – blake
    Sep 26, 2017 at 18:27
  • @blake The sed solution would be easiest to modify: sed -n '/is the/s/^.*blah:\([0-9]*\);.*$/\1/p' file | sort -u
    – Kusalananda ♦
    Sep 26, 2017 at 18:33
  • Perfect! Thank you Kusalananda - that worked perfectly
    – blake
    Sep 26, 2017 at 18:43
0 
cat file | grep "is the" | awk -F':' '{print $2}'|awk -F';' '{print $1}'|sort -u
Improve this answer
2
  • You can combine cat|grep|awk|awk with gawk -F '[:;]' '/is the/ {print $2}' file
    – glenn jackman
    Sep 26, 2017 at 19:14
  • And then you can get the unique ids by using an associative array: gawk -F '[:;]' '/is the/ {id[$2]} END {for (i in id) print i}' file
    – glenn jackman
    Sep 26, 2017 at 19:14
0

Try this:

grep "is the" file | sed 's/.*blah://;s/;.*//' | sort -u

Explanation:

  1. grep gets all lines with "is the" (in any part of the line)
  2. sed remove all before ":" and after ";" (you could use sed -e 's/.*blah://' -e 's/;.*//' instead for best understanding)
  3. sort sorts lines
Improve this answer
2
  • Thanks Egor - I should have been more clear. in some of the lines there are multiple numbers followed by a colon. I need to pull out ONLY the numbers following the "blah:" string
    – blake
    Sep 26, 2017 at 18:32
  • @blake i corrected the answer according your comments
    – Egor Vasilyev
    Sep 26, 2017 at 18:50

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .