0

Here is an example block of text from a file:


Now is the time for all blah:1; to come to the aid
Now is the time for all blah:1; to come to the aid
Now is the time for all blah:1; to come to the aid
Now is the time for all blah:10; to come to the aid
Go to your happy place  blah:100; to come to the aid
Go to your happy place  blah:4321; to come to the aid
Go to your happy place  blah:4321; to come to the aid
Now is the time for all blah:4321; to come to the aid
Now is the time for all blah:9876; to come to the aid
Now is the time for all blah:108636; to come to the aid
Now is the time for all blah:1194996; to come to the aid

Question: How would I extract all unique numbers from lines that have "is the" in them?

I've tried using grep -o -P -u '(?<=blah:).*(?=;)' but it doesn't like the semi colon

  • 1
    Ok a little clarification: the 'print $2' solution won't work as the text preceding "blah:" varies up to appx 30 characters and includes many special characters that get interpreted as separate fields – blake Sep 26 '17 at 18:16
  • I need to pull out only the numbers immediately following the "blah:" string because each line has multiple numbers followed by a colon – blake Sep 26 '17 at 18:29
  • 1
    You should put all your requirements in the question (you can edit your question) – glenn jackman Sep 26 '17 at 19:47
  • If you're happy with one or several of the answers, upvote them. If one is solving your issue, accepting it would be the best way of saying "Thank You!" :-) – Kusalananda Sep 26 '17 at 19:51
5

You're looking for the \K directive to forget about the stuff you just matched.

grep -oP 'is the.*?blah:\K\d+'

Then sort -u

| improve this answer | |
  • grumble grumble, hidden requirements, mutter, mutter – glenn jackman Sep 26 '17 at 19:47
  • 1
    The story of every software developer... – Kusalananda Sep 26 '17 at 19:48
3

Using sed:

$ sed -n '/is the/s/^.*blah:\([0-9]*\);.*$/\1/p' file | sort -u
1
10
108636
1194996
4321
9876

The substitution replaces the contents of all lines containing the string is the with the number between blah: and ;. Lines not containing the string are ignored.

| improve this answer | |
  • This works great however I ran into a problem where some lines have multiple numbers followed by a colon, so I need to pull out ONLY the numbers immediately following the "blah:" string – blake Sep 26 '17 at 18:27
  • @blake The sed solution would be easiest to modify: sed -n '/is the/s/^.*blah:\([0-9]*\);.*$/\1/p' file | sort -u – Kusalananda Sep 26 '17 at 18:33
  • Perfect! Thank you Kusalananda - that worked perfectly – blake Sep 26 '17 at 18:43
0
cat file | grep "is the" | awk -F':' '{print $2}'|awk -F';' '{print $1}'|sort -u
| improve this answer | |
  • You can combine cat|grep|awk|awk with gawk -F '[:;]' '/is the/ {print $2}' file – glenn jackman Sep 26 '17 at 19:14
  • And then you can get the unique ids by using an associative array: gawk -F '[:;]' '/is the/ {id[$2]} END {for (i in id) print i}' file – glenn jackman Sep 26 '17 at 19:14
0

Try this:

grep "is the" file | sed 's/.*blah://;s/;.*//' | sort -u

Explanation:

  1. grep gets all lines with "is the" (in any part of the line)
  2. sed remove all before ":" and after ";" (you could use sed -e 's/.*blah://' -e 's/;.*//' instead for best understanding)
  3. sort sorts lines
| improve this answer | |
  • Thanks Egor - I should have been more clear. in some of the lines there are multiple numbers followed by a colon. I need to pull out ONLY the numbers following the "blah:" string – blake Sep 26 '17 at 18:32
  • @blake i corrected the answer according your comments – Egor Vasilyev Sep 26 '17 at 18:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.