Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

Say I have lines like this:

*[234]*
*[23]*
*[1453]*

where * represents any string (except a string of the form [number]). How can I parse these lines with a command line utility and extract the number between brackets?

More generally, which of these tools cut, sed, grep or awk would be appropriate for such task?

share|improve this question

4 Answers 4

up vote 10 down vote accepted

If you have GNU grep, you can use its -o option to search for a regex and output only the matching part. (Other grep implementations can only show the whole line.) If there are several matches on one line, they are printed on separate lines.

grep -o '\[[0-9]*\]'

If you only want the digits and not the brackets, it's a little harder; you need to use a zero-width assertion: a regexp that matches the empty string, but only if it is preceded, or followed as the case may be, by a bracket. Zero-width assertions are only available in Perl syntax.

grep -P -o '(?<=\[)[0-9]*(?=\])'

With sed, you need to turn off printing with -n, and match the whole line and retain only the matching part. If there are several possible matches on one line, only the last match is printed. See Extracting a regex matched with 'sed' without printing the surrounding characters for more details on using sed here.

sed -n 's/^.*\(\[[0-9]*\]\).*/\1/p'

or if you only want the digits and not the brackets:

sed -n 's/^.*\[\([0-9]*\)\].*/\1/p'

Without grep -o, Perl is the tool of choice here if you want something that's both simple and comprehensible. On every line (-n), if the line contains a match for \[[0-9]*\], then print that match ($&) and a newline (-l).

perl -l -ne '/\[[0-9]*\]/ and print $&'

If you only want the digits, put parentheses in the regex to delimit a group, and print only that group.

perl -l -ne '/\[([0-9]*)\]/ and print $1'

P.S. If you only want to require one or more digits between the brackets, change [0-9]* to [0-9][0-9]*, or to [0-9]+ in Perl.

share|improve this answer
    
All good, other than that he wants to "extract the number between brackets". I think "except [number]" means except [0-9] –  Peter.O Mar 8 '12 at 0:06
1  
@Peter.O I understood “except [number]” to mean that there aren't other parts of the line of that form. But I edited my answer to show how to print only the digits, just in case. –  Gilles Mar 8 '12 at 0:39
1  
Those perl regex asserts look really useful! I've been reading about them after seeing you use both backward and forward assertions, even in grep (I'd switched off to the the fact you can choose a regex engine). I'll be devoting a bit more time to perl's regex from here on. Thanks... PS.. I just read in man grep... "This is highly experimental and grep -P may warn of unimplemented features." ... I hope that doesn't mean unstable(?) ... –  Peter.O Mar 8 '12 at 6:16

You can't do it with cut.

  1. tr -c -d '0123456789\012'
  2. sed 's/[^0-9]*//g'
  3. awk -F'[^0-9]+' '{ print $1$2$3 }'
  4. grep -o -E '[0-9]+'

tr is the most natural fit for the problem and would probably run the fastest, but I think you would need gigantic inputs to separate any of these options in terms of speed.

share|improve this answer
    
For sed, ^.* is greedy and consumes all but the last digit, and + needs to be \+ or else use the posix \([0-9][0-9]*\).... and in any case 's/[^0-9]*//g' works just as well, ... Thanks for the tr -c` example, but isn't that trailing \012 surperfluous? –  Peter.O Mar 8 '12 at 0:48
    
@Peter Thanks for catching that. I'd have sworn I tested the sed example. :( I've changed it to your version. Regarding \012: it is needed otherwise tr will eat the newlines. –  Kyle Jones Mar 8 '12 at 0:58
    
Aha... I was seeing it as \0, 1, 2 (or even \, 0, 1, 2). I'm not well enough attuned to octal it seems.. Thanks. –  Peter.O Mar 8 '12 at 6:48

If you mean extract a set of consecutive digits between non-digit characters, I guess sed and awk are the best (although grep is also able to give you the matched characters):

sed: you can of course match the digits, but it's perhaps interesting to do the opposite, remove the non-digits (works as far as there is only one number per line):

$ echo nn3334nn | sed -e 's/[^[[:digit:]]]*//g'
3344

grep: you can match consecutive digits

$ echo nn3334nn | grep -o '[[:digit:]]*'
3344

I don't give an example for awk because I have null experience with it; it is interesting to note that, although sed is a swiss knife, grep gives you a simpler, more readable way to do this, which also works for more than one number on each input line (the -o only prints the matching parts of the input, each one on its own line):

$ echo dna42dna54dna | grep -o '[[:digit:]]*'
42
54
share|improve this answer
    
Just as a comparison, here is a sed eqivalent of the "more than one number per line" example grep -o '[[:digit:]]*' . . . sed -nr '/[0-9]/{ s/^[^[0-9]*|[^0-9]*$//g; s/[^0-9]+/\n/g; p}' ... (+1) –  Peter.O Mar 7 '12 at 21:10

Since it has been said that this cannot be done with cut, I will show that it is easily possible to produce a solution that is at least not worse than some of the others, even though I do not endorse the use of cut as the "best" (or even a particularly good) solution. It should be said that any solution not looking specifically for *[ and ]* around the digits makes simplifying assumptions and is therefore prone to failure on examples more complex than then one given by the asker (e.g. digits outside *[ and ]*, which should not be shown). This solution checks at least for the brackets, and it could be extended to check the asterisks as well (left as an exercise to the reader):

cut -f 2 -d '[' myfile.txt | cut -f 1 -d ']'

This makes use of the the -d option, which specifies a delimiter. Obviously you could also pipe into the cut expression instead of reading from a file. While cut is probably pretty fast, since it is simple (no regex engine), you have to invoke it at least twice (or a few more time to check for *), which creates some process overhead. The one real advantage of this solution is that it is rather readable, especially for casual users not well versed in regex constructs.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.