How do I extract the digits from a line and save it into a variable?

Question

I feel like this is a really easy question, and when I Google, I find lots of answers for part of the problem, but when I try to put them together, it doesn't work and I can't figure out why.

Here's the scenario:

1. I have a file with a lot of text in it.
2. One of those lines matches this pattern: foo = 1700;
3. I want to extract 1700
4. I want to save it into a bash script variable so I can refer to it later in the script.

I cannot get past step 3. Here's what I've tried:

sed -nE 's/^foo = //p' file | sed -nE 's/;//p'

This prints out:

1700

Great, but what if I need to trim white space or something? If I can't use */+, I wouldn't know how to do that. I learned that you can't use */+ in a substitution on another answer, so I can't figure out how to do this. I looked into the man page of grep, and I didn't see any option for groups when I search for that word. I think I know how to solve this in awk, but I've always found its regex functions to be a little clunky and for the commandline scripts to require too many escapes, so ideally that's not the only way to solve this.

Answer 1

1. To start with, here's how to capture the numeric value:

   $ echo 'foo = 1700;' | sed -n -e 's/^foo = \([0-9]\+\).*/\1/p'
   1700
   

   That's using sed's default Basic Regular Expressions (BRE). You can also use Extended Regular Expressions (ERE) with sed's -E option:

   echo 'foo = 1700;' | sed -n -E -e 's/^foo = ([0-9]+).*/\1/p'
   1700
   

   The sub-expression [0-9]+ inside the parentheses (...) captures one-or-more digits. This is called a "capture group" and is used in the replacement with the \1 (which is the first capture group - if there are multiple capture groups, they can be used as \1, \2, \3, etc).

   In this case, the sed script tries to replace the entire line with just the \1 capture group and if that succeeded, print the modified line.

2. Next, you want to get sed's output into a variable. You do that with with command substitution. e.g.

   $ myvar=$(echo 'foo = 1700;' | sed -n -E -e 's/^foo = ([0-9]+).*/\1/p')
   $ echo $myvar
   1700
   

3. To use this in your script, just use your file as an argument to sed instead of piping echo ... into it.

   myvar=$(sed -n -E -e 's/^foo = ([0-9]+).*/\1/p' file)
   

4. To trim white space, or to cope with lines that might have optional leading whitespace, or optional whitespace around the =, etc:

   myvar=$(sed -n -E -e 's/^[[:space:]]*foo[[:space:]]*=[[:space:]]*([0-9]+).*/\1/p' file)
   

   Note that some versions of sed (GNU sed, at least. maybe others) understand perl's \s, so you can shorten that to:

    myvar=$(sed -n -E -e 's/^\s*foo\s*=\s*([0-9]+).*/\1/p' file)
   

Answer 2

For completeness, with grep implementations that support -o and perl-like regular expressions with -P, you can do:

grep -Po 'foo\s*=\s*\K\d+'

Where:

• \s matches any whitespace character
• * 0 or more of the preceding atom. So \s* for instance matches 0 or more whitespace characters.
• \d matches a decimal digit (generally the same as [0123456789], though not [0-9] which often matches a lot more characters).
• + matches one or more of the preceding atom.
• \K resets the start of the matched portion (what to Keep, or in the case of grep -o what to output).

So, that will print all the sequences of one or more digits that follow foo= with any amount of whitespace allowed on either side of the =, even if more than one occur on a given line.

With pcregrep, you can also specify a number after -o to print what's matched by the give capture group, instead of the whole matched portion:

pcregrep -o1 'foo\s*=\s*(\d+)'

Portably, you can actually use the real thing: perl:

perl -lne 'print $1 for m{foo\s*=\s*(\d+)}g'

Answer 3

Assuming you want to pick out a numeric foo value,

echo 'foo = 1700;' | awk '$1=="foo" {print $NF+0}'
1700

By default awk splits on whitespace (not just a single space). NF is the number of fields, in this case 3; $NF is the string value of the 3rd whitespaced field. The +0 converts this string 1700; to the numeric value 1700.

It will work on lines such as foo = 1700; but not lines such as foo=1700;. I wasn't sure from your question whether you were simply concerned about removing extra whitespace, or that your data might have no whitespace and all, with the = and ; being the only boundary points. If you want to ignore any whitespace, regardless of whether there is some or none, you would be better off with sed,

echo 'foo=1700;' | sed -n 's/^foo *= *//p' | sed -e 's/;$//' -e 's/ *$//'
1700

Answer 4

Using gawk:

Assuming file input as:

$ cat input
foo = 1700
foo=17
foo           2000
foobarfoo = 200
foo = foo = 14 243
foo =
200 = foo

This gawk command would do:

awk '{if(match($0, /\<foo\s*=\s*[0-9]+/)){ 
a=substr($0,RSTART,RLENGTH);sub(/foo\s*=\s*/, "",a); print a}}' input

Or with this gawk command:

awk '{if(match($0, /\<foo\s*=\s*[0-9]/))
{ l=RLENGTH-1;match($0, /foo\s*=\s*[0-9]+/)
print substr($0,RSTART+l,RLENGTH-l); }}' input

If pattern is simple as in foo = 1700, i.e., a space after foo and equal sign, then the above command can be shortened:

awk '{if(match($0, /\<foo = [0-9]+/)) print substr($0,RSTART+6,RLENGTH-6);}' input

Another method:

awk '/\<foo\s*=\s*[0-9]+/ {print gensub(/(.*)(foo\s*=\s*)([0-9]+)(.*)/, "\\3", "g") }' input

In this command, all captured groups are replaced by third captured group using backreferencing(\\3). gensub() is a gawk builtin function.

Answer 5

awk '{for(i=1;i<=NF;i++){if($i ~ /foo/ && $0 ~ /foo.*=.*[0-9]*/){gsub(";","",$(i+2));print $(i+2)}}}' filename

Tested and worked fine

Answer 6

awk '/foo = /{print $NF}' file

... or if you want to save output to var

VAR=$(awk '/foo = /{print $NF}' file)