Parsing a Hadoop version string with Bash

Question

we have this CLI syntax

hdp-select | grep hadoop-client
hadoop-client - 2.6.4.0-91

the final goal is to get the number as example:

2640

we capture the last number & remove the - and remove the .

so I did

 hdp-select | grep hadoop-client | awk '{print $3}' | sed s'/-/ /g' | awk '{print $1}' | sed s'/\.//g'
 2640

but this is ugly solution

I will happy to know other elegant solution

Answer 1

With sed

hdp-select | sed '/^hadoop-client - /!d;s///;s/-.*//;s/\.//g'

Answer 2

Optimized solution with single awk program:

hdp-select | awk -F'-' '/^hadoop-client/{ gsub(/[[:space:].]+/, "", $3); print $3; exit }'

The output:

2640

Answer 3

You can easily do that in bash:

input='2.6.4.0-91'
input=${input%-*}
input=${input//./}
echo "$input"
2640

Answer 4

With awk: Use -, . and space as separators and print columns 5 to 8.

hdp-select | awk -F '[-. ]' '/hadoop-client/ {print $5$6$7$8}'

Output:

2640

Answer 5

Another sed, arguably simpler than the other (two commands vs three):

hdp-select | sed -n 's/hadoop-client - \(.*\)-.*/\1/;s/\.//gp'

Answer 6

You can use the column tool from the util-linux package:

column -t -s . -o '' <<'eof'
1.0.0
eof

Or:

echo 1.0.0 | column -t -s . -o ''

Result:

100

Answer 7

Here's how I would do it:

input="hadoop-client - 2.6.4.0-91"
if [[ "$input" =~ ([0-9\.]+) ]] ; then
    value=${BASH_REMATCH[1]} 
    value=${value//./}
    echo "Matched '$value'"
else
    echo "Unparseable"
fi

It's not one line, but that's an advantage for me. It's readable, and uses only the tools it needs to - a regular expression to extract the digits of the version number, and then a replacement to strip out the '.' characters. Additionally, it gives a basic form of validation that what was given is what was expected - if you wanted to be more cautious, you'd throw in hadoop-client\ -\ before the bracket in the regex to check that the input really is what you think it should be.

Relies on regular expressions in bash, but as they've been around since bash version 3 (2004), that's probably ok.

Answer 8

It is posible to do it directly in the shell (bash):

$ re='hadoop-client - ([0-9]+).([0-9]+).([0-9]+).([0-9]+)-'
$ a=$(hdp-select)
$ (IFS='' ; [[ $a =~ $re ]] && echo "${BASH_REMATCH[*]:1}" )
2640

If you are limited to older shells, use a similar regex with expr:

$ expr "$(hdp-select)" : '.*client - \([0-9]\+\.[0-9]\+\.[0-9]\+\.[0-9]\+\)-.*' | sed 's/\.//g'

Answer 9

This solution is suitable for someone who knows regular expressions and does not know sed or awk.

expr and tr come from "GNU Core Utilities". Unfortunately, expr uses basic regular expression and each regular expression is anchored with ^.

expr "$(hdp-select)" : '.*oop-client - \(\([0-9]\+\.\?\)\+\)-[0-9]\+' | tr -d '.'

Result:

2640