I have a file containing only two lines, with the following structure:

$ cat /tmp/pwpower.log
000D6F0000D34227, -114.10
000D6F0001A405C4, -130.09

The values are power values of my solar plant. Negative value means generation.

I would need the values extracted via grep/sed/awk - whatever is the smartest way. I need to have both values extracted separately and without the minus sign.

What I do now is kind of stupid but it works - I'm sure many of you will have smarter ways for me :-) Here of course I only see the values plus Minus.

To get the first value:

cat /tmp/pwpower.log |grep -o "\-.*" | head -n 1

To get the second value:

cat /tmp/pwpower.log |grep -o "\-.*" | tail -n1

And related question, is there a simple way to take these STRINGs and transform so that I can calculate the SUM ?


All values:

$ awk -F '[ -]*' '$0=$NF' /tmp/pwpower.log

Value on first line:

$ awk -F '[ -]*' 'NR==1{print $NF;exit}' /tmp/pwpower.log

Value on second line:

$ awk -F '[ -]*' 'NR==2{print $NF;exit}' /tmp/pwpower.log

Sum of all values:

$ awk -F '[ -]*' '{sum+=$NF} END{print sum}' /tmp/pwpower.log
  • 1
    Your FS doesn't need to be that complicated: -F- will do. – glenn jackman Jun 17 '14 at 23:15
  • For more complex input, if you don't know if preceding fields have spaces, you can use a comma as the field separator and run the whole thing through tr -d "- " first. – Jason C Jun 18 '14 at 1:03
  • @glennjackman The way I interpret the question is that the value can be non-negative (no generation) which breaks with just -F-. – Adrian Frühwirth Jun 18 '14 at 11:50
  • Indeed, I hadn't read that closely. – glenn jackman Jun 18 '14 at 13:40

You can use cut to select the 2nd column of numbers, and paste -sd+ to create a series of numbers to add together. The tool bc can then be used to do the calculation.

$ cut -d',' -f2 pwpower.log | paste -sd+ | bc

How it works

Selects the numbers from the 2nd column.

$ cut -d',' -f2 pwpower.log 

Re-formats them into a single line with a + sign in between each number:

$ cut -d',' -f2 pwpower.log | paste -sd+
 -114.10+ -130.09

Performs the calculation:

$ cut -d',' -f2 pwpower.log | paste -sd+ | bc

To get the absolute value:

$ cut -d',' -f2 pwpower.log | sed 's/-//g' | paste -sd+ | bc

If the format of the file pwpower.log is guaranteed you can have cut do the omitting of the minus sign:

$ cut -d'-' -f2 pwpower.log | paste -sd+ | bc

A KISS approach

$ awk '{print -$2; t+=-$2}; END{print t}' pwpower.log 
  • 1
    KISS? What is that? – Bernhard Jun 17 '14 at 10:42
  • Keeping it Simple and Stupid ;) – steeldriver Jun 17 '14 at 10:49
  • @sim yes but unless I misread, the OP specifically asked for the sign to be removed – steeldriver Jun 17 '14 at 11:36
  • @steeldriver - ah yes my bad, I missed that sentence buried in there. – slm Jun 17 '14 at 13:05
  • 4
    Keep it simple, stupid. To imply when you don't keep it simple, you are being stupid. – Matt Jun 17 '14 at 13:34

I like your grep command, but it could be improved to remove the minus sign, and work in cases when there is no minus sign. Extended regular expressions available in GNU grep with the -E flag allow us to match a number more precisely.

It is slightly more efficient not to use cat, but pass the file name as an argument to the first command and let it read the file. It also occurs to me that if you are dealing with just the first or last lines from the file, it makes more sense to use the head or tail commands first so that you only have to match one line with grep.

First value:

$ head -n 1 /tmp/pwpower.log | grep -oE '[0-9\.]+$' 

Last value:

$ tail -n 1 /tmp/pwpower.log | grep -oE '[0-9\.]+$'

Sum (with awk command from here):

$ grep -oE '[0-9\.]+$' /tmp/pwpower.log | awk '{s+=$1} END {print s}'
[root@ip-10-186-149-181 ~]# cut -d '-' -f2 /tmp/pwpower.log | paste -sd+ | bc

This will do the calculation without the minus.

I think cut is faster than awk, in general


awk is the right tool, but the number probably can be positive (right?), which means you do not want to use the minus sign as a field separator. Instead, use the comma as the field separator, then negate each value numerically -- awk will automatically convert strings to numbers for you:

$ awk -F, '{ print -$2 }' < /tmp/pwpower.log

If there happen to be any positive numbers, they'll come out negative. If you only want the sum, awk can do that too:

$ awk -F, '{ sum += -$2 } END { print sum }' < /tmp/pwpower.log
  • In awk you can use sqrt($2^2) as a trick to get absolute value. – Jason C Jun 18 '14 at 0:57
  • @JasonC That's clever, but in context I believe it would be the wrong thing to do. – zwol Jun 18 '14 at 13:14

To sum the two values:

(awk -F- '{printf "%s+", $2}' /tmp/pwpower.log; echo 0) | bc -l
  • This all is a bit redundant. Why on earth nog use the calculation options of awk? – Bernhard Jun 17 '14 at 10:42
  • 1
    yeah, true, but i like bc =) – chaos Jun 17 '14 at 10:44
  • 1
    Ohw, well, then why use awk? echo $(cut -d- -f2 file | tr '\n' '+')0 | bc – Bernhard Jun 17 '14 at 10:51
  • @Bernhard Well, why use cut? Use [insert_cmd_here] instead, generate a subshell loop, invent new maths, use a human cluster or mental arithmetic, pipe my thoughts into bc. Why do we do things? No reason to make my answer bad. – chaos Jun 18 '14 at 18:05

You can use sed also

$-sed -r 's/[^-]+.(.*)/\1/g' /tmp/pwpower.log

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.