8

I am trying to manipulate a file which contains numbers in scientific notation, but without the e symbol, i.e. 1.2e+3 is written as 1.2+3.

The easiest thing I thought of doing with awk was to replace + with e+, using the gsub function and do my calculation in the new file. The same goes for the minus case. So a simple fix could be done using the following command

awk '{gsub("+", "e+", $1); print $1, $2, $3, $4, $5}' file_in

and do the same in all the columns.

However the file contains also negative numbers which makes things a bit more complicated. A sample file can be seen bellow

 1.056000+0 5.000000-1 2.454400-3 2.914800-2 8.141500-6
 2.043430+1 5.000000-1 2.750500-3 2.698100-2-2.034300-4
 3.829842+1 5.000000-1 1.969923-2 2.211364-2 9.499900-6
 4.168521+1 5.000000-1 1.601262-2 3.030919-2-3.372000-6
 6.661784+1 5.000000-1 5.250575-2 3.443669-2 2.585500-5
 7.278104+1 5.000000-1 2.137055-2 2.601701-2 8.999800-5
 9.077287+1 5.000000-1 1.320498-2 2.961020-2-1.011600-5
 9.248130+1 5.000000-1 3.069610-3 2.786329-2-6.317000-5
 1.049935+2 5.000000-1 4.218794-2 3.321955-2-5.097000-6
 1.216283+2 5.000000-1 1.432105-2 3.077165-2 4.300300-5

Any idea on how to manipulate and calculations with such a file?

2
  • 2
    How do you want to make calculations with a format like that 2.698100e-2-2.034300e-4 ?
    – ctac_
    Mar 5, 2019 at 8:44
  • 3
    This looks like it's probably meant to be parsed as fixed-width column data. The apparent whitespace between columns is just an artifact of the number format displaying positive values with a leading space instead of a plus sign. Mar 5, 2019 at 12:28

2 Answers 2

14

Is this output correct?

 1.056000e+0 5.000000e-1 2.454400e-3 2.914800e-2 8.141500e-6
 2.043430e+1 5.000000e-1 2.750500e-3 2.698100e-2-2.034300e-4
 3.829842e+1 5.000000e-1 1.969923e-2 2.211364e-2 9.499900e-6
 4.168521e+1 5.000000e-1 1.601262e-2 3.030919e-2-3.372000e-6
 6.661784e+1 5.000000e-1 5.250575e-2 3.443669e-2 2.585500e-5
 7.278104e+1 5.000000e-1 2.137055e-2 2.601701e-2 8.999800e-5
 9.077287e+1 5.000000e-1 1.320498e-2 2.961020e-2-1.011600e-5
 9.248130e+1 5.000000e-1 3.069610e-3 2.786329e-2-6.317000e-5
 1.049935e+2 5.000000e-1 4.218794e-2 3.321955e-2-5.097000e-6
 1.216283e+2 5.000000e-1 1.432105e-2 3.077165e-2 4.300300e-5

Code:

perl -lne 's/(\.\d+)(\+|\-)/\1e\2/g; print' sample

Explanation:

  • -lne take care of line endings, process each input line, execute the code that follows

  • s/(\.\d+)(\+|\-)/\1e\2/g:

    • substitute (s)
    • (.\d+)(\+|\-) find two groups of (a dot and numbers) and (a plus or minus)
    • \1e\2 substitute them with the first group then e then the second group
    • g globally - don't stop at the first substitution in each line, but process all possible hits
  • print print the line

  • sample input file

This one adds space if it's missing. In fact it puts space between the numbers regardless. Ie. if there were two spaces in some case, there would be only one in the output.

perl -lne 's/(\.\d+)(\+|\-)(\d+)(\s*)/\1e\2\3 /g; print' sample

Most of it is similar to the previous one. The new thing is the (\d+) group nr 3 and the (\s*) group nr 4. * here means optional. In the substitution no \4 is used. There's a space instead.

The output is this:

 1.056000e+0 5.000000e-1 2.454400e-3 2.914800e-2 8.141500e-6 
 2.043430e+1 5.000000e-1 2.750500e-3 2.698100e-2 -2.034300e-4 
 3.829842e+1 5.000000e-1 1.969923e-2 2.211364e-2 9.499900e-6 
 4.168521e+1 5.000000e-1 1.601262e-2 3.030919e-2 -3.372000e-6 
 6.661784e+1 5.000000e-1 5.250575e-2 3.443669e-2 2.585500e-5 
 7.278104e+1 5.000000e-1 2.137055e-2 2.601701e-2 8.999800e-5 
 9.077287e+1 5.000000e-1 1.320498e-2 2.961020e-2 -1.011600e-5 
 9.248130e+1 5.000000e-1 3.069610e-3 2.786329e-2 -6.317000e-5 
 1.049935e+2 5.000000e-1 4.218794e-2 3.321955e-2 -5.097000e-6 
 1.216283e+2 5.000000e-1 1.432105e-2 3.077165e-2 4.300300e-5 
4
  • Thank you very much for the answer! Yes it seems correct!! Can you explain what you did, for future reference?
    – Thanos
    Mar 5, 2019 at 7:25
  • Is it also possible to separate the last column ($5$) from the previous one with a space?
    – Thanos
    Mar 5, 2019 at 7:33
  • You are perfect! Thank you very much for your help!
    – Thanos
    Mar 5, 2019 at 7:44
  • @Thanos See the update. And notice I added a backslash before . in the first group. This is correct. Without this backslash the dot would not mean a literal dot.
    – user147505
    Mar 5, 2019 at 7:47
2

You could also use sed, e.g.:

<infile sed -E 's/([0-9])([+-])([0-9])/\1e\2\3/g' | awk '{ print $1 + 0 }'

However, this does not take into account that the columns in OP's listing are sometimes not separated. Here is a workaround with appropriate precision:

<infile sed -E 's/.{11}/& /g'       |
sed -E 's/([0-9])([+-])/\1e\2/g'    |
gawk '{ print $1 + 0 }' OFMT='%.7g'

Output:

1.056
20.4343
38.29842
41.68521
66.61784
72.78104
90.77287
92.4813
104.9935
121.6283
2
  • This removes resolution from the numbers, and I'm not sure it will work when a negative value is next to another like the example in the question 2.698100-2-2.034300-4
    – pipe
    Mar 5, 2019 at 14:31
  • @pipe: You are right, I missed that detail. I have added a workaround by adding space. Wrt. precision, I used the OFMT variable to set awk's precision to the same as the input's
    – Thor
    Mar 7, 2019 at 10:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.