I am running Ubuntu 14.04.1 LTS 64-bit with Bash 4.3.11(1)-release I have a program called harminv producing output as follows:

$ h5totxt hsli0.126.h5 | harminv -vt 0.1 -w 2-3 -a 0.9 -f 200 
# harminv: 1902 inputs, dt = 0.1
frequency, decay constant, Q, amplitude, phase, error
# searching frequency range 0.31831 - 0.477465
# using 200 spectral basis functions, density 6.60692
-2.14026, 3.511909e-05, 30471.5, 0.922444, 1.26783, 1.383955e-06
2.14013, 2.052504e-05, 52134.7, 0.920264, -1.27977, 3.426846e-07
# harminv: 2/6 modes are ok: errs <= 1.000000e-01 and inf * 3.426846e-07
, amps >= 0, 9.000000e-01 * 0.922444, |Q| >= 10

When the -v(verbose) option is omitted I have a much neater output as follows:

$ h5totxt hsli0.126.h5 | harminv -t 0.1 -w 2-3 -a 0.9 -f 200 
frequency, decay constant, Q, amplitude, phase, error
-2.14026, 3.511909e-05, 30471.5, 0.922444, 1.26783, 1.383955e-06
2.14013, 2.052504e-05, 52134.7, 0.920264, -1.27977, 3.426846e-07

I would like to be able to extract the positive numbers in the first column of the output in both cases, but have no idea on how to do it, except that I can use sed or awk. I would be grateful if someone points me in the right direction, and my aim is to record each positive number to make a plot against some other variable.


Using sed

This will print only lines that start with a positive number:

sed -n 's/^\([[:digit:]][^ ,]*\).*/\1/p'

Combined with one of your pipelines, it would look like:

h5totxt hsli0.126.h5 | harminv -vt 0.1 -w 2-3 -a 0.9 -f 200 | sed -n 's/^\([[:digit:]][^ ,]*\).*/\1/p'

How it works

  • -n

    This tells sed not to print any line unless we explicitly ask it to.

  • s/^\([[:digit:]][^ ,]*\).*/\1/p

    This tells sed to look for lines that start with a positive number and print only that number.

    In a regex, ^ matches only at the beginning of a line. [[:digit:]] matches any digit. [^ ,]* matches anything that follows that digit except a space or a comma. This is all grouped with parenthesis so that we can refer to the number later as \1. The whole line is then replaced with the number and, with the p option, we tell sed to print it.

    One used to use [0-9] to match digits. With the advent of unicode fonts, that is no longer reliable. The expression [[:digit:]], however, is unicode safe.

Alternative using extended regex

If you are using GNU sed (which is true of all linux systems), then the -r option can be used to get extended regular expressions. With extended regex, parens used for grouping do not need to be escaped:

sed -rn 's/^([[:digit:]][^ ,]*).*/\1/p'

On OSX or other BSD systems, use -E in place of -r.

Using awk

This does the same but using awk:

awk -F, '/^[[:digit:]]/{print $1}'

Combined with your pipeline:

h5totxt hsli0.126.h5 | harminv -vt 0.1 -w 2-3 -a 0.9 -f 200 | awk -F, '/^[[:digit:]]/{print $1}'
  • How does it select only positive numbers? I think the first number can be selected with awk after one more pipe, right? The solution with awk does not produce any output by the way. – Vesnog Feb 19 '15 at 22:09
  • 1
    @Vesnog OK. I updated it to print only the first number. I added an explanation for the sed option in the updated answer. Let me know if you have questions. – John1024 Feb 19 '15 at 22:17
  • Thanks I really appreciate your effort. I got the same result adding another pipe with cut as follows: h5totxt hsli0.126.h5 | harminv -vt 0.1 -w 2-3 -a 0.9 -f 200 | sed -n '/^[[:digit:]]/p' | cut -d , -f 1 – Vesnog Feb 19 '15 at 22:19
  • 1
    @Vesnog [0-9] only works if the character set that you are using has all the digits in order starting with 0 and ending with 9. That was true of ASCII. It is not generally true of unicode. The \(...\) are grouping. This is the same as vim unless you use the \v option in your vim regex. – John1024 Feb 19 '15 at 22:29
  • 1
    Strictly speaking, this captures all numbers that are ≥ 0; i.e., positive or equal to zero. – Scott Feb 19 '15 at 23:29

Given the input you show, the following should work:

sed -n 's/[^[:digit:]]/\


sed 's/[^[:digit:]].*//;/./!d'

...with some seds you could also write it like...

sed -n 's/[^0-9]/\n&/;/.\n/P'


sed 's/[^0-9].*//;/./!d'

...and maybe even - depending on your input data set - with GNU sed, like...

sed -n 's/\W/\n&/;/.\n/P'


sed 's/\W.*//;/./!d'

Because regular language is basically a description of complements, you can almost always turn a regular expression on its head. Sometimes it makes for less work when you do.

So if you're searching the head of a string for a specific pattern that is of unknown length, it can be more simple just to look for the first part of the string which does not match your pattern.

The first example above inserts a \newline character before the first non-numeric character it encounters on a line. It then checks if, having done so (if it did so), there is at least one character between the insertion and the head of the line. If not, it does not print, but if so, it prints only up to the \newline it inserted.

The next example is similar - it just strips the longest string that it can from a line which begins with a character that does not match your pattern, then deletes all blank lines from output.

The rest are just shorthands for doing more of the same as some seds might interpret them, though the first two adhere pretty strictly to POSIX sed syntax specification (though it maybe [[:digit:]] is overkill, because, as I understand it, UTF-8 is an ASCII superset and most languages which do not incorporate Arabic numerals are also different enough from the one in which I write this to require other modifications to make this workable anyway).

All of the examples - depending on implementation and input as noted - should print only the first consecutive sequence of digit matches which begin at the head of the line.

Thinking about it, though - since it seems you're delimiting on spaces and commas anyway - I suppose it could also be written:

sed -n 'y/, -/\n\n\n/;/^[0-9]/P'

...which does hardly any actual regular expression matching at all - as the y/// function translates characters rather than grouping them on patterns. The regexp matcher is only called for to test the result.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.