I'm finding really hard to use sed command, plus I can't seem to find well written tutorials.

Let me say that I worked with regular expression in other languages (Python, JavaScript, Java), so that shouldn't be the problem.

So, here are my questions (a "theoretical" one and a more practical one):

  1. are regular expressions used in sed exactly the same as the ones used by Python/JS/Java? I read about BREs and EREs, but how much are they different? Shouldn't ERE be an extension of BRE?

  2. if I want to, say, just extract something from a piped output, what's the sed syntax to do that?

Details about the second question: say I have the output of uptime piped with sed:

uptime | sed ...

Given an example output from uptime: 18:13 up 5:12, 2 users, load averages: 0,45 0,37 0,40, I want to parse the single uptime in the form of two separated numbers (hours and minutes), and then I want to display them in the form of xxhyym (xx are the hours, yy the minutes).

And to finish, here's what I'd do in Python:

hh, mm = re.match(r'\s+ up \s+(\d{1,2}):(\d{1,2})').groups()
print '%sh%sm' % (hh, mm)
  • 2
    Just a heads up: both your python and @vonbrand's sed will fail if the uptime is longer than a day (unless you both really mean to ignore days). – tink Mar 26 '13 at 17:37

Traditional unix tools support either BRE or ERE (basic or extended regular expressions). POSIX codifies both. Wikipedia explains them. Most modern tools extend ERE, often with additional features first introduced in Perl (which is known as PCRE).

ERE extends the functionality of BRE, but doesn't extend the syntax. In BRE, only the characters \[.*^$ have a special meaning, and some operators such as grouping \(…\) use backslashes. In ERE, +?|() are also special, and backslash followed by a non-alphanumeric character is never special.

BRE doesn't have Python/PCRE's \d and \s. You can express these character sets with the traditional set constructs and character classes: \d is [[:digit:]] and \s is [[:space:]]. Note the double brackets: one to indicate a character set and one to indicate a character class; for example “letters, dashes or underscores” can be written [-_[:alpha:]].

BRE's don't have a + operator (some sed implementations support \+ as an extension to the BRE syntax); X+ is the same as XX*. Groups and match counts need additional backslashes.

The BRE equivalent of Python's \s+ up \s+(\d{1,2}):(\d{1,2}) is thus [[:space:]][[:space:]]* up [[:space:]][[:space:]]*\([[:digit:]]\{1,2\}\):\([[:digit:]]\{1,2\}\). Note that you're matching too much whitespace: \s+ and a space means at least two whitespace characters.

You'll need to match the whole line, as sed's s command rewrites the line. There isn't a separate command to write out a string assembled from saved groups. Correcting for the extra whitespace, the analog of your Python snippet is:

uptime | sed 's/^.*[[:space:]][[:space:]]*up[[:space:]][[:space:]]*\([[:digit:]]\{1,2\}\):\([[:digit:]]\{1,2\}\).*$/\1h\2m/'

Unlike the Python snippet, this extracts the first match rather than the last match, but it doesn't matter here.

The output of uptime sticks to space characters and ASCII digits, so you can simplify the regex:

uptime | sed 's/^.* up  *\([0-9]\{1,2\}\):\([0-9]\{1,2\}\).*$/\1h\2m/'

This will only match the output of uptime if the machine has been up for less than 1 day. I'll leave matching the number of days as an exercise. (Hint: write two expressions: sed -e s/AS ABOVE/\1h\2m/ -e 's/EXERCISE/\1d\2h\3m/')

  • Note that ERE's + can be written \{1,\} in BREs. – Stéphane Chazelas Mar 27 '13 at 0:28

Each tool uses (mostly) their own RE library. Even among different sed versions you will find differences here. Two popular standards are POSIX standard regular expressions, many tools accept those (at least with some options), another popular set is the Perl Compatible Regular Expression library (PCRE). But the last ones are quite a bit different from "vanilla" REs...

In your case:

uptime | sed -e 's/^ \([0-9][0-9]\):\([0-9][0-9]\).*$/\1h\2m/'

(Tried on Fedora 18, sed-4.2.1-10.fc18.x86_64, GNU sed).

Update: What is wrong with the copious documentation on GNU sed's homepage? Or this tutorial? The info documentation for GNU sed is a bit long-winded, but complete.

  • Mmm ok, that's worse than I imagined it. I hoped I could use some syntax like \d{1,2} which I really find useful. Anyway, on OS X the command you wrote returns the whole output of uptime... LOL. Anyway, thank you for the answer, I'm gonna study sed in a deeper way. – whatyouhide Mar 26 '13 at 17:43
  • If you want that kind of syntax (not universal), you might want to consider perl, it has very rich patterns and is quite one-liner friendly. And no, I haven't ever claimed to be a sed expert. – vonbrand Mar 26 '13 at 17:58
  • No I wasn't saying that, I was just saying that sed interests me so I'm gonna study it deeply probably :). – whatyouhide Mar 26 '13 at 18:10
  • Oh sure you can use something like \d{1,2},namely [0-9]\{1,2\}. The syntax of sed can be a bit off, and the regexps are not as powerful, but I find it very useful for many tasks on the command line – jpmuc Mar 26 '13 at 20:05
  • 1
    @juanpa, for this problem that is more typing (and more thinking ;-) than my solution. – vonbrand Mar 26 '13 at 20:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.