1

I would like to know what it really does with following scripts used with sed command.

sed -e 's:<F0_M>:<o,f0,male>:' \
          -e 's:<F0_F>:<o,f0,female>:' \
          -e 's:([0-9])::g' \
          -e 's:<sil>::g' \
          -e 's:([^ ]*)$::' | \

First and second scripts looks like we are transforming text of type <F0_F> to <o,f0,female>. But what about the last three where it involves '::','g' and '$' sign. In most of the documentations they have used '\' and '/' in most of scripts. But here they have used ':' instead of slashes. Can some one explain above three scripts?

4

The standard delimiter used in sed commands is /, as in commands like this:

sed -e s/foo/bar/g < input > output

However, if the s command is followed by a different character, that becomes the delimiter for that particular expression.

The use of non-/ delimiters is common when the delimiter itself needs to appear in the command, and so would require careful attention to escaping. For example, a / delimiter is annoying to deal with in scripts that deal with Unix paths.

That doesn't appear to be the case here, so I assume the author of that command just prefers : as a delimiter in sed commands.

Your command has five expressions:

s:<F0_M>:<o,f0,male>:

This replaces the first instance of <F0_M> on each line of the input with <o,f0,male> in the output. If there is more than one match in the input on that line, the subsequent ones will be left alone.

The single quotes just prevent the shell from interpreting any of the characters in the expression. They're all passed literally to the sed command.

s:<F0_F>:<o,f0,female>:

Similar to the above case, only apparently for the other gender.

s:([0-9])::g

Removes all single digits in parentheses from the input lines.

Unlike the previous two expressions, this expression affects all instances on each line because of the trailing g, which means "global".

Note that it only works on single digits. It won't do anything to (42), for example.

s:<sil>::g

Removes all <sil> instances from each line of the input when writing to the output.

s:([^ ]*)$::

Removes a parenthesized character at the end of the line if it doesn't contain a space. Also removes an empty pair of parentheses at the end of a line.

There are whole books on these topics, sed and regular expressions. A single answer really is not the right place to learn the whole topic.

The above expression is actually a bit tricky in that regard: $ pins a regular expression (or regex for short) to the end of a line, and ^ to the beginning, but the ^ in that expression means something different.

I recommend that you read Mastering Regular Expressions by Jeffrey Friedl.

  • So, having :: means we are replacing whatever the content before : with null. Like we are going to drop that value in the new context. And 'g' will be used to apply that to all the matches we get through out our context? – udani Jan 17 '16 at 8:17
  • Yes, sorry, I assumed you knew enough sed to recognize something like s/foo//, so all I had to do was point out that s:foo:: does the same thing. Both mean "Remove the first instance of foo on each line of the input when writing to the output." – Warren Young Jan 17 '16 at 8:18
  • Well I am just going through each of these commands while studying a code. Can you explain the meaning of ([^ ]*)$ as well. It says [^ ]* is used to match all the words (Eg: won't want ) But what about the $ sign? In code it says [it 's -> it's : merge words that contain apostrophe] But how does this happen? – udani Jan 17 '16 at 8:32
  • @udani: See edited answer. – Warren Young Jan 17 '16 at 8:36
  • Wow, that was the kind of explanation I expected....thank you so much for your time on answering my question .... :) [It looks like I have missed the authors explanation for the last regex which say (...) -> null : remove utterance names from end-lines of train] But thanks again for the above explanation. It means alot for me. :) – udani Jan 17 '16 at 8:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.