0

I am new to regex and sed, and am trying to create what I thought would be a straightforward regex: I want to remove word-final letter if it's an 'o'.

  • Input string: Hello Hello
  • Expected output: Hell Hell

The good news: I can remove the 'o' when it is at the end of the string:

$ echo 'Hello Hello' |sed 's/\(.*\)o/\1/g'
Hello Hell
$ echo 'Hello Hello' |sed 's/\(.*\)o$/\1/g'
Hello Hell

The bad news: I cannot remove it from words earlier in the string. I have tried this with all the anchor symbols I can think of. The result is that none of the word-final 'o's is removed:

$ echo 'Hello Hello' |sed 's/\(.*\)o\b/\1/g'
Hello Hello
$ echo 'Hello Hello' |sed 's/\(.*\)o\>/\1/g'
Hello Hello
$ echo 'Hello Hello' |sed 's/\(.*\)o\W/\1/g'
Hello Hello
$ echo 'Hello Hello' |sed 's/\(.*\)o\s/\1/g'
Hello Hello

Can you please help me regain my sanity by telling me what I'm doing wrong?

Update: I get the dictinct impression that my machine produces different results than some other people's. I am using the terminal window on my Macbook. If anyone can shed some light on this, please tell me.

  • sed -e 's/o\>//g' You are welcome – AlexP Nov 20 '17 at 20:09
  • Thanks you for answering! I see why you've made this more compact. However, this exact command still results in no deletion at all on my machine. Here is a copy and paste of command and output: $ echo 'Hello Hello' |sed -e 's/o\>//g' Hello Hello – Triplesmeg Nov 20 '17 at 20:41
  • It seems I am using correct commands but somehow they don't work on my system – Triplesmeg Nov 20 '17 at 20:42
1
echo 'Hello Hello' | sed 's/o$//'

seems more useful to me than your

echo 'Hello Hello' | sed 's/\(.*\)o$/\1/g'

In your question is says that the output of

echo 'Hello Hello' | sed 's/\(.*\)o\b/\1/g'

was Hello Hello but for me it is Hello Hell. You can correct that to

echo 'Hello Hello' | sed 's/\([^o]*\)o\b/\1/g'

but

echo 'Hello Hello' | sed 's/o\b//g'

seems better to me.

  • Thank you! Unfortunately the two lines you are suggesting still result in 'Hello Hello' on my machine. (I have copied and pasted so I cannot have made a typo.) It seems my commands are alright but my machine behaves differently for some reason or other... – Triplesmeg Nov 20 '17 at 20:45
  • You may need to use [[:>:]] in place of \> or \b for the word boundary in OSX sed - see for example sed whole word search and replace – steeldriver Nov 20 '17 at 22:30
1

Removing the o at the end of words is removing a o between a word character and a non-word character (or the EOL) so:

sed -r 's/(\w)o(\W|$)/\1\2/g'
  • I like this thinking a lot. – CoreyJJohnson Nov 27 '17 at 18:32
1

I'm wondering if somehow space isn't your word delimeter. Try something like the following:

$ echo hello hello | sed -e 's/o / /g;s/o$//'
hell hell

The problem with this example is that you'll also have to do the same for . and , and any other word delimeter. Match o followed by another specific char with [] like o[ \.,]. For some reason this doesn't work for EOL$, so add another search string with ;. Example:

$ echo hello hello, hello. toot hello | sed -e 's/o\([ \.,]\)/\1/g;s/o$//'
hell hell, hell. toot hell
$ echo $SHELL
/bin/bash
$ sed --version
sed (GNU sed) 4.4
$ set | grep IFS
IFS=$' \t\n'
  • Note that the only command that is potentially affected by $IFS in there is echo $SHELL as you forgot to quote the variable expansion. Other than that, the value of $IFS is not relevant here. – Stéphane Chazelas Nov 27 '17 at 19:44
  • The shell sends the arguments to sed via the pipeline. I believe that $IFS still applies here. – CoreyJJohnson Nov 30 '17 at 13:29
  • No, the shell passes arguments to echo and echo writes the concatenation of them with space and a newline character to the pipe. $IFS is involved in the read builtin command and unquoted parameter and arithmetic expansions and command substitutions and the expansion of "$*" and "${array[*]}" and that's about it. – Stéphane Chazelas Nov 30 '17 at 13:36
  • Interesting. I'm having trouble wrapping my head around what you're saying but it fully makes sense. Thanks. – CoreyJJohnson Nov 30 '17 at 18:19
0

I have tried this with all the anchor symbols I can think of.

It's not the anchors, but the fact that you have a greedy match with the asterisk. The \(.*\)o expression matches as long a string as it can, so it will eat everything up to the last o. It might match earlier o's too.

But then, capturing something and then returning it back is useless, you could just remove the \(.*\) and the \1 completely.

So, these would (at least in GNU sed) remove o's at the end of words:

sed 's/o\>//g' 
sed 's/o\b//g' 

This, of course only at the end of string:

sed 's/o$//g' 

And this will remove an o, along with a following non-word-character (e.g. the space after Hello):

sed 's/o\W//g' 

If your sed doesn't support \</\> or \b, you'll have to do something else. This would match o followed by a non-alphanumeric character, or the end of line:

$ echo "jello, jello" | sed -E -e 's/o([^[:alnum:]]|$)/\1/g'
jell, jell

This works e.g. in the sed that comes with OS X/macOS.


Perl regexes support adding a question mark to * or + to make them non-greedy. Then they would match the shortest possible string:

echo "jello, jello" | perl -pe 's/(.*?)o/$1/g'
jell, jell
  • Thank you! I was looking into this 'greedy' business but hadn't yet figured it out. Unfortunately on my Mac terminal I still get incorrect results with the commands you are suggesting. (As with other people's suggestions.) It seems for some reason my machine behaves differently. Both commands you mention to remove the word-final o's result in 'Hello Hello' (no deletion at all). – Triplesmeg Nov 20 '17 at 20:55
  • @Triplesmeg, the command line tools that come with Macs aren't exactly as featureful, compared to the GNU tools you'd have on a Linux system. I don't think the sed there supports \b etc. But then, I don't think they're standard anyway, so there's that. – ilkkachu Nov 21 '17 at 9:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.