7

I want to use sed's transliterate (y///) to replace one set of characters by another.

I would expect this to work just as well as using the tr program.

$ echo '[]{}abc' | tr '[ab}' 'gefh' 
g]{hefc

However, when I go to perform this same operation with sed, I see the following error:

$ echo '[]{}abc' | sed 'y/[ab}/gefh/' 
sed: 1: "y/[ab}/gefh/": unbalanced brackets ([])

This makes some sense, as I expect to need to escape the [ character. However, when I do try and escape that, I receive the following, different error:

$ echo '[]{}abc' | sed 'y/\[ab}/gefh/' 
sed: 1: "y/\[ab}/gefh/": transform strings are not the same length

My current work-around is to either (1) just use tr or (2) insert a "dummy character" in the right-hand side of the transliteration whose job is to do nothing but match the escape character.

$ echo '[]{}abc' | sed 'y/\[ab}/_gefh/' 
g]{hefc

This is however unsatisfying and suspicious. It's also not very safe, e.g. when \ is in the input string.

$ echo '[]{}abc\' | sed 'y/\[ab}/_gefh/' 
g]{hefc_

What's the correct way to escape a character in a sed transliteration without sed treating the escape character itself a part of the translation?

0

2 Answers 2

6

Assuming you are on macOS (the only system whose native sed I can get to display this issue, although I haven't checked on FreeBSD whence macOS's sed comes):

$ echo '[]{}abc' | sed 'y/[ab}/gefh/'
sed: 1: "y/[ab}/gefh/": unbalanced brackets ([])
$ echo '[]{}abc' | sed 'y/\[ab}/gefh/'
sed: 1: "y/\[ab}/gefh/": transform strings are not the same length
$ echo '[]{}abc' | sed 'y/\[ab}/\gefh/'
g]{hefc

So, one solution is to

  1. Escape the [ in the first string to avoid having an unbalanced bracket, and
  2. Make the two strings equal length by adding a "no-op" backslash to the second string too.

Or,

  • You could also enclose both strings in [...], which, upon reflection, may be the safest way to deal with this as it can be done in a mechanical manner without caring about where in the string the [ is located:

    $ echo '[]{}abc' | sed 'y/[[ab}]/[gefh]/'
    g]{hefc
    
  • Or install GNU sed via e.g. Homebrew on macOS, or FreeBSD's package system, and use that.

I would treat this as a bug in this sed implementation.

1
  • 3
    FreeBSD 11.2 has the same bug. BusyBox 1.30.1 has a similar bug. At a guess, they reuse code from parsing character sets in regular expressions, and they parse character classes (things like [:alnum:]), even though they don't use the result of that parsing (correctly: y/[:alnum:]/…/ is supposed to replace the characters :almnu[], not alphanumerical characters). Nov 16, 2021 at 18:58
4

What you're doing is the correct way: [ is always supposed to be an ordinary character in sed y///. This is different from tr where [ can be part of a character class like [:alpha:].

Unfortunately, several implementations of sed evidently have a bug that causes them to try to parse balanced brackets in sed. I've observed the bug you describe with FreeBSD 11.2 and with BusyBox 1.30.1.

Using backslashes is tricky: the behavior of backslash+character is not standard except when the character is \, n or the delimiter. So while you can perhaps use it to work around the bug in your specific implementation, the resulting code probably won't work with another implementation.

A portable workaround is to temporarily swap [ with some other character that you don't want to change, and in the same replacement translate ] to itself. To avoid likely possible buggy parses, this character shouldn't be ], ^ or :. When you're doing the swap, you'll need to construct a string with balanced brackets and with something between them. FreeBSD doesn't like [] in the replacement either; an easy way to work around this is to put an extra character before the ]. For example, I'll use B as the temporary [.

y/[B_]/B[_]/; y/Bab}/gefh/; y/[B_]/B[_]/

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .