Left and right square brackets treated differently by sed/bash

Question

I have a file with the following contents:

[root@server list]# cat filenames.txt
[AAA','ACMEDEMO2','ACMEDEMO3','ACMEDEMO4','RENTCOGH','TESTENT','DORASINE','LOKAWINK','BBB]
[root@qa4app01 list]#

I want this as a list to use in my python script, so I am trying to change AAA' to 'AAA'and 'BBB to 'BBB' and I thought I would use sed and replace [ with [' and ] with '] .

To confirm, I tried this:

[root@server list]# cat filenames.txt | sed "s/]/']/g"
[AAA','ACMEDEMO2','ACMEDEMO3','ACMEDEMO4','MENSCOGH','TESTENT','DORASINE','LOKAWINK','BBB']
[root@server list]#

It worked and I was able to replace the ] with a ']. So, for AAA I just need to change replace right square brackets and the single quote in the sed with a left square bracket and tried this:

[root@server list]# cat filenames.txt | sed -e "s/]/']/g" -e "s/[/['/g"
sed: -e expression #2, char 8: unterminated `s' command
[root@server list]#

Okay. I thought for some reason appending sed commands are not working properly and to check this I tried the sed on the left square bracket separately:

[root@server list]# cat filenames.txt | sed  "s/[/['/g"
sed: -e expression #1, char 8: unterminated `s' command
[root@server list]#

Strange. It looks as if sed is treating left and square brackets differently. I was able to get away without escaping the [, while sed refuses to work with ] without the an escape character. I eventually got what I want with adding escape characters to the left square bracket like below:

[root@server list]# cat filenames.txt | sed -e "s/]/']/g" -e "s/\[/\['/g" ['AAA','ACMEDEMO2','ACMEDEMO3','ACMEDEMO4','MENSCOGH','TESTENT','DORASINE','LOKAWINK','BBB'] [root@server list]#

I am in the process of writing a python + shell script which will be handling files with numerous instances of both the square brackets. My question is whether the right and left square brackets are treated differently by sed or bash.

sed version is GNU sed version 4.2.1.

P.S.: I have decided to use escape characters with both the brackets to play safe, but want to be absolutely sure it won't break the script if I don't use the escape character (for readability reasons) at places where I will be working only with right square bracket ]

Answer 1

[ introduces a character class and ] closes it.

If sed sees only ] (i.e. the closing command), it will be OK, sed assumes this is not special command.

Using only [ (without a closing ]) confuses sed, so it must be escaped.

On a side note, there is no need for

• g (global) flag (only one substitution per line)
• escape [ in replacement.

I managed your change with

sed -e "s/]/']/" -e "s/\[/['/" 

Answer 2

Archemar explained why this failed on [, here's another way of making this particular replacement:

$ sed "s/^./['/;s/.$/']/" file
['AAA','ACMEDEMO2','ACMEDEMO3','ACMEDEMO4','RENTCOGH','TESTENT','DORASINE','LOKAWINK','BBB']

Or, using & which means "whatever you just matched":

$ sed "s/^./&'/;s/.$/'&/" file
['AAA','ACMEDEMO2','ACMEDEMO3','ACMEDEMO4','RENTCOGH','TESTENT','DORASINE','LOKAWINK','BBB']