3

I am trying to add missing quotes at the ends of some lines in a text file.

I find that the regex [^\"]$ suffices to find lines with missing terminal doublequotes and so tried the following replacement using a backreference (which tbh I've never used before). Using parens around the 'capture group' I hoped that sed would allow backreference to that group, but

sed  's|([^\"]$)|\1\"|g' bigfile.tsv

hits

sed: -e expression #1, char 17: invalid reference \1 on `s' command's RHS

and same if I don't escape the replacement quotes

sed  's|([^\"]$)|\1"|g' bigfile.tsv

(tho now its char 16 that's offensive) . How does the backreference go? https://xkcd.com/1171/

2 Answers 2

8

When you run sed without -E, then the expression is a basic regular expression and the capture groups must be written as \(...\). When you use -E to enable extended regular expressions, capture groups are written (...).

The \ inside [...] is literal, so your expression would also avoid adding a double quote on lines ending with \. Some of the other escaping is also unnecessary.

Therefore, you may write your sed command as

sed 's/\([^"]\)$/\1"/'

or as

sed -E 's/([^"])$/\1"/'

Or, using &:

sed 's/[^"]$/&"/'

The & in the replacement part of the expression will be substituted by the part of the input that matched the regular expression.

A couple of other alternatives that does not use a capture group:

sed '/[^"]$/ s/$/"/'

This applies s/$/"/ to all lines that matches /[^"]$/.

Or, alternatively,

sed '/"$/ !s/$/"/'

This applies s/$/"/ to all lines that don't match /"$/ (there's a slight difference from the other approaches here in that it also adds a " to empty lines).

Note that in all cases, the g flag at the end is definitely not needed.

1
  • Thanks for the detailed answer, this cleared up some mysteries for me Nov 13, 2018 at 13:13
2

Try sed -e 's|\([^\"]$\)|\1\"|g' bigfile.tsv.

3
  • This still does not add the missing " to lines ending with backslash and it needlessly uses g at the end (an anchored expression can only match once).
    – Kusalananda
    Nov 12, 2018 at 15:04
  • I just provided a hint about the error message; I did not intend to solve the whole problem for the author. Mostly because I think the author will learn more when doing it himself or herself.
    – U. Windl
    Nov 12, 2018 at 15:09
  • @U.Windl They may, or they may not ever notice.
    – Kusalananda
    Nov 12, 2018 at 15:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.