0

I'm looking for a sed (or other Unix utility) command to find all instances of two string literals being concatenated, such that the command will replace these concatenations with the corresponding combination of the two string literals.

Specifically I am trying to automate these edits for Go and Java source code.

The command should not break if \" occurs in either string literal, or if + occurs in either string literal.

The command should handle any number of whitespace characters (or lack of whitespace characters) before and after the + character.

Some examples:

  • "foo" + "bar" => "foobar"
  • "\"foo" + "bar" => "\"foobar"
  • "foo"+"bar" => "foobar"
  • "something else with " + "other words" => "something else with other words"
  • a + "bar" + "baz" + c => a + "barbaz" + c
  • "" + "bar" + "" + "foo"+ x => "barfoo"+ x

(I won't mind if the an answer uses a programming language instead of a Unix tool.)

4
  • Can there be something like c = '"'; s = "+"; c2 = '"' as well? IOW, what language should we implement a tokeniser for? May 6 at 21:42
  • @StéphaneChazelas The languages are Go and Java, both of which require string literals are enclosed in double quotes. May 7 at 0:46
  • @thanasisp Technically speaking, for Go and Java, the concatenation can expand over multiple lines, but I don't mind not satisfying that edge case. I don't understand what you mean by "Can + exist into fields". As for adding a minimal reproducible example, I think my question and the examples I give are sufficient. If you have more questions, I am happy to answer them. May 7 at 0:50
  • But AFAIK, java does have some 'c' single quoted chars. It also has some /* comments */ where " could occur unmatched. You could also have your plussed strings on different lines. Your sample could be addressed with something as simple as sed 's/"[[:blank:]]*\+[[:blank:]]*"//g' May 7 at 6:12

2 Answers 2

2

Could be something like like:

sed -E '
  :1
    s/^(([^"]+|"(\\.|[^"\\])*")*"(\\.|[^"\\])*)"[[:blank:]]*\+[[:blank:]]*"/\1/
  t1'

On:

"foo" + "bar"
"\"foo" + "bar"
"foo"+"bar"
"something else with "    +   "other words"
"foo\"+" + "+" + "bar"
"1"+"\"+"+"2"

It gives:

"foobar"
"\"foobar"
"foobar"
"something else with other words"
"foo\"++bar"
"1\"+2"

Now, if we had to process any java code, that would not be enough, you'd have to account for " possibly be embedded in comments or in '"' or have your additions on several lines like in:

/*
   "-quoted strings ("A" + "B") have been converted to "AB"
*/
'"' // blah " + ""
c = '"'; s = "+"; c2 = '"';
f("foo" +
  "bar")

To address those, you'd have to add matching for comments, '...' tokens and process the input in a multiline fashion. Doable, but more complicated.

1

One way with POSIX sed plus making use of shell variables to make regex composition tractable.

#> one quoted chunk
qF='"
  \(
    [^\"]* \( [\]. [^\"]* \)*
  \)
"'

#> concatenate operator
plus='
  [[:blank:]]* [+] [[:blank:]]*
'

#> string add operation
add=" $qF $plus $qF "

#> de-whitespace regex
re=${add//[$IFS]/}

##> 
sed -e '
  :loop
    s/'"$re"'/"\1\3"/
  t loop
' file

Outputs:-

"foobar"
"\"foobar"
"foobar"
"something else with other words" 
a + "barbaz" + c
"barfoo"+ x

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.