0

I'm looking for a sed (or other Unix utility) command to find all instances of two string literals being concatenated, such that the command will replace these concatenations with the corresponding combination of the two string literals.

Specifically I am trying to automate these edits for Go and Java source code.

The command should not break if \" occurs in either string literal, or if + occurs in either string literal.

The command should handle any number of whitespace characters (or lack of whitespace characters) before and after the + character.

Some examples:

  • "foo" + "bar" => "foobar"
  • "\"foo" + "bar" => "\"foobar"
  • "foo"+"bar" => "foobar"
  • "something else with " + "other words" => "something else with other words"
  • a + "bar" + "baz" + c => a + "barbaz" + c
  • "" + "bar" + "" + "foo"+ x => "barfoo"+ x

(I won't mind if the an answer uses a programming language instead of a Unix tool.)

Improve this question
4
  • Can there be something like c = '"'; s = "+"; c2 = '"' as well? IOW, what language should we implement a tokeniser for?
    – Stéphane Chazelas
    May 6 at 21:42
  • @StéphaneChazelas The languages are Go and Java, both of which require string literals are enclosed in double quotes.
    – Shane Bishop
    May 7 at 0:46
  • @thanasisp Technically speaking, for Go and Java, the concatenation can expand over multiple lines, but I don't mind not satisfying that edge case. I don't understand what you mean by "Can + exist into fields". As for adding a minimal reproducible example, I think my question and the examples I give are sufficient. If you have more questions, I am happy to answer them.
    – Shane Bishop
    May 7 at 0:50
  • But AFAIK, java does have some 'c' single quoted chars. It also has some /* comments */ where " could occur unmatched. You could also have your plussed strings on different lines. Your sample could be addressed with something as simple as sed 's/"[[:blank:]]*\+[[:blank:]]*"//g'
    – Stéphane Chazelas
    May 7 at 6:12

2 Answers 2

Reset to default
2

Could be something like like:

sed -E '
  :1
    s/^(([^"]+|"(\\.|[^"\\])*")*"(\\.|[^"\\])*)"[[:blank:]]*\+[[:blank:]]*"/\1/
  t1'

On:

"foo" + "bar"
"\"foo" + "bar"
"foo"+"bar"
"something else with "    +   "other words"
"foo\"+" + "+" + "bar"
"1"+"\"+"+"2"

It gives:

"foobar"
"\"foobar"
"foobar"
"something else with other words"
"foo\"++bar"
"1\"+2"

Now, if we had to process any java code, that would not be enough, you'd have to account for " possibly be embedded in comments or in '"' or have your additions on several lines like in:

/*
   "-quoted strings ("A" + "B") have been converted to "AB"
*/
'"' // blah " + ""
c = '"'; s = "+"; c2 = '"';
f("foo" +
  "bar")

To address those, you'd have to add matching for comments, '...' tokens and process the input in a multiline fashion. Doable, but more complicated.

Improve this answer
1

One way with POSIX sed plus making use of shell variables to make regex composition tractable.

#> one quoted chunk
qF='"
  \(
    [^\"]* \( [\]. [^\"]* \)*
  \)
"'

#> concatenate operator
plus='
  [[:blank:]]* [+] [[:blank:]]*
'

#> string add operation
add=" $qF $plus $qF "

#> de-whitespace regex
re=${add//[$IFS]/}

##> 
sed -e '
  :loop
    s/'"$re"'/"\1\3"/
  t loop
' file

Outputs:-

"foobar"
"\"foobar"
"foobar"
"something else with other words" 
a + "barbaz" + c
"barfoo"+ x
Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.