1

I want to find the string inside double quotes or single quotes in a text file(the text file is multiline),

for example:

I have a 
test "foo bar1" test2 "foo\"bar2", 
"foo 'bar3", 
'foo bar4', 'foo \'bar5', 'foo "bar6',

It will output

foo bar1
foo\"bar2
foo 'bar3
foo bar4
foo \'bar5
foo "bar6

the hard point is:

  1. The text file is multiline.
  2. It may have escaped double or single quotes inside quotes.
  3. The double quotes may have single quotes inside.
  4. The single quotes may have double quotes inside.
  5. The quotes must be paired match.
8
  • 4
    What have you tried so far? – annahri Jun 21 '20 at 23:54
  • , and , can be replaced by newline? And after this replacement there can be only one element per line? – Hauke Laging Jun 22 '20 at 0:03
  • the seperator between words may be not ,, the seperator can be others. – John Chen Jun 22 '20 at 0:09
  • 2
    You should make clear what can be relied on. Obviously there can be quoted " and '. Can there be quoted \ , too? I guess noone will feel like writing a parser for all possible kinds of input. – Hauke Laging Jun 22 '20 at 0:14
  • for now just consider " and '. – John Chen Jun 22 '20 at 0:32
4

We can use match-time code interpolation feature of Perl (??{ match time regex }) to tackle this. Essentially what it does is, based on what quote matched, it places the corresponding, valid regex fir that quote, such that the regex engine will natch the pair of that quote.

$ perl -lne '
    print substr($&, 1, -2+length($&))
      while
         /(?:(["'\''])(??{q<(?:[^\\\\>.$1.q<]|\\\\.)*>.$1}))/gx;
' file

Results:

foo bar1
foo\"bar2
foo 'bar3
foo bar4
foo \'bar5
foo "bar6

A smoother rewrite of the above is as follows:

$ perl -lne '
    BEGIN {
       $genRE = sub {
          my $openingQ = shift;
          # look in the Notes below for why
          qq<(?:[^\\\\${openingQ}]|\\\\.)*>
       };
    }
    print $2
      while 
        /
         (["'\''])               (?#: opening quote) 
          ((??{ $genRE->($1) })) (?#: run of in between quote pair stuff) 
         \1                      (?#: corresponding closing quote)
        /gx;
' file

Notes::

  • "........" is matched by /"[^"]*"/
  • "...... \"......" is matched by /"(?:[^\\"]|\\.)*"/
  • similar ly for the single quote as well.
3

Another perl approach:

perl -lne 'print $2 while m{(["'\''])((?:\\.|(?!\1).)*+)\1}g'

Here using a negative look ahead operator in (?!\1). to match characters other than the one matched by the first capture group. You could also simply cover the '...' and "..." cases separately with:

perl -lne 'print $1 while m{(?|"((?:\\.|[^"])*+)"|'"'((?:\\\.|[^'])*+)')}g"
0

This is difficult. I do not have a solution. I am not even sure what the best tool for this task is.

I have come close:

$ grep -oP '((?<!\\)"\K.*?(?=(?<!\\)"))|'"((?<!\\\\)'\K.*?(?=(?<!\\\\)'))" input

foo bar1
foo\"bar2
foo 'bar3
foo bar4
, 
foo \'bar5
, 
foo "bar6

The problem with multiple matches per line is that the closing quote of the earlier sting is matched as starting quote of the in-between text. And I cannot block that with a positive look-behind for an even number of quotes because the look-behind must have a fixed length. At least for grep.

Furthermore the matching of several ' within " (or the other way round) is interesting, to say the least.

Maybeawk is the better tool for this. With it you can check which quote type comes first, jump to the next and check if it is preceded by a backslash.

1
  • thank you for your answer. – John Chen Jun 23 '20 at 2:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.