I have a csv file with data similar to this:

"1b579a5e-9701-40eb-bd36-2bc65169da99","week14_Friday-019","6907eaad-1aff-4d26-9088-ba20374b67c0","2181-019","f20af5bb-c716-42e0-9b9d-cbolf5bfecea","15-BIO-2001","COLLEGE Bio 1","d39330be-df56-4365-8fb4-37e68d040c52","Which engine has the smaller efficiency?","{choices:[a","b","c","d],"type:MultipleChoice}","{solution:[0],"selectAll:false,{"selectMultiple:false",}"type:MultipleChoice}","2016-04-25 00:30:19.000","1922ac5a-6ff6-4ea4-9078-6df4d85d294f","{solution:[0],"type:MultipleChoice}","1","1116911f-8ee5-45c3-b173-a6be681bb15a","FakeLastName","FakeFirstName","FakeName@mail.edu","Student"

I want to remove the double-quotes ", but only if they are inside of the curly braces {}, preferably with sed or awk. Desired output is:

"1b579a5e-9701-40eb-bd36-2bc65169da99","week14_Friday-019","6907eaad-1aff-4d26-9088-ba20374b67c0","2181-019","f20af5bb-c716-42e0-9b9d-cbolf5bfecea","15-BIO-2001","COLLEGE Bio 1","d39330be-df56-4365-8fb4-37e68d040c52","Which engine has the smaller efficiency?","{choices:[a,b,c,d],type:MultipleChoice}","{solution:[0],selectAll:false,{selectMultiple:false,}type:MultipleChoice}","2016-04-25 00:30:19.000","1922ac5a-6ff6-4ea4-9078-6df4d85d294f","{solution:[0],type:MultipleChoice}","1","1116911f-8ee5-45c3-b173-a6be681bb15a","FakeLastName","FakeFirstName","FakeName@mail.edu","Student"

Any help would be greatly appreciated. Thanks!

  • @Thephrastus - regexp is almost always applied by a proper stateful language such as sed, ed, ex, awk, perl, and others.
    – mikeserv
    Aug 7 '18 at 19:37

It would be easier with sed:

sed -e :1 -e 's/\({[^}]*\)"\([^}]*}\)/\1\2/g; t1'

Or perl:

perl -pe 's{\{.*?\}}{$& =~ s/"//gr}ge'

Note that it assumes there's no nested {...}.

To handle nested {...}, you can use perl's recursive regexp capabilities:

perl -pe 's(\{(?:[^{}]++|(?0))*\})($& =~ s/"//gr)ge'

With sed, working our way outwards to escape the inner {...}s before removing the "s:

sed 's/_/_u/g
     s/\({[^{}]*\){\([^{}]*\)}/\1_<\2_>/g; t1
     s/\({[^}]*\)"\([^}]*}\)/\1\2/g; t2
     s/_</{/g; s/_>/}/g;s/_u/_/g'
  • 1
    Stéphane Chazelas, This worked great, except I do actually have nested {...} I updated the description input/desired output. Please update when you have a chance? Thanks! Aug 7 '18 at 16:36
  • 1
    @BrockWinfrey, see edit. Aug 7 '18 at 19:04

Try this using sed:

$ sed -r ' :L; s/(\{[^"}]*)"(([^"}]*")*)([^"}]*\})/\1\2\4/g; tL; ' file

awk for nested braces:

    awk -F{ '
            {for (i=2; i<=NF; i++)  {if (1 == n = split ($i, T, "}")) n++ 
                                     for (j=1; j<n; j++)    {RS = index ($i, T[j])
                                                             L = gsub (/"/, _, T[j])
                                                             $i = substr ($i, 1, RS-1)  T[j]   substr ($i, RS + length (T[j]) + L)
    1' OFS="{" file

Clumsy, but yields the output specified, at least for singly nested braces.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.