0

I would like to edit a huge single line file with thousands of occurrences of some string 'string_string': until the next appearance of a comma , (including this comma) and removing those occurrences from the file.

I assume that either sed or awk can be used to do that. They are advertised as tools to manipulate strings/character streams but also to be more applicable for multiline files.

Since both awk and sed commands can be some kind of cryptic and I would like to learn while solving daily occurrences of different problems I would like you to give a brief explanation of the resulting command itself.

My first approach was to let vim run a recorded sequence but this is running for 3h now and not even close to the end - even if it would at some point in time solve the problem I would like to know a better and more efficient way.

Requested example:

['string_string': <asdffds.1j2_3>, 'abd_dfA': 212, 'kajaj': <asdffdsa>, 'string_string': <fdjjdjd.asjsk2222>, 'jsjsjsj': 32.23], 
['string_string': <asdffds.1j2_3>, 'abd_dfA': 212, 'kajaj': <asdffdsa>, 'string_string': <fdjjdjd.asjsk2222>, 'jsjsjsj': 32.23]

Result:

[ 'abd_dfA': 212, 'kajaj': <asdffdsa>,  'jsjsjsj': 32.23], 
[ 'abd_dfA': 212, 'kajaj': <asdffdsa>,  'jsjsjsj': 32.23]
Improve this question
2
  • 1
    Well, I see. Just for completeness: Is there any chance that the string_string entry is the last element of such an [ ... ] array? In that case it would not have the "next ," as delimiting character.
    – AdminBee
    May 9 at 11:33
  • @AdminBee good question - but can't happen. Probably something the example lacks... (leaves room for improvement). Maybe I should have put a comma even there so the question doesn't arise.
    – baxbear
    May 9 at 11:36

2 Answers 2

Reset to default
3

If your sed can handle lines longer than LINE_MAX (which is as low as 1024 bytes on some systems), you could do:

sed "s/'string_string':[^,]*,//g" < your-file

If not, you can always use perl instead:

perl -pe "s/'string_string':.*?,//g" < your-file
Improve this answer
1
  • 1
    or awk: awk "{gsub(/'string_string':[^,]*,/,null)}1", just for completeness.
    – Ed Morton
    May 9 at 13:43
1

One way , using string processing to monitor the positions of string_string & comma , one after the other. In each pair:

perl -plse '
  substr($_,$p-1,$q-$p+1,"")
           while
  ++($p=index($_,$s,$q-$p)) &&
  ++($q=index($_,",",$p));
' -- -s="'string_string':" file
Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.