4

I can't find the answer to this anywhere. It sounds simple but I'm starting to think maybe it's not.

I want sed to remove all the CATs before the STOP in this string:

two CAT two four CAT CAT seven one STOP four CAT two CAT three

So the output I'm hoping for will be:

two two four seven one STOP four CAT CAT two CAT three

There could be any number of CATs anywhere in the string. The stop marker can be anywhere too, but just one of them, and always spelled STOP.

(Edit: as pointed out below my question is ambiguous - must CAT have adjacent spaces or can any chars border it? Maybe only non-alphanumeric chars are ok? Presenting my actual use case was intense (a big bash function) so I simplified, too much. Readers please bear in mind that solutions below may make different assumptions about adjacency. Thanks)

4
  • 3
    I imagine an awk-based answer would be more straight-forward; would that be acceptable?
    – Jeff Schaller
    Jun 3 '20 at 19:17
  • Everything here is great. I've gone with the one that entirely uses a single sed pipeline (see its comment to know why.) If any readers want that too, scroll down cos it's not very upvoted atm.
    – user224016
    Jun 4 '20 at 16:18
  • (1) Please specify what should be done with CAT, (i.e., CAT followed by punctuation), CATCH, BOBCAT and LOCATE. (2) steeldriver's answer IS a single sed command — see the comments.  Even Dale Hagglund agrees. Jun 5 '20 at 0:42
  • Thank you G-Man for also pointing that out, I really messed up quite a lot there so I'll edit what I can. Re adjacent characters to CAT, my simplified question is ambiguous. I'm not sure which solutions here have assumed spaces or have not. I'll edit the question to say this. In the actual use case (complex bash function about tags in filenames) CAT is illegal if bordered by any alphanumeric characters and will have been filtered out earlier on.
    – user224016
    Jun 5 '20 at 15:49
10

You could replace one at a time in a loop, until there are no more CATs before the STOP:

$ echo 'two CAT two four CAT CAT seven one STOP four CAT two CAT three' |
    sed -e :a -e '/CAT.*STOP/s/CAT //;ta'
two two four seven one STOP four CAT two CAT three
7
  • 3
    I really wonder why this has been downvoted.
    – Quasímodo
    Jun 3 '20 at 19:58
  • Because better using the right tool for the right situation. Cutting of 'labels' is a task for awk, not for sed Jun 3 '20 at 20:01
  • 2
    I see no problem with using sed for this. A while loop is a while loop, whether or not it actually includes the keyword "while".
    – chepner
    Jun 4 '20 at 11:41
  • @user224016 there's no "drop out into bash" here - unless you mean the echo command? that's just to provide some input, for testing purposes Jun 4 '20 at 17:37
  • @user224016 I'll agree that you may be misunderstanding here. this is a "pure sed" answer and the echo is just how you are getting your input string into sed. Additionally I suspect any difference in speed would be negligible. Jun 4 '20 at 17:54
5

With any awk:

awk '{while($0~/CAT .*STOP/)sub(/CAT /,"")}1' file
$ echo 'two CAT two four CAT CAT seven one STOP four CAT two CAT three' |
  awk '{while($0~/CAT .*STOP/)sub(/CAT /,"")}1'
two two four seven one STOP four CAT two CAT three
5

With perl

perl -pe 's/CAT (?=.*STOP)//g'

this will replace CAT only if there is STOP is present later in the line

4

A gawk solution using STOP as a field separator and gsub out the CATs

echo 'two CAT two four CAT CAT seven one STOP four CAT two CAT three' |
  awk -F"STOP" 'BEGIN{OFS="STOP"}{gsub("CAT ","",$1)}1'

or

  awk -F"STOP" '{gsub("CAT ","",$1); print $1 FS $2}'
2

(If this is more important that just a sed puzzle, I highly recommend not doing it with sed at all. You can write this easily in python and very clearly, unlike the obscure answer here.)

You can use a loop in sed as shown below. Explanations and caveats below the code.

s/STOP/@/
: loop
s/^\([^@]*\)CAT\(.*\)$/\1\2/
t loop
s/@/STOP/
p

Run this as

$ sed -n -f t.sed

and it will fix each line as you described. Note that if no STOP is present, the code as written will remove all occurrences of CAT from the input line. Also, this code assumes that @ cannot occur in your input lines. If it does, you'll need to find another marker character.


So, what's going on here? Let's walk through the code:

s/STOP/@/

It is difficult in sed to write a regexp that matches the absence of something, except for the case of a single character, eg x, in which case [^x]* does the job. So, replace our sentinel STOP with a single character that we know to be unused in the rest of the line. If there is no such character, life gets a lot harder, and then we remember that sed is really not the right tool for this job.

: loop
s/^\([^@]*\)CAT\(.*\)$/\1\2/
t loop

This is the key part. : loop declares a label in your sed script that you can branch back to later. Next, in the s/// line, try to find a CAT not preceded an @ marker and replace it, preserving the text before and after. If the substitution happened, t loop will branch back to loop to try again. If the substitution failed, ie, if no such CAT is found, the branch back to loop is not taken.

s/@/STOP/
p

Restore the actual STOP text and print out the final line.

5
  • I have no trouble with doing this sort of thing to explore sed to see how far it can be pushed. Indeed, I usually get a kick out of solving problems like this. But, I'll warn again that I would never use sed for this sort of problem in any real world context. If I have access to sed, I certainly have access to awk, and possibly perl or python, all of which are better choices. Jun 4 '20 at 16:12
  • as written, this answer leaves extra spaces in the output. Jun 4 '20 at 17:59
  • Yeah, I noticed that just after I posted, but it was already kind of late. Not sure what I'd do, but I might start with a simple fix-up after the loop to squash multiple spaces. That keeps the main s/// as simple as possible, but might not be what's desired. And like I said, as the rules get more complicated, I'd move away from sed almost immediately. Jun 4 '20 at 18:07
  • I just noticed @steeldriver's post, which is more direct than mine. I still wouldn't use sed for this problem but his answer is the better one and should probably be accepted instead of mine. Jun 4 '20 at 18:10
  • Thank you Dale Hagglund for pointing this out, I really messed up a bit here. I've edited things above to try and address this.
    – user224016
    Jun 5 '20 at 15:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.