Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I want to write a perl one-liner that replaces every instance of two specific consecutive strings that may or may not be separated by whitespace.

For instance, say my two strings are john paul and george and I want to replace consecutive instances of these strings (in this order) with pete. Running the one-liner on

$ cat ~/foo

john paulgeorge
john paul george
john paul

    george

george john paul

should result in

$ cat ~/foo

pete
pete
pete

george john paul

The only thing I've thought of is

$ perl -p -i -e 's/john paul\s*george/pete/g' ~/foo

but this results in

$ cat ~/foo

pete
pete
john paul

    george

george john paul

Is there a way to alter my one-liner?

share|improve this question
1  
yeah! Pete Best would be an improvement... – mikeserv Jan 15 at 19:59
    
@mikeserv haha done – Brian Fitzpatrick Jan 15 at 20:06
    
i dunno perl, but with bsd or gnu sed: sed -E 's/((george|john|paul) *){2}/pete/g' should work... – mikeserv Jan 15 at 20:16
    
you're running line-based, like awk or sed. unset $/; see perldoc.perl.org/perlvar.html – Jeff Schaller Jan 15 at 20:26
1  
@mikeserv, I think you misread the question. john paul is a single string, not two strings. And that sed command wouldn't handle matches broken across multiple lines. – Wildcard Jan 15 at 22:31
up vote 5 down vote accepted

The only thing you need to add to your one-liner is the option to slurp the file as a single string:

perl -0777 -p -i -e 's/john paul\s*george/pete/g' ~/foo
#    ^^^^^

See http://perldoc.perl.org/perlrun.html#Command-Switches

share|improve this answer
    
Accepted this one as it most easily corrects my code. Thank you! – Brian Fitzpatrick Jan 15 at 22:32

perl's -n and -p options put variants of while (<>) { ... } around your program, which makes them process input linewise. If you want to replace across multiple lines, you need to read the whole thing into a string, which you need to do yourself.

perl -e 'local $/;$_=<>;s/john paul\s*george/pete/g;print'

This undefines $/, the record separator, so that <> slurping won't do line splitting any more, reads the entire input into $_ at once, and then does the replacement on that long string. You have to do your own printing, too.

There's not much magic here any more - it's just writing a complete Perl program in a slightly uncomfortable way. -i will still work for in-place replacement, though.

If you have a large file this is going to be fairly inefficient (or exhaust your memory), but that seems more or less unavoidable without building a better parser. You can also see perldoc -q 'entire file' for other alternatives and a lot of telling you you don't really mean it.

share|improve this answer
    
There must be a way in Perl to do what I did in sed, right? I slurped multiple lines only when there is a possibility of a match. – Wildcard Jan 15 at 22:23

With sed you can do this without slurping the entire file:

sed -e ':top' -e 's/john paul[[:space:]]*george/pete/g;$b' -e '/john paul[[:space:]]*$/!b' -e 'N;btop' input

This is much lighter on memory usage; it only slurps multiple lines when there is a possibility of a multi-line match starting from the current line. And then it only slurps until either the match is found, or until there is no further possibility of a match.

As a bonus, it's POSIX-compliant. (Perl isn't part of POSIX.) Thanks to mikeserv for pointing this out in the comments.

Explanation:

:top sets a label named top.

s/john paul[[:space:]]*george/pete/g does the substitution you want for whatever is in the pattern space. (Default is line by line.)

$b skips to the end and prints if the current line is the last line of the file.

/john paul[[:space:]]*$/!b:

The pattern /john paul[[:space:]]*$/ will match john paul at the end of the pattern space followed by any amount of whitespace (but nothing other than whitespace), then ! inverts the pattern. So the effect here is to execute the b command (skip to the end of the script, thus printing the pattern space, reading the next line from the file, and starting from the top of the script) only if there is no possibility of a multi-line match starting with the current pattern space.

N appends the next line from the file to the pattern space (after appending a newline).

btop branches to the :top label without clearing out the pattern space.

share|improve this answer
1  
there's nothing specific to GNU in this that I see... i would recommend <input in the general case, though. sed -e:top -e'$!N; s/....stuff....; ttop' -e'P;D' is more simple and usually more efficient as well, though. – mikeserv Jan 15 at 22:36
    
@mikeserv, thanks, fixed. I got [[:space:]] and \s mixed up; the latter is GNU-specific, the former is not. – Wildcard Jan 15 at 22:44

Another sed:

s=[:space:]
sed -e:t -e$\!"N;s/john paul[$s]*george/pete/g;/\n/"\!tt -e"P;D" <in >out

That will handle any/all occurrences of your string in a single substitution, and only buffer as little as is absolutely necessary. It works via a sliding window on input, and only branches back to pull in newlines if the previous substitution successfully replaced your string, and, as a result, removed a newline character in the process.

The weird ! quoting is only necessary in a default (read: insane) interactive (ba|z|t?c)sh shell, but is generally not a problem in a scripted shell (unless you've got a csh variant).

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.