7

I have three files

~/naive-file.txt
~/old-text.txt
~/new-text.txt

I want to find every instance where the contents of ~/old-text.txt occurs in ~/naive-file.txt and replace those occurrences with the contents of ~/new-text.txt. I'm sure this is doable with sed or awk but I can't seem to figure out the right command. Is this possible?

For instance, suppose the content of ~/naive-file.txt is

$ cat ~/naive-file.txt
Sed id ligula quis est convallis tempor.

This is the old text.

It might have multiple lines and some special characters like these \ { & % #)!
etc...


Nunc aliquet, augue nec adipiscing interdum, lacus tellus malesuada massa, quis
varius mi purus non odio.

Suppose the content of ~/old-text.txt is

$ cat ~/old-text.txt
This is the old text.

It might have multiple lines and some special characters like these \ { & % #)!
etc...

Suppose the content of ~/new-text.txt is

$ cat ~/new-text.txt
This is the new text.

It could also have multiple lines and special characters like these \ { & %
etc...

Running my desired command would then produce

Sed id ligula quis est convallis tempor.

This is the new text.

It could also have multiple lines and special characters like these \ { & %
etc...


Nunc aliquet, augue nec adipiscing interdum, lacus tellus malesuada massa, quis
varius mi purus non odio.
  • first string from old-text.txt to be replaced with first string from new-text.txt and so on? – iruvar May 25 at 21:47
  • @iruvar I'll add an example to clarify. – Brian Fitzpatrick May 25 at 21:49
  • @iruvar Example added. – Brian Fitzpatrick May 25 at 21:57
  • Are the number of lines the same in old and new files? – Rakesh Sharma May 26 at 9:11
  • No. I probably should have made that clear in my example. – Brian Fitzpatrick May 26 at 15:29
5

Perl to the rescue!

Read the replacements pairs into a hash. Then read the input line by line and try to replace the matches.

#!/usr/bin/perl
use warnings;
use strict;

open my $ot, '<', 'old-text.txt' or die $!;
chomp( my @lines = <$ot> );
open my $nt, '<', 'new-text.txt' or die $!;
my %replace;
@replace{@lines} = <$nt>;
chomp for values %replace;

my $regex = join '|', map quotemeta, @lines;
open my $in, 'naive-file.txt' or die $!;
while (<$in>) {
    s/($regex)/$replace{$1}/;
    print;
}

If some of the strings to be replaced are substrings of other strings to be replaced, you need to sort the strings in the regular expression by their length descending, i.e.

my $regex = join '|', map quotemeta, sort { length $b <=> length $a } @lines;
  • what is the purpose of $regex, coz it's not being used here? – Rakesh Sharma May 26 at 9:55
  • Good point, fixed. – choroba May 26 at 10:19
  • 2
    Isn’t this line-by-line while the OP wanted to replace one big bunch of bytes with another big bunch? – Roman Odaisky May 26 at 14:47
  • So how should one match the blocks to replace? Are they separated by two newline? – choroba May 26 at 19:15
4

Bash

Replace first match:

target=$(cat naive-file.txt)
old=$(cat old-text.txt)
new=$(cat new-text.txt)
echo "${target/"$old"/"$new"}"

Replace all match:

echo "${target//"$old"/"$new"}"

Replace beginning match:

echo "${target/#"$old"/"$new"}"

Replace end match:

echo "${target/%"$old"/"$new"}"
2

Here is the GNU awk one-liner:

awk 'NR==FNR{old[++k]=$0}FILENAME=="new-text.txt"{new[FNR]=$0}
     FILENAME=="naive-file.txt"{for(i=1;i<k;i++)if(old[i]==$0)$0=new[i];print}'\
     old-text.txt new-text.txt naive-file.txt

Might not be suitable for very large files as all patterns are first stored into arrays.

Output:

Sed id ligula quis est convallis tempor.

This is the new text.

It could also have multiple lines and special characters like these \ { & %
etc...


Nunc aliquet, augue nec adipiscing interdum, lacus tellus malesuada massa, quis
varius mi purus non odio.
  • for efficiency reasons you want to be breaking out of the loop after a match is found? Of course that also changes the behaviour of the program if the contents of old and new files are such that a replacement string could itself subsequently qualify for replacement – iruvar May 25 at 23:27
  • @iruvar I assumed from OP question that the patterns are unique and form whole lines. One can break the loop or even (instead of the loop) take and remove the first element from the array if patterns appear in the same order - that should be much faster for large files. – jimmij May 26 at 0:02
0
$ perl -0777ne '
     $A[@ARGV] = $_;
     @ARGV and next;
     my($naive, $new, $old) = @A;
     while ( index($naive,$old,$p) > -1 ) {
        substr($naive, index($naive,$old,$p), length($old)) = $new;
        $p = index($naive,$old,length($old)) ;
      }
      print $naive;
' old.txt new.txt naive.txt

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.