This is my file1:

#{ VOL       @home    }
#database    daba
#relation    tcdeatid
#copy           1
#{ version        0 }
#.lruno := 72
#.infno := 1
#.tid.noel := 101
#.tid.info := 64
#.tid.setnr := 1225              <--- (1225 number will be changed)
#.typeidm := 1
#.sourcetable := 2
#.lruno := 72
#.infno := 205
#.tid.noel := 101
#.tid.info := 76
#.tid.setnr := 5625              <--- (5625 number will be changed)
#.typeidm := 1
#.sourcetable := 2

In file2, I have the correct entries:

#.tid.info := 3345              <--- (I want to put this number in file1 in place of 64)
#.tid.setnr := 1254              <--- (I want to put this number in file1 in place of 1225)
#.tid.info := 5567              <--- (I want to put this number in file1 in place of 76)
#.tid.setnr := 9056              <--- (I want to put this number in file1 in place of 5625)

I want to make changes in file1, directed by data from file2.

I have tried sed "s/tid.setnr := 1225/tid.setnr := 1254/g" file1 > modified_file1

But instead of doing it manually, I want my script to read the new tid.info and tid.setnr values from file2 and modify the respective lines in file1.  The first line in file2 should replace the entire first line in file1 that contains tid.info, the second line in file2 should replace the second line in file1 that contains tid.setnr, and so on.

  • 3
    Welcome to Stack Exchange.  Your question sounds a little like this to me: "I have an apple and an orange, and I want magic to happen, resulting in a banana, or something like that."  Please explain a bit more about the operation you are trying to perform.  For example, will file2 always be a list of tid.setnr values and nothing else, or can file2 contain any of the tags that are in file1?  Can file2 contain tags that aren't in file1?  (If so, how do you know where to put them?)  What if file1 has three tid.setnr values and file2 has only two?  … … … … … … … …  Also, what have you tried? – G-Man Says 'Reinstate Monica' Jun 6 '15 at 3:51
  • 1
    P.S. Please edit your question to add details and make it clearer; do not respond to my comment with more comments. – G-Man Says 'Reinstate Monica' Jun 6 '15 at 3:55

One way to do this with sed is to generate a sed script:

exec 3< file2
grep -n "tid\.setnr" file1 | while IFS=: read n junk
        IFS= read -r line <&3  ||  break
        printf "%sc%s\n%s\n" "$n" '\' "$line"
done > "$tmpfile"
exec 3<&-
sed -f "$tmpfile" file1 > modified_file1
rm -f "$tmpfile"
  • exec 3< file2 opens file2 for reading on file descriptor 3.
  • The grep -n finds the lines in file1 that contain tid.setnr, by line number.  This gets piped into a while loop.
  • while IFS=: read n junk

    • while … read … means read a line at a time, repeatedly, as long as there is information to read (i.e., stopping when it reaches the end of the data).
    • IFS=: read n junk means read everything up to the first : (which will be the line number from grep -n) into n, and the rest of the line (the old tid.setnr data line) into junk, which we ignore, because we don’t care about it.
  • IFS= read -r line <&3 reads a line from file descriptor 3 (file2), doing everything that we know how to do to tell the shell not to mangle it, and put it into a variable called line.
  • … || break says, if the above read fails (presumably due to end of file), break out of the loop.
  • The printf writes a sed change command, addressed to line number n, saying that that line should be replaced with the content on the line variable.
  • done > "$tmpfile" marks the end of the while loop and specifies that the standard output of the entire loop is $tmpfile.  This will end up looking like this:

    #.tid.setnr := 1254
    #.tid.setnr := 9056
    • Note that the while loop terminates as soon as it gets an EOF from either input.  Therefore, if file1 has more tid.setnr lines than file2, the extra ones will go unchanged (i.e., c commands for them will not be generated).  Similarly, if file2 has more tid.setnr lines than file1, the extra ones will be ignored.  Unfortunately, this discrepancy will not be reported, although adding that capability would not be very hard.
  • exec 3<&- closes file descriptor 3.

  • sed -f "$tmpfile" file1 > modified_file1 runs sed on file1, reading commands from $tmpfile and writing output to modified_file1.

This should do what you want.  Obviously, change the filenames if you want.  You should run this “as is” once, and then look at the modified_file1 file (or simply change the sed command not to redirect its output, so it writes to the screen) and verify that the output is what you want.  Then you can change the sed command to sed -i to edit file1 in place (if that’s what you want to do).

If the sed command gives an error like “Too many line numbers”, try splitting the script file ($tmpfile) into smaller files.  I suggest starting with a size under 100 commands; e.g., 90 commands, which is 180 lines, since each command is two lines.  You can do this manually, with a text editor1, but there’s a tool written specifically for this job.  It is, intuitively, called split.  The command

split --lines=180 "$tmpfile"

will split the script into files in the current directory called xaa, xab, xac, ….  The first n−1 will be 180 lines long; the last one will be whatever it needs to be (≤ 180) to make up the total.  For example, if you have 500 instances of tid.setnr, then your script will be 1000 lines long, and you will get six x files — xaa, xab, xac, xad, and xae will each have 180 lines, and xaf will have 100.  Now do

sed -f xaa file1 > modified_file1aa

If you still get “Too many line numbers”, go back and try it again with a smaller number of --lines.  If it doesn’t report an error, look at modified_file1aa and see whether it looks like the first 90 tid.setnr lines have been changed.  If it looks OK, then do

sed -f xab modified_file1aa > modified_file1ab
sed -f xac modified_file1ab > modified_file1ac
sed -f xad modified_file1ac > modified_file1ad
sed -f xae modified_file1ad > modified_file1ae
sed -f xaf modified_file1ae > modified_file1af

modified_file1af is now your final modified_file1.

If you want to play experiment, you can

  • Try larger numbers of --lines until you find out what the maximum is.
  • Try

    sed -f xaa -f xab -f xac -f xad -f xae -f xaf file1 > modified_file1_test

    but that probably won’t work.

  • If you do the above experiments, I encourage you to tell us the results.

If you only need to do this one time, you’re done.  But, if you need to do this repeatedly, change the last two lines of the code block at the top of this answer to

split --lines=180 "$tmpfile"            <--- (Using a number that works for you, of course)
cp file1 modified_file1
for f in x*
        sed -i -f "$f" modified_file1
rm -f "$tmpfile" x*

As I mentioned earlier, the -i option tells sed to edit the specified file in place (i.e., write the changes back out to the input file).  Or, even simpler,

split --lines=180 "$tmpfile"
for f in x*
        sed -i -f "$f" file1
rm -f "$tmpfile" x*

if you don’t need to keep your original file1 intact.

Note that the usage synopsis for split is


and its default behavior is to create files called PREFIXaa, PREFIXab, PREFIXac, etc.  In other words, the default PREFIX is x.  If you might have other files whose names begin with x, you should define prefix to be something that will be unique (e.g., prefix=Nainita.$$. or prefix=/tmp/Nainita.$$.) and then change the above to

split --lines=180 "$tmpfile" "$prefix"
for f in "$prefix"*
        sed -i -f "$f" file1
rm -f "$tmpfile" "$prefix"*

1 If you’re a glutton for punishment, you could do it with sed — but why play with fire?

  • Many thanks for your reply... I will be in touch after implementing your solution... But I have a question.... I have understand what you said... but I have to load e\with the corrected value of file 1 into the database without changing its file format... If I try your solution, then is the file format remain unchanged? – pmaipmui Jun 6 '15 at 6:00
  • I don't understand your concern.  The output (modified_file1) will be exactly the same as file1 in lines 1-12, 14-21, and 23-28 (file1 is 28 lines long).  Line 13 of modified_file1 will be line 1 from file2.  Line 22 of modified_file1 will be line 2 from file2. – Scott Jun 6 '15 at 6:28
  • Yes... thanks again... it is cleared now... – pmaipmui Jun 6 '15 at 8:32
  • 1
    I think this might be unnecessarily complicated. – mikeserv Jun 6 '15 at 12:17
  • My comment wasn't meant to remark on length or breadth but complexity. The problem with this answer - and the reason I wrote mine - is that you're defying the basic principles of sed's design just to write its script. That's what makes it complicated. It's a stream editor - it's easier to use with streams. Anyway, doing it this way doesn't fit sed well - which is why you have to shoehorn it in - but ed or ex could do the whole thing in similar fashion with a few keystrokes. Because I trade i/o en-route, my answer is both clear and brief. Learning how took a long while, though. – mikeserv Jun 6 '15 at 16:02

With GNU sed's R command:

sed -e 's/^#.tid.setnr :=.*//;tA;b;:A;R file2' -e 'd' file1
  • You've made it unnecessarily complicated. sed '/^#\.tid\.setnr/{R file2 (Enter) d}' file1 seems to be good enough. – Scott Jun 6 '15 at 7:42
  • 2
    Thank you. Or with GNU sed: sed -e '/^#.tid.setnr/{R file2' -e 'd}' file1 – Cyrus Jun 6 '15 at 7:45
  • Wow!  I had no idea that { and } could be broken apart into separate scripts (separate -e arguments) like that.  It's not clear from the documentation.  Is that also a GNU extension? – G-Man Says 'Reinstate Monica' Jun 6 '15 at 22:53
  • @G-Man: A few weeks ago I came across it by a typo. sed (/usr/bin/sed, /usr/xpg4/bin/sed and GNU's /usr/bin/gsed) from Solaris 11 can separate scripts, too. – Cyrus Jun 7 '15 at 5:25
  • 1
    @G-Man - It's specified behavior: Multiple -e and -f options may be specified. All commands shall be added to the script in the order specified, regardless of their origin... and The application shall not present a script that violates the restrictions of a text file except that the final character need not be a \newline. In most cases you use the null read at the tail of an -e arg to represent a newline - like for delimiting write files or [aic]. It usually won't work in an s/// though. – mikeserv Jun 7 '15 at 7:16
{   sed '/^#\.tid\.setnr/!d;=;g;G' |
    paste  -d'c\\\n' - - - ./addfile
}   <./mainfile | sed -f - ./mainfile

It just prepends the relevant line numbers from ./mainfile to each line in ./addfile as separated by the command string c\ and a <newline> before passing the results onto another sed as a script for editing ./mainfile. Given your example data, the script generated looks like:

#.tid.setnr := 1254              <--- (I want to put this number in file1 in place of 1225)
#.tid.setnr := 9056              <--- (I want to put this number in file1 in place of 5625)

...which instructs sed to change lines 13 and 22 of its output to match the appended lines for each command.

  • same error is coming. "too many line numbers" @mikeserv – pmaipmui Jun 16 '15 at 11:44

I am found of perl. This script is a crude one that seems to solve the problem:

use strict;

# subroutine to load a test file into an array
sub load{
   my $file = shift;
   open my $in, "<", $file or die "unable to open $file : $!";
   my @data = <$in>;
   chomp @data;
   foreach (@data) { s/\cM//g;}
   return @data;

my $file1 = shift || die "usage: $0 [file1] [file2]\n";
my @file1_data = &load($file1);
my $file2 = shift || die "usage: $0 [file1] [file2]\n";
my @file2_data = &load($file2);

my $i = 0;
foreach( @file1_data )
    if( $i < @file2_data ) 
        my @s = split / /, $file2_data[$i];
        if( @s )
            if( /^$s[0]/ )
            $_ = $file2_data[$i++];
    print "$_\n";

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.