7

I am searching for a nice method of merging two or more files line by line using the zipper method (also called late merge). Assuming we have three files, the result should look like this:

line1 file1
line1 file2
line1 file3
line2 file1
line2 file2
line2 file3
...

EDIT

I wrote a little python script capable of doing this:

#!/usr/bin/python

import sys, itertools

fileList = []
for file in sys.argv[1:]:
    f = open(file, "r")
    fileList.append(f.read().split("\n"))

for z in itertools.izip_longest(*fileList):
    print "\n".join([i for i in z if i is not None])

I still wonder if there is any standard tool or a clever combination of them doing the same thing.

6

I usually use paste from coreutils for this sort of thing:

paste -d'\n' file1 file2 file3
|improve this answer|||||
  • Nice, a standard tool. But unfortunately paste prints lots of empty lines if the files are of unequal length. – scai Aug 12 '12 at 12:50
  • Depending on what you need, you could remove the blank lines by appending: | grep -v '^$'. – Thor Aug 12 '12 at 13:38
  • That's correct, but of course only works if the specified files don't have empty lines themselves. – scai Aug 12 '12 at 14:57
2

i wrote a little perl script, that do this

#!/usr/bin/perl

do { open($fh[$_], "<$ARGV[$_]") or die("'$ARGV[$_]' does not exist") } for(0..$#ARGV);


for($i=0;;$i++) {
    $j=$#ARGV+1;

    $fh = $fh[$i%$j];
    if ( $_ = <$fh> ) {
        print $_;
    } else {
        $end |= 2**($i%$j);
    }

    if($end == (2**($j))-1) {
        last;
    }
}

close($_) for(@fh);

save it to a file an call it like

script.pl file1 file2 file3 ... > merge

That is at least one possibility to solve your task.

|improve this answer|||||
  • Thanks for your nice script, but unfortunately I can only accept one answer so I chose the one using a standard tool. – scai Sep 2 '12 at 10:57
2

Another Python version that does not need to read all files into memory at once:

paddy$ more f[123].tmp
::::::::::::::
f1.tmp
::::::::::::::
line1 file1
line2 file1
line3 file1
::::::::::::::
f2.tmp
::::::::::::::
line1 file2
line2 file2
line3 file2
line4 file2
::::::::::::::
f3.tmp
::::::::::::::
line1 file3
line2 file3
line3 file3
line4 file3
line5 file3
paddy$ python2.7 -c 'import sys, itertools
files = [open(fname) for fname in sys.argv[1:]]
sys.stdout.write("".join("".join(lines) for lines in itertools.izip_longest(*files, fillvalue="") ))' f[123].tmp
line1 file1
line1 file2
line1 file3
line2 file1
line2 file2
line2 file3
line3 file1
line3 file2
line3 file3
line4 file2
line4 file3
line5 file3
paddy@paddy-ThinkPad-T61:~$ 

Replace izip_longest with zip_longest and it will work in Python 3.x too.

|improve this answer|||||
  • Thanks for your nice script, but unfortunately I can only accept one answer so I chose the one using a standard tool. – scai Sep 2 '12 at 10:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.