7

I am searching for a nice method of merging two or more files line by line using the zipper method (also called late merge). Assuming we have three files, the result should look like this:

line1 file1
line1 file2
line1 file3
line2 file1
line2 file2
line2 file3
...

EDIT

I wrote a little python script capable of doing this:

#!/usr/bin/python

import sys, itertools

fileList = []
for file in sys.argv[1:]:
    f = open(file, "r")
    fileList.append(f.read().split("\n"))

for z in itertools.izip_longest(*fileList):
    print "\n".join([i for i in z if i is not None])

I still wonder if there is any standard tool or a clever combination of them doing the same thing.

3 Answers 3

6

I usually use paste from coreutils for this sort of thing:

paste -d'\n' file1 file2 file3
3
  • Nice, a standard tool. But unfortunately paste prints lots of empty lines if the files are of unequal length.
    – scai
    Aug 12, 2012 at 12:50
  • Depending on what you need, you could remove the blank lines by appending: | grep -v '^$'.
    – Thor
    Aug 12, 2012 at 13:38
  • That's correct, but of course only works if the specified files don't have empty lines themselves.
    – scai
    Aug 12, 2012 at 14:57
2

Another Python version that does not need to read all files into memory at once:

paddy$ more f[123].tmp
::::::::::::::
f1.tmp
::::::::::::::
line1 file1
line2 file1
line3 file1
::::::::::::::
f2.tmp
::::::::::::::
line1 file2
line2 file2
line3 file2
line4 file2
::::::::::::::
f3.tmp
::::::::::::::
line1 file3
line2 file3
line3 file3
line4 file3
line5 file3
paddy$ python2.7 -c 'import sys, itertools
files = [open(fname) for fname in sys.argv[1:]]
sys.stdout.write("".join("".join(lines) for lines in itertools.izip_longest(*files, fillvalue="") ))' f[123].tmp
line1 file1
line1 file2
line1 file3
line2 file1
line2 file2
line2 file3
line3 file1
line3 file2
line3 file3
line4 file2
line4 file3
line5 file3
paddy@paddy-ThinkPad-T61:~$ 

Replace izip_longest with zip_longest and it will work in Python 3.x too.

1
  • Thanks for your nice script, but unfortunately I can only accept one answer so I chose the one using a standard tool.
    – scai
    Sep 2, 2012 at 10:58
2

I wrote a little Perl script, that do this

#!/usr/bin/perl

do { open($fh[$_], "<$ARGV[$_]") or die("'$ARGV[$_]' does not exist") } for(0..$#ARGV);


for($i=0;;$i++) {
  $j=$#ARGV+1;

  $fh = $fh[$i%$j];
  if ( $_ = <$fh> ) {
    print $_;
  } else {
    $end |= 2**($i%$j);
  }

  if($end == (2**($j))-1) {
    last;
  }
}

close($_) for(@fh);

Save it to a file and call it like:

script.pl file1 file2 file3 ... > merge

That is at least one possibility to solve your task.

1
  • Thanks for your nice script, but unfortunately I can only accept one answer so I chose the one using a standard tool.
    – scai
    Sep 2, 2012 at 10:57

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .