Merge files line by line

Question

I have 10 files and I want to copy the first line of every file, the the second of every file and so into one big file.

file1.txt is

A 
B 
C 
D 

file2.txt is

E 
F
G
H

file3.txt is

I 
J
K
L

I want to have

A 
E
I
B
F
J
C
G
K
....

Thanks

Answer 1

You can use paste for that, just set the delimiter to a newline:

$ paste -d'\n' file1 file2 file3
A 
E 
I 
B 
F
J
C 
G
K
D 
H
L

Answer 2

Here's an alternate to paste (tested with GNU pr and a shell that supports ANSI-C quoting)

pr -mts$'\n' <files>

One advantage is that -s option will work for multicharacter separator as well.

Since sed is tagged, you can use R command (available on GNU sed, not sure about other implementations).

sed -e 'R f2' -e 'R f3' f1

Answer 3

#!/usr/bin/perl

use strict;

# how many files to open? 10 in the question, 3 in examples.
my $numfh=3;

# a counter for the number of open file handles
my $openfh=$numfh;

# open an array of filehandles, one for each input file.
my @fh = ();
for my $i (1..$numfh) {
  open($fh[$i],"<","file$i.txt") || die "Couldn't open file$i.txt for read: $!";
};

# open the output file.
open (my $out,">","bigfile.txt") || die "Couldn't open bigfile.txt for write: $!";

# repeat until there are no more open file handles.
until ($openfh < 1) {
  for my $i (1..$numfh) {
    if (eof($fh[$i])) {
      # if an input file is eof, close it and decrement openfh counter.
      $openfh--;
      close($fh[$i]);
    } else {
      # print a line of input from the current input file to the output file.
      print $out scalar readline $fh[$i]
    };
  };
}

Save this as, e.g., merge.pl and make it executable with chmod +x merge.pl. Then run it like so:

$ ./merge.pl

Output:

$ cat bigfile.txt 
A
E
I
B
F
J
C
G
K
D
H
L

and here's a version using @ARGV and printing to STDOUT for @terdon:

#!/usr/bin/perl

use strict;

my @fh = ();
my $i=1;

for my $f (@ARGV) {
  open($fh[$i++], "<", $f) || die "Couldn't open $f for read: $!";
};

my $numfh=$#fh; my $openfh=$numfh;

until ($openfh < 1) {
  for my $i (1..$numfh) {
    if (eof($fh[$i])) {
      $openfh--;
      close($fh[$i]);
    } else {
      print scalar readline $fh[$i]
    };
  };
}

Or using a hash to hold the file handles instead of an array:

#!/usr/bin/perl

use strict;

my %fh = ();

for (@ARGV) {
  open($fh{$_}, "<", $_) || die "Couldn't open $_ for read: $!";
};

while (keys %fh) {
  for my $f (@ARGV) {
    next unless (defined($fh{$f}));
    if (eof($fh{$f})) {
      close($fh{$f});
      delete($fh{$f});
    } else {
      print scalar readline $fh{$f}
    };
  };
}

Run either as:

$ ./merge.pl file[123].txt > bigfile.txt

The output is identical to the hard-coded version.

Answer 4

Using GNU sed we can use the R read command to pull in the top lines progressively from the input files to get a merged output.

sed -e "
  $(printf 'R%s\n' file{2..9}.txt file10.txt)
" file1.txt

---

perl -lne '
  push @{$A[@ARGV]}, $_}{
  for my $i (0..$#{$A[0]}){
    print for map { $A[$_][$i] } reverse 0..$#A
  }
' file{1..9}.txt file10.txt

Here in this we Store the files in an AoA aka 2D array @A. The first index refers to the file position in the positional arguments list and the second index is for data of that particular file.In the end block }{ we are zipping the array by picking one one element from each array.