1

I have 10 files and I want to copy the first line of every file, the the second of every file and so into one big file.

file1.txt is

A 
B 
C 
D 

file2.txt is

E 
F
G
H

file3.txt is

I 
J
K
L

I want to have

A 
E
I
B
F
J
C
G
K
....

Thanks

4 Answers 4

11

You can use paste for that, just set the delimiter to a newline:

$ paste -d'\n' file1 file2 file3
A 
E 
I 
B 
F
J
C 
G
K
D 
H
L
0
3

Here's an alternate to paste (tested with GNU pr and a shell that supports ANSI-C quoting)

pr -mts$'\n' <files>

One advantage is that -s option will work for multicharacter separator as well.


Since sed is tagged, you can use R command (available on GNU sed, not sure about other implementations).

sed -e 'R f2' -e 'R f3' f1
1
#!/usr/bin/perl

use strict;

# how many files to open? 10 in the question, 3 in examples.
my $numfh=3;

# a counter for the number of open file handles
my $openfh=$numfh;

# open an array of filehandles, one for each input file.
my @fh = ();
for my $i (1..$numfh) {
  open($fh[$i],"<","file$i.txt") || die "Couldn't open file$i.txt for read: $!";
};

# open the output file.
open (my $out,">","bigfile.txt") || die "Couldn't open bigfile.txt for write: $!";

# repeat until there are no more open file handles.
until ($openfh < 1) {
  for my $i (1..$numfh) {
    if (eof($fh[$i])) {
      # if an input file is eof, close it and decrement openfh counter.
      $openfh--;
      close($fh[$i]);
    } else {
      # print a line of input from the current input file to the output file.
      print $out scalar readline $fh[$i]
    };
  };
}

Save this as, e.g., merge.pl and make it executable with chmod +x merge.pl. Then run it like so:

$ ./merge.pl

Output:

$ cat bigfile.txt 
A
E
I
B
F
J
C
G
K
D
H
L

and here's a version using @ARGV and printing to STDOUT for @terdon:

#!/usr/bin/perl

use strict;

my @fh = ();
my $i=1;

for my $f (@ARGV) {
  open($fh[$i++], "<", $f) || die "Couldn't open $f for read: $!";
};

my $numfh=$#fh; my $openfh=$numfh;

until ($openfh < 1) {
  for my $i (1..$numfh) {
    if (eof($fh[$i])) {
      $openfh--;
      close($fh[$i]);
    } else {
      print scalar readline $fh[$i]
    };
  };
}

Or using a hash to hold the file handles instead of an array:

#!/usr/bin/perl

use strict;

my %fh = ();

for (@ARGV) {
  open($fh{$_}, "<", $_) || die "Couldn't open $_ for read: $!";
};

while (keys %fh) {
  for my $f (@ARGV) {
    next unless (defined($fh{$f}));
    if (eof($fh{$f})) {
      close($fh{$f});
      delete($fh{$f});
    } else {
      print scalar readline $fh{$f}
    };
  };
}

Run either as:

$ ./merge.pl file[123].txt > bigfile.txt

The output is identical to the hard-coded version.

5
  • Why hardcode the number of files instead of just iterating over @ARGV? And why hard code the output file name instead of just printing to STDOUT? Or the input file name for that matter. I guess if you're going to reinvent the wheel (and I do love me some perl-wheel-reinventions) it may as well be better in some way than paste and this seems like it is just more convoluted, limited and slower but with no benefits.
    – terdon
    Commented Apr 21, 2021 at 14:00
  • @terdon Because the OP said they had 10 files and the examples implied they were named file1.txt to file10.txt. and once you've hard-coded the input files, hard-coding bigfile.txt is obvious and fun. Also because it was an opportunity to do something that wasn't just yet another while(<>) loop :-). The only real point of this answer is that it's an example of how to open multiple files in perl and read from & write to them - i.e. it's a starting point for programs that can do a lot more than just paste.
    – cas
    Commented Apr 21, 2021 at 14:09
  • And that's the part I really like (I am going to steal that filehandle loop/array approach, very nice!). But can't you just do exactly the same thing using @ARGV so the user can do ./merge.pl file1 file2 differentprefix3 foobar asd etc?
    – terdon
    Commented Apr 21, 2021 at 14:15
  • oh, absolutely, you can do it with @ARGV. it's actually easier like that. And you can use $outfile=shift and open that for write before looping over the remaining filename arguments if you don't want to use STDOUT for some reason. But i liked the idea of hard-coded file names for a change.
    – cas
    Commented Apr 21, 2021 at 14:18
  • @terdon the filehandle array is nothing special - fairly commonly used. a FH is a scalar, and arrays can hold scalars, so it's kind of obvious.
    – cas
    Commented Apr 21, 2021 at 14:20
1

Using GNU sed we can use the R read command to pull in the top lines progressively from the input files to get a merged output.

sed -e "
  $(printf 'R%s\n' file{2..9}.txt file10.txt)
" file1.txt

perl -lne '
  push @{$A[@ARGV]}, $_}{
  for my $i (0..$#{$A[0]}){
    print for map { $A[$_][$i] } reverse 0..$#A
  }
' file{1..9}.txt file10.txt

Here in this we Store the files in an AoA aka 2D array @A. The first index refers to the file position in the positional arguments list and the second index is for data of that particular file.In the end block }{ we are zipping the array by picking one one element from each array.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .