The problem with all the pipelines is that you are essentially doubling the work. No matter how fast the decompression is, the data still need to be shuttled to another process.
Perl has PerlIO::gzip which allows you to read gzipped streams directly. Therefore, it might offer an advantage even if its decompression speed may not match that of unpigz
:
#!/usr/bin/env perl
use strict;
use warnings;
use autouse Carp => 'croak';
use PerlIO::gzip;
@ARGV or croak "Need filename\n";
open my $in, '<:gzip', $ARGV[0]
or croak "Failed to open '$ARGV[0]': $!";
1 while <$in>;
print "$.\n";
close $in or croak "Failed to close '$ARGV[0]': $!";
I tried it with a 13 MB gzip compressed file (decompresses to 1.4 GB) on an old 2010 MacBook Pro with 16 GB RAM and an old ThinkPad T400 with 8 GB RAM with the file already in the cache. On the Mac, the Perl script was significantly faster than using pipelines (5 seconds vs 22 seconds), but on ArchLinux, it lost to unpigz:
$ time -p ./gzlc.pl spy.gz
1154737
real 4.49
user 4.47
sys 0.01
versus
$ time -p unpigz -c spy.gz | wc -l
1154737
real 3.68
user 4.10
sys 1.46
and
$ time -p zcat spy.gz | wc -l
1154737
real 6.41
user 6.08
sys 0.86
Clearly, using unpigz -c file.gz | wc -l
is the winner here both in terms of speed. And, that simple command line surely beats writing a program, however short.