Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

When handling log files, some end up as gzipped files thanks to logrotate and others not. So when you try something like this:

$ zcat *

you end up with a command line like zcat xyz.log xyz.log.1 xyz.log.2.gz xyz.log.3.gz and then with:

gzip: xyz.log: not in gzip format

Is there a tool that will take the magic bytes, similar to how file works, and use zcat or cat depending on the outcome so that I can pipe the output to grep for example?

NB: I know I can script it, but I am asking whether there is a tool out there already.

share|improve this question

6 Answers 6

up vote 18 down vote accepted

zless

It seems a pity about zcat, as libz has an API that supports reading from both compressed and uncompressed files transparently. But the manpage does say that zcat is equivalent to gunzip -c.

share|improve this answer
    
Thank you for this alternative. I could have thought of that, could I not? ;) ... oh well. Spot on, +1 and accept (also because you have less rep than the other answerer). –  0xC0000022L May 27 '13 at 20:10

Try it with -f or --force:

zcat -f -- *

man zcat for details.

Also:

so that I can pipe the output to grep for example

You have zgrep for that:

zgrep -- PATTERN *
share|improve this answer
    
Thanks, that's an interesting alternative to the zless solution. Nice and +1. –  0xC0000022L May 27 '13 at 20:08
3  
Note that both zless and zgrep are scripts that do call gzip -cdfq (that is zcat -fq). –  Stéphane Chazelas May 28 '13 at 12:50

There is a drop-in replacement for ztools (zcat, zgrep, ..) called zutils that unites all the decompression tools independently of the backend. So with the same command you can read plain, lzma, gzipped, xz files transparently.

It's available in debian wheezy or newer, probably in redhat/centos too.

The project's page is here nongnu.org

A blog post explaining the usage of the util here (noone.org)

share|improve this answer

This works fine in RHEL 5.x where zcat is a binary. It fails in RHEL 6.x (and Ubuntu 12.x) where zcat is a script. This used to work fine.

I wouldn't be using zcat at all but zgrep won't properly handle uncompressed files either.

share|improve this answer

What about wrapper?

$ cat xcat.sh 
#!/bin/bash

for i in $@;do 
        [ ! -z "$(file -i $i | grep "gzip")" ] && zcat $i || cat $i
done

$ bash xcat.sh plain.txt gzipped_text.gz
share|improve this answer

You just combine discrete zcat/cat commands enclosed in parenthesis.

( zcat /var/log/lol.log.3.gz; zcat /var/log/lol.log.2.gz; cat /var/log/lol.log.1; cat /var/log/lol.log ) | grep "lolololol"

Make sure you sequence your logfiles from the highest number going down in order to make sure the dates are in order. The lowest archive number going up will just stack them ridiculously.

share|improve this answer
    
Thanks, but how is this transparent in any way of the meaning? –  0xC0000022L Oct 30 at 10:31
    
Define transparent ? –  Dmitri DB Oct 30 at 16:52
    
Ah, I think I get it -- You mean agnostic to whether or not we're dealing with a compressed archive or not. Yeah, I'm pretty sure all the other answers here may be more applicable to that end. Guess I'll have to try using them someday! –  Dmitri DB Oct 30 at 16:54

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.