Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

When handling log files, some end up as gzipped files thanks to logrotate and others not. So when you try something like this:

$ zcat *

you end up with a command line like zcat xyz.log xyz.log.1 xyz.log.2.gz xyz.log.3.gz and then with:

gzip: xyz.log: not in gzip format

Is there a tool that will take the magic bytes, similar to how file works, and use zcat or cat depending on the outcome so that I can pipe the output to grep for example?

NB: I know I can script it, but I am asking whether there is a tool out there already.

share|improve this question

7 Answers 7

up vote 24 down vote accepted

zless

It seems a pity about zcat, as libz has an API that supports reading from both compressed and uncompressed files transparently. But the manpage does say that zcat is equivalent to gunzip -c.

share|improve this answer
    
Thank you for this alternative. I could have thought of that, could I not? ;) ... oh well. Spot on, +1 and accept (also because you have less rep than the other answerer). –  0xC0000022L May 27 '13 at 20:10

Try it with -f or --force:

zcat -f -- *

Since zcat is just a simple script that runs

exec gzip -cd "$@"

with long options that would translate to

exec gzip --stdout --decompress "$@"

and, as per the man gzip (emphasize mine):

-f --force
      Force compression or decompression even if the file has multiple links
      or the corresponding file already exists, or if the compressed data is
      read from or written to a terminal. If the input data is not in a format
      recognized by gzip, and if the option --stdout is also given, copy the
      input data without change to the standard output: let zcat behave as cat.

Also:

so that I can pipe the output to grep for example

You could use zgrep for that:

zgrep -- PATTERN *

though see Stéphane's comment below.

share|improve this answer
    
Thanks, that's an interesting alternative to the zless solution. Nice and +1. –  0xC0000022L May 27 '13 at 20:08
4  
Note that both zless and zgrep are scripts that do call gzip -cdfq (that is zcat -fq). –  Stéphane Chazelas May 28 '13 at 12:50

There is a drop-in replacement for ztools (zcat, zgrep, ..) called zutils that unites all the decompression tools independently of the backend. So with the same command you can read plain, lzma, gzipped, xz files transparently.

It's available in debian wheezy or newer, probably in redhat/centos too.

The project's page is here nongnu.org

A blog post explaining the usage of the util here (noone.org)

share|improve this answer

This works fine in RHEL 5.x where zcat is a binary. It fails in RHEL 6.x (and Ubuntu 12.x) where zcat is a script. This used to work fine.

I wouldn't be using zcat at all but zgrep won't properly handle uncompressed files either.

share|improve this answer

What about wrapper?

$ cat xcat.sh 
#!/bin/bash

for i in $@;do 
        [ ! -z "$(file -i $i | grep "gzip")" ] && zcat $i || cat $i
done

$ bash xcat.sh plain.txt gzipped_text.gz
share|improve this answer

Opens both compressed and non-compressed, in chronological order.

ls -v syslog* | tac | xargs zcat -f | less
share|improve this answer
    
It gives wrong order with more than ten log files (syslog.10.gz ...) –  Vanni Aug 19 at 22:51
    
Good catch. -v should fix that. –  Ryan 2 days ago
    
ls -rv to avoid tac. For log files, less $(ls -rv syslog*) with your LESSOPEN env var set properly works well. You can do search across files with esc-n to find next match, ignoring file boundaries. –  Peter Cordes 2 days ago

Copy and paste (or put it at the end of your ~/.bashrc file) this bash function:

logs() { zcat -f $(ls -rv "$1"*) | less; }

Now you can type for example logs /var/log/syslog or logs /var/log/nginx/access.log to see all the syslog or nginx log messages from oldest to newest with less.

You can then search something typing /something and hitting n for next.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.