I'm trying to write a bash script that will go into a directory loop through the .gz files and delete them if they are empty (ie the uncompressed file contained within in empty.

I've got a couple of questions:

  • Is there a standard file size of a compressed (gz) empty file I can check for?
  • Or is there a better way to check if a gz contains an empty file without decompressing it with a bash script?

I was trying to use the following code to acheive this but it relies on the filesize being 0 i think.

for f in dir/*
    if [[ -s $f ]]

Unfortunately the gzip contains the orginal filename, so the its size will vary without for different empty files.

gunzip -c $f | head -c1 | wc -c

will echo 1 for files that are non-zero in uncompressed size, and 0 for compressed empty files.

for f in dir/*
    if [[ $(gunzip -c $f | head -c1 | wc -c) == "0" ]] 

Might do what you want?

| improve this answer | |
  • This is the correct approach, since the use of head avoids completely decompressing giant gzip files just to find out if they have one or more bytes. If there really are zero uncompressed bytes in the file, then it will complete very quickly. Using gunzip -l is not guaranteed to work, since you can have multiple gzip streams in a file, and gzip -l will only tell you about the last one. Also a gzip file with an exact multiple of 4 GB of uncompressed data will show up as zero length to gzip -l. – Mark Adler May 29 '12 at 22:01

You could add -l to gzip|gunzip|zcat to print out information about the gzipped file and then test against the uncompressed file size.

$ gunzip -l file.empty.gz 
compressed        uncompressed  ratio uncompressed_name
        31                   0   0.0% file.empty
| improve this answer | |

The filename (or the absence of the filename within the compressed file) will change the size of the files so you can't look for a standard size, e.g.

$ touch file1 file2-longer-name file3
$ gzip file1 file2-longer-name
$ gzip --no-name file3
$ ls -l file*.gz
-rw-r--r-- 1 cwarden cwarden 26 May 29 10:33 file1.gz
-rw-r--r-- 1 cwarden cwarden 38 May 29 10:33 file2-longer-name.gz
-rw-r--r-- 1 cwarden cwarden 20 May 29 10:33 file3.gz

I would probably do something like this:

for f in dir/*; do
    zcat $f | head -1 | ifne -n rm -f $f > /dev/null
| improve this answer | |

There is probably a better way but this should give you the file size of the uncompressed version of the compressed file:

export fsize=$(gunzip -l ${f} | tail -1 | awk -F " " '{print $2}' )

| improve this answer | |
for i in *.gz
     [ `zcat "$i" | head -n 1 | wc -c ` -eq 0 ] && rm -i "$i"

The zcat "$i" | head -n 1 | wc -c will count the number of characters in the first line of the file, this avoids uncompressing the whole file by just looking at the beginning of it.

I also added a -i to the rm command to ask for confirmation before deleting the file, but you could remove that if it's unecessary in your case.

| improve this answer | |
  • 3
    Don't do for x in $(ls ...), just use for x in *.gz. It's more whitespace-safe and more readable. – Kevin May 29 '12 at 17:45
  • @Kevin, neither is whitespace-safe. Use IFS=$(echo -e "\n\r"); for i in $(ls -1 *gz); do.... – xn. May 29 '12 at 17:55
  • 1
    @xn. incorrect, the glob in for x in *.gz is a standard glob, it's perfectly whitespace-safe. Yours, on the other hand, won't handle filenames with \n\r in the name. – Kevin May 29 '12 at 18:01
  • P.S. Better to use $'\n\r' instead of $(echo -e "\n\r"). – Kevin May 29 '12 at 18:03
  • @Kevin: didn't know about the $'\n\r' syntax. Thanks. Without changing IFS, how do you avoid having to quote $i? bit.ly/LCg1C6 – xn. May 29 '12 at 18:08

Not the answer you're looking for? Browse other questions tagged or ask your own question.