How can I uncompress gz files in random directories and store the decompressed files in those same directories?

Question

CentOS 5.9

I've got a server with a foo.ext.gz file located in a variety of random directories.

Example:

• /opt/fooapp/foosubdirectory/foo_randomnumber/blah/blah/foo.ext.gz
• /opt/fooapp/foosubdirectory/foo_differentrandomnumber/blah/blah/foo.ext.gz
• /opt/fooapp/foosubdirectory/foo_anotherdifferentrandomnumber/blah/blah/foo.ext.gz

I'd like to run a bash command that will:

1. Locate foo.ext.gz files
2. Extract the contents of the gz file in the same directory that its respective GZ file resides in
3. Keep the original gz file intact.

If I was doing this manually, I'd start with find / -iname foo.ext.gz. After that, I'd copy the directory the file resides in and type something like:

gunzip -c /opt/fooapp/foosubdirectory/foo_12345/blah/blah/foo.ext.gz  > /opt/fooapp/foosubdirectory/foo_12345/blah/blah/foo.ext

The problem here is that I need to manually do this process for several dozen files/directories.

Is there a way I can leverage xargs or a for loop?

Answer 1

Although you can parse the output from a find you have to take care of spaces etc. Unfortunately gunzip does not have a --keep/-k flag for keeping (like bzip2 and xz do have).

If would make a small script gunzipkeep that takes one parameter (the gzipped file) and does the decompression, put that script somewhere in your $PATH and call it with:

find /opt/fooapp/foosubdirectory -name "foo.ext.gz" -print0 | xargs -0 --norun-if-empty --max-args 1 gunzipkeep

The script could be something like:

#!/bin/bash
inname=$1
outname=${inname%.gz}

gunzip -c "$inname" > "$outname"

Answer 2

It is difficult to do this without using an helper script (or a bash function) as done in another answer but not impossible. Here using -execdir option of find and some bash parameter expansion.

find /opt/fooapp/foosubdirectory -name '*.gz' -execdir /bin/bash -c 'pwd ; echo ${0%.gz}; cp ${0} ${0%.gz}.tmp.gz ; gunzip ${0%.gz}.tmp.gz ; mv ${0%.gz}.tmp ${0%.gz}' {} \;

[edit] NOTE: you need a recent version of bash (for this particular parameter exparsion), some older versions does not have this features. I tested this on a V 3.2.x

[Edit] NOTE2 : The -execdir expression, as far I know, is present into GNU find (and other modern implementations) but not into older ones. I testet this on GNU find v 4.2.x

The same rewitten for readability & comment:

find /opt/fooapp/foosubdirectory 
     -name '*.gz'
     -execdir /bin/bash -c '_bash_command_string_ ' {} \;
# This ^ will run bash from the subdirectory containing the matched file

  _bash_command_string_  --> 
      pwd ;                       # we are working in this subdir 
      echo ${0%.gz};              # this is matched filename (minus final .gz)
      cp ${0} ${0%.gz}.tmp.gz ;   # copy the .gz file as .tmp.gz
      gunzip ${0%.gz}.tmp.gz ;    # gunzip the .tmp.gz as .tmp
      mv ${0%.gz}.tmp ${0%.gz}    # rename .tmp as matched filename (minus final .gz)

This solution is interesting as an clever hack but probably too complex to be used in pratice.

See Bash Reference - Shell Parameter Expansion , search ${parameter%word} .

Answer 3

With bash ≥4, run shopt -s extglob to make **/ traverse directories recursively. (Beware that this traverses symbolic links to directories. In zsh, you don't need any special setup, and **/ doesn't descend into symlinks, but ***/ does.) Then a simple loop suffices:

err=
for z in **/*.gz; do
  gunzip <"$z" >"${z%.gz}" &&
  touch -r "$z" "${z%.gz}" ||    # if you want to retain the file's modification time
  err=1
done
if [ -n "$err" ]; then echo >&1 'Watch out, there were errors!'; fi

With only a POSIX sh, invoke a shell from find. It's more difficult to extract the error status — check if the command produces anything on stderr.

find . -name '*.gz' -exec sh -c '
  for z do gunzip <"$z" >"${z%.gz}" && touch -r "$z" "${z%.gz}"; done
' _ {} +

Answer 4

find . -name \*.gz | parallel gzip -dc {} \> {.}

• {.} is an input line without an extension
• The default delimiter is just linefeed

find . -name \*.gz | while read f; do gzip -dc "$f" > "${f%.gz}"; done

• Add IFS= or -r if the input lines can start or end with characters in IFS or if they can contain backslashes