Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. It only takes a minute to sign up.
Sign up to join this community
I have a directory of logs that I would like to set up a job to compress using gzip. The issue is I don't want to recompress the logs I've already compressed.
I tried using ls | grep -v gz | gzip, but that doesn't seem to work.
Is there a way to do this? Basically I want to gzip every file in the directory that does not end in .gz.
You can just do:
gzip *
gzip will tell you it skips the files that already have a .gz ending.
If that message gets in the way you can use:
gzip -q *
What you tried did not work, because gzip doesn't read the filenames of the files to compress from stdin, for that to work you would have to use:
ls | grep -v gz | xargs gzip
You will exclude files with the pattern gz anywhere in the file name, not just at the end.¹ You also have to take note that parsing the output of ls is dangerous when you have file names with spaces, newlines, etc., are involved.
A more clean solution, not relying on gzip to skip files with a .gz ending is, that also handles non-compressed files in subdirectories:
find . -type f ! -name "*.gz" -exec gzip {} \;
¹ As izkata commented: using .gz alone to improve this, would not work. You would need to use grep -vF .gz or grep -v '\.gz$'. That still leaves the danger of processing ls' output
$ so it's clearer. And you didn't escape the . anyway, so it's only going to exclude files that start with gz (since they don't have "any character before gz")
$ going to exclude (-v) all files that have names consisting of three or more characters including any character followed by "gz" anywhere in the filename. So it will include files with names that start with "gz".
Oct 10, 2014 at 22:07
gz anywhere in the name (except the start). So it's better to anchor it anyway. Having not used $, it read as though you didn't realize what . means in a regex, hence the rest of the confusing comment.