I have one doubt with this:

for i in ./*.gz
gunzip -c $i | head -8000 | gzip > $(basename $i)

I get an "unexpected end of file" error for gzip, and I don't get the 8000 at all, only a few tens of lines. I've checked the integrity of my files and they are fine. Curiously, if I run for individual files:

gunzip -c file1.gz | head -8000 | gzip > file1.gz

I get the expected result. With this script I can obtain the first 8000 lines from a compressed file with 10708712 lines, and compress again. The new file overwrites the original file, but that's ok.

  • 3
    If the individual invocation worked for you you just got lucky. Redirecting back into a file that you're reading is usually not a good idea. – tink Oct 29 '14 at 19:45

You should write the output to a temporary file and then rename:

for i in ./*.gz 
  gunzip -c "$i" | head -8000 | gzip > "$i.tmp"
  mv -f "$i.tmp" $(basename "$i") 

That your version sometimes works is if you gunzip buffers enough while reading.

| improve this answer | |
  • It works, thanks a lot. I only add the option -f to mv – zorbax Oct 29 '14 at 20:07
  • 1
    Thanks for feedback. I added the -f to the script (for those that don't make it to the comments ;-) ) – Anthon Oct 29 '14 at 20:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.