I have a server application creating log files and rotating these files based on size. The files are time stamped at creation. The application then deletes the oldest file when X number of files in the directory is reached.

The application is not capable of compressing, only rotation and deletion.

The files are rotating rapidly and I need to be able to pull the files off from a remote server and compress them as quickly as possible due to space constraints on the storage server.

EDIT: Due to resource limitations, I do not want to compress on the application server.

I have looked into rsync but based on the requirement to compress immediately after the file transfer to the storage server, rsync does not seem to fit the requirements. Can anyone suggest an alternative?

Thanks in advance.

  • Can you compress them before transferring them? gzip all but the most recent file, then rsync -a *.gz host:/path/to/logs. – DopeGhoti Apr 4 '18 at 15:58
  • No, sorry I should have mentioned that I do not want to compress on the application server because I want to keep resource utilization at a minimum, network bandwidth is not a concern. – krdot Apr 4 '18 at 16:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.