-1

Currently I am using the following command to create an archive with files older than 7 days:

find /var/tunningLog/ -type f -mtime +7 -print0 | tar -czf "/var/tunningLog/$(date '+%Y-%m-%d').tar.gz" --null -T - && echo "OK" || echo "NOK"

But it is taking to long (currently /var/tunningLog/ has 49G). Is there any way to speed up the process or to improve the command? Thx

2

Since you're creating your archive of /var/TuningLog in /var/TuningLog, the archive you create contains the archive you created a week ago, or earlier. You have an ever growing archive of archives.

Either modify your find to exclude your archives (\! -name '*.tar.gz') if that doesn't exclude wanted files.

OR

Save your archives elsewhere.

2
  • Thanks @waltinator, also I've replaced find ... exec with find ...| xargs and it had improved the execution time. – dejdej Jan 15 at 9:30
  • xargs and tar don't work together well. If you have too many filenames and xargs runs a second tar -cvf ... – waltinator Jan 15 at 20:52
0

Using a find | tar pipe is something from the stoneage of UNIX when the only way to reuse code was to combine commands via pipelines.

This is slow because it requires to run stat(2) twice for every file.

  • Once from within the find(1) command to identify the files of interest

  • Another time from within tar(1) because tar needs the meta data for archiving

Since 33 years, there is another way to combine reusable code by using shared libraries and since 16 years, there is libfind that includes all features of the find command in a way that allows to reuse the functionality.

Since you are using non-standard tar options like -T, it seems that you are using gtar instead of tar and gtar is known for it's ineffective code for creating the tar headers.

I recommend you to have a look at star, the oldest free tar implementation that includes libfind support since 2005 and includes support for a -find option that permits to use find(1) syntax to the right side of -find on the command line.

Check e.g. the man page from the schilytools web page at: http://schilytools.sourceforge.net/man/man1/star.1.html

This leads to a command line similar to:

star -c -f /tmp/xxx.tar -find . -type f -mtime +7
1
  • 1
    thanks for info @schily – dejdej Jan 28 at 14:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.