2

We have an rsync job that copies thousands of tiny little files to a server. After the files are on that server, we use retrospect to back them up to tape. Retrospect is starting to choke due to the large number of little files. I'm looking for a way to archive the files after the rsync so retrospect only has to dump one file to tape. I don't want to tar all the files everyday as that just adds to much stress to the disk and we may miss our backup window. After the backup is done, rsync will need to run again, but I don't want it to copy every file back, so it will need to be able to view all the files before update.

Thoughts?

  • 2
    tar can write directly to tapes. It is the Tape ARchiver. – jordanm Jul 10 '15 at 17:20
2

The reason youd use rsync to sync from ServerA -> ServerB is if users/services need access to the files on the ServerB system. If it's purely for backup as is suggested and users don't need access to read the files then why use rsync? Tar has incremental options and you can pipe it over the network as you would with rsync.

http://www.gnu.org/software/tar/manual/html_node/Incremental-Dumps.html

tar -c -z -g /var/tar/$path.snar -f - /$path \
    | ssh backupsrv "cat > serverA-$path.tar.gz"

Alternatively switch to a backup solution that will allow you to put the backup client on ServerA directly and remove unnecessary complication.

Free ones include Amanda, and Bacula.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.