archive files and directories before transfer

Question

I was wondering what is the advantage of making all the files and diretories into a archive file for transfer, such as by cpio, instead of transferring them directly, such as by cp, scp?

Thanks and regards!

Answer 1

There are a few reasons you might want to tar up a bunch of files before transfer:

• Compression: You will get better compression by compressing one large file rather than many small files. At least scp can compress on the fly, but is on a file by file basis.

• Connections: At least with scp, it makes a new connection for each file it transfers. This can greatly slow down the throughput if you are transferring many small files.

• Restart: If your transfer protocol allows restarting a transfer in the middle, it might be easier than figuring out which file was in progress when the transfer was interrupted.

• Permissions: Most archiving programs let you retain the ownership and permissions of files, which may not be supported in your transfer program.

• File location: If the destination location is at the end of a long path, or hasn't been decided, it might be useful to transfer an archive to the destination and decide latter where the files should go.

• Integrity: Its easier to compute and check the checksum for a single archive file than for each file individually.

Answer 2

I do this because its often easier to handle one big file, then thousands of small chunks.

I don't have to calculate checksums for every file, most of the time it is enough to just create a checksum for one archive.

Also it is easier for me to preserve file permissions.

These are just some of my reasons.

Answer 3

If using rsync is possible, the advantage of archiving first is almost gone.

• Compression: rsync has built-in compression (option -z).
• Connection: The number of connection (generally 1) doesn't increase with the number of files transfered.
• Restart: rsync only transmit the difference, so incremental transfer is simple. You can choose to keep partial files so it is faster to resume the transmission.
• Integrity: Check-sum is used internally, and if you are paranoid about it, you can force the -c option, so it is checked again.