I need to copy large files over the network, but I am not sure about the command line to use for it. I have been using rsync, but I feel maybe I could zip the files first?
rsync already provides compression when you specify option
-z. You should avoid applying this option for already compressed files or binaries as it will just introduce additional overhead.
When using rsync with the ssh protocol (the default) and you have only limited CPU resources and security is not an issue, you can also try specifying
-e ssh -c arcfour to use a less expensive encryption algorithm (RC4). On modern CPUs this won't make any big difference, though.
Other interesting arguments you may want to use:
-vPfor a nice progress bar and keep partially transfered files when interrupted, so you can resume from the partial file the next time you run
-ato preserve most (but not all!) file meta data such as permissions and modification times
I'm not sure I understand whether you have a problem compressing files or you have a problem transferring files, or you just have a general question. I'll go with the later.
If you have several files in a folder, you could tar them up in a single archive (.tar) file:
tar -cvf myfile.tar myfolder
You may then compress this file using a very good compressor, such as lzma:
lzma myfile.tar. Note that you can also instruct tar to compress with the
-j (bzip2), or
--lzma options, in which case you execute a single command, for example:
tar --lzma -cvf myfile.tar.lzma myfolder.
You may then copy your file using
scp, and uncompress it (
lzma -d myfile.tar.lzma) and then extract it (
tar -xvf myfile.tar) once on the destination system. If you used a single command, the single command for extraction would be:
tar --lzma -xvf myfile.tar.lzma.
You may also instruct rsync to compress using the
-z option, and not worry about creating a single-file archive beforehand.