If only have enough space to do a small number of files, you can do the archiving in chunks, and it can even be automated if you want. Assuming your files are stored something like this:
.
+-dir0000
| +-file0000.gz
| +-file0001.gz
| [...]
+-dir0001
| +-file1000.gz
| +-file1001.gz
[...]
For each directory, run (from the shared root of the files):
tar rf /path/to/archive_name.tar dir_name
rm -r dir_name
If you're OK with automating this (I tried it and it worked, but you get error messages from find
which can be ignored) try the following:
find * -maxdepth 1 -type d -exec sh -c "tar rf /path/to/archive_name '{}' && rm -r '{}'" \;
(As usual, try this on a test directory structure first so you can verify it's doing what you want!)
I quoted the {}
in case your directories have spaces. If you have a flat file structure, then you can do something similar, only without -type d
. Make sure not to use +
at the end of -exec
, or else find
will try to do a whole bunch of files/directories at a time, which defeats the point of breaking the archiving into pieces in the first place!
Eventually all the files will be moved into the archive. It'll be slow, but it can be done.