I need to backup a fairly large directory, but I am limited by the size of individual files. I'd like to essentially create a tar.(gz|bz2) archive which is split into 200MB maximum archives. Clonezilla does something similar to this by splitting image backups named like so:

sda1.backup.tar.gz.aa
sda1.backup.tar.gz.ab
sda1.backup.tar.gz.ac

Is there a way I can do this in one command? I understand how to use the split command, but I'd like to not have to create one giant archive, then split it into smaller archives, as this would double the disk space I'd need in order to initially create the archive.

up vote 96 down vote accepted

You can pipe tar to the split command:

tar cvzf - dir/ | split --bytes=200MB - sda1.backup.tar.gz.

On some *nix systems (like OS X) you may get the following error:

split: illegal option -- -

In that case try this (note the -b 200m):

tar cvzf - dir/ | split -b 200m - sda1.backup.tar.gz.

If you happen to be trying to split file to fit on a FAT32 formatted drive use a byte limit of 4294967295. For example:

tar cvzf - /Applications/Install\ macOS\ Sierra.app/ | \
split -b 4294967295 - /Volumes/UNTITLED/install_macos_sierra.tgz.
  • 2
    Great! How can I then extract the archive? – Naftuli Kay Jan 18 '13 at 19:42
  • 22
    Will cat sda1.backup.tar.gz.* | tar xzvf - do the job? – Naftuli Kay Jan 18 '13 at 19:45
  • 3
    Yes, it should. split by default sets the names the files so that when sorted by LOCALE (which is done by shell globbing) will be in the correct order. – jordanm Jan 18 '13 at 19:47
  • 4
    Without verbose, just do tar czf ... without the v and merge by cat backup.tar.gz.* | tar tar xzf - without v. I see no benefit of the verbose output here by v. – Léo Léopold Hertz 준영 Jul 9 '16 at 12:15
  • 1
    Just helped a friend by packing Xcode onto a FAT32 formatted flash drive with: tar cvzf - Xcode.app/ | split -b 2000m - /Volumes/PH/xcode/xcode.tgz (used from cd /Applications/) Thank you very much :) – ecth Dec 8 '17 at 12:06

On OS X (also a Unix), the split command works slightly differently:

$ tar cvzf - foo | split -b 2500m - foo.tgz.

Just to add: As the maximum allowed file size in vfat/fat32 is 2^32 minus 1 (4294967295 bytes), the split command with the maximum allowed file size on such file system is:

split -b4294967295 -d my_input_file my_output_file_splitted
serega@serega-sv:~$ tar -c  -M --tape-length=1024 --file /tmp/pseudo-tape.tar --new-volume-script=/tmp/new-volume.sh --volno-file=/tmp/volno /tmp/stuff-to-archive 
tar: Removing leading `/' from member names
moving /tmp/pseudo-tape.tar to /tmp/archive.1
moving /tmp/pseudo-tape.tar to /tmp/archive.2
moving /tmp/pseudo-tape.tar to /tmp/archive.3

You'll need a script for automation moving pseudo-tape.tar file to a new name:

serega@serega-sv:~$ cat /tmp/new-volume.sh 
dir="/tmp"
base_name="pseudo-tape.tar"
next_volume_name=`echo -n "archive."; cat $dir/volno`
echo "moving $dir/$base_name to $dir/$next_volume_name"
mv "$dir/$base_name" "$dir/$next_volume_name"
  • I haven't downvoted your answer, because I am happy to see one that uses -M --tape-length. However, this answer does ignore the OP's request for a solution that uses gzip or bzip2 compression. – sampablokuper Mar 15 at 13:28
  • Caution: you are not talking about tar but rather about a tar clone called gtar (GNU tar). This tar clone supports to create multi volume archives but with a noticeable probability is unable/unwilling to extract from those multi volume archives as it incorrectly claims that a follow up volume is not the right continuation part. – schily Sep 1 at 11:32

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.