0

I'm to backup 7GB on 9GB drive. Means, I have about 2GB of empty space only.

But tar related solutions require first - get everything into 1 file, next - split it.

Is there any possibility to archive (and probably but not necessarily compress) and split, to have an ability to transfer to a remote drive chunk by chunk.

I mean

  • while 1st file is ready, I'm to move it to an external drive and delete from the system.

  • when 2nd file is ready, I'm to move ....

0
1

Break it into multiple archives at source.

Use du to discover some of the largest directories and archive/compress these first (use tar | gzip or a direct tar compression option). Copy that .tgz file to a remote drive, check the size, then remove from your main disc.

Repeat for other large directories until you you have archived maybe 5GB.

Finally, archive and compress the base directory but use the exclude option in tar to name the directories already archived.

Alternatively, you can tar and gzip through pipes, and then pipe into ssh to write it to your remote drive. Something like (check options):

tar cfz - myBaseDir | ssh myRemote 'cat > myArchive.tg z'
2
  • This solution is obvious, unfrtunately. The case, the system has gazillion of relatively small files and folders, so I've already backed up all the biggies. But the rest is still on the go. – Alexey Abraham May 4 '20 at 11:25
  • 1
    I'm not saying to back up individual files.The aim is to work high enough up the directory tree to get a reasonably sized subtree. However, if some directory has a gazillion files at the same level, you can use find with -name patterns to get subsets of these of the appropriate size (either files or directories), send those name lists to files, and pass each one into tar with the -T (--files-from) option. Did you also consider the tar | ssh option -- that uses no space at all on the source machine. – Paul_Pedant May 4 '20 at 16:14
0

As far as I know, there is no one-liner for that. But I've written a small bash script, which put all files into several tar packages.

#!/bin/bash

package_i=1
all_files=$(find . -name "*" -type f)

# stop on every 100 MB for now
max_size=104857600

for f in $all_files; do
        #
        # add file to current tar file
        #
        tar -rvf package${package_i}.tar $f

        #
        # check if the size is more than 100 MB
        # if so, then wait till current package tar
        # was moved by user to another storage
        # and then proceed
        #
        size_of_current_package=$(stat --printf="%s" package${package_i}.tar)
        if [ $size_of_current_package -ge $max_size ]; then
                read -p "Will wait when you moved package${package_i}.tar to another storage ..."
                package_i=$(( $package_i + 1 ))
                echo "=== Package iterator is now '${package_i}' ==="
        fi
done

The script adds file for file into the current packageX.tar file (package1.tar, package2.tar, etc.). If that packageX.tar file exceeds some amount of bytes (in this example this amount is 100MB or 104857600 Bytes), then it will wait till the user copied the packageX.tar file to another storage and proceed after the user hit Enter.

2
  • isn't the varialbe i unused? – Jakub Jindra May 22 '20 at 18:32
  • @JakubJindra Yes, you are right. I've corrected the script. Thank you for the hint. – MrPython May 22 '20 at 20:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.