0

I have to write 17 Tb to tape:

ssh some_host 'tar -cz /' | dd bs=20b of=/dev/tape

of course 17 Tb doesn't fit to one tape so I need automatically change it when "no room" error occurs. I have robot changer and "mtx next" works fine. I also need to write label to log when tape changed, so I prefer to write hook script on this event.
"tar" has "change tape script" feature, but I run tar on another host.

Also, coping 17 Tb to local host is not the option. sshfs is not an option, as bad. And please don't offer huge backup solutions.

What i need is pipe tool like dd which able to run some script on 'no room' error and proceed after. Specifying block size is also important as tape drive requires some values.

3
  • If you can install rmt on the system with the tape drive, you can have tar access the drive that way. If you have GNU tar, you can give it the --rsh-command='ssh ...' option if you don't want to run rsh. If you have LTO tapes, a blocking factor of 20 may be too small to keep the tape streaming; 126 is what we used with LTO4. But I think some rmt implementations restrict you to 20-block transfer sizes, so you may want to look at @schily 's implementation of rmt Commented Sep 25, 2015 at 14:48
  • Thank you Mark, why didnt you make an answer, just comment ? rmt is a good choice, but how to divide it's time? I have about twenty servers who needs backup. How can I queue them to use rmt?
    – xoid
    Commented Sep 29, 2015 at 7:34
  • Sometimes I suggest one or a few alternatives in comments in case the asker can't install some of the suggested software. I'll write an answer now. Commented Sep 29, 2015 at 16:09

1 Answer 1

1

If you can install rmt on the system with the tape drive, you can have tar access the drive over the network. By default tar will use the rsh protocol to run rmt on the tape server, but if you have GNU tar, you can give it the --rsh-command='ssh tapeserver /usr/sbin/rmt' option.

If you have LTO tapes, a blocking factor of 20 may be too small to keep the tape streaming; 126 is what we used with LTO4. But I think some rmt implementations restrict you to 20-block transfer sizes, so you may want to look at @schily's implementation of rmt.

In a comment you asked

rmt is a good choice, but how to divide it's time? I have about twenty servers who needs backup. How can I queue them to use rmt?

If the backup commands for each server can be packaged into a shell script, there are probably some flexible batch queuing systems out there that can guarantee sequential processing of them, but I don't know of any offhand and I realize you don't want to have a lot of complexity here.

As a start, you could try something like this, on a system that can ssh to all the servers:

#!/bin/sh
lock=/var/run/doalldumps.lock
status=/var/run/doalldumps.status
for s in $(cat ~/servers)
do
(
    flock -e 9
    echo started $s at $(date) > $status
    ssh $s -n command-to-do-backups
    echo finished $s at $(date) > $status
) 9> $lock
done

Alternatively, a simple way to serialize access to the tape drive is to use flock to lock a file on the server with the tape drive. You could use this in the tar --rsh-command option:

tar ... --rsh-command='ssh tapeserver flock -e /var/run/tape.lock /usr/sbin/rmt'
3
  • Great solution Mark. I have read rmt man and it seems rmt can't change volume as it filled. Can I run tar on some server with option --volume-change-script and when rmt on tape server will fill the volume, will rmt run this script? Script should do some log record "this backup task half-written to volume with this barcode" and then"mtx next" to change a volume.
    – xoid
    Commented Sep 30, 2015 at 18:27
  • I've never seen an rmt that is capable of running scripts, sorry. Commented Sep 30, 2015 at 18:33
  • So, no luck to backup 17 Tb filesystem with rmt. I writing my own binary which operates mostly as dd, read from stdin and writing to specified file with specified buffer size (I hope this is the same as block size in tar). And in case of write errror 28 (disk full) it closes the file, runs script, opens file again and proceeds further.
    – xoid
    Commented Oct 1, 2015 at 11:37

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .