What's the best way to backup files via network on Linux and Solaris machines?

Question

I wonder what the best way to backup files VIA network

I have Solaris machines with Oracle 10i with VERITAS cluster, machines are connected to EMC storage

/data/oracle directory is mounted on the EMC storage

What I want is to backup /data/oracle directory (70G) on some backup machine VIA network (include soft links files), while the reliability of copying files is very very important

I checked and find some ideas to do that

For example the first option is to use rsync

  rsync   -WavH –progress   /data/oracle $backup_server_ip:/Backup_dir

The second option: using is transfer files with tar option so we do both sides transfer files and compress files

  cd /directory_that_we_want_to_backup
  tar cpf - . | bzip2 -c | ssh  $backup_server_ip  \
     "cd /Backup_dir && bzip2 -d | tar xpf -"and so on

I need advice what is more reliable from the options below, and maybe there are other good options

Answer 1

For Solaris-only, look at "ZFS send" (assuming Solaris 10 update 2 or later).

The process is to first take a snapshot, and then "send" it to another place. Eg:

zfs snapshot POOL/dataset@backup1
zfs send POOL/dataset@backup1 | ssh $USER@$HOST zfs receive $TARGETPOOL/$TARGETDATASET

The above solution have rock solid reliability.

rsync is your second best option, but gives you the added benefit of being cross-platform. rsync does not ship with Solaris by default but can be easily installed from, eg Solaris Free Ware.

You should clean up your snapshots when you do not want them any longer. Keeping them around for a little while is good - you can use them to send incremental streams.

To do incremental streams, your second (and subsequent) backups will look like this:

zfs snapshot POOL/dataset@backup2
zfs send -i backup1 POOL/dataset@backup2 | ssh $USER@$HOST zfs receive $TARGETPOOL/$TARGETDATASET
zfs destroy POOL/dataset@backup1

Note: Destroy the previous-to-last snapshot, allowing you to do an incremental again next time!!!

Having many hundreds or thousands of snapshots lying around can be problematic. Firstly it severely affects the performance of management utilities, such as zfs list. Secondly, as data changes under them, they start to hog space (They initially don't take up any disk space)

Tied with rsync is using cpio (in a similar way to how you use tar) ... cpio handles more strange files better than tar. In particular it handles sparse files, will re-create hard links if possible, and handles other file types properly.

find /backup_directory | cpio -o | ssh $USER@$HOST cpio -idm /target_directory

cpio -o = Create backup (Copy Out)

cpio -i = Restore from backup (Copy in) The -d and -m are to re-create the directory structure and maintain permissions/modification times, etc, respectively.

tar is everybody's favourite because .... well, there is no good reason. tar works fine if you know you will only ever have plain files.

Answer 2

There are several possible backup solutions. I am not sure which one are cross-platform.

You could try to have a look at duplicity it should be compatible with any POSIX system though it was tested mainly on GNU/Linux.

BackupPC is another contender, written in Perl so expected to work on Solaris too.

And finally, Bacula, it is an open source network backup program that runs on a variety of OS: Linux, FreeBSD or Solaris to name a few. I would really recommend that one, though it is a bit tougher as the other previous alternatives to configure. The guys behind this product offer also professional support if needed.

Other resources: see AskUbuntu