Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I wonder what the best way to backup files VIA network

I have Solaris machines with Oracle 10i with VERITAS cluster, machines are connected to EMC storage

/data/oracle directory is mounted on the EMC storage

What I want is to backup /data/oracle directory (70G) on some backup machine VIA network (include soft links files), while the reliability of copying files is very very important

I checked and find some ideas to do that

For example the first option is to use rsync

  rsync   -WavH –progress   /data/oracle $backup_server_ip:/Backup_dir

The second option: using is transfer files with tar option so we do both sides transfer files and compress files

  cd /directory_that_we_want_to_backup
  tar cpf - . | bzip2 -c | ssh  $backup_server_ip  \
     "cd /Backup_dir && bzip2 -d | tar xpf -"and so on

I need advice what is more reliable from the options below, and maybe there are other good options

share|improve this question
2  
Please don't cross-post (see the faq). –  Mat Mar 4 '13 at 6:14

3 Answers 3

For Solaris-only, look at "ZFS send" (assuming Solaris 10 update 2 or later).

The process is to first take a snapshot, and then "send" it to another place. Eg:

zfs snapshot POOL/dataset@backup1
zfs send POOL/dataset@backup1 | ssh $USER@$HOST zfs receive $TARGETPOOL/$TARGETDATASET

The above solution have rock solid reliability.

rsync is your second best option, but gives you the added benefit of being cross-platform. rsync does not ship with Solaris by default but can be easily installed from, eg Solaris Free Ware.

You should clean up your snapshots when you do not want them any longer. Keeping them around for a little while is good - you can use them to send incremental streams.

To do incremental streams, your second (and subsequent) backups will look like this:

zfs snapshot POOL/dataset@backup2
zfs send -i backup1 POOL/dataset@backup2 | ssh $USER@$HOST zfs receive $TARGETPOOL/$TARGETDATASET
zfs destroy POOL/dataset@backup1

Note: Destroy the previous-to-last snapshot, allowing you to do an incremental again next time!!!

Having many hundreds or thousands of snapshots lying around can be problematic. Firstly it severely affects the performance of management utilities, such as zfs list. Secondly, as data changes under them, they start to hog space (They initially don't take up any disk space)

Tied with rsync is using cpio (in a similar way to how you use tar) ... cpio handles more strange files better than tar. In particular it handles sparse files, will re-create hard links if possible, and handles other file types properly.

find /backup_directory | cpio -o | ssh $USER@$HOST cpio -idm /target_directory

cpio -o = Create backup (Copy Out)

cpio -i = Restore from backup (Copy in) The -d and -m are to re-create the directory structure and maintain permissions/modification times, etc, respectively.

tar is everybody's favourite because .... well, there is no good reason. tar works fine if you know you will only ever have plain files.

share|improve this answer
    
Does the zfs snapshot thing need some cleanup after successfully "sending" it over? (Side point: that procedure might very well be rock solid, but if you do that kind of thing blindly on a live database, the only thing you'll receive at the other end is a solid rock of unusable data :-) ) –  Mat Mar 4 '13 at 9:59
    
Not necessarily true. You will end up with a database that needs to go through recovery, same as if the server lost power. Put your entire database in a single ZFS pool and you can use recursive snapshots, which are a) atomic operations, and b) preserve write order. Better to put your database into online backup mode temporarily if you can. I will edit my answer to add a comment about cleaning up the snapshot afterwards. –  Johan Mar 4 '13 at 10:09
    
+1 for zfs send. I used this for backing up approximately 700GB of data between two data centers. The initial copy took a while, but each incremental change is small and fast. –  bahamat Jun 21 '13 at 0:11

There are several possible backup solutions. I am not sure which one are cross-platform.

You could try to have a look at duplicity it should be compatible with any POSIX system though it was tested mainly on GNU/Linux.

BackupPC is another contender, written in Perl so expected to work on Solaris too.

And finally, Bacula, it is an open source network backup program that runs on a variety of OS: Linux, FreeBSD or Solaris to name a few. I would really recommend that one, though it is a bit tougher as the other previous alternatives to configure. The guys behind this product offer also professional support if needed.

Other resources: see AskUbuntu

share|improve this answer

I had good experiences using rdiff-backup around 2005. Development since 2009 is down to nothing, but this may just be because the project is now quite stable: it has been in development since 2002. There still seems to be activity on the mailing list, though. This uses the same underlying algorithm as rsync. While it is not as fancy and full-featured as some other backup solutions, it has the advantage of being quite flexible and customizable, and the basic functionality is quite simple.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.