Tag Info

Hot answers tagged

33

As long as you don't move the file across file-system borders, the operation should be safe. This is due to the mechanism, how »moving« actually is done. If you mv a file on the same file-system, the file isn't actually touched, but only the file-system entry is changed. $ mv foo bar actually does something like $ ln foo bar $ rm foo This would ...


27

Use rsync, and pass -u if you want to only update files that are newer in the original directory, or --ignore-existing to skip all files that already exist in the destination. rsync -au /local/directory/ host:/remote/directory/ rsync -a --ignore-existing /local/directory/ host:/remote/directory/ (Note the / on the source side: without it rsync would ...


19

rsync is probably the best tool for this. There are a lot of options on this command so read man page. I think you want the --checksum option or the --ignore-times


17

I heard it (sorry, I forget where) as typing the sync command three times (as in: S Y N C Return, wait for the prompt, repeat, repeat). I also read that the origin was a particular system where it would take a couple of seconds for the disk to finish flushing its buffers, even after it had told the operating system everything was fine. Typing the command ...


17

scp will overwrite the files if you have write permissions to them. In other words: You can make scp effectively skip said files by temporarily removing the write permissions on them (if you are the files' owner, that is).


17

Rsync doesn't try to resolve conflicts. This isn't its job. Rsync doesn't even have a way to detect that both sides have modified the file, because it has no information about any common ancestor. With the default options, the source file is unconditionally copied to the destination, overwriting the destination file. With the option -u, the source file is ...


13

The tool of choice for unidirectional synchronization is rsync, and the tool of choice for bidirectional synchronization is Unison. Both require the executable to be available on both sides. If you can make a file executable on the server side, drop the unison binary and make it executable. If you have Linux, *BSD, Solaris or Mac OS X locally, you can ...


11

There are plenty of ways for an unprivileged user to slow down a system and running sync is far from being the more efficient. On the other hand, having the file systems data committed to disk is quite a legitimate request so forbidding users (and thus their processes) to do it would be excessive. In any case, I disagree about your "unnecessary disk writes" ...


10

rsync is able to do this. rsync --ignore-existing <src> <dest> You can perform also various kinds of updates. Just have a look at the man page.


10

There are a number of factors that might make a software clock run slow or fast. Clocks on virtual servers are especially prone to a whole class of these problems. 12 seconds a day is pretty bad until you come across virtual boxes with clocks that run at 180–200% speed! Clocks on laptops that suspend can suffer from time-keeping issues too. You should ...


10

Using rsync can accomplish this. Based on the type of system you have, you will need to donwload it: sudo yum install rsync (RPM Based) sudo apt-get install rsync (Debian Based) Then using this, here is the command you will need to use: rsync -a source destination Or rsync -r source destination Where -r stands for copying data recursively (but don’t ...


9

Keep the files under version control. This has multiple benefits, including facilitating keeping files synchronized (commit on one machine, update on the others) and keeping a history of changes (so you can easily find out what broke a program that worked last month). I use CVS and synchronize the repositories with Unison or sneakernet, but that's because ...


9

rsync is actually not recommended for two-way sync (by it's developers). The already recommended unison will be a better solution for you. Also keep in mind that you probably have to deal with a lot of Mac specific HFS+ stuff when using rsync. So be sure to use a properly patched rsync3 on the Mac or expect problems with modification dates (when using the ...


9

You want bi-directional sync. Take a look at unison, which does this: http://www.cis.upenn.edu/~bcpierce/unison/ For example, on Debian/Ubuntu: $ sudo apt-get install unison $ unison MAC/ LINUX/ If you have trouble with permissions (example ext4 -> FAT): $ unison -perms 0 vlc-2.2.0/ /media/sf_vlc/vlc Contacting server... Looking for changes Reconciling ...


9

I believe you can use rsync to do this. The key observation would be in needing to use the --existing and --update switches. --existing skip creating new files on receiver -u, --update skip files that are newer on the receiver A command like this would do it: $ rsync -avz --update --existing src/ dst Example Say ...


8

I'll go with Gilles and point you to Unison as suggested by hasen j. Unison was DropBox 20 years before DropBox. Rock solid code that a lot of people (myself included) use every day -- very worthwhile to learn. Still, join needs all the publicity it can get :) This is only half an answer, but I have to get back to work :) Basically, I wanted to ...


8

Have a look at How do I connect to a pc through another pc using ssh You create a new ~/.ssh/config entry with the name tunnelb: Host tunnelb HostName hostB User user ProxyCommand ssh user@hostA nc %h %p If you have a recent version of ssh you can use Proxycommand ssh user@hostA -W %h:%p instead. This is preferred as it does not rely on nc Now you can ...


8

The right thing to do is something like mdadm --add /dev/md0 /dev/sdb1. Use the correct array in place of md0 and the correct partition in place of sdb1. The key thing is the array is running. Its completely unambiguous which data to copy: the data that is currently running. If you have bitmaps enabled, the resync will be fairly fast as it'll only copy what ...


7

There's a utility called unison: http://www.cis.upenn.edu/~bcpierce/unison/ Description from site: Unison is a file-synchronization tool for Unix and Windows. It allows two replicas of a collection of files and directories to be stored on different hosts (or different disks on the same host), modified separately, and then brought up to date by ...


7

you can remount with sync option and then remount it back with async: mount -o remount,sync /mountpoint mount -o remount,async /mountpoint Using remount option will not mess with processes using remounted filesystem.


7

use rsync: rsync -a --ignore-existing cosmo_sim_9 /dest/disk/cosmo_sim_9 --ignore-existing will cause it to skip existing files on the destination, -a will make it recursive, preserving if possible permission/ownership/group/timestamp/links/special devices. you can do it for all directories by using a bash for loop: for dir in cosmo_sim_* ; do rsync -a ...


7

Since you say you're using node.js, I assume you'd be using fs.rename() (or fs.renameSync()) to rename the files. This node.js method is documented to use the rename(2) system call, which does not touch the file itself in any way, but merely changes the name under which it is listed in the file system: "rename() renames a file, moving it between ...


6

Check out lsyncd. Lsyncd watches a local directory trees event monitor interface (inotify). It aggregates and combines events for a few seconds and then spawns one (or more) process(es) to synchronize the changes. By default this is rsync. Lsyncd is thus a light-weight live mirror solution that is comparatively easy to install not requiring new ...


6

Why not use rsync instead? rsync -a /branch2/media/ /branch1/media/ The reason why mv can't move /branch2/media/cd/ to /branch1/media is because /branch1/media already has a cd/ in it. mv refuses to clobber non-empty directories.


6

A filesystem overlay would probably work better for this. Using something like aufs you can create a 'virtual' directory out of several combined directories. You can configure whether writes should propagate back to the original directories, or use a copy-on-write method to leave the originals alone, or disallow writes altogether. However to answer your ...


6

Use rsync's --ignore-existing flag. From man rsync: --ignore-existing skip updating files that exist on receiver To do this between two distinct servers you'd do something like this: rsync --ignore-existing -avz -e ssh source remoteuser@remoteserver:destination If you are truly going to be running this "all the time" (ie. continuously) like you ...


6

You should take a look at rdiff-backup it does rsync underneath, but with the additional intelligence that you require (and it makes incremental backups capable of rolling back, but you can configure to switch that off). The latest release is old (2009), but that is a sign of stability.


6

The go to tool for syncing data is rsync. You can sync either at the directory level or just the contents of a directory like so: Examples directory sync 24 9 * * * rsync -a /home/fan/Data /media/T/ contents sync 24 9 * * * rsync -a /home/fan/Data/ /media/T/Data/ The first example will sync the directory Data from /home/fan to the directory /media/T. ...



Only top voted, non community-wiki answers of a minimum length are eligible