Tag Info

Hot answers tagged

32

As long as you don't move the file across file-system borders, the operation should be safe. This is due to the mechanism, how »moving« actually is done. If you mv a file on the same file-system, the file isn't actually touched, but only the file-system entry is changed. $ mv foo bar actually does something like $ ln foo bar $ rm foo This would ...


19

Use rsync, and pass -u if you want to only update files that are newer in the original directory, or --ignore-existing to skip all files that already exist in the destination. rsync -au /local/directory/ host:/remote/directory/ rsync -a --ignore-existing /local/directory/ host:/remote/directory/ (Note the / on the source side: without it rsync would ...


18

rsync is probably the best tool for this. There are a lot of options on this command so read man page. I think you want the --checksum option or the --ignore-times


15

I heard it (sorry, I forget where) as typing the sync command three times (as in: S Y N C Return, wait for the prompt, repeat, repeat). I also read that the origin was a particular system where it would take a couple of seconds for the disk to finish flushing its buffers, even after it had told the operating system everything was fine. Typing the command ...


12

The tool of choice for unidirectional synchronization is rsync, and the tool of choice for bidirectional synchronization is Unison. Both require the executable to be available on both sides. If you can make a file executable on the server side, drop the unison binary and make it executable. If you have Linux, *BSD, Solaris or Mac OS X locally, you can ...


10

rsync is able to do this. rsync --ignore-existing <src> <dest> You can perform also various kinds of updates. Just have a look at the man page.


10

There are plenty of ways for an unprivileged user to slow down a system and running sync is far from being the more efficient. On the other hand, having the file systems data committed to disk is quite a legitimate request so forbidding users (and thus their processes) to do it would be excessive. In any case, I disagree about your "unnecessary disk writes" ...


10

Using rsync can accomplish this. Based on the type of system you have, you will need to donwload it: sudo yum install rsync (RPM Based) sudo apt-get install rsync (Debian Based) Then using this, here is the command you will need to use: rsync -a source destination Or rsync -r source destination Where -r stands for copying data recursively (but don’t ...


9

scp will overwrite the files if you have write permissions to them. In other words: You can make scp effectively skip said files by temporarily removing the write permissions on them (if you are the files' owner, that is).


9

Keep the files under version control. This has multiple benefits, including facilitating keeping files synchronized (commit on one machine, update on the others) and keeping a history of changes (so you can easily find out what broke a program that worked last month). I use CVS and synchronize the repositories with Unison or sneakernet, but that's because ...


8

I'll go with Gilles and point you to Unison as suggested by hasen j. Unison was DropBox 20 years before DropBox. Rock solid code that a lot of people (myself included) use every day -- very worthwhile to learn. Still, join needs all the publicity it can get :) This is only half an answer, but I have to get back to work :) Basically, I wanted to ...


8

You want bi-directional sync. Take a look at unison, which does this: http://www.cis.upenn.edu/~bcpierce/unison/


8

There are a number of factors that might make a software clock run slow or fast. Clocks on virtual servers are especially prone to a whole class of these problems. 12 seconds a day is pretty bad until you come across virtual boxes with clocks that run at 180–200% speed! Clocks on laptops that suspend can suffer from time-keeping issues too. You should ...


8

Have a look at How do I connect to a pc through another pc using ssh You create a new ~/.ssh/config entry with the name tunnelb: Host tunnelb HostName hostB User user ProxyCommand ssh user@hostA nc %h %p If you have a recent version of ssh you can use Proxycommand ssh user@hostA -W %h:%p instead. This is preferred as it does not rely on nc Now you can ...


7

rsync is actually not recommended for two-way sync (by it's developers). The already recommended unison will be a better solution for you. Also keep in mind that you probably have to deal with a lot of Mac specific HFS+ stuff when using rsync. So be sure to use a properly patched rsync3 on the Mac or expect problems with modification dates (when using the ...


7

There's a utility called unison: http://www.cis.upenn.edu/~bcpierce/unison/ Description from site: Unison is a file-synchronization tool for Unix and Windows. It allows two replicas of a collection of files and directories to be stored on different hosts (or different disks on the same host), modified separately, and then brought up to date by ...


7

The right thing to do is something like mdadm --add /dev/md0 /dev/sdb1. Use the correct array in place of md0 and the correct partition in place of sdb1. The key thing is the array is running. Its completely unambiguous which data to copy: the data that is currently running. If you have bitmaps enabled, the resync will be fairly fast as it'll only copy what ...


7

use rsync: rsync -a --ignore-existing cosmo_sim_9 /dest/disk/cosmo_sim_9 --ignore-existing will cause it to skip existing files on the destination, -a will make it recursive, preserving if possible permission/ownership/group/timestamp/links/special devices. you can do it for all directories by using a bash for loop: for dir in cosmo_sim_* ; do rsync -a ...


7

Since you say you're using node.js, I assume you'd be using fs.rename() (or fs.renameSync()) to rename the files. This node.js method is documented to use the rename(2) system call, which does not touch the file itself in any way, but merely changes the name under which it is listed in the file system: "rename() renames a file, moving it between ...


6

Check out lsyncd. Lsyncd watches a local directory trees event monitor interface (inotify). It aggregates and combines events for a few seconds and then spawns one (or more) process(es) to synchronize the changes. By default this is rsync. Lsyncd is thus a light-weight live mirror solution that is comparatively easy to install not requiring new ...


6

A filesystem overlay would probably work better for this. Using something like aufs you can create a 'virtual' directory out of several combined directories. You can configure whether writes should propagate back to the original directories, or use a copy-on-write method to leave the originals alone, or disallow writes altogether. However to answer your ...


6

Use rsync's --ignore-existing flag. From man rsync: --ignore-existing skip updating files that exist on receiver To do this between two distinct servers you'd do something like this: rsync --ignore-existing -avz -e ssh source remoteuser@remoteserver:destination If you are truly going to be running this "all the time" (ie. continuously) like you ...


5

There are numerous options for programs or even file systems that handle synchronization. I still use the ridiculously old unison program to keep some of my home directories in sync. There are other programs similar to this as well. For easier situations that only require one way coping, rsync does the job nicely. For cross platform synchronization, the ...


5

Using rsync --exclude is the more obvious choice here, but if you really want to just send a few files or folders and have something specific to exclude, you can use shell globing with scp. First make sure you have the right globing options set in your shell. For bash run shopt -s extglob and for zsh use setopt ksh_glob. Then something like this: scp ...


5

Use rsync, and specify --ignore-existing if you do not want to update existing files: rsync -e ssh -av --ignore-existing /my/source/directory jim@foreign.machine.egg:/data/ A popular additional option I usually add is --progress so you see what's going on. If you expect interruptions, also say --partial, but I'm not sure how "partial" and ...


5

Why not use rsync instead? rsync -a /branch2/media/ /branch1/media/ The reason why mv can't move /branch2/media/cd/ to /branch1/media is because /branch1/media already has a cd/ in it. mv refuses to clobber non-empty directories.


5

wget is a great tool. Use wget -m http://somesite.com/directory -m --mirror Turn on options suitable for mirroring. This option turns on recursion and time-stamping, sets infinite recursion depth and keeps FTP directory listings. It is currently equivalent to -r -N -l inf --no-remove-listing.


5

Like rsync, but use zsync to get from an httpd server.



Only top voted, non community-wiki answers of a minimum length are eligible