20

I have:

  • a Linux server that I connect via SSH on IP 203.0.113.0 port 1234

  • a home computer (behind a router), public IP 198.51.100.17, which is either Debian or Windows+Cygwin

What's the easiest to have a folder /home/inprogress/ synchronized (in both directions), a bit like rsync, but with a filesystem watcher, so that each time a file is modified, it is immediately replicated on the other side? (i.e. no need to manually call a sync program)

I'm looking for a command-line / no-GUI solution, as the server is headless.

Is there a Linux/Debian built-in solution?

|improve this question|||||
  • 7
    You are describing syncthing. – Kusalananda♦ Nov 27 '18 at 12:06
  • There's lsync, but I don't know if it works usefully for bidirectional sync. – Ulrich Schwarz Nov 27 '18 at 12:07
  • lsync, csync2, inotify+rsync, but I would prefer using them in a local network setting. – Rui F Ribeiro Nov 27 '18 at 12:11
  • 2
    At the filesystem level, this sounds like OCFS2, or even RAID 1 over NBD – roaima Nov 27 '18 at 12:17
  • 3
    One-way sync is easy. Bi-directional sync implies conflict resolution (yes, it WILL happen at some point), which in turns means some kind of UI (though not necessary a GUI). – jcaron Nov 27 '18 at 15:49
23

Following @Kusalananda's comment, I finally spent a few hours testing Syncthing for this use case and it works great. It automatically detects changes on both sides and the replication is very fast.

Example: imagine you're working locally on server.py in your favorite Notepad software, you hit CTRL+S (Save). A few seconds later it's automatically replicated on the distant server (without any popup dialog).

One great thing I've noticed is that you don't have to think about the IP of the home computer and server with Syncthing: each "device" (computer, server, phone, etc.) has a unique DeviceID and if you share the ID with another device, it will find out automatically how they should connect to each other.

To do:

  • Home computer side (Windows or Linux):

    Use the normal Syncthing in-browser configuration tool

  • VPS side:

    First connect the VPS with a port forwarding:

    ssh <user>@<VPS_IP> -L 8385:localhost:8384

    The latter option will redirect the VPS's Syncthing web-configuration tool listening on port 8384 to the home computer's port 8385.

    Then run this on VPS:

    wget https://github.com/syncthing/syncthing/releases/download/v0.14.52/syncthing-linux-amd64-v0.14.52.tar.gz 
tar xvfz syncthing-linux-amd64-v0.14.52.tar.gz
nohup syncthing-linux-amd64-v0.14.52/syncthing &

    Then on the home computer's browser, open http://localhost:8385 : this will be the VPS's Syncthing configuration!

Other solution I tried:

Additional advantages of Syncthing I've just noticed:

  • you can reduce fsWatcherDelayS in the config.xml from 10 to 2 seconds so that after doing CTRL+S, 2 seconds later (+the time to upload, i.e. less than 1 second for a small text file) it's on the other computer

  • if you sync two computers which are in the same local network (by just giving the DeviceID to each other, no need to care about local IP addresses), it will automatically notice that it doesn't need to transit via internet, but it can deal locally. This is great and allows a very fast speed transfer (4 MB/s!) sync of phone <--> computer both connected to the same home router via WiFi... ...whereas it would be stuck at 100 KB/s on ADSL with a Dropbox sync! (my ADSL is limited at 100 KB/s on upload)

|improve this answer|||||
  • @sudodus A friend already recommended it to me a few weeks ago, and I had tried it for computer <-> phone sync, but I didn't imagine it would work so well for a development server too! Edit your code, hit save, 1 2 3 it's on the other computer! – Basj Nov 27 '18 at 16:26
  • 1
    +1. I had the same problem for quite a while, used rsync + some scripts for years, and finally tried syncthing, csync, unison, and a few others 2 months ago. Works great, with any number of computers, and even when some of them don't have a permanent network connection. Can recommend 100%. – Guntram Blohm supports Monica Nov 27 '18 at 19:41
  • 1
    Great that you have tested all these different solutions @GuntramBlohm! If you have a few minutes to post an answer to give your feedback / comparison between syncthing, rsync, csync, unison, etc. the pros / cons for each, it would be super interesting for future reference! – Basj Nov 27 '18 at 19:54
16

Sounds like Unison should do the job.

Unison is a file-synchronization tool for OSX, Unix, and Windows. It allows two replicas of a collection of files and directories to be stored on different hosts (or different disks on the same host), modified separately, and then brought up to date by propagating the changes in each replica to the other.

It does have an optional GUI that can make resolving conflicts somewhat easier, but everything can also be done using a text-based user interface. You can also predefine how to resolve conflicts for fully unattended operation.

There's a file watcher (fsmonitor) component to trigger a sync whenever needed. Search for "repeat watch" in the manual for details.

Looks like Debian has the right version (2.48+) packaged out of the box.

|improve this answer|||||
  • Thanks! Is the file watcher fsmonitor included out of the box with Unison, or do we have to install this tool and connect it with Unison manually? – Basj Nov 27 '18 at 12:31
  • 1
    @Basj It's included in recent enough versions and it's really trivial to set up. I have added a link to the manual. – TooTea Nov 27 '18 at 12:35
  • I used it in the past in an old Mac. It just works with minimum hassle. No idea if it is appropriate for server scenarios though. – Rui F Ribeiro Nov 27 '18 at 14:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.