Sync a folder between 2 computers, with a filesystem watcher so that each time a file is modified, it is immediately replicated

Question

I have:

• a Linux server that I connect via SSH on IP 203.0.113.0 port 1234

• a home computer (behind a router), public IP 198.51.100.17, which is either Debian or Windows+Cygwin

What's the easiest to have a folder /home/inprogress/ synchronized (in both directions), a bit like rsync, but with a filesystem watcher, so that each time a file is modified, it is immediately replicated on the other side? (i.e. no need to manually call a sync program)

I'm looking for a command-line / no-GUI solution, as the server is headless.

Is there a Linux/Debian built-in solution?

Answer 1

Following @Kusalananda's comment, I finally spent a few hours testing Syncthing for this use case and it works great. It automatically detects changes on both sides and the replication is very fast.

Example: imagine you're working locally on server.py in your favorite Notepad software, you hit CTRL+S (Save). A few seconds later it's automatically replicated on the distant server (without any popup dialog).

One great thing I've noticed is that you don't have to think about the IP of the home computer and server with Syncthing: each "device" (computer, server, phone, etc.) has a unique DeviceID and if you share the ID with another device, it will find out automatically how they should connect to each other.

To do:

• Home computer side (Windows or Linux):

  Use the normal Syncthing in-browser configuration tool

• VPS side:

  First connect the VPS with a port forwarding:

  ssh <user>@<VPS_IP> -L 8385:localhost:8384
  

  The latter option will redirect the VPS's Syncthing web-configuration tool listening on port 8384 to the home computer's port 8385.

  Then run this on VPS:

  wget https://github.com/syncthing/syncthing/releases/download/v0.14.52/syncthing-linux-amd64-v0.14.52.tar.gz 
  tar xvfz syncthing-linux-amd64-v0.14.52.tar.gz
  nohup syncthing-linux-amd64-v0.14.52/syncthing &
  

  Then on the home computer's browser, open http://localhost:8385 : this will be the VPS's Syncthing configuration!

---

Other solution I tried:

• SSHFS using this tutorial. Please note that in this tutorial they don't use sshfs-win but win-sshfs instead (these are two different projects). I tried both, and I couldn't make any of them work (probably a problem with my VPS configuration).

• Here is an interesting reference too: https://softwarerecs.stackexchange.com/questions/13875/windows-sshfs-sftp-mounting-clients

---

Additional advantages of Syncthing I've just noticed:

• you can reduce fsWatcherDelayS in the config.xml from 10 to 2 seconds so that after doing CTRL+S, 2 seconds later (+the time to upload, i.e. less than 1 second for a small text file) it's on the other computer

• if you sync two computers which are in the same local network (by just giving the DeviceID to each other, no need to care about local IP addresses), it will automatically notice that it doesn't need to transit via internet, but it can deal locally. This is great and allows a very fast speed transfer (4 MB/s!) sync of phone <--> computer both connected to the same home router via WiFi... ...whereas it would be stuck at 100 KB/s on ADSL with a Dropbox sync! (my ADSL is limited at 100 KB/s on upload)

Answer 2

Sounds like Unison should do the job.

Unison is a file-synchronization tool for OSX, Unix, and Windows. It allows two replicas of a collection of files and directories to be stored on different hosts (or different disks on the same host), modified separately, and then brought up to date by propagating the changes in each replica to the other.

It does have an optional GUI that can make resolving conflicts somewhat easier, but everything can also be done using a text-based user interface. You can also predefine how to resolve conflicts for fully unattended operation.

There's a file watcher (fsmonitor) component to trigger a sync whenever needed. Search for "repeat watch" in the manual for details.

Looks like Debian has the right version (2.48+) packaged out of the box.