0

I am using Unison to synchronize files between several clients. Each client is identical, meaning that whenever one client updates a certain file, all other clients must be updated consequently.

The files are stored in a centralized cloud server. Each client has non-root SSH access to the centralized cloud server. There is no link between the clients.

It's important that ownership of the files is preserved. For this reason, I am using --rsync-path="rsync --fake-super" below. This stores the owner/group in the extended file attributes, so ownership on the client can be restored during synchronization afterwards. That said, if there is a better method to preserve ownership, feel free to let me know, as this might also eliminate the problem below.

A relevant snippet from the configuration is as following:

copythreshold = 0
copyprog      = /usr/bin/rsync -avzX --rsync-path="rsync --fake-super" --inplace -e ssh
copyprogrest  = /usr/bin/rsync -avzX --rsync-path="rsync --fake-super" --inplace --partial -e ssh

I observe the following behavior:

  1. When a file is created, rsync as configured in copyprog is used to transfer the files.

    This is great, because now the newly created file has the user.rsync.%stat attribute set (which holds the owner/group) on the cloud server. A consecutive synchronization on the other clients will indeed preserve the ownership.

  2. However, when the file is updated, rsync as configured in copyprog is not used. I believe Unison does some custom built-in transfer logic instead.

    This is not so great, because now the user.rsync.%stat attribute is lost on the cloud server. A consecutive synchronization on the other clients will now loose the owernship.

Is it possible to configure Unison such that copyprog is also always used for updates? The documentation mentions:

If you set copythreshold to 0, Unison will use the external copy utility for all whole-file transfers.

Unfortunately, nothing is mentioned about updates.

Improve this question
3
  • Did you try using unison -debug all to get more information on what is being done? – meuh Sep 11 '18 at 14:26
  • @meuh Yes, that's how I debugged the behavior as described in the question. For a new file, I noticed that rsync was used, while for an updated file, rsync was not used. Should I add that debug output to the question? – pbillen Sep 11 '18 at 15:01
  • If you can reduce the output to just a few relevant lines, please add it. I see unison is written in the OCaml language, so it might be difficult to understand what it is doing. – meuh Sep 11 '18 at 15:04
1

I reproduced the problem you see, and looked through the very verbose debug output, and the OCaml copy.ml code (I am not familiar with this language) but did not see any obvious reason for why it could not use the copyprog setting.

However, this issue from February 2017 says that copyprog only works for new files (copyprogrest is for continuing an interrupted copyprog). A fix for that would probably also solve your problem. It is now marked as an enhancement. You might like to post a new issue there giving your use case.

Improve this answer
2
  • Thanks a lot for taking your time to reproduce this. I was looking to that same code, but am having troubles understanding OCaml as well. I'll create a post on the github repo and refer to this question. Thanks! – pbillen Sep 11 '18 at 19:44
  • It can be found here: github.com/bcpierce00/unison/issues/219 – pbillen Sep 11 '18 at 19:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.