1

The following is problem of file management that I would like to sort out in Linux. It's not exactly a problem of version control, rather one of synchronisation between computers.

Situation

  • There are three computers (say A, B. C).

  • These computers are not always on at the same time

  • These computers can be quickly connected one to another using the NFS protocol

  • Normally but not strictly, the computers are used with different aims in mind.

  • I have categorized a large number of files (binaries and plain text) in three directory trees: dir-A, dir-B, dir-C. As a starting point, these three trees are in each of the three computers. More in details

    • dir-A contains the documents that I usually handle when using on computer A;
    • dir-A also gets backed up on computer A only;
    • The version of dir-A in computer A should be the 'master copy'. The namesake copies in the other computers are there for the sake of convenience and redundancy.
    • Likewise, for the other two pairs of directories and computers.

Incidentally, the document categorization is pretty fine because I can retrieve directory and files from the command line swiftly. So I would like to stick to command line and scripts, rather than resorting to packaged software for managing libraries.

Intention

The picture gets more complicated because sometimes, when working on computer A while B and C are off,

  1. I receive new documents fitting the categories in dir-B and dir-C, so that I store them in the local copies of them;
  2. I might fitfully rearrange the content of dir-B and dir-C locally to improve the categorization. This will boil down to creating, removing, renaming, merging subdirectories, moving around files, deleting clutter, and so on and so forth.

However, the intention is that the 'master copy' of dir-B (dir-C) has to live in computer B (C). So, at a chosen moment, the modifications done on computer A need to be carried over to computer B (C).

Finally, this should work also the other way round, when I edit convenience copies of dir-A in B and C.

Question

I am thinking of the optimal strategy to have maximum flexibility with the given constraints. Task 1 is a simple synchronization problem (I'd use rsync), but task 2 is more complex.

Hence, I wonder

  • Are there tools/utilities in Linux that I could use to implement the both task 1 and 2?
  • How would you solve this?
  • Can the problem be actually solved?
  • Perhaps should I drop any constraint to make it workable?
0

This is not entirely a Unix/Linux problem, you can do it with any level of sophistication on any OS.

Lemme restate your problem:

  • {1} You have N computers that may or may not be connected to each other in any given moment, generally considered to comprise 1 master and several slaves
  • {2} The resources on those computers are de-synchronized over time, renamed etc.
  • {3} You work on one computer at a time
  • {4} You want to re-synchronize the computers

Solution:

{1} implies that you need a batch reconciliation job running at fixed intervals when at least two of the computers are connected.

{2} means that you have to track renamings, re-categorizations, etc. to be able to find out matching resources

{3} leads one to conclude that the authoritative record is found by comparing modification times across computers.

  • Key prerequisite for being able to compare times for the transactions to find the latest, authoritative record is having accurate clocks on all N computers. NTP is your friend here.

  • Modification record is provided by the inotify mechanism.

  • Decision on what to move where can be taken after identifying conflicting records in the N lists of files. I recommend GNU datamash as an early candidate for this task.

  • Moving files around is the easiest task. rsync is versatile enough.

Finally, since I'm not paid for this, I'll leave the mundane task of writing the scripts as an exercise for the reader.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.