1

I have a folder structure that I want to copy using rsync, in which the structure is a series of backups from which I only want to copy the latest (which can be identified using a symbolic link), like so:

/backups
    /foo
        /2019-05-01
        /2019-06-01
        /latest -> 2019-06-01
    /bar
        /2019-05-20
        /2019-06-20
        /latest -> 2019-06-20

And so-on; what I would like to do is, using a single rsync command, copy only the latest backup from each folder, while otherwise mimicking the structure. To this end I came up with the following command:

rsync -rptgoDLm --include '*/' --include '/*/latest/**' --exclude '*' user@remote:/backups /some/local/path

Basically an archive with --copy-links and some include/exclude trickery to select only the /*/*/latest branches (plus -m to avoid a bunch of empty directory structures). This works fine except for one problem; if any of these branches contains symbolic links, then these are also copied as their targets, rather than just as plain symbolic links.

What I really need to be able to do is only use --copy-links behaviour with the latest symbolic links, while using another behaviour (e.g- that of --links) for any others that are encountered.

Is such a thing a possible with a single rsync command? My aim is to be able to run this without having to know what the immediate contents of /backups is, so that if I add anything new (e.g- /backups/baz) then it will copy automatically.

Update: To clarify, the resulting directory structure I'd like to see on the destination would be:

/backups
    /foo
        /latest
    /bar
        /latest

i.e- I only need the latest version of each backup on the target (the target itself will handle the backup history in another way).

|improve this question
  • Will your destination always have copies of the previous directories, i.e. foo/2019-05-01 and bar/2019-05-20 in your example? – roaima Jun 1 at 10:07
  • Can you diagram the resulting directory structure you want in /some/local/path? – Jim L. Jun 1 at 13:42
  • @JimL. I've updated the question to reflect the intended destination structure; basically I want to treat the latest symbolic links as directories and copy only them to the destination, without resolving any other links contained within. – Haravikk Jun 2 at 16:31
  • @roaima I've updated the question to give a sample of the intended structure I want on the destination; basically I want to treat the latest symbolic links as directories and copy only them to the destination, but without resolving any symbolic links contained within. – Haravikk Jun 2 at 16:34
4

This seems to fit your requirements

rsync -avR user@remote:'/backups/*/latest/' /some/local/path/

The -R flag effectively "copies" the entire source path into the destination and the resulting backups will land as /some/local/path/backups/{whatever}/latest.

If you find you don't want the entire source path you can include /./ to indicate the point from which the path should be copied across to the destination. For example, /backups/./foo/latest/ would result in /some/local/path/foo/latest/ (i.e. the /backups component has been skipped). There's lots more detail in the documentation (man rsync).

|improve this answer
  • I need an additional / after latest, but it works like a charm! – Freddy Jun 2 at 20:42
  • Aha! This does the trick; like Freddy I had to add a trailing slash (so /backups/*/latest/), I also found in the documentation for -R that I can control the path information with ./, so for example can do /backups/./*/latest/ to only get the structure starting from the wildcard. – Haravikk Jun 4 at 8:21
  • @Freddy ah yes, I'd missed off the -k flag, but appending a trailing slash would work even better. I've fixed my answer, thank you. – roaima Jun 4 at 8:40
  • @Haravikk yes that's right, but I didn't mention the /./ feature because your target path in your question seemed to match the source and I didn't want to add unnecessary complexity. I've updated my answer for future reference. – roaima Jun 4 at 8:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.