2

I have a folder structure that I want to copy using rsync, in which the structure is a series of backups from which I only want to copy the latest (which can be identified using a symbolic link), like so:

/backups
    /foo
        /2019-05-01
        /2019-06-01
        /latest -> 2019-06-01
    /bar
        /2019-05-20
        /2019-06-20
        /latest -> 2019-06-20

And so-on; what I would like to do is, using a single rsync command, copy only the latest backup from each folder, while otherwise mimicking the structure. To this end I came up with the following command:

rsync -rptgoDLm --include '*/' --include '/*/latest/**' --exclude '*' user@remote:/backups /some/local/path

Basically an archive with --copy-links and some include/exclude trickery to select only the /*/*/latest branches (plus -m to avoid a bunch of empty directory structures). This works fine except for one problem; if any of these branches contains symbolic links, then these are also copied as their targets, rather than just as plain symbolic links.

What I really need to be able to do is only use --copy-links behaviour with the latest symbolic links, while using another behaviour (e.g- that of --links) for any others that are encountered.

Is such a thing a possible with a single rsync command? My aim is to be able to run this without having to know what the immediate contents of /backups is, so that if I add anything new (e.g- /backups/baz) then it will copy automatically.

Update: To clarify, the resulting directory structure I'd like to see on the destination would be:

/backups
    /foo
        /latest
    /bar
        /latest

i.e- I only need the latest version of each backup on the target (the target itself will handle the backup history in another way).

4
  • Will your destination always have copies of the previous directories, i.e. foo/2019-05-01 and bar/2019-05-20 in your example? Commented Jun 1, 2019 at 10:07
  • Can you diagram the resulting directory structure you want in /some/local/path?
    – Jim L.
    Commented Jun 1, 2019 at 13:42
  • @JimL. I've updated the question to reflect the intended destination structure; basically I want to treat the latest symbolic links as directories and copy only them to the destination, without resolving any other links contained within.
    – Haravikk
    Commented Jun 2, 2019 at 16:31
  • @roaima I've updated the question to give a sample of the intended structure I want on the destination; basically I want to treat the latest symbolic links as directories and copy only them to the destination, but without resolving any symbolic links contained within.
    – Haravikk
    Commented Jun 2, 2019 at 16:34

1 Answer 1

5

This seems to fit your requirements

rsync -avR user@remote:'/backups/*/latest/' /some/local/path/

The -R flag effectively "copies" the entire source path into the destination and the resulting backups will land as /some/local/path/backups/{whatever}/latest.

If you find you don't want the entire source path you can include /./ to indicate the point from which the path should be copied across to the destination. For example, /backups/./foo/latest/ would result in /some/local/path/foo/latest/ (i.e. the /backups component has been skipped). There's lots more detail in the documentation (man rsync).

4
  • I need an additional / after latest, but it works like a charm!
    – Freddy
    Commented Jun 2, 2019 at 20:42
  • Aha! This does the trick; like Freddy I had to add a trailing slash (so /backups/*/latest/), I also found in the documentation for -R that I can control the path information with ./, so for example can do /backups/./*/latest/ to only get the structure starting from the wildcard.
    – Haravikk
    Commented Jun 4, 2019 at 8:21
  • @Freddy ah yes, I'd missed off the -k flag, but appending a trailing slash would work even better. I've fixed my answer, thank you. Commented Jun 4, 2019 at 8:40
  • @Haravikk yes that's right, but I didn't mention the /./ feature because your target path in your question seemed to match the source and I didn't want to add unnecessary complexity. I've updated my answer for future reference. Commented Jun 4, 2019 at 8:41

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .