1

I have been working on this for several days and no matter what I do I can't get it to work. I've read practically every Q&A on StackExchange network about rsync but still not found the answer.

Basically I am writing a website deploy tool and it uses rsync to move files from a source Git repo to deploy Git repo; the destination. Committing to the deploy repo actually deploys the code to the web server so I am delaying with rsync between two different directories.

It is very important that any files that the developer deleted in the source repo also get deleted in the deploy repo. However, there are selected files that are in the deploy repo that need to not be deleted Specifically this are files that the hosting provider provides and are needed in production but that are problematic to maintain in the source repo.

I understand from what I have read in man, here, and elsewhere that if I add the names of the files that are in the deploy/destination repo that I want to keep to an --exclude-from file

Here is what I am doing, basically:

rsync --archive --verbose \
      --exclude ".git" --exclude ".git*" \
      --exclude-from="/tmp/excludes.txt" --delete-after \
      /dir/to/source/repo/ \
      /dir/to/deploy/repo

And for example purposes let's say this is the contents of /tmp/excludes.txt:

/foo/bar/baz.php

Now when I have tried this using a trivial example locally it always seems to work as expected. But when I use it with CircleCI in a real-world deploy scenario, rsync always deletes /foo/bar/baz.php after running the above command.

Can you see what I am doing wrong? Thanks in advance.

UPDATE - THE ABOVE DOES WORK

Turns out the logic in my script running in CircleCI was at fault, and I just could not see it. I was running multiple rsync commands with different roots (essentially subdirectories of my main root) because of different rules needed, and one list of exclude files with filepaths that were all based on the main root. Ugh.

Anyway dear reader, the upshot is that the above example indeed works as I describing how I wanted it to work.

  • Is /foo/bar/baz.php actually /dir/to/deploy/repo/foo/bar/baz.php? If not, what is the relationship between the target directory and this exclusion? – roaima Sep 2 '18 at 18:52
  • @roaima Yes, your assumption was correct; /foo/bar/baz.php is actually /dir/to/deploy/repo/foo/bar/baz.php. Wow, you are active here. I was just reading your answer on another question. – MikeSchinkel Sep 2 '18 at 18:54
  • 1
    @roaima Okay, not this is really, really F-ing weird. I added --backup and --backup-dir I learned about from one of your answers to see if I could look for the files there, and now it is working as expected. Does that many any sense?!?!?!? – MikeSchinkel Sep 2 '18 at 18:59
  • The rsync command in the question should not delete files on the target at all since you don't use --delete nor --delete-excluded. Du you have rsync set up as an alias for something? – Kusalananda Sep 2 '18 at 19:14
  • 1
    Kusalananda the --delete-after implies --delete. – roaima Sep 2 '18 at 21:54
0

Just to close the loop, if anyone reading this wants to know how to use rsync to mirror the destination directory with the source directory except for specific named files this is how you would do it:

rsync --archive --verbose \
      --exclude-from="/tmp/excludes.txt" --delete-after \
      /dir/to/source/repo/ \
      /dir/to/deploy/repo

This is the contents of /tmp/excludes.txt where the named file within has a full filepath of /dir/to/deploy/repo/foo/bar/baz.php:

/foo/bar/baz.php

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.