I know what you are thinking right now "Just use rsync" but unfortunately this system does not have rsync, and I would like to come up with another solution.
- The source Server has a Upload folder containing 140 000+ Files (No Sub Directories)
- The destination Server needs the Upload folder containing 140 000+ Files
Let's call it Migrating with only Post-It Notes... Now to complicate matters, Server 1 keeps getting new files every day, due to new Uploads or Thumbnails being generated, so that idea in your head right now "just copy the file from 1 to 2" will not work, as once I am done with the Copy of about 20+GB I can start again, as there is already new file on the source Server...
My Solution Idea
- Copy the complete folder from source to destination
- Find the latest create date in the destination Server and use as starting point on the source Server
- Copy all files since the last create date from the source to the destination (delta)
- Setup a cron to do this as often as possible.
find /uploads/* -mtime -1
bash: /bin/find: Argument list too long
Now before I start writing a bash script loop , I was wondering if there is someone out there that could suggest another way of doing this without a bash script... good old low level CLI.