0

We need to periodically archive some big files older than 2 days to a NAS while keeping their directory tree structure. Those files are kept for 7 days in the source directory.

At first we used find for this:

find ${SOURCE_DIR} -type f -mtime +2 -exec ksh -c 'mkdir -p $(dirname ${DEST_NAS_DIR}$0) && cp -p $0 ${DEST_NAS_DIR}$0' {} \;

However we noticed that the script is copying already archived files, thus each execution takes too much time.

cp doesn't have the -n / --no-clobber option. So, how can I avoid overwriting the same files in destination? any idea?

Regards!

6
  • Can you set the files to be non-writeable?
    – terdon
    Jul 1 at 12:29
  • 1
    use rsync instead of find+cp
    – White Owl
    Jul 1 at 12:35
  • @WhiteOwl Note that rsync can't filter by timestamp metadata.
    – Kusalananda
    Jul 1 at 12:38
  • 1
    @Kusalananda, Really? That is a news to me. Then the my eyes deceives me, then I see how rsync is copying just the updated files?
    – White Owl
    Jul 1 at 12:53
  • @WhiteOwl Sure, but how would you tell rsync to only copy files that have been modified more than two days ago, which is what the user appears to want to do? What rsync can't do is what find does with the -mtime test.
    – Kusalananda
    Jul 1 at 13:19

2 Answers 2

2

You can try to cheat with something like:

...yes n|cp -i -p $0 ${DEST_NAS_DIR}$0' {}...

Unfortunately I do not have AIX to test it but this will ask for overwrite any existing file and sending "n" will prevent it.

P.S. I do not have currently where to test it so please do it before work with real files.

2
  • Shouldn't we double-quote the variables?
    – Jeff Schaller
    Jul 1 at 13:55
  • @JeffSchaller, correct. In my post I just provide a idea how to avoid files overwrite :) Jul 1 at 14:10
2

If rsync is available on your system, you may use its --ignore-existing flag:

find ${SOURCE_DIR} -type f -mtime +2 \
  -exec rsync --ignore-existing '{}' ${DEST_NAS_DIR} \;

Possibly the -u flag might be interesting - it would check if the sender has newer versions of existing files, too, and update them if so.

See if you want --archive mode activated: it means to be recursive and preserve several information regarding times, ownership and more. Check man rsync for more details.

1
  • 1
    And see here for a method how to minimize number of rsync program calls.
    – FelixJN
    Jul 1 at 18:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.