For example, if I have a java process that (once every 2 hours) takes a backup of a large file, like 500mb, and places it into folder A on HDD A, and I want to schedule a cron script to copy the entire folder A to HDD B once per day... what happens if the cron script happens to fire while the java process is in the middle of writing to folder A? Will it wait? Will it just ignore the file that isn't done being written yet?


In general, Unix commands treat files the same, whether they are open and being written to or not. So if a cron job is using cp to copy a directory, what gets copied depends solely on what is in the directory at the time the cp command examines the directory. If a file is only partially written at the time cp visits it, a partially written copy will be produced. If you want to prevent such occurrences, you need to implement a locking scheme amongst cooperating processes to prevent concurrent access to the directory.

  • +1. But if you're trying to make a backup, use rsync instead. You can use it to perform incremental copying. – James Youngman May 9 '12 at 22:34
  • You do not need a locking scheme. You need an atomic update scheme. What I did for a similar problem is to copy new files into a tmp subdirectory then move them into place after the copy is complete. If you copy subdirectories then use a directory at the same level. Just make sure it is on the same filesystem. – Zan Lynx Nov 19 '12 at 18:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.