0

I'm using Java's Files.walkFileTree to scan a particular directory periodically for *.csv files and then reading them.

I know that mv is atomic ( at least within the same device ) so 'mv'ing a file shouldn't be a problem in terms of race condition.

If I do a cp instead, does cp first do a copy to a different filename, and then do a atomic mv to the original name ? Or will it cause a race condition by doing a direct read-write ?

I have the same question for extracting files to another directory using tar & gz .

PS. In case the answer is OS-dependent, I am working on CentOS 6. Also if possible can the answer be mentioned for Windows too ?

  • If you're concerned about reading a file that is still being written to, you could check the timestamp and ignore files less than X minutes old. Or repeatedly check the file size every few seconds to see when it stops changing. On linux, the best option would probably be to use inotify (there are java libraries for it) to tell you when a directory's contents change and when files in that directory are closed - i.e. wait for notification rather than periodic polling. – cas Jul 26 '17 at 1:42
2

For GNU cp and GNU tar (the standard implementations on most Linux distros, including CentOS), they both create the file in-place. I'm pretty sure the same is true of most other implementations as well. It's not too hard to write a wrapper script to make it functionally atomic though. I'm not sure about Windows, but I don't think that it provides any more guarantees that POSIX does for filesystem operation atomicity.

It's also worth noting also that mv is only atomic if it doesn't cross a filesystem boundary, otherwise it becomes equivalent to running cp and then unlinking the source file.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.