2

I accidentally launched 32 processes (one process per core on my virtual machine) to mv some files between two ext4 filesystems, but only some of them were successfully mv'd:

for i in `seq 1 32`; do
    mv /path/to/ext4-1/pattern* /path/to/ext4-2/ &
done

What exactly happens when multiple processes try to mv the same files?

3

mv within the same filesystem is atomic for each file, but across filesystem, it's a much more complex operation (assuming that you're only moving regular files and that the destinations don't exist initially):

  1. Remove the destination file if it exists.
  2. Open the source file for reading. Some implementations may perform this before the previous step.
  3. Create the destination file. Some implementations may truncate it if it exists at that point. Some implementations may perform this before the previous step.
  4. Read some data from the source file and write it to the destination file.
  5. Replicate the metadata of the source file on the destination file. Some implementations may perform part or all of this after the next step.
  6. Close the source file and the destination file.
  7. Remove the source file.

For the most part, if two mv processes attempt to move the same file at the same time, they'll both copy the data: the instance first to start will create a file, the second instance will delete that file and create a new one. However, if you're unlucky, it is possible to lose data. For example:

  1. Instance 1 performs the copy and closes the two files (steps 1–6).
  2. At that point, instance 2 reaches this file and removes the destination (step 1), but is preempted before it can open the source file.
  3. Instance 1 proceeds to remove the source file (step 7).
  4. Instance 2 goes on to step 2, but the file no longer exists.

I think that if a file is present at the destination, it will have the correct content. But as we've seen, it is possible for a file to disappear.

  • @roaima POSIX mandates that it's deleted. I haven't reviewed all Unix variants out there but I expect they're compliant on that point. That's necessary, for example, if the destination file already exists and belongs to a different user. – Gilles 'SO- stop being evil' Dec 6 '15 at 14:32
  • Thank you for the clarification. It's always good to learn something new. [I'll delete my comment shortly, unless you get there first. Regards.] – roaima Dec 7 '15 at 15:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.