I've been getting this strange error message in one of my build scripts - cp fails, returning the error "File exists". I've even tried using cp -f, which should overwrite the file if it exists, but the error still shows up. Running cp to overwrite existing files works perfectly when I do it manually. What could cause this error?

up vote 20 down vote accepted

This turned out to be caused by a race condition. cp checks if the destination file already exists, and if not - overwrites it. The problem was happening because this cp command was being run twice in parallel, which caused the file in question to sometimes appear after checking whether it exists, but before the attempt to create the file. The strace output looks like this:

# Command was "cp a b"
stat("b", 0x7fff89510620)               = -1 ENOENT (No such file or directory)
stat("a", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
stat("b", 0x7fff895103a0)               = -1 ENOENT (No such file or directory)
# File b will be created at this point in time
open("a", O_RDONLY)                     = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
open("b", O_WRONLY|O_CREAT|O_EXCL, 0644) = -1 EEXIST (File exists)

Here's some bash code used to catch this:

#!/bin/bash

touch a

f() {
  while true; do
    rm -f b
    strace -o /tmp/cp${BASHPID}.trace cp a b || break
  done
}

cleanup() {
  kill -9 %1 %2
}

f &
f &

trap cleanup exit

wait

This same error can occur with mkdir -p or any other action that attempts to overwrite a file. Using flock can help avoid race conditions in cases like this.

Your Answer

 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.