1

I've got this script which copies files listed in a csv recursively from a main folder. The copying aspect of the script works perfectly. However, I now want to add some error detection functionality. The "if" loop I have added below doesn't work as I would like. I have added a fake file to my csv list for testing purposes, and the script reports that all files have copied successfully, even though one of the files doesn't exist. Could someone suggest how I can get the script to report the file(s) it has not successfully found/copied from the csv and specify the file names? EDIT ** As there are 4000+ files listed in the CSV, I would like the script to print a summary of any copy failures at the end of the script, rather than for each files copied during the process. This would make it easier to identify/summarize any issues.

while IFS=, read -r file rest
do
find $sourcefolder -name "${file}" -exec cp '{}' $destfolder \;
done < $csv
if [ $? -eq 0 ]
then
echo "All files from CSV copied succesfully"
else "File from CSV not copied as not found"
fi
1

The error code returned by find has nothing to do with whether any files were found. It will only return a failure code if the command itself failed to run. What you are looking for is something like:

while IFS=, read -r file rest
do
  find $sourcefolder -name "${file}" -exec cp '{}' $destfolder \;
  ## Check if the file was copied
  [[ -e "$destfolder/$file" ]] || echo "File $file was not copied"
done < "$csv"
echo "Finished"

That way, the script will tell you about any files that weren't successfully copied.

If you only want to print the error at the end of the script, you can save the bad file names in a variable and print it after the loop has finished:

while IFS=, read -r file rest
do
  find "$sourcefolder" -name "$file" -exec cp '{}' "$destfolder" \;
  ## Check if the file was copied
  [[ -e "$destfolder/$file" ]] || badfiles=$(printf "%s\n%s" "$file" "$badfiles")
done < "$csv"

## Check if all the files were correctly copied
if [[ -z "$badfiles" ]]; then
        echo "All files were succesfuly copied."
else
        printf 'Some files were not copied:\n%s\n' "$badfiles"
fi

You can also extend this to avoid overwriting files with the same name:

while IFS=, read -r file rest
do
  find "$sourcefolder" -name "$file" -print0 |
    while IFS= read -d '' -r filepath; do
     c="";
     while [[ -e "$destfolder/$file$c" ]]; do
       ((c++))
     done
     cp "$filepath" "$destfolder/$file$c"
  done
  ## Check if the file was copied
  [[ -e "$destfolder/$file" ]] || badfiles=$(printf "%s\n%s" "$file" "$badfiles")
done < "$csv"

## Check if all the files were correctly copied
if [[ -z "$badfiles" ]]; then
        echo "All files were succesfuly copied."
else
        printf 'Some files were not copied:\n%s\n' "$badfiles"
fi

This will fail if the file names in your csv can contain newlines. If that is a possibility, change the first while loop to:

while IFS= read -d"," -r file rest; do ...

Note, however, that this will fail if any line of the csv contains nothing but a file name (so no comma). You can't have it both ways, if you can have lines with no commas use the 1st version and if you can have file names with newlines, use the second.

  • Thanks terdon, the only issue with this is that there are 4000+files and as the error is printed after each copy, it would be difficult for me to monitor if one of the files was not located. I will update my question to include the fact that I would like a fail summary at the end, rather than after each copy action. – neilH Feb 29 '16 at 13:59
  • @bms9nmh OK, that's easy enough. Give me a second to edit. Also note that I had an bug in the original version and the message was printed for files that were copied, not those that weren't. And is having the same file name a potential problem? – terdon Feb 29 '16 at 14:00
  • ok thanks for that i'll wait and see your edit, I'm not 100% sure about the same name issue, its definitely a possibility though now you mention it. – neilH Feb 29 '16 at 14:05
  • @bms9nmh OK, the simplest solution would be to use cp -i (assuming you're on a Linux system or have access to GNU cp) which will prompt you before overwriting. That, however, means that you will have to interact with the script. If that's an issue, I can make it so that it will move file to file1, then file2 etc. What do you prefer? – terdon Feb 29 '16 at 14:09
  • @bms9nmh I went with the version that increments the file name. – terdon Feb 29 '16 at 14:18
1
  1. Your loop is incorrect, since you do nothing with rest

  2. As terdon suggested, check after each copy, instead of once after loop

  3. Negate the existence of file check, for correct comment
  4. And heed terdon's warning, if you have same named files in your source tree, you will only get one copy in destination.

Try this,

while read -d, -r file
do
   find $sourcefolder -name "${file}" -exec cp '{}' $destfolder \;
   ## Check if the file was copied
   [[ ! -e "$destfolder/$file" ]] && echo "File $file was not copied"
done < "$csv"
echo "Finished"

Update:

This solution works if your CSV is like this

fileA,fileB,fileC,fileD, ...

However since you say terdon's solution works your file must look like this

fileA,valueA1,valueA2,valueA3
fileB,valueB1,valueB2,valueB3
fileC,valueC1,valueC2,valueC3
fileD,valueD1,valueC2,valueD3

and the remainder of line after filename is simply discarded.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.