CSV copy script error reporting in bash

Question

I've got this script which copies files listed in a csv recursively from a main folder. The copying aspect of the script works perfectly. However, I now want to add some error detection functionality. The "if" loop I have added below doesn't work as I would like. I have added a fake file to my csv list for testing purposes, and the script reports that all files have copied successfully, even though one of the files doesn't exist. Could someone suggest how I can get the script to report the file(s) it has not successfully found/copied from the csv and specify the file names? EDIT ** As there are 4000+ files listed in the CSV, I would like the script to print a summary of any copy failures at the end of the script, rather than for each files copied during the process. This would make it easier to identify/summarize any issues.

while IFS=, read -r file rest
do
find $sourcefolder -name "${file}" -exec cp '{}' $destfolder \;
done < $csv
if [ $? -eq 0 ]
then
echo "All files from CSV copied succesfully"
else "File from CSV not copied as not found"
fi

Answer 1

The error code returned by find has nothing to do with whether any files were found. It will only return a failure code if the command itself failed to run. What you are looking for is something like:

while IFS=, read -r file rest
do
  find $sourcefolder -name "${file}" -exec cp '{}' $destfolder \;
  ## Check if the file was copied
  [[ -e "$destfolder/$file" ]] || echo "File $file was not copied"
done < "$csv"
echo "Finished"

That way, the script will tell you about any files that weren't successfully copied.

If you only want to print the error at the end of the script, you can save the bad file names in a variable and print it after the loop has finished:

while IFS=, read -r file rest
do
  find "$sourcefolder" -name "$file" -exec cp '{}' "$destfolder" \;
  ## Check if the file was copied
  [[ -e "$destfolder/$file" ]] || badfiles=$(printf "%s\n%s" "$file" "$badfiles")
done < "$csv"

## Check if all the files were correctly copied
if [[ -z "$badfiles" ]]; then
        echo "All files were succesfuly copied."
else
        printf 'Some files were not copied:\n%s\n' "$badfiles"
fi

You can also extend this to avoid overwriting files with the same name:

while IFS=, read -r file rest
do
  find "$sourcefolder" -name "$file" -print0 |
    while IFS= read -d '' -r filepath; do
     c="";
     while [[ -e "$destfolder/$file$c" ]]; do
       ((c++))
     done
     cp "$filepath" "$destfolder/$file$c"
  done
  ## Check if the file was copied
  [[ -e "$destfolder/$file" ]] || badfiles=$(printf "%s\n%s" "$file" "$badfiles")
done < "$csv"

## Check if all the files were correctly copied
if [[ -z "$badfiles" ]]; then
        echo "All files were succesfuly copied."
else
        printf 'Some files were not copied:\n%s\n' "$badfiles"
fi

This will fail if the file names in your csv can contain newlines. If that is a possibility, change the first while loop to:

while IFS= read -d"," -r file rest; do ...

Note, however, that this will fail if any line of the csv contains nothing but a file name (so no comma). You can't have it both ways, if you can have lines with no commas use the 1st version and if you can have file names with newlines, use the second.

Answer 2

1. Your loop is incorrect, since you do nothing with rest

2. As terdon suggested, check after each copy, instead of once after loop

3. Negate the existence of file check, for correct comment
4. And heed terdon's warning, if you have same named files in your source tree, you will only get one copy in destination.

Try this,

while read -d, -r file
do
   find $sourcefolder -name "${file}" -exec cp '{}' $destfolder \;
   ## Check if the file was copied
   [[ ! -e "$destfolder/$file" ]] && echo "File $file was not copied"
done < "$csv"
echo "Finished"

Update:

This solution works if your CSV is like this

fileA,fileB,fileC,fileD, ...

However since you say terdon's solution works your file must look like this

fileA,valueA1,valueA2,valueA3
fileB,valueB1,valueB2,valueB3
fileC,valueC1,valueC2,valueC3
fileD,valueD1,valueC2,valueD3

and the remainder of line after filename is simply discarded.