I was provided a great bit of code for searching and copying files listed in a csv from a directory and its subfolders in an answer to my previous question.

However, some files are not copied. The code I used was:

while IFS=, read -r file rest
find "$sourcefolder" -name "$file" -exec cp -i '{}' "$destfolder" \;

[[ -e "$destfolder/$file" ]] || badfiles=$(printf "%s\n%s" "$file" "$badfiles")
done < "$csv"

if [[ -z "$badfiles" ]]; then
    echo -e "\e[32mAll files were succesfuly copied."
echo -e "\e[0m"


echo -e "\e[31m"
printf 'Some files were not copied, please see below list:\n%s\n'
echo -e "\e[34m"
printf "$badfiles"


echo -e "\e[0m \n\n"

For some reason, 5 of the files listed are not located and are skipped over. The script somehow misses them and doesn't report them in its error reporting. I know the error logging works because if I make up a completely random file name that doesn't exist and include it in the csv, the script reports it as not being copied. But there seems to be some issue with these particular entries in the csv. There are no issues with the files' or directories' permissions.

The output of ls -l on the 5 problem files is shown below:

-rwxrwxrwx 1 archive archive 29067850 Mar  8  2008 bap_02_08_0184_01_tomorrow's_world.mp4
-rwxrwxrwx 1 archive archive 148025274 May 21  2010 ./Ascent_Media_MP4files/MP4 FILES/OralHistory_HistoryOFScience_Walter.mp4
-rwxrwxrwx 1 archive archive 57990878 Nov 29  2008 ./Flash_Files/Tomorrows World_TX_01_04_1976_LSF1056Y.mp4
-rwxrwxrwx 1 archive archive 203758186 Jul 17  2009 ./Ascent_Media_MP4files/MP4 FILES/not so much a programme more a way of life - LCA6710B - 29_01_1965.mp4
-rwxrwxrwx 1 archive archive 179529394 May 25  2010 ./Ascent_Media_MP4files/MP4 FILES/SundayNight_WHAuden_LMA6900T_TX28_11_1965.mp4

The offending entries in the csv file as well as those above and below them are:

Tomorrows World_TX_18_09_68_TX_11_12_68_LSF5910A_LSF5911T.mp4,KA0004107-001.mp4
Tomorrows World_TX_07_05_1969_LSF5931B.mp4,KA0004117-001.mp4

Billy Bunter_TX_09_09_1956_LCH9311A.mp4,HPR000185-001.mp4

Tomorrows World_TX_04_01_1967_LSF5856D.mp4,KA0003210-001.mp4
Tomorrows World_TX_16_10_1975_and_23_10_1975_LSF6428R.mp4,KA0003211-001.mp4
Tomorrows World_TX_01_04_1976_LSF1056Y.mp4,KA0003213-001.mp4
Nationwide_TX_ 06_10_1970_ LCA3040B.mp4,KA0003215-001.mp4

Preservation Stockshots_LN1221-LN1288.mp4,KA0004516-001.mp4
not so much a programme more a way of life - LCA6710B - 29_01_1965.mp4,KA0004587-001.mp4
pioneers of social change - 10 to sum up - ESB6385P - 26_11_1962.mp4,KA0004625-001.mp4

Its Magic - TX 29_11_1957 - LLV5261E.mpg,KA0004351-001.mp4
Tomorrows World - LSF1072E - 27_05_1976.mp4,KA0003196-001.mp4
  • The files are correctly copied when I test this on my machine. My guess is that they're not where you think they are. Try making a csv file with only the offending lines and running while IFS=, read -r file rest; do echo "$file : $(find "$sourcefolder" -name "$file")"; done. That will show you were each of those files is found and if it is found. Also, it turns out cp -i doesn't work with while, it will just ignore the prompt and move to the next file, so it's useless here. My bad. – terdon Mar 4 '16 at 13:55
  • Hi, Terdon. They are in the directory where there meant to be- I've run a script with the code you suggested and it locates them fine. One thing I have noticed is that when I moved the problem listings to the bottom of the csv something strange happended. I made a note of the files which above and below the ones I moved before I moved them. On the next run the files which had been moved to the bottom copied across ok, but files in a similar position to the problem files original position did not copy. It's very strange. – neilH Mar 4 '16 at 14:43
  • Have you created the csv in Windows by any chance? – terdon Mar 4 '16 at 14:47
  • No, but it was generated from an excel spreadsheet which would have been created in Windows (by someone else) but I opened it in Libre Calc as a .xslx – neilH Mar 4 '16 at 14:49
  • 1
    Well, if the problem gone away when you move things around, I am willing to bet you have some non-printing character there screwing things up. Try grepping for one of the bad file names and passing it through od. For example: grep "bap_02_08_0184_01_tomorrow's_world.mp4" file.csv | od -c". Add the output to your question. – terdon Mar 4 '16 at 16:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.