1

I was provided a great bit of code for searching and copying files listed in a csv from a directory and its subfolders in an answer to my previous question.

However, some files are not copied. The code I used was:

while IFS=, read -r file rest
do
find "$sourcefolder" -name "$file" -exec cp -i '{}' "$destfolder" \;

[[ -e "$destfolder/$file" ]] || badfiles=$(printf "%s\n%s" "$file" "$badfiles")
done < "$csv"

if [[ -z "$badfiles" ]]; then
    echo -e "\e[32mAll files were succesfuly copied."
echo -e "\e[0m"

exit
else

echo -e "\e[31m"
printf 'Some files were not copied, please see below list:\n%s\n'
echo -e "\e[34m"
printf "$badfiles"

fi

echo -e "\e[0m \n\n"
exit

For some reason, 5 of the files listed are not located and are skipped over. The script somehow misses them and doesn't report them in its error reporting. I know the error logging works because if I make up a completely random file name that doesn't exist and include it in the csv, the script reports it as not being copied. But there seems to be some issue with these particular entries in the csv. There are no issues with the files' or directories' permissions.

The output of ls -l on the 5 problem files is shown below:

-rwxrwxrwx 1 archive archive 29067850 Mar  8  2008 bap_02_08_0184_01_tomorrow's_world.mp4
-rwxrwxrwx 1 archive archive 148025274 May 21  2010 ./Ascent_Media_MP4files/MP4 FILES/OralHistory_HistoryOFScience_Walter.mp4
-rwxrwxrwx 1 archive archive 57990878 Nov 29  2008 ./Flash_Files/Tomorrows World_TX_01_04_1976_LSF1056Y.mp4
-rwxrwxrwx 1 archive archive 203758186 Jul 17  2009 ./Ascent_Media_MP4files/MP4 FILES/not so much a programme more a way of life - LCA6710B - 29_01_1965.mp4
-rwxrwxrwx 1 archive archive 179529394 May 25  2010 ./Ascent_Media_MP4files/MP4 FILES/SundayNight_WHAuden_LMA6900T_TX28_11_1965.mp4

The offending entries in the csv file as well as those above and below them are:

Tomorrows World_TX_18_09_68_TX_11_12_68_LSF5910A_LSF5911T.mp4,KA0004107-001.mp4
TomorrowsWorld_LSF5910A_LSF5911T_18_09_1968_11_12_1968.mp4,KA0004107-002.mp4
bap_02_08_0184_01_tomorrow's_world.mp4,KA0004117-002.mp4
bap_02_08_0184_02_tomorrow's_world.mp4,KA0004117-003.mp4
Tomorrows World_TX_07_05_1969_LSF5931B.mp4,KA0004117-001.mp4
bap_02_08_0863_tomorrow's_world.mp4,KA0004118-002.mp4

Billy Bunter_TX_09_09_1956_LCH9311A.mp4,HPR000185-001.mp4
ZooQuestToGuiana_6_LGF6998A_18_10_1955.mp4,HPR000211-001.mp4
OralHistory_HistoryOFScience_Walter.mp4,HPR000214-001.mp4
ORalHistory_HistoryOfScience_NevillMott.mp4,HPR000215-001.mp4
OralHistory_HistoryOfScience_FredSanger.mp4,HPR000216-001.mp4

Tomorrows World_TX_04_01_1967_LSF5856D.mp4,KA0003210-001.mp4
Tomorrows World_TX_16_10_1975_and_23_10_1975_LSF6428R.mp4,KA0003211-001.mp4
Tomorrows World_TX_01_04_1976_LSF1056Y.mp4,KA0003213-001.mp4
Panorama_TX_20_11_1962_LCA6829R.mp4,KA0003214-001.mp4
Nationwide_TX_ 06_10_1970_ LCA3040B.mp4,KA0003215-001.mp4

Preservation Stockshots_LN1221-LN1288.mp4,KA0004516-001.mp4
Whickers_World_Quest_For_Beauty_TX_22_10_1966_LCA8262X.mp4,KA0004556-001.mp4
not so much a programme more a way of life - LCA6710B - 29_01_1965.mp4,KA0004587-001.mp4
Further_Adventures_Of_The_Musketers_1_Defiance_TX_21_05_1967_LDL5239W.mp4,KA0004592-001.mp4
pioneers of social change - 10 to sum up - ESB6385P - 26_11_1962.mp4,KA0004625-001.mp4

Its Magic - TX 29_11_1957 - LLV5261E.mpg,KA0004351-001.mp4
2MOROSWORLD_LSF1015P_06_03_1975.mp4,KA0003171-001.mp4
Tomorrows World - LSF1072E - 27_05_1976.mp4,KA0003196-001.mp4
ManAlive_LivingInSin_LGF5720B_02_03_1966.mp4,HPR000153-001.mp4
SundayNight_WHAuden_LMA6900T_TX28_11_1965.mp4,KA0005555-001.mp4
  • The files are correctly copied when I test this on my machine. My guess is that they're not where you think they are. Try making a csv file with only the offending lines and running while IFS=, read -r file rest; do echo "$file : $(find "$sourcefolder" -name "$file")"; done. That will show you were each of those files is found and if it is found. Also, it turns out cp -i doesn't work with while, it will just ignore the prompt and move to the next file, so it's useless here. My bad. – terdon Mar 4 '16 at 13:55
  • Hi, Terdon. They are in the directory where there meant to be- I've run a script with the code you suggested and it locates them fine. One thing I have noticed is that when I moved the problem listings to the bottom of the csv something strange happended. I made a note of the files which above and below the ones I moved before I moved them. On the next run the files which had been moved to the bottom copied across ok, but files in a similar position to the problem files original position did not copy. It's very strange. – neilH Mar 4 '16 at 14:43
  • Have you created the csv in Windows by any chance? – terdon Mar 4 '16 at 14:47
  • No, but it was generated from an excel spreadsheet which would have been created in Windows (by someone else) but I opened it in Libre Calc as a .xslx – neilH Mar 4 '16 at 14:49
  • 1
    Well, if the problem gone away when you move things around, I am willing to bet you have some non-printing character there screwing things up. Try grepping for one of the bad file names and passing it through od. For example: grep "bap_02_08_0184_01_tomorrow's_world.mp4" file.csv | od -c". Add the output to your question. – terdon Mar 4 '16 at 16:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.