Bash Script to cp files from a list

Question

I have a large amount of files I need to copy to Box.com using DAV2fs. I've found I'm only able to copy a couple gigs at a time, and then I have to wait for a couple minutes while box catches up or it errors out.

So what I did was I made a list of all the files/directories using "find ./ > outfile.txt"

I want to iterate though the list of files, and after say 100 copies(or whatever) wait 10 minutes. How would I do that? without using cp -r, when the outfile.txt looks like:

/dirctory1
/directory1/file.txt
/directory1/file2.txt

cp omits the directory so file and file2 never get copied. If I do cp -r then it will do directory1 and all it's contents so my file count will be off.

EDIT:

To clarify, I'm interested in the cp portion. Specifically how to get cp to create a directory without doing -r as -r will throw off my count.

Answer 1

Try this. (Not tested.)

destination=     # assign 
i=0
while read line; do
    cp --parents "$line" "$destination"
    ((i++))
    [ $i -eq 100 ] && sleep 600 && i=0
done < outfile.txt
# or: done < <(find ... and so on) # to avoid creating the temp file

Answer 2

Change the "echo cp" to be your real cp and the following should do it. It'll sleep for 10 minutes after every 100 files. It assumes that none of your filenames have/had newline characters in them, such that each line in your outfile.txt corresponds to a filename.

#!/usr/bin/env bash
count=0
while IFS= read -r file
do
  echo cp "$file" dest/
  count=$((++count))
  [[ $(($count % 100)) == 0 ]] && sleep $((10 * 60))
done < outfile.txt