2

The Situation:
I want to copy a directory recursively to an external hard drive. The directory contains a lot of files (at least 100,000).

The Problem:
External hard drives tend to get quite hot when heavily used like in my task (usage over several hours). That is bad for the life expectancy. So since time is not an issue in my case I would like that the copying would take some rest between copying the files to allow the drive to cool down a bit.

At the moment I use ionice -c3 nice nice cp -r which at least reduced performance loss to all other running tasks. But it does not address the heat problem.

Any suggestions?

Using other commands than cp like rsync would be ok to (if applicable from command line) but so far I could not find any command or option that allows me to wait x seconds between each file copy.


Additional Information: The external hard drive is the final destination of the data not a way to transfer the data to another computer.

  • 2
    This sounds to me a little bit like an XY problem; consider something like a fan or other ways to physically cool the drive? – Jeff Schaller Jul 6 '16 at 12:15
  • 2
    The only correct answer is to solve the cooling problem of your hardware. – frostschutz Jul 6 '16 at 12:16
  • @frostschutz: Try to add a fan to an external usb hard drive which cannot be plugged open without the risk of damaging it. So modifying the hardware is not really an option. – MrSmith42 Jul 6 '16 at 12:17
  • 1
    buy a better USB drive box or external drive. one with cooling that works. what you have now is dangerous junk that will inevitably kill whatever (non-SSD) drive you put in it. – cas Jul 7 '16 at 5:29
  • @cas: That exactly the problem. Most external USB drives are not designed to deal with multiple hours non stop activity. – MrSmith42 Jul 7 '16 at 8:03
4

Not particularly elegant, but you could run your copy command and then run a loop that pauses it for, say 3 minutes every 20 minutes:

  1. Launch your copy command in the background

    cp -r /path/to/dir /path/to/external/drive &
    
  2. Run this loop which will stop/restart it:

    while ps -p $! >/dev/null; do 
        kill -SIGCONT $!; 
        sleep 20m; 
        kill -s SIGSTOP $!; 
        sleep 3m; 
    done
    
  • 3
    You could rsync or pipe tar through pv with a rate limitation. Not sure if that will help the drive to cool down though, a lower data rate does not mean idle time for the drive. – frostschutz Jul 6 '16 at 12:17
2

rsync has an option to limit the throughput; for example --bwlimit=1 is 1 kibibyte/second. Use a suffix of K M or G to multiply the number, the default being K. The man page talks about sockets, but it applies to local copies too.

You can try running sudo smartctl -a /dev/sdx on your disk sdx to see if there are any indications of it suffering from over-temperature, though not all disks and buses implement SMART.

You can try hdparm -M 128 (check your man page for your version of this command) on your disk to do acoustic management which should slow down head movement to reduce noise, but also heat.

0

I thought of one way. Unfortunately it doesn't resume so it kind of assumes you don't mind leaving your computer on. Not tested either.

EDIT: woops, terdon's revised answer is much better.

SRC=src # source directory
DEST=dest # target directory
LOCKFILE=/var/tmp/copy.lock

BATCH_LEN=5m # 5 minutes
BATCH_DELAY=5m

touch "$LOCKFILE"

(cd "$SRC" && tar -cf - .) |
(cd "$TMP" && while [ -e "$LOCKFILE" ]; do
    dd bs=4K & (sleep $BATCH_LEN; kill %1 >/dev/null)
    sleep $BATCH_DELAY
 done) |
(cd "$DEST" && tar -xpf - ; rm "$LOCKFILE")

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.