When moving files between two external drives, are they temporarily written to the internal hdd of the computer?

Question

While moving a big chunk of data between two external USB drives, I notice my laptop is slowed down. It was my understanding that the files are not written to any intermediate location (such as /tmp or similar) unless there is a shortage of free RAM. Am I wrong?

Answer 1

If you have a copy such as this, or its GUI equivalent,

cp -a /media/external/disk1/. /media/external/disk2/

the data is read from the first disk's filesystem and written directly to the second. There is no intermediate write to another storage location. If you are seeing slow speeds it may be that the two disks are sharing the same USB controller and contending for access to the bus.

Anything more than that and you will have to provide further details, such as the make/model of computer, its bus topology, etc.

Answer 2

There is no concept of internal and external: The operating system knows not of the box, and what is in or out of it. The only file-system that is special (but only a little bit) is the root file-system, and this can be mounted read-only. You can still do almost anything with a read-only root file-system (but not change the content of the root file-system).

Answer 3

TL,DR: it's probably the cache.

Copying or moving data between two removable drives doesn't store the data on an internal drive. The copying program reads a chunk of data from the source drive, writes it to the target drive, and repeats with the next chunk of data. A sophisticated program may try to make the copy faster, at the expense of heavier load during the copy, by copying multiple chunks in parallel.

Accessing drives takes some CPU time, both for the filesystem and for the actual drive access. The cost of the filesystem management is small if you have a lot of small files and negligible for large files. The cost of drive access depends on the quality of the USB controller and of the driver. I think USB controllers and their Linux drivers generally use DMA reasonably efficiently so there's not much CPU cost here either, but I know very little about USB and may be wrong here.

The data that is being copied is cached like any other data that is read from a disk. If you're copying a large amount of data (relative to the size of your RAM), the data that you're copying will displace frequently-accessed files and will even cause part of the memory of applications to be moved to swap. This can cause a significant performance hit. The system doesn't know that the data that it's reading from the source disk is only meant to be written out immediately, and doesn't need to be kept in RAM, so it tends not to make the best decision in terms of cache allocation.

When you copy a large amount of data on Linux, try using nocache (available as a package in major distributions). This is a command line utility that runs a command in a mode where the data that it reads will mostly not go into the cache. For example:

nocache mv /media/source/dir /media/target/dir

Using the nocache prefix can work on a GUI application, but only if the application starts normally. Some desktop environments arrange to start applications via a central process (e.g. KDE with kdeinit), or may open a window in an existing process, and in these cases nocache won't have any effect.

If you're concerned about privacy, it's unlikely, but possible that some of the data could end up in the swap space. It's unlikely because the copying program only keeps each chunk of data in memory for a small amount of time, and swap is used in priority for parts of the memory that haven't been accessed for a long time.