0

When copying a 2 GB file from HDD to USB stick, the initial speed is about 80 MB/sec, and decreases steadily to 10 MB/sec or even less.

Why is this so? The answers I've found says it's related to cache. But if the file is being copied only once, what does the cache have to do with it? I thought cache was used for information copied more than once.

I'm using Nautilus on Debian/Gnome, if it makes a difference.

EDIT - Adding some benchmarks

Before copying a 2.4 GB mp4 file, I repeatedly ran some checksums on it, to see how long it takes only to read the file, and also expecting to put it on the disk cache:

sha512sum file.mp4 (3x)
19s
8s
6s

sha256sum file.mp4 (3x)
10s
10s
10s

sha512sum file.mp4 (3x)
6s
6s
6s

It seems the file went to cache.

Next, I copied it to the USB stick. The default mounting on Debian 10 is asynchronous. The amount copied each minute, the remaining time and the overall speed were:

few sec 985 MB  13sec left  116 MB/s
1min    1.1 GB  1 min left  18.6 MB/s
2min    1.4 GB  1 min left  11.7 MB/s
3min    1.7 GB  1 min left  9.7 MB/s
4min    2.1 GB  35sec left  8.7 MB/s
4min45  "done"
eject
(it doesn't inform how long it will take)
+2min22 "ejecting" = ~7min total

2403758161÷427 = 5629410.2 = 5.63 MB/s = 5.37 MiB/s

Then I mounted the USB stick with the option -sync, and registered the same info each minute:

few sec         20min left  2 MB/s
30s             9 min left  3.9 MB/s
1min    272 MB  7 min left  4.5 MB/s
2min    500 MB  7 min left  4.2 MB/s
3min    744 MB  6 min left  4.1 MB/s
4min    1.1 GB  5 min left  4.4 MB/s
5min    1.4 GB  3 min left  4.6 MB/s
6min    1.7 GB  2 min left  4.6 MB/s
7min    2.0 GB  1 min left  4.6 MB/s
8min    2.3 GB  36sec left  4.7 MB/s
8m26s   done

eject
instantaneous

2403758161÷506 = 4750510.2 = 4.75 MB/s = 4.53 MiB/s

Conclusions:

The checksum indicates that the time to copy the file to memory is much smaller than the total copy time. So it seems incorrect to use it as part of the overall speed estimation.

Synchronous copy seems to be a little slower, but at least it correctly informs the time remaining.

Improve this question

2 Answers 2

Reset to default
0

The HDD is much faster than the USB stick, so when the file is read from the HDD it goes into the cache to free the HDD up for other tasks. The data is then sent to the USB stick at a speed it can handle.

Cache's are used for files that are referenced more than once, but that's not their only purpose. At some point, as in this case, the file has to be read for the first time. Now, if you were to copy the same file again pretty soon, the cache would deliver it instead of the HDD. After a short while of not being referenced again, the file would be purged from the cache, allowing the cache to handle new data.

Improve this answer
8
  • If the USB stick is the limiting speed, still it makes no sense to watch a continuously decreasing copy speed. The speed should be, all the time, the USB speed.
    – Rodrigo
    Dec 20, 2020 at 18:00
  • Even if Nautilus is showing the average speed between copy/paste, we should have only two phases (copy from cache vs. copy from HDD), since the cache size is much smaller than 2 GB. Instead, the decrease in speed seems to be linear.
    – Rodrigo
    Dec 20, 2020 at 18:02
  • @Rodrigo the overall speed will be at most that of USB. It will start fast, because of the cache. How slowly it goes to the actual USB speed depends on many system factors, among them the way your application calculates data transfers and how often things are actually written to USB. Depending on the size, you could have an instantaneous transfer, but then have to write the transfer time when you want to eject the drive (because only then the system might transfer the data).
    – Eduardo Trápani
    Dec 20, 2020 at 19:20
  • @Rodrigo You may be confusing CPU cache with disk cache. The latter can use almost all available RAM -- certainly much more than 2GB even on my Laptop. mount does have a -sync option, but I would expect serious impact on throughput.
    – Paul_Pedant
    Dec 20, 2020 at 19:56
  • 1
    @Rodrigo - also, it's not just an average - it's an average over time. So, the speed will decrease in a linear manner until such time as the average reaches the maximum speed that the USB can handle - or slightly above that and possibly/probably rounded to the nearest x%. Decreasing in a linear fashion would be the expected result if it's an average over time.
    – KGIII
    Dec 20, 2020 at 19:58
0

Data will be written to cache/write-buffer, this happens fast. Then it is copied out to the secondary-storage device (the USB), this happens more slowly. Soon you run out of cache/write-buffer, at this point writes to cache/write-buffer reduces to the write speed of the secondary-storage (USB).

You can do the copy with no caching. The main benefit of this is that other (possibly more important) data is not dropped from the cache.

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .