5

I am running ddrescue from disk A to disk B.

I had expected the slower of the disks would be running at full speed all the time and the other be running enough to keep up.

Imagine my surprise when vmstat 5 (i.e. the numbers below are 5 second averages) gave this output:

procs -----------memory----------    ---swap-- -----io----      -system--    -----cpu-----
 r b  swpd   free   buff     cache   si so     bi     bo        in   cs      us sy id wa st
 2 0  608260 234672 21308688 4346420 0  0      116890 31        1374 2884    1  4  85 10 0
 0 1  608260 232820 21309700 4348712 0  0      134248 53        1546 3354    1  4  85 10 0
 3 1  608260 247120 21267028 4374552 0  0      134272 0         2162 5534    4  4  83 9  0
 1 2  608260 244292 21265720 4374092 0  0      134246 139792    2518 5139    4  7  71 19 0
 0 2  608260 244480 21224808 4417508 0  0      64845  129203    1726 7227    3  5  72 20 0
 0 2  608260 247056 21224816 4414024 0  0      0      106422    850  1089    1  3  74 22 0
 0 2  608260 239648 21224824 4418628 0  0      0      111522    920  1455    2  3  75 21 0
 2 1  608260 291152 21224852 4377064 0  0      0      97719     984  1407    3  3  72 22 0
 0 1  608260 230956 21312888 4351932 0  0      99738  21        1264 3855    1  3  85 11 0
 0 1  608260 244772 21297528 4353928 0  0      131123 16        1451 3080    1  4  86 9  0
 1 0  608260 254376 21263872 4378864 0  0      62694  84199     1202 1989    2  4  86 8  0
 0 1  608260 250532 21263896 4382276 0  0      0      139168    954  974     1  3  88 8  0
 0 1  608260 242984 21297132 4355128 0  0      21710  70264     925  1487    1  2  90 7  0
 0 1  608260 224980 21310664 4359112 0  0      129997 29        1498 3221    1  4  85 10 0
 0 2  608260 237668 21270804 4387544 0  0      129997 106189    1848 3093    1  6  79 14 0
 1 1  608260 228084 21303840 4365168 0  0      129997 100277    1944 3129    1  6  75 18 0
 0 2  608260 245196 21227500 4424640 0  0      62643  84207     1220 5027    1  4  83 11 0
 0 2  608256 259756 21227516 4408504 0  0      0      87639     972  10750   1  3  73 22 0
 0 2  608248 236968 21227524 4429560 0  0      0      96103     955  9584    1  3  73 23 0
 0 1  608248 241404 21283160 4371248 0  0      78541  42        1186 3315    2  3  85 11 0
 1 1  608244 248900 21263160 4373180 0  0      129998 44        1595 4237    3  4  82 11 0
 0 1  608192 229964 21212448 4390868 6  0      130147 75        2490 9510    9  5  74 12 0
 0 1  608192 247132 21177192 4409944 0  0      62738  78208     1297 2931    1  4  86 9  0

So what we see is that disk A is running full speed for a while, then taking a break; almost the same time disk B starts running at full speed, and when it takes a break, disk A takes over again.

This would make sense if ddrescue had a 2 GB buffer: Read 2 GB, write 2 GB, repeat. But it does not: It takes up around 2 MB of RAM.

It could also be a "time buffer", so it reads for 20-30 seconds and then writes for 20-30 seconds. But again: I have not asked ddrescue to do any such thing, and the memory usage of ddrescue remains constant.

When the reading disk stops, ddrescue does not update the screen. Only when the reading starts again, does ddrescue update the screen. It does, however, take 20% CPU time constantly.

The system is idle otherwise (in the order of 1 MB/s to the root disk now and then), and neither disk A nor disk B is mounted.

What am I seeing here, and why does the slower disk not run at 100% utilization all the time?

Edit

For fun I tried dd (not ddrescue) and cat:

dd if=/dev/sdc of=/dev/sdf bs=64k
cat /dev/sdc >/dev/sdf

The results were almost identical: For this vmstat 5 gave more what I would expect. The writing disk takes a while to get into gear, but then it writes at the same speed as the reading disk (with cat the writing disk even got into gear faster):

procs -----------memory----------   ---swap-- -----io----     -system-- ------cpu-----
 r b swpd   free   buff     cache   si so     bi     bo       in   cs   us sy id wa st
 1 2 606928 245344 20762212 4440244 0  0      130662 26235    1727 3460 1  5  83 12 0
 1 1 606928 244132 20762812 4439672 0  0      130637 123438   2146 3380 1  7  73 19 0
 0 2 606928 227948 20805904 4412948 0  0      130662 30634    1741 3282 1  5  83 12 0
 0 2 606928 235468 20770744 4438160 0  0      129024 115610   2040 3374 1  6  75 17 0
 0 2 606928 243064 20766380 4439116 0  0      128691 131280   2129 3343 1  7  74 18 0
 0 1 606928 233528 20800160 4413956 0  0      128691 67756    1966 3434 1  6  78 16 0
 0 2 606928 230792 20775552 4438440 0  0      128666 69637    1974 4296 3  5  79 13 0
 0 2 606928 254492 20752076 4437996 0  0      127693 133437   2209 3595 1  7  73 18 0
 3 0 606928 233820 20767232 4440488 0  0      127693 137047   2335 5781 4  6  71 19 0
 1 1 606928 232712 20796324 4414648 0  0      127693 88403    2147 3568 1  6  77 16 0
 0 2 606928 226688 20772316 4442504 0  0      127181 141011   2255 3552 1  7  74 18 0
 2 1 606928 221332 20779576 4441668 0  0      125414 114102   2109 3646 2  6  74 18 0
 1 1 606928 225360 20776584 4440432 0  0      125389 126106   2122 3288 1  6  73 20 0
 1 1 606928 226712 20773400 4442740 0  0      125389 119439   2065 3291 1  6  74 18 0
 0 3 606928 247892 20748848 4446264 0  0      125133 103741   2014 3281 2  6  68 24 0
 0 2 606928 238596 20755732 4448696 0  0      115226 135512   2127 3230 2  6  71 20 0
 0 2 606928 234152 20757924 4448044 0  0      123445 131901   2216 4040 3  6  73 18 0
 1 2 606928 252320 20740332 4446900 0  0      123392 135939   2193 3433 1  7  73 19 0
 0 2 606928 245396 20747396 4446904 0  0      123443 128925   2259 5655 5  6  71 18 0
 1 3 606928 222832 20766180 4449504 0  0      123162 139278   2234 3591 1  7  74 18 0

Edit2

I tried changing:

  • /proc/sys/vm/dirty_expire_centisecs
  • /proc/sys/vm/dirty_background_ratio

None of these gave any visible difference. However, changing /proc/sys/vm/dirty_background_ratio from 10 to 1 made ddrescue behave much more like cat:

procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 0  0 604700 306296 20348532 4373480    0    0 71526 102216 1545 2561  1  4 84 11  0
 0  1 604700 242724 20393720 4393508    0    0 123392 87694 1892 3152  2  6 80 13  0
 0  1 604700 230028 20402612 4393308    0    0 125389 124578 2139 3512  1  6 80 13  0
 1  0 604700 228944 20402148 4394620    0    0 125389 126991 2216 4064  2  6 78 14  0
 0  1 604696 253744 20341280 4431316    0    0 125415 128649 2683 6258  5  6 75 13  0
 1  1 604696 235080 20407744 4384972    0    0 125389 118890 2383 4816  4  6 76 14  0
 0  1 604696 285636 20346160 4398800    0    0 71475 109890 1809 10541  3  5 79 13  0
 2  2 604696 240168 20368092 4421908    0    0 114816 87582 1961 4467  3  5 79 12  0
 0  2 604696 240296 20377552 4411108    0    0 125389 121069 2251 4723  4  6 78 13  0
 1  1 604696 232420 20387084 4408712    0    0 125414 122238 2533 4376  2  6 78 13  0
 0  2 604696 238956 20387196 4402336    0    0 125389 123819 2659 5142  2  7 77 15  0
 2  0 604696 236288 20372312 4420324    0    0 124826 125246 2371 5450  4  6 74 16  0
 0  1 604696 293840 20335076 4399684    0    0 71450 107597 1882 5107  3  5 78 14  0
 1  1 604696 222776 20397184 4406132    0    0 108467 76212 2294 7814  4  5 80 12  0
 2  0 604696 230164 20397256 4398928    0    0 125372 121854 2527 4427  2  6 78 14  0
 0  1 604696 236932 20387916 4403792    0    0 125414 122394 2309 5088  3  6 78 13  0
 0  1 604696 242700 20388340 4396072    0    0 125414 128671 2355 5179  4  6 78 13  0
 0  1 604696 233416 20388316 4404064    0    0 125389 128662 2369 5511  5  6 76 14  0
 0  1 604696 237948 20402284 4384688    0    0 84250 104036 1919 5008  5  4 81  9  0
 1  2 604696 232280 20392248 4402312    0    0 125389 102426 2090 4531  3  5 78 14  0

You can still see a significant drop every 30 seconds or so (every 6 lines).

  • 1
    don't use such a large buffer. 1-20mb is ideal. – Ipor Sircer Jan 17 '17 at 16:19
  • 1
    dd bs=10M ...... – Ipor Sircer Jan 17 '17 at 16:42
  • 1
    Also consider lower level hardware, the disks might be on the same controller, transferring over the same bus, and each disk has its own buffer. And on the other hand you have common interrups by other software trying to use the disks. – Dalvenjia Jan 17 '17 at 19:19
  • 1
    Per info ddrescue it does use the kernel cache. You can also only expect about 50% duty-cycle per disk (read-write-repeat). Testing here, even with mounted disks I approached 85-90% of advertised throughput on the slower drive, when in use, and even when using the same drive for in/out it was at the same levels. Memory cache took all that was available to run the RAM at near 100% the whole time. Override cache with -idirect and -odirect if you computer will handle it, but you'll take a performance HIT for it! Better off to tweek -cluster-size= for your system (higher = better). – Gypsy Spellweaver Jan 18 '17 at 10:02
  • 1
    @GypsySpellweaver I would expect cat also to use the normal kernel cache. What puzzles me is that cat does what I would expect: Full speed on both drives after a few seconds. How does ddrescue's use of cache differ from cat? – Ole Tange Jan 18 '17 at 13:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.