Consider a 100MB raw block device as a simple example. That is 204800 blocks of 512 bytes each for a total of 102760448 bytes.
The challenge is to shift the first 98MB (200704 blocks) so there is a gap of 2MB (4096 blocks) in front of it. To do this in-place requires that nothing is written to a sector that has not been read. One way to achieve this is to introduce a buffer:
$ dd if=/dev/sdj2 count=200704 | mbuffer -s 512 -b 4096 -P 100 | dd of=/dev/sdj2 seek=4096
The expectation is that
mbuffer will store 4096 blocks before passing anything to the writer, thus ensuring that nothing is written to an area that has not been read and that the writer lags the reader by the size of the buffer. The buffer should allow the reader and writer to operate as fast as possible within those constriants.
However, it doesn't seem to work reliably. I've tried using real devices but it never works on them, whereas experiments with a file worked on my 64-bit box but not on my 32-bit box.
First, some preparation:
$ dd if=/dev/sdj2 count=200704 | md5sum 0f0727f6644dac7a6ec60ea98ffc6da9 $ dd if=/dev/sdj2 count=200704 of=testfile
This doesn't work:
$ dd if=/dev/sdj2 count=200704 | mbuffer -s 512 -b 4096 -P 100 -H | dd of=/dev/sdj2 seek=4096 summary: 98.0 MiByte in 4.4sec - average of 22.0 MiB/s md5 hash: 3cbf1ca59a250d19573285458e320ade
This works on 64-bit system but not on 32-bit system:
$ dd if=testfile count=200704 | mbuffer -s 512 -b 4096 -P 100 -H | dd of=testfile seek=4096 conv=notrunc summary: 98.0 MiByte in 0.9sec - average of 111 MiB/s md5 hash: 0f0727f6644dac7a6ec60ea98ffc6da9
How can this be done reliably?
I have read other questions about buffering and looked at
mbuffer. I could only get the latter to work with the required buffer size.
Using intermetiate storage is an obvious solution to the problem that always works but it isn't practical when sufficient spare capacity isn't available.
Test platforms running Arch Linux with
mbuffer version 20140302.