I've place the bitmap on an external EXT4 drive (1xHDD) but I would expect the file to be written too whenever data is changed on my RAID10 array, right?
I even did checksum of the bitmap file, and it's surely isn't changed.
Size of file is only 2KB (bitmap chunk-size is set to 64MB)

  • I tried to stop and start the array, no change, no writing to the file
  • If the file is not accessible when array starts I'll get the error mdadm: Could not open bitmap file /mnt/1xHDD/R10_bitmap.dat which is good

Status of the mdadm process with cat /proc/mdstat : enter image description here

Array details with mdadm --detail /dev/md127 : enter image description here

Check of bitmap file with mdadm -X /mnt/1xHDD/R10_bitmap.dat

      Filename : /mnt/1xHDD/R10_bitmap.dat
         Magic : 6d746962
       Version : 4
          UUID : 39f863d1:4d247162:c3672454:1765abe2
        Events : 2914
Events Cleared : 2914
         State : OK
     Chunksize : 64 MB
        Daemon : 5s flush period
    Write Mode : Normal
     Sync Size : 624877568 (595.93 GiB 639.87 GB)
        Bitmap : 9535 bits (chunks), 0 dirty (0.0%)

Content of my mdadm.conf:

HOMEHOST <system>
ARRAY /dev/md/SRV01:R10_HDD metadata=1.2 bitmap=/mnt/1xHDD/R10_bitmap.dat name=SRV01:R10_HDD UUID=39f863d1:4d247162:c3672454:1765abe2

By the way, any comments in regards to placing the bitmap file on a single HDD? Tried to just rename it and the array continues without problems, although mdadm --details... will tell the bitmap has been deleted.

Additional note: Moving to external bitmap the write performance on this RAID10 array increased from 135MB/s to 370MB/s! (using direct IO, i.e. page-cache not in use).

  • 1
    Did you check with /proc/sys/vm/block_dump ? if the array is in perfect sync the bitmap will pretty much also always be in sync state so it should be hard to observe bitmap changes... unless the bitmap sports its own last-updated timestamp but it probably doesn't – frostschutz May 29 '19 at 8:19
  • Tried to remove a member drive, delete and add files: bitmap-file still not change. Then re-add the removed member drive. The array is resynced within a 30-90 seconds minutes. I assume the above should result in an array not in sync and at least that would change the bitmap file? I'll try to play around with the block_dump and see if that will enlighten something – MrCalvin May 29 '19 at 8:26
  • Hmm...when I write to the array I do get an 7809.656092] md127_raid10(2642): WRITE block 274432 on sde1 (8 sectors) from dmesg. There are nothing else on the sde1 than the bitmap file. Strange the size is so small. And timestamp doesn't change (oh...I mounted the drive with noatime, maybe that's why :-P) Did a checksum again, and now the file has changed! I guess it's working bette that it looks :-) – MrCalvin May 29 '19 at 8:38
  • If I rename the bitmap file (mdadm does report it as deleted), dmesg still shows [ 8438.595740] md127_raid10(2642): WRITE block 274432 on sde1 (8 sectors) Now I'm confused! – MrCalvin May 29 '19 at 8:43
  • 1
    Your bitmap file should be small, it’s tracking 64MiB chunks so it only needs 9535 bits (as indicated in the mdadm output); so rewriting the full bitmap would only involve 1192 bytes, plus metadata and journal (since it’s on an ext3 file system). I’m not sure the md layer does anything special to ensure the writes are flushed, so the cache may be masking writes. – Stephen Kitt May 29 '19 at 8:47

I seems the external bitmap-file was working after all.

As pointing out by Stephen Kitt in the comment, the file is usually pretty small, especially if you're using a large bitmap chunk-size. (I expected it to be much bigger).

Why I didn't observe the change of the bitmap file was because I forgot I mounted the filesystem with the EXT4 option noatime (will not update the modify date-time) and when I did an MD5 checksum of the file, the array was at the time in sync, so no changes.

Secondly about stability/risk of placing the external bitmap on a "insecure" 1xdrive:
Taking down the drive where the bitmap file was located, at runtime, didn't introduce any instability to the running and affected array. mdadm just report the bitmap file has been "deleted", but the array continue to work fine. Although is will not go online when restarting the array, e.g. at reboot, until you reconfigure the bitmap (either to none, internal or to another external bitmap file).
All-in-all I find it safe to place it on a none-mirrored disk.

It is way underestimated in the mdadm documentation and in all the mdadm guides I've been reading how big the performance impact is using the default internal bitmap, at least on both RAID10 and RAID5.

Just did a quick benchmark on R5:
Bitmap: Internal, write: 267MBs
Bitmap: None, write: 434MBs
(Total test-size 20G, blocksize 512K, page-cache enabled)

|improve this answer|||||

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.