On fresh drbd setup (no filesystem on drbd0, just synced) I have invalidated "node2" state and full sync started:
root@node1:~# drbdadm invalidate-remote drbd0 root@node1:~# drbd-overview 0:drbd0/0 SyncSource Secondary/Secondary UpToDate/Inconsistent [>....................] sync'ed: 0.2% (153188/153464)Mfinish: 1:04:59 speed: 40,180 (40,180) K/sec
and having drbd0 sync from node1 to node2 in progress I have set it primary on node1 for a moment, created ext4, and set it back to standby and then I went to node2 (inconsitent) and made it as primary without any warnings (sic!)
root@node2:~# drbd-overview 0:drbd0/0 SyncTarget Secondary/Secondary Inconsistent/UpToDate [====>...............] sync'ed: 26.7% (112500/153464)Mfinish: 0:47:58 speed: 40,000 (40,296) want: 74,040 K/sec root@node2:~# drbdadm primary drbd0 root@node2:~# drbd-overview 0:drbd0/0 SyncTarget Primary/Secondary Inconsistent/UpToDate [====>...............] sync'ed: 27.1% (111912/153464)Mfinish: 0:50:37 speed: 37,724 (40,252) want: 75,600 K/sec root@node2:~#
next, I mounted the ext4 from inconsistent device filesystem without any problems:
root@node2:~# mount /dev/drbd0 /mnt
then in that /mnt I created a few thousands of rsa key pairs (all the time on inconsistend drbd0 on node2 and sync from node1 in progres)
then I unmounted /mnt on node2 and mounted it on node1 and I see there all the files (sync node1->node2 still in progress) which I have created on the inconsistent state drbd0 on node2.
next I created another thousands of files on node1 and remounted /mnt to node2 and I see all the files which were created on node1 and node2 (during full sync on drbd0 on node2 in porgress = inconsistent state of it)
I have checked all files checksums in each step and all were ok.
How it works?? Is that by design? Why there was no split-brain state when I activated drbd0 on inconsistent device? What is the real state of drbd0 device on inconsistent node that is can be used as a primary for read/write? Was the resync done in both ways: from node1-node2 full sync (as node2 was invalidated in first step) and from inconsistent node node2 -> node1 the changes/files I created when I mounted it there?
After sync completed, final status is ok but I am still confused...
root@node2~# drbd-overview 0:drbd0/0 Connected Primary/Secondary UpToDate/UpToDate /mnt ext4 148G 1.7G 145G 2%
thank you in advance for any explaination,