So I have gotten to the point where I had all services running when I configured the cluster, but after reboot I am getting the following :

Full list of resources:

 virtual_ip     (ocf::heartbeat:IPaddr2):       Started node1
 webserver      (ocf::heartbeat:apache):        Stopped
 Master/Slave Set: WebDataClone [WebData]
     Masters: [ node2 ]
     Slaves: [ node1 ]
 WebFS  (ocf::heartbeat:Filesystem):    Started node2
 sqlfs  (ocf::heartbeat:Filesystem):    Stopped
 Master/Slave Set: SQLDataClone [SQLData]
     Masters: [ node1 ]
     Slaves: [ node2 ]

Failed actions:

    sqlfs_start_0 on node1 'unknown error' (1): call=26, status=complete, last-rc-change='Sat Jun  4 00:57:27 2016', queued=0ms, exec=32ms
    sqlfs_start_0 on node2 'unknown error' (1): call=34, status=complete, last-    rc-change='Sat Jun  4 00:57:27 2016', queued=0ms, exec=59ms

This is what I used for my SQL Configuration :

 resource sqldata {
 protocol C;
 disk   /dev/sdb2;
 device /dev/drbd1;

 startup {
wfc-timeout 30;
outdated-wfc-timeout 20;
degr-wfc-timeout 30;

 net {
cram-hmac-alg sha1;
shared-secret sync_disk;

 syncer {
rate 10M;
al-extents 257;
on-no-data-accessible io-error;
verify-alg sha1;

 on node1.freesoftwareservers.com {
   flexible-meta-disk internal; 

 on node2.freesoftwareservers.com {
   meta-disk internal;


Start DRBD and sync :

drbdadm -- --overwrite-data-of-peer primary sqldata
drbdadm primary --force sqldata
watch cat /proc/drbd

Modified my.cnf gets changed on BOTH servers :

service mysqld stop
sudo nano /etc/my.cnf
Change in /etc/my.cnf :

#bind-address =
#Make Virtual_IP
bind-address =
#Where DRBD will Mount
Mount and Populate with Data :

Mount and Populate drbd1 :

/sbin/mkfs.ext4 /dev/drbd1
mkdir /mnt/drbd1
mount /dev/drbd1 /mnt/drbd1 
mv /var/lib/mysql_drbd/* /mnt/drbd1/
chcon -R --reference=/var/lib/mysql /mnt/drbd1
umount /dev/drbd1


pcs cluster start --all
pcs cluster cib sqlfs_cfg
pcs -f sqlfs_cfg resource create sqlfs ocf:heartbeat:Filesystem device="/dev/drbd1" directory="/var/lib/mysql_drbd" fstype="ext4"
pcs -f sqlfs_cfg constraint colocation add sqlfs with SQLDataClone INFINITY with-rsc-role=Master
pcs -f sqlfs_cfg constraint order promote SQLDataClone then start sqlfs
pcs -f sqlfs_cfg constraint colocation add webserver with sqlfs INFINITY
pcs -f sqlfs_cfg constraint order sqlfs then webserver
pcs -f sqlfs_cfg constraint
pcs -f sqlfs_cfg resource show
pcs cluster cib-push sqlfs_cfg

sqlfs_drbd_cfg :

pcs cluster start --all
pcs cluster cib sqlfs_drbd_cfg
pcs -f sqlfs_drbd_cfg resource create SQLData ocf:linbit:drbd drbd_resource=sqldata op monitor interval=60s
pcs -f sqlfs_drbd_cfg resource master SQLDataClone SQLData master-max=1 master-    node-max=1 clone-max=2 clone-node-max=1 notify=true
pcs -f sqlfs_drbd_cfg resource show
pcs cluster cib-push sqlfs_drbd_cfg

Set Prefered Location on node1 AKA Primary :

pcs constraint location sqlfs prefers node1=50

Any thoughts on how to troubleshoot this? Again, it worked during initial config, and broke after reboot...


  • I would grep the logs (/var/log/messages or /var/log/syslog) for 'sqlfs' and post any errors that were printed to stdout (gets logged by Pacemaker) by the start operation. That will probably give us more clues. Jun 5, 2016 at 17:36

1 Answer 1


I don't see mysql defined in the Pacemaker; are you starting it manually after the cluster starts or does it start at boot?

If it's starting at boot, it could be starting before Pacemaker, and therefore holding a lock on the mountpoint (where it's data should be) which would cause the Filesystem resource to not be able to mount the filesystem there.

Kind of a shot in the dark, but if that's the case, you should disable mysql from starting at boot and then define it in the cluster.

  • @FreeSoftwareServers I'm glad it was helpful. I have alerts setup for certain keywords that are HA cluster related so I should see your new post when it comes along... perhaps you've noticed this pattern ;) Jun 7, 2016 at 21:59
  • disclaimer: I currently work for LINBIT (the original creators and maintainers of DRBD), so there would be a bit of an ethics problem if I accepted money for DRBD/Pacemaker assistance. Head over to linbit.com/en/p/products if you're interested in paid support, or you can get more focused assistance from the HA community using one of the various channels listed here: drbd.org/en/community/connecting. The #clusterlabs channel on freenode is also a very good place to ask clustering questions. Jun 8, 2016 at 14:30

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .