2

So I have gotten to the point where I had all services running when I configured the cluster, but after reboot I am getting the following :

Full list of resources:

 virtual_ip     (ocf::heartbeat:IPaddr2):       Started node1
 webserver      (ocf::heartbeat:apache):        Stopped
 Master/Slave Set: WebDataClone [WebData]
     Masters: [ node2 ]
     Slaves: [ node1 ]
 WebFS  (ocf::heartbeat:Filesystem):    Started node2
 sqlfs  (ocf::heartbeat:Filesystem):    Stopped
 Master/Slave Set: SQLDataClone [SQLData]
     Masters: [ node1 ]
     Slaves: [ node2 ]

Failed actions:

    sqlfs_start_0 on node1 'unknown error' (1): call=26, status=complete, last-rc-change='Sat Jun  4 00:57:27 2016', queued=0ms, exec=32ms
    sqlfs_start_0 on node2 'unknown error' (1): call=34, status=complete, last-    rc-change='Sat Jun  4 00:57:27 2016', queued=0ms, exec=59ms

This is what I used for my SQL Configuration :

 resource sqldata {
 protocol C;
 disk   /dev/sdb2;
 device /dev/drbd1;

 startup {
wfc-timeout 30;
outdated-wfc-timeout 20;
degr-wfc-timeout 30;
  }

 net {
cram-hmac-alg sha1;
shared-secret sync_disk;
  }

 syncer {
rate 10M;
al-extents 257;
on-no-data-accessible io-error;
verify-alg sha1;
  }

 on node1.freesoftwareservers.com {
   address  192.168.1.218:7788;
   flexible-meta-disk internal; 
  }

 on node2.freesoftwareservers.com {
   address  192.168.1.221:7788;
   meta-disk internal;
  }

 }

Start DRBD and sync :

drbdadm -- --overwrite-data-of-peer primary sqldata
drbdadm primary --force sqldata
watch cat /proc/drbd

Modified my.cnf gets changed on BOTH servers :

service mysqld stop
sudo nano /etc/my.cnf
Change in /etc/my.cnf :

#bind-address = 127.0.0.1
#Make Virtual_IP
bind-address = 192.168.1.215
#datadir=/var/lib/mysql
#Where DRBD will Mount
datadir=/var/lib/mysql_drbd
Mount and Populate with Data :

Mount and Populate drbd1 :

/sbin/mkfs.ext4 /dev/drbd1
mkdir /mnt/drbd1
mount /dev/drbd1 /mnt/drbd1 
mv /var/lib/mysql_drbd/* /mnt/drbd1/
chcon -R --reference=/var/lib/mysql /mnt/drbd1
umount /dev/drbd1

THIS IS THE CONFIG THAT SEEMS TO FAIL :

pcs cluster start --all
pcs cluster cib sqlfs_cfg
pcs -f sqlfs_cfg resource create sqlfs ocf:heartbeat:Filesystem device="/dev/drbd1" directory="/var/lib/mysql_drbd" fstype="ext4"
pcs -f sqlfs_cfg constraint colocation add sqlfs with SQLDataClone INFINITY with-rsc-role=Master
pcs -f sqlfs_cfg constraint order promote SQLDataClone then start sqlfs
pcs -f sqlfs_cfg constraint colocation add webserver with sqlfs INFINITY
pcs -f sqlfs_cfg constraint order sqlfs then webserver
pcs -f sqlfs_cfg constraint
pcs -f sqlfs_cfg resource show
pcs cluster cib-push sqlfs_cfg

sqlfs_drbd_cfg :

pcs cluster start --all
pcs cluster cib sqlfs_drbd_cfg
pcs -f sqlfs_drbd_cfg resource create SQLData ocf:linbit:drbd drbd_resource=sqldata op monitor interval=60s
pcs -f sqlfs_drbd_cfg resource master SQLDataClone SQLData master-max=1 master-    node-max=1 clone-max=2 clone-node-max=1 notify=true
pcs -f sqlfs_drbd_cfg resource show
pcs cluster cib-push sqlfs_drbd_cfg

Set Prefered Location on node1 AKA Primary :

pcs constraint location sqlfs prefers node1=50

Any thoughts on how to troubleshoot this? Again, it worked during initial config, and broke after reboot...

Thanks!

  • I would grep the logs (/var/log/messages or /var/log/syslog) for 'sqlfs' and post any errors that were printed to stdout (gets logged by Pacemaker) by the start operation. That will probably give us more clues. – Matt Kereczman Jun 5 '16 at 17:36
2

I don't see mysql defined in the Pacemaker; are you starting it manually after the cluster starts or does it start at boot?

If it's starting at boot, it could be starting before Pacemaker, and therefore holding a lock on the mountpoint (where it's data should be) which would cause the Filesystem resource to not be able to mount the filesystem there.

Kind of a shot in the dark, but if that's the case, you should disable mysql from starting at boot and then define it in the cluster.

  • I am understanding more, thank you, I didn't even think about that, but now that I think about how I set up Apache, it makes sense, first I added the httpd deamon to the cluster to be managed, then the WebFS/DRBD device. But I am Having an error starting MySQL daemon, it says not installed.... Going to start a new thread, will post link if your up for helping more! – FreeSoftwareServers Jun 7 '16 at 2:46
  • @FreeSoftwareServers I'm glad it was helpful. I have alerts setup for certain keywords that are HA cluster related so I should see your new post when it comes along... perhaps you've noticed this pattern ;) – Matt Kereczman Jun 7 '16 at 21:59
  • I am actually looking for some paid assistance for some MySQL DRBD configs, I cant find contact info in your profile, perhaps you could contact me, freesoftwareservers@gmail.com – FreeSoftwareServers Jun 8 '16 at 14:00
  • disclaimer: I currently work for LINBIT (the original creators and maintainers of DRBD), so there would be a bit of an ethics problem if I accepted money for DRBD/Pacemaker assistance. Head over to linbit.com/en/p/products if you're interested in paid support, or you can get more focused assistance from the HA community using one of the various channels listed here: drbd.org/en/community/connecting. The #clusterlabs channel on freenode is also a very good place to ask clustering questions. – Matt Kereczman Jun 8 '16 at 14:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.