I want to implement the fault tolerance in a CentOS environment.


Scenario: Two different systems connected via network having CentOS 6.0.

What I want: Now I want to setup Fault Tolerance in for both systems. So that even if one machine got any problem my running applications/servers never go down.

I found kemari and RDMA tools to implement that in VMs. But I don't know, from where I do start. I mean what to download, how to configure and how to run the VM by this?

If not in VMs, please give me some idea to implement this task only on two physical machines rather than virtually.

  • What applications are running on the CentOS VMs? You typically do the load balancing up higher in the stack, rather then at the VM level. This might be of use too: linux-kvm.org/wiki/images/0/0d/0.5.kemari-kvm-forum-2010.pdf – slm May 15 '14 at 12:32
  • This video shows a live migration using Kemari. youtube.com/watch?v=YrVNZG77PhY. Is this what you're ultimately asking for guidance on how to go about setting it up? – slm May 15 '14 at 12:53
  • This looks to be a more appropriate Kemari main site then the SF site you linked to: osrg.net/kemari – slm May 15 '14 at 13:00
  • BTW, Kemari is in development from what I can tell, are you looking to do this in a production scenario or just to play with it? – slm May 15 '14 at 13:04
  • I am first playing with it and after that I am wishing to deploy in production. – Tejas May 16 '14 at 4:17

I would actually start here with the KVM website. They have a page titled: Migration which looks to have everything you need. In my research I found very little regarding Kemari, other than a few videos on youtube, some slide decks from various conferences, the Kemari website, and a few forum posts.

Looking at the tutorial videos and the KVM website's migration page, the example/howto seems to be all you need.

Example / HOWTO

A is the source host, B is the destination host:

TCP example:

  1. Start the VM on B with the exact same parameters as the VM on A, in migration-listen mode:

      B: <qemu-command-line> -incoming tcp:0:4444 (or other PORT))
  2. Start the migration (always on the source host):

      A: migrate -d tcp:B:4444 (or other PORT)
  3. Check the status (on A only):

      A: (qemu) info migrate                   
  • please go through updated question – Tejas May 17 '14 at 5:56

Your scenario looks like an HA (High Availability) Cluster solution.

In short:

At first check that your machines can communicate over the network (set accourding iptable rules) and take a look at your selinux configuration.

Do on both nodes:

yum groupinstall "High Availability"

On one node install the cluster management web interface Luci

yum install luci

Enable all needed services on both nodes

chkconfig modclusterd on
chkconfig ricci on
chkconfig cman on
chkconfig rgmanager on

Enable Luci on the management box

run chkconfig luci

Create an initial config file for 2 machines (/etc/cluster/cluster.conf) by running:

ccs_tool create -2 hacluster

You have to do the configurations on the first node only, they will be synced to the second one automaticly. Now edit /etc/cluster/cluster.conf and change the node names to your needs (the names have to match their host names)

Check your config file for errors


Then copy it to the other node (you have to do this only once). When your cluster is up and running you can do this with

cman_tool version -r

Start all cluster services

service cman start
service modclusterd start
service rgmanager start
service ricci start

Check for connection between the nodes and synchronization with


That your cluster can be used properly you have to setup its VIP (virtual IP Address). Here you have to take care that its either in the same network as your other services are or you have to setup some routing/ virtual devices that your cluster can be reached.

Edit your /etc/cluster/cluster.conf:

< rm >
    < failoverdomains/ >
    < resources/ >
    < service autostart="1" exclusive="0" name="IP" recovery="relocate" >
        < ip address="" monitor_link="on" sleeptime="10"/ >
    < /service >
< /rm >

Increment config_version="n" so the cluster knows that you have something changed and apply the changes tho the cluster

cman_tool version -r

For further information about clustering read here (Red Hat)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.