1

This is what I have done

I have a VM which contains:

  1. A Ceph client
  2. A TGT server
  3. DHCP server

the machine exposes the image (RHEL) present in Ceph and PXE boots the remote node (which is the initiator).

I have cloned the VM and closed the DHCP server such that the cloned VM has both the Ceph client and the TGT server. So now I have two TGT servers exposing the same Target and have included the IP's of both the TGT servers in the tftpboot file such as:

#!ipxe
  sleep 23
  set keep-san 1
  ifconf --configurator=dhcp net0
  sanboot iscsi:10.20.30.1:tcp:3260:1:kumo-dan-installscript-img37 \
          iscsi:10.20.30.2:tcp:3260:1:kumo-dan-installscript-img37
  boot

And the target exposed are:

[root@bmi-introspect conf.d]# tgt-admin -s

Target 1: kumo-dan-installscript-img37
    System information:
        Driver: iscsi
        State: ready
    I_T nexus information:
        I_T nexus: 4
            Initiator: iqn.2010-04.org.ipxe:4c4c4544-0034-4410-8056-cac04f4c3032 alias: localhost.localdomain
            Connection: 0
                IP Address: 10.20.30.70
    LUN information:
        LUN: 0
            Type: controller
            SCSI ID: IET     00010000
            SCSI SN: beaf10
            Size: 0 MB, Block size: 1
            Online: Yes
            Removable media: No
            Prevent removal: No
            Readonly: No
            SWP: No
            Thin-provisioning: No
            Backing store type: null
            Backing store path: None
            Backing store flags:
        LUN: 1
            Type: disk
            SCSI ID: IET     00010001
            SCSI SN: beaf11
            Size: 10737 MB, Block size: 512
            Online: Yes
            Removable media: No
            Prevent removal: No
            Readonly: No
            SWP: No
            Thin-provisioning: No
            Backing store type: rbd
            Backing store path: bmi-introspect/kumo-dan-installscript-img37
            Backing store flags:
    Account information:
    ACL information:
        ALL

Once the node boots up I see two drives:

  1. sdb wherein the boot drive is present
  2. sda which is another drive (To be used as multipath) 3 .I install device-multimapper-multipath
  3. mpathconf --enable --with_multipathd y
  4. mpathconf --enable
  5. Made following changes in the /etc/multipath.conf file:

    (uncommenting the defaults settings)
    
    defaults {
            polling_interval        10
            path_selector           "round-robin 0"
            path_grouping_policy    multibus
            uid_attribute           ID_SERIAL
    #       prio                    alua
            path_checker            readsector0
            rr_min_io               100
            max_fds                 8192
            rr_weight               priorities
            failback                immediate
            no_path_retry           5
            user_friendly_names     yes
            find_multipaths         yes
    }
    
  6. systemctl start multipathd; systemctl enable multipathd

  7. now on multipath -ll I do not see any devices, neither does lsblk show the drive as mpath
  8. when I reboot the node I see the one drive as a multipath drive (lsblk)

Now, I do:

$ multipath -a /dev/sdb (which is where my boot drive is)
$ dracut --force -H --add multipath

... after which I shutdown the system and reboot it.

  1. Logging into the node I See:

    [root@localhost ~]# multipath -l
    mpatha (360000000000000000e00000000010001) dm-0 IET     ,VIRTUAL-DISK
    size=10G features='1 queue_if_no_path' hwhandler='0' wp=rw
    `-+- policy='round-robin 0' prio=0 status=active
      |- 7:0:0:1 sda 8:0  active undef unknown
      `- 8:0:0:1 sdb 8:16 active undef unknown
    
    [root@localhost ~]# multipath -ll
    mpatha (360000000000000000e00000000010001) dm-0 IET     ,VIRTUAL-DISK
    size=10G features='1 queue_if_no_path' hwhandler='0' wp=rw
    `-+- policy='round-robin 0' prio=1 status=active
      |- 7:0:0:1 sda 8:0  active ready running
      `- 8:0:0:1 sdb 8:16 active ready running
    
    [root@localhost ~]# lsblk
    NAME        MAJ:MIN RM  SIZE RO TYPE  MOUNTPOINT
    sda           8:0    0   10G  0 disk
    +-mpatha    253:0    0   10G  0 mpath
      +-mpatha1 253:1    0  476M  0 part  /boot
      +-mpatha2 253:2    0    5G  0 part  /
    sdb           8:16   0   10G  0 disk
    +-mpatha    253:0    0   10G  0 mpath
      +-mpatha1 253:1    0  476M  0 part  /boot
      +-mpatha2 253:2    0    5G  0 part  /
    

NOTE: Multipathing is working fine even one of the TGT server goes down the node dynamically shifts over the second server.

Problems

  1. Is this the correct way to do it or am I missing out on anything?
  2. When ever I try to reboot the node it waits for both the TGT servers to be up and does not boot up, unless both the servers are reachable (Thus killing the purpose).

Is there a way of fixing this?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.