0

I'm using RHEL 8.7 on both the client and the server, a clean install from rhel-8.7-x86_64-dvd.iso.

The /etc/nfs.conf and /etc/nfsmount.conf files are unmodified.

On the nfsserver

mkdir /data
chmod /data
systemctl disable firewalld --now

The contents of /etc/exports:

/data   *(rw,async,no_root_squash)
# exportfs -s
/data  *(async,wdelay,hide,no_subtree_check,sec=sys,rw,secure,no_root_squash,no_all_squash)

On the nfs client:

# mkdir /data
# mount 192.168.1.1:/data /data
# mount | grep data

192.168.1.1:/data on /data type nfs4
(rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.1.2,local_lock=none,addr=192.168.1.1)

How can I get RDMA to work?

Am I correct in that I should be seeing proto=rdma in the mount on the client side, rather than proto=tcp? If not, how do I verify RDMA is in effect?`

3
  • 1
    Hello the procedure for RHEL 7.9 is here unix.stackexchange.com/questions/684194/…, should be the same on RHEL 8.7 except if your Network cards doesn't support it.
    – admstg
    Feb 7, 2023 at 17:22
  • haha, thanks for reminding me
    – ron
    Feb 7, 2023 at 18:07
  • that explains the deja vu feeling I was having, typing this question for rhel8
    – ron
    Feb 7, 2023 at 18:09

1 Answer 1

2

for those interest, in RHEL 8.7 this is the deal

  • clean install from rhel-8.7-x86_64-dvd.iso
  • On the NFS Server side you must edit /etc/nfs.conf and uncomment rdma=n and set that to rdma=y. This does not need to be done on the nfs client side.
    • this is at the bottom of the [nfsd] block.
    • you don't need to modify anything else in this file, everything else will be commented out.
  • on the nfs client, you must do mount -o rdma <server>:/<folder> <folder> in order to see proto=rdma when validating your mount otherwise it will show as proto=tcp. You do not need to do mount -o rdma,port=20049.
  • on both the server side and client side, be sure in /etc/rdma/modulefiles/rdma.conf does not have xprtrdma {NFS over RDMA client support} nor svcrdma {NFS over RDMA server support} commented out. By default they should not be.
    • By default only rds_rdma is commented out, this is not required to be uncommented to get nfs proto=rdma to happen. Whether rds_rdma is beneficial or not I do not know at this point in time.
3
  • rdma=y was the missing part from official Red Hat documentation. May 10, 2023 at 21:10
  • note: /proc/fs/nfsd/portlist must contain rdma <rdma-port>. if not then a client mount -o rdma` will never work. After boot in RHEL 8.8 this does not happen, one must manually do a service nfs-server restart to see the entry in the portlist file.
    – ron
    Jul 3, 2023 at 13:51
  • note: there's also seems to be an undocumented choice in /etc/nfs.conf in that one would logically think either rdma=n or rdma=y. You need to do rdma=nfsrdma there.
    – ron
    Jul 3, 2023 at 13:52

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .