- With RHEL 8.8 currently, and RHEL 9.x, the latest NFS version is 4.2.
- When NFS 4 was introduced, it did away with a few things in NFS3 one of which was multiple port numbers:
- NFS4 mandates all traffic now exclusively TCP uses the single well known port 2049.
- you can find more mostly reputable articles stating the same thing.
- I have confirmed this by having only TCP 2049 open in firewalld for NFS 4.1 in RHEL 7.9; it does not use port 111 or any other unless you change the default configurations of
/etc/sysconfig/nfs. And in fact when I did get rdma working (over port 20049) that the rdma protocol specifically bypasses firewalld, an inherent aspect of why rdma saves cpy cycles and is faster i suppose.
The NFS insecure option in
/etc/exportssets the server to listen to a request from any port on the client. Changing it to 'secure' (default) makes sure that the server will listen to only requests originating from ports 1-1024 of the client. Thus an unauthorized user on the client is kept from starting an NFS dialogue. For reference : https://security.stackexchange.com/questions/246527/what-is-insecure-about-the-insecure-option-of-nfs-exports
The default is secure vs insecure when doing an NFS4 export if neither is mentioned in
With security rules it is oftentimes stated The NFS server must not have the insecure file locking option enabled..
First with the
secure option in play, be default, the *will only operate on secure ports less than 1024` seems to be completely not true since NFS4 runs on port 2049. The number 2049 is greater than 1024... what am I missing?
With regards to
RDMA which by convention happens on port
20049 there seems to be a little missed fact that one needs to explicitly state the insecure option in
/etc/exports if a
mount -o rdma is to be used otherwise the mount always happens as
proto=tcp and not
proto=rdma with no indication why.
I did validate that, using
MLNX_OFED_LINUX-23.04-126.96.36.199-rhel8.8-x86_64.iso installed in place of the Redhat
InfiniBand Support packages that a
mount -o rdma,port=1023 does work with a
mount on the client side showing
However one must also do (with MLNX only?) an
echo rdma 20049 > /proc/sys/nfsd/portlist. Or in the case with secure export an echo rdma 1023. Does anyone know how/why these values are not in
/proc/sys/nfsd/portlist in the first place and why I must do them manually ? And then what is the correct way to put those numbers there, so that after boot my
/etc/fstab nfs mounting of my data folder as rdma happens successfully? The MLNX instructional pdf falls short.
I have been banging my head against the wall getting RDMA to work, there seems to be a lot of shortcoming with NFS overall, and I have a paid for cluster mgr software that has RDMA placeholders for configuration, but all mounts are always proto=tcp. So if anyone can provide any information on anything described would be helpful, I will + any answer.
Also: I will end up doing
secure and choose some port number 1023 and below to satisfy security rules. How do I choose a proper number in that range? As ron nobody my understanding was I should never use port numbers below 1000 or 1024 for stuff I set up?
update: it appears that the
/etc/exports parameter of secure or insecure is inconsequential. What matters is having
rdma 20049 in
/proc/fs/nfsd/portlist on the nfsserver. With that, or any number, it appears to work with the secure exportfs.