Port 248 is registered by John Kelly from Bell & Howell for
bhfs. Normally, you should contact him before using the port he registered. However, the e-mail is no longer valid and it is impossible to track him down. Also, there are no known implementations of bhfs. As I understood, it was linked to some sort of micro fiches file system from Bell and Howell, but my memory fails me in the details and might even be wrong.
The ports 249-255 are reserved by Jon Postel, the father of the Internet. He died October 1998, so he won't be using these ports anymore. See rfc2468.
The problem with using unregistered ports is, that someone else may have the same idea about using that port. That means, that in some (exceptional, I admit) circumstances you might get a port conflict.
If it is just a configuration on servers that you manage, you should not have any problems using port 249.
RedHat says about the "secure" option:
The secure option is the server-side export option used to restrict exports to “reserved” ports. By default, the server allows client communication only from “reserved” ports (ports numbered less than 1024), because traditionally clients have only allowed “trusted” code (such as in-kernel NFS clients) to use those ports. However, on many networks it is not difficult for anyone to become root on some client, so it is rarely safe for the server to assume that communication from a reserved port is privileged. Therefore the restriction to reserved ports is of limited value; it is better to rely on Kerberos, firewalls, and restriction of exports to particular clients.
Note that is the from port, not the port RDMA listens to.