5

I have been doing some elaborate testing to determine if our NetApp NFS (v3 protocol) server in async mode properly honors the client's fsync request. I've hit a wall when I discovered that Linux (RHEL 6, kernel 2.6.32-431.5.1) never issues a COMMIT op at all!!! That fact was uncovered by use of both the nfsstat tool, and by the nfstrace tool. Not a single COMMIT.

This seems to violate NFS semantics:

Version 3 clients use COMMIT operations when flushing safe asynchronous writes to the server during a close(2) or fsync(2) system call, or when encountering memory pressure.

What is going on?

Notes:

The mountpoint is definitely mounted with the async operation (which is the default).

To generate fsync requests, I used Postgresql's test_fsync tool. It uses a variety of ways to issue syncs and reports on the benchmarks so you can determine which is best for you on your system.

Timing differences with test_fsync indicate the fsync function takes longer to execute on async mounts than sync mounts, presumably because with sync mounts, data is being flushed all the time, and it's only when fsync is called that the data is flushed. However, the timing differences is quite erratic and I might just be encountering performance transience.

Mounting the server with the sync option changed nothing.

UPDATE: The plot thickens.

On Ubuntu/Mint 17, Linux kernel 3.13.0, (nfs version: 1.2.8), i set up a loopback mountpoint, with both sync and async options and re-ran the tests. Speed differences definitely show a difference between sync and async. nfsstat shows that after each run of pg_fsync_test, exactly 1 COMMIT transpired.

W.T.F.

1

A colleague at work dug up a possible answer:

the Netapp technical report tr-3183 (Using Red Hat Client with NetApp Storage over NFS) says:

Data ONTAP's NFS server promotes all write requests to FILE_SYNC, whether the client requests UNSTABLE, DATA_SYNC or FILE_SYNC writes. Thus the client never has to send a COMMIT request when it is writing to NetApp storage because all the writes are logged in the NVRAM and the clients get an acknowledgement for the writes right away. Hence the writes with NetApp storage are asynchronous in nature and are much faster.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.