Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. It only takes a minute to sign up.
Sign up to join this community
man ntfs-3g makes the big_writes mount option seem like a Good Thing:
big_writes
This option prevents fuse from splitting write buffers into 4K
chunks, enabling big write buffers to be transferred from the
application in a single step (up to some system limit, generally
128K bytes).
No down sides are listed... when would I want to avoid mounting with big_writes?
It's a good question, really. From what I can see here there's probably no downsides to it.
Back in 2008 when big_writes was first added to FUSE Miklos said:
Prior to 2.6.26 fuse only supported single page write requests. In theory all fuse filesystem should be able support bigger than 4k writes, as there's nothing in the API to prevent it. Unfortunately there's a known case in NTFS-3G where big writes cause filesystem corruption. There could also be other filesystems, where the lack of testing with big write requests would result in bugs.
To prevent such problems on a kernel upgrade, disable big writes by default, but let filesystems set a flag to turn it on.
The specific problem is documented in NTFS-3G's param.h as:
/* * Use of big write buffers * * With small volumes, the cluster allocator may fail to allocate * enough clusters when the volume is nearly full. At most a run * can be allocated per bitmap chunk. So, there is a danger when the * number of chunks (capacity/(32768*clsiz)) is less than the number * of clusters in the biggest write buffer (131072/clsiz). Hence * a safe minimal capacity is 4GB */ #define SAFE_CAPACITY_FOR_BIG_WRITES 0x100000000LL
And currently it does defend against this limit by verifying that the fs capacity is at least 4GB before requesting big_writes. So it is safe.
In fact, Tuxera recommends turning it on if NTFS-3G is eating up your CPU:
Workaround: using the mount option “big_writes” generally reduces the CPU usage, provided the software requesting the writes supports big blocks.
Since libfuse 3.0.0 (2016) big_writes is always on:
The
-o big_writesmount option has been removed. It is now always active. File systems that want to limit the size of write requests should use the-o max_write=<N>option instead. commit
However, libfuse3 is a whole new generation, and ntfs-3g does not support building against it at all. I did inadvently backport the new 1MiB write buffer (libfuse 3.6.0) into ntfs-3g's internal mini libfuse2, so if you want to use that you can grab it from here:
Yes, I did remember to refactor the write size / volume size limit. And yes, it is faster with big file writes even on modern Ryzen.
PS: I am planning to add libfuse3 support to ntfs-3g, but there's no real deadline for it. Tuxera is demonstrably not interested in such a change, and coming up with a name for a fork is a bit too much for me.
