test file: a.txt 1.2G
monitor command: iostat -xdm 1
The first scene: cp a.txt b.txt #b.txt is not exist
The second scene: cp a.txt b.txt #b.txt is exist
Why the first scene don't consume IO, but the second scene consume IO?
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. It only takes a minute to sign up.Sign up to join this community
It could well be that the data had not been flushed to disk during the first
cp operation, but was during the second.
vm.dirty_background_bytes to something small, like 1048576 (1 MiB) to see if this is the case; run
sysctl -w vm.dirty_background_bytes=1048576, and then your first
cp scenario should show I/O.
Except in cases of synchronous and/or direct I/O, writes to disk get buffered in memory until a threshold is hit, at which point they begin to be flushed to disk in the background. This threshold doesn't have an official name, but it's controlled by
vm.dirty_background_ratio, so I'll call it the "Dirty Background Threshold." From the kernel docs:
Contains the amount of dirty memory at which the background kernel flusher threads will start writeback.
dirty_background_bytesis the counterpart of
dirty_background_ratio. Only one of them may be specified at a time. When one sysctl is written it is immediately taken into account to evaluate the dirty memory limits and the other appears as 0 when read.
Contains, as a percentage of total available memory that contains free pages and reclaimable pages, the number of pages at which the background kernel flusher threads will start writing out dirty data.
The total available memory is not equal to total system memory.
There's a second threshold, beyond this one. Well, more a limit than a threshold, and it's controlled by
vm.dirty_ratio. Again, it doesn't have an official name, so we'll call it the "Dirty Limit". Once enough data has been "written", but not committed to the underlying block device, further attempts to
write will have to wait for write I/O to complete. (The precise details of what data they'll have to wait on is unclear to me, and may be a function of the I/O scheduler. I don't know.)
Disks are slow. Spinning rust especially so, so while the R/W head on a disk is moving to satisfy a read request, no write requests can serviced until the read request completes and the write request can be started. (And vice versa)
This is why we buffer write requests in memory and cache data we've read; we move work from the slow disk to faster memory. When we eventually go to commit the data to disk, we've got a good quantity of data to work with, and we can try to write it in a way that minimizes seek time. (If you're using an SSD, replace the concept of disk seek time with reflashing of SSD blocks; reflashing consumes SSD life and is a slow operation, which SSDs attempt--to varying degrees of success--to hide with their own write caching.)
We can tune how much data gets buffered before the kernel attempts to write it to disk using
If the amount of data you're writing is too great for how quickly it's reaching disk, you'll eventually consume all your system memory. First, your read cache will go away, meaning fewer read requests will be serviced from memory and have to be serviced from disk, slowing down your writes even further! If your write pressure still doesn't let up, eventually like memory allocations will have to wait on your write cache getting freed up some, and that'll even more disruptive.
So we have
vm.dirty_ratio); it lets us say, "hey, wait up a minute, it's really time we got data to the disk, before this gets any worse."
Putting a hard stop on I/O is very disruptive, though; disk is already slow from the perspective of reading processes, and it can take several seconds to several minutes for that data to flush; consider
vm.dirty_bytes's default of 20. If you have a system with 16GiB of RAM and no swap, you might find your I/O blocked while you wait for 3.4GiB of data to get flushed to disk. On a server with 128GiB of RAM? You're going to have services timing out while you wait on 27.5GiB of data!
So it's helpful to keep
vm.dirty_ratio, if you prefer) fairly low, so that if you hit this hard threshold, it will only be minimally disruptive to your services.
With these tunables, you're always trading between throughput and latency. Buffer too much, and you'll have great throughput but terrible latency. Buffer too little, and you'll have terrible throughput but great latency.
On workstations and laptops with only single disks, I like to set
vm.dirty_background_bytes to around 1MiB, and
vm.dirty_bytes to between 8MiB and 16MiB. I very rarely find a throughput benefit beyond 16MiB for single-user systems, but the latency hangups can get pretty bad for any synchronous workloads like web browser data stores.
On anything with a striped parity array, I find some multiple of the array's stripe width to be a good starting value for
vm.dirty_background_bytes; it reduces the likelihood of needing to perform a read/update/write sequence while updating parity, improving array throughput.
vm.dirty_bytes, it depends on how much latency your services can suffer. Myself, I like calculating the theoretical throughput of the block device, use that to calculate how much data it could move in 100ms or so, and setting
vm.dirty_bytes accordingly. A 100ms delay is huge, but it's not catastrophic (in my environment.)
All of this depends on your environment, though; these are only a starting point for finding what works well for you.