I have Ubuntu 18.04 server with 500G SSD disk which has LVM and dm-crypt on it. I recently noticed that the number of bytes written to disk (as reported by vmstat -d or iostat) is unrealistically high. After monitoring the system I/O I found that the giant spike in disk writes happens once a week when fstrim.service runs:

bytes written

From the logs, it looks like every week when fstrim runs it reports that basically all free space was written to the disk, even though the system is almost at idle and has just under 10Gb written in a week at most.

Is this an expected behavior? I always thought that only new free blocks since last fstrim run should be discarded, but not the entire free space each time. This puts absurdly high wear on SSD (judging by media wearout value as reported by disk). Or is it somehow related to the presence of dm-crypt?

The disk does support TRIM:

hdparm -I /dev/sda | grep TRIM
  *    Data Set Management TRIM supported (limit 8 blocks)
  *    Deterministic read ZEROs after TRIM

And discard pass through is also enabled in dm-crypt:

dmsetup table
  silverbox--vg-swap: 0 19529728 linear 253:0 917964800
  silverbox--vg-root: 0 917962752 linear 253:0 2048
  sda3_crypt: 0 937496576 crypt aes-xts-plain64 00...0 0 8:3 4096 1 allow_discards
  • 1
    I suspect that it does not burden the ssd too much, as all it has to do is lookup internal info on the blocks being trimmed to see if they are already marked free. So it only does internal table reads and no writes if there is nothing new. This is why it is so fast. These internal tables may even be heavily cached in ram inside the ssd, so perhaps even no flash reads are done. – meuh Jun 18 '19 at 9:15
  • Please add the tags trim and fstrim. – wfjm Sep 8 '19 at 14:02

ATA trim command only changes meta data in the disk drive, it definitely does not do any low level writes to the memory cells. If the disk supports deterministic trim a trimmed block is returned with zeros, this however is done by the controller based on the new meta data status and not because the cells were actually erased at the time of the trim command.

trim commands are unfortunately counted as writes in all kernel statistics I know about. So iostat or sar or /sys/fs/ext4/*/lifetime_write_kbytes give the sum of true writes and trims. See also question on superuser.

fstrim, when run once a week, seems to release the whole unused disk space. If you have, for example, a 1 TB disk which is 50% used, the default fstrim activity appears in the statistics as 500 GB written per week or 70 GB/day.

Bottom line is: the write statistics is easily dominated by trims counted as writes, especially for moderately filled file systems.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.