ls -lh and du -h giving different file sizes when truncate is used

Question

I am trying to find out a way to reduce the size of a log file while it is being used.

So for a file I tried this:

$ yes > yeslog

So data is being written to yeslog continuously.

After I while I check the sizes with ls -alh yeslog and du -h yeslog and they report the same size.

Then I do truncate -s 0K yeslog, then I check the sizes again with ls and du. But now, du is showing a reduced size, while ls is showing the same size as before.

Why is this happening and is truncate really doing its job?

Answer 1

This is a variant of How does linux manage the offsets of files. Both ls and du are correct, they are measuring different things.

When you run truncate, it reduces the file to 0 bytes. However yes immediately writes to it again, at the offset following where it had written before; all the missing data is replaced with zeroes, and entirely missing blocks are replaced sparsely. As a result, the apparent size of the file keeps increasing, but the actual disk space it consumes on disk goes back to 0 every time you run truncate, and while it increases again when yes writes to it, the sparse blocks aren’t counted.

ls shows the apparent size of the file by default, whereas du shows the disk space consumed, so ls will show a larger value than du after the first truncate which loses an entire block. You can get ls to show the allocated size with the -s flag.

If you ask the shell to append to yeslog (yes >> yeslog), then the file will be opened with O_APPEND and yes will write from the beginning of the file after it’s truncated, instead of continuing to write at the same offset as before.