Why memmap IO are ignored in /proc/[PID]/io

Question

I have a problem /proc/[PID]/io and memmap I need to profile application IO that uses python library astropy.
One of the problem I have is that IO read/write total bytes are incorrect in /proc/[PID]/io.

I have a script copy.sh :

#!/bin/bash
cp huge.fits huge.2.fits
#python copyfits.py
#python copyfitsMemmapOFF.py
sleep 5 
cat /proc/$$/io

I comment the cp line to and the line I want to run for each test.

copyfits.py contains:

from astropy.io import fits    
hdulist = fits.open('huge.fits', mode='readonly')
hdulist.writeto('huge.2.fits')
hdulist.close()

copyfitsMemmapOFF.py contains:

    from astropy.io import fits    
    hdulist = fits.open('huge.fits', mode='readonly', memmap=False)
    hdulist.writeto('huge.2.fits')
    hdulist.close()

heres the IO results for each solution:

cp huge.fits huge.2.fits

rchar: 9749929202
wchar: 9749551680

python copyfits.py

**rchar: 8399421**
wchar: 9749551685

python copyfitsMemmapOFF.py

rchar: 9757502959
wchar: 9749551685

I understand that memmap being off will lead to inconsistent IO results when using this variable to monitor how much the application read files. How/Why memmap are not counted in standard IO, and how can I found out about those IO ?
Because if it's kernel instead of application reading file, the file is still accessed.

Answer 1

/proc’s io only tracks “explicit” I/O, i.e. I/O performed using a small number of system calls. For reads, those are read, readv, preadv, sendfile, and copy_file_range; you can see the stats accumulated using add_rchar in fs/read_write.c.

I/O for memory-mapped files is quite different; when reading, it relies on page-fault handling, with a number of optimisations to increase performance (read-ahead etc.) You can track that to some extent by looking at page faults in /proc/${pid}/stat (fields 10 and 12). The hard part is figuring out how much data is read with each page fault; my experiments suggest 64KiB per fault but I haven’t found hard data to back that up (and it probably depends on circumstances).

I don’t know of any ready-to-use way of tracing mapped I/O from the point of view of the process (i.e. bytes read into the process, regardless of whether any block I/O actually occurred).

Accounting for memory-mapped reads accurately turns out to be quite a tough problem, mainly because of the way accounting has to reflect intention. io counts bytes which a program explicitly asked to read or write; but when a process maps a file into its memory, reads occur with a granularity determined by the kernel, not the reading process. You could read only one byte every 4KiB, and the kernel will read in the entire file for you — what should the accounting reflect? It can’t easily reflect bytes really read in memory by the process, that would have a huge performance impact (on top of being impossible to implement across all architectures).