I have a problem /proc/[PID]/io and memmap
I need to profile application IO that uses python library astropy.
One of the problem I have is that IO read/write total bytes are incorrect in /proc/[PID]/io.
I have a script
#!/bin/bash cp huge.fits huge.2.fits #python copyfits.py #python copyfitsMemmapOFF.py sleep 5 cat /proc/$$/io
I comment the
cp line to and the line I want to run for each test.
from astropy.io import fits hdulist = fits.open('huge.fits', mode='readonly') hdulist.writeto('huge.2.fits') hdulist.close()
from astropy.io import fits hdulist = fits.open('huge.fits', mode='readonly', memmap=False) hdulist.writeto('huge.2.fits') hdulist.close()
heres the IO results for each solution:
cp huge.fits huge.2.fits
rchar: 9749929202 wchar: 9749551680
**rchar: 8399421** wchar: 9749551685
rchar: 9757502959 wchar: 9749551685
I understand that memmap being off will lead to inconsistent IO results when using this variable to monitor how much the application read files.
How/Why memmap are not counted in standard IO, and how can I found out about those IO ?
Because if it's kernel instead of application reading file, the file is still accessed.