I have a large tar file (60GB) containing image files. I'm using
mmap() on this entire file to read in these images, which are accessed randomly.
mmap() for the following reasons:
- Thread safety -- I cannot seek an ifstream from multiple threads.
- I can avoid extra buffering.
- I get some caching (in the form of a requested page already being resident.)
The question is what happens when I've read every image in that 60GB file? Certainly not all of the images are being used at once -- they're read, displayed, and then discarded.
mmap() call is:
mmap(0, totalSize, PROT_READ, MAP_SHARED | MAP_NORESERVE, fd, 0);
Here's the question: does the kernel see that I've mapped read-only pages backed by a file and simply purges the unused pages on memory pressure? I'm not sure if this case is recognized. Man pages indicate that
MAP_NORESERVE will not require backing swap space, but there doesn't seem to be any guarantee of what happens to the pages under memory pressure. Is there any guarantee that the kernel will purge my unneeded pages before it, say, purges the filesystem cache or OOM's another process?