1

I'm following this answer, trying to generate some major page faults with mmap:

#include <fcntl.h>
#include <stdio.h>
#include <sys/mman.h>
#include <sys/stat.h>

int main(int argc, char ** argv) {
  int fd = open(argv[1], O_RDONLY);
  struct stat stats;
  fstat(fd, &stats);
  posix_fadvise(fd, 0, stats.st_size, POSIX_FADV_DONTNEED);
  char * map = (char *) mmap(NULL, stats.st_size, PROT_READ, MAP_SHARED, fd, 0);
  if (map == MAP_FAILED) {
    perror("Failed to mmap");
    return 1;
  }
  int result = 0;
  int i;
  for (i = 0; i < stats.st_size; i++) {
    result += map[i];
  }
  munmap(map, stats.st_size);
  return result;
}

I tried to map a 1.6G file then read but only 1 major page fault occurred.

Major (requiring I/O) page faults: 1
Minor (reclaiming a frame) page faults: 38139

When I read data randomly by

// hopefully this won't trigger extra page faults
unsigned int idx = 0;
for (i = 0; i < stats.st_size; i++) {
  result += map[idx % stats.st_size];
  idx += i;
}

the page faults surged to 16415

Major (requiring I/O) page faults: 16415
Minor (reclaiming a frame) page faults: 37665

Is there something like prefetching in kernel to preload mmap data? How can I tell this by /usr/bin/time or perf?

I'm using gcc 6.5.0 and Ubuntu 18.04 with 4.15.0-54-generic.

5
  • What's in your 1.6GB file? Does it have holes in it? (like, does du on the file report 1.6GB or much less than that?) How are you reading on the first case, producing only 1 major page fault?
    – filbranden
    Aug 10, 2019 at 0:52
  • @filbranden I just randomly picked a large file, hoping to generate numerous major page faults. The file happened to be cuda_9.1.85_387.26_linux.run, the installer for CUDA 9.1. du reported exactly 1.6GB. I used the code provided in the link. I'll edit the question to copy it here. By the way, what does have holes in it mean?
    – zingdle
    Aug 10, 2019 at 8:11
  • There's this thing called "sparse" files, where you have blocks of all zeroes that are not taking space in disk, clearly not the case here... I think you might be up to something with your comment on readahead, there might be something there.
    – filbranden
    Aug 10, 2019 at 12:49
  • Though, one thing you want to check is how this behaves if you clear the page cache in between runs. echo 1 >/proc/sys/vm/drop_caches as root. Do you get major page faults then? Do you see different behavior depending on linear vs random reads?
    – filbranden
    Aug 10, 2019 at 12:52
  • Clearing the cache didn't help here. It is the same result as before. "Prefetching" is just my personal guess and I don't have any confidence about this.
    – zingdle
    Aug 10, 2019 at 14:51

1 Answer 1

2

Yes, the kernel does readahead by default (what you called prefetching), see https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/mm/filemap.c?h=v5.5#n2476

You can disable readahead on this memory region by calling posix_madvise() after mmap() with the POSIX_MADV_RANDOM advice.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .