Unix & Linux

Questions tagged [mmap]

Ask Question

All about using memory mapped files. Questions on programming should be asked on Stack Overflow SE.

42 questions
Newest
Active
Bountied
Unanswered
Filter by
Sorted by
Tagged with
0
votes
2answers
45 views

What is lazy swap reservation?

I am reading the Linux Progamming Interface. 49.9 MAP_NORESERVE and Swap Space Overcommitting Some applications create large (usually private anonymous) mappings, but use only a small part ...
0
votes
0answers
52 views

Mmap failed with operation not permitted

I've compiled application on Linux, but I got this error wiringPiSetup: mmap (GPIO) failed: Operation not permitted After that: I've added iomem=relaxed to grub cat /proc/cmdline BOOT_IMAGE=/...
1
vote
1answer
47 views

Mapping several files into one [closed]

I need to make two files appear as one : +----------------+ +------------------+ | | | | |...
0
votes
1answer
40 views

User-space swap on mmap()-ped files

Imagine a system with very small ram (nearly embedded). It is enough - for nearly all tasks, but with an exception. There is a task X, which requires a huge amount of RAM what the machine has not. ...
1
vote
1answer
65 views

Mmaping tremendously large files

I have a very large disk drive (2TB), but not very much RAM (8GB). I'd like to be able to run some big data experiments on a large file (~200GB) that exists on my disk's filesystem. I understand that ...
0
votes
0answers
31 views

Discrepancy between linux man page and implementation of mmap

I am trying to compile a program that requires the mmap(2) to support the flag MAP_SYNC, but my compiler reports that no such thing exists. Now, I manually verified the header file for mmap and found ...
1
vote
1answer
152 views

How do I create a temporary file in memory?

I need to create some (small) temporary files in my zsh scripts, and I want them to live on RAM so as to avoid disk IO. What are the idiomatic ways of doing this? I want it to work on Linux and macOS, ...
1
vote
1answer
108 views

Weird major page fault number when reading sequentially / randomly in mmap region

I'm following this answer, trying to generate some major page faults with mmap: #include <fcntl.h> #include <stdio.h> #include <sys/mman.h> #include <sys/stat.h> int main(...
4
votes
1answer
748 views

mmap: effect of other processes writing to a file previously mapped read-only

I am trying to understand what happens when a file, which has been mapped into memory by the mmap system call, is subsequently written to by other processes. I have mmaped memory with PROT_READ ...
0
votes
1answer
39 views

mmap - map to address above 2^48

I understand that the used address space of 64-bit PCs is [0,2^48), but can I use mmap to map a file to an address above 248? I wrote the following code but found out the mapped address was still ...
1
vote
1answer
113 views

unlink/rm target of an open symlink

I've come across an issue where processes that load symlinked .so files crash if I update the .so while the processes are still running. I have long-running processes that load shared libraries (.so)...
0
votes
0answers
65 views

Is a mmaped file considered an open file in linux/unix?

Hi I feel confused about open file and mmapped file. I know that if a file is accessed using system call such as read/write will return a file descriptor, and be considered as an open file? How about ...
6
votes
1answer
320 views

ld.so.cache and libc.so.6 memory-mapped for every call?

Playing with strace, it appears to me that ld.so.cache and libc.so.6 are opened and mapped to memory for almost every process. At least those processes that I experimented with. Doesn't this mean ...
2
votes
0answers
303 views

Is there a standard for the Linux user-space memory map?

I'm contemplating a project where I would use a memory mapped file that is shared between multiple processes, and in order to be able to use pointer values in the mapped region in all those processes*,...
3
votes
1answer
4k views

How to use dd if=/dev/mem in place of devmem ?

It seems like root@testbox:~# dd if=/dev/mem bs=1 count=4 skip=2149646336 | hd should be nearly equivalent to root@testbox:~# devmem 2149646336 32 But, while devmem works great, that dd invocation ...
0
votes
2answers
294 views

Does a forked process copy all mapped memory when writing?

I understand that a forked process maps the parent's memory and copies it when writing to it. Does it copy just what it needs written or does it copy the entire mapped memory ?
1
vote
1answer
579 views

Monitoring page cache / memory mapped files access

I have an app that uses multiple memory mapped files. If I check the major page faults numbers (with /proc/<pid>/stat), they skyrocket. I was wondering if it's possible to monitor somehow what ...
7
votes
4answers
2k views

what is the purpose of memory overcommitment on Linux?

I know about memory overcommitment and I profoundly dislike it and usually disable it. A well written program could malloc (or mmap which is often used by malloc) more memory than available and crash ...
1
vote
0answers
222 views

journalctl's mmap() calls make it incredibly slow

Whenever I run journalctl, it runs astonishingly slowly. It often takes minutes or even hours to return results. I've experienced some other performance problems on the affected machine that seemed to ...
2
votes
1answer
3k views

program stack size

I learned default stack size for each process is limited to 8MB and mmap_base is calculated based on stack size in rlimit and random value. Code below is mmap_base function which calculates mmap_base ...
9
votes
1answer
10k views

Understanding MMAP

I was going through documentation regarding MMAP here and tried to implement it using this I have a few doubts regarding its implementation. Does MMAP provide a mapping of a file and returns a ...
2
votes
0answers
311 views

How to find part of the code which creates Anonymous private memory mapped region

I'm analyzing /proc/pid/smaps output for C++ process and I find large number of private memory mapped regions. Number of memory mapped regions keep growing based on the traffic. Is there a way I can ...
5
votes
1answer
2k views

If I mmap a file from tmpfs, will it double the memory usage?

I am working on a system where we lock files in memory using mmap and MAP_LOCKED and MAP_POPULATE for performance. If we do this with a file that is in tmpfs, will it use the existing tmpfs memory ...
0
votes
1answer
364 views

Do regular reads benefit from mmap?

If I use mmap on some file descriptor fd, will regular calls to read() be "aware" that there is a memory mapping of the file and read from memory directly? What if mlock() is used after mmap(). Is it ...
7
votes
1answer
184 views

How is the page cache mapped in the kernel on 64-bit x86 architectures?

On a modern 64-bit x86 Linux, how is the mapping between virtual and physical pages set up, kernel side? On the user side, you can mmap in pages from the page cache, and this will map 4K pages in ...
6
votes
1answer
691 views

OOM from iterating over very large memory map

I'm writing a simulator that relies on generating a (potentially) very large problem domain. As the data can't fit in RAM, I use 4 four memory mapped files to hold it. This is a 64-bit application ...
1
vote
0answers
62 views

The same program which uses 'mmap' behaves differently on the different machines

I am running an application which uses a huge memory mapped file on two different machines (CentOS 7). I found that the same program's performance is significantly different when it's running on two ...
2
votes
1answer
91 views

Doubt about mmap

I'm trying to build a cache package for golang that relies on mmap to store large amounts of content in memory, even more than the system has. The cache doesn't need to be persistant; every time the ...
1
vote
1answer
232 views

Pagemap on memory mapped devices not working

I am trying to find the physical addresses of heap variables, stack variables and memory mapped peripheral addresses using the /proc/{pid}/pagemap file using the steps detailed in the file: http://lxr....
3
votes
2answers
452 views

Can I add to /proc/self?

I want to create a number of named memory regions in my program, and mmap them somewhere so that other processes can read them. I can't guarantee that only one instance of my program will run at a ...
1
vote
3answers
1k views

mmap and slow DMA transfers

I have a process that reads data from a hardware device using DMA transfers at a speed of ~4 * 50MB/s and at the same time the data is processed, compressed and written to a 4TB memory mapped file. ...
0
votes
1answer
167 views

Linux Swap Space

I have 32GB memory. SSD and NonSSD drives. I installed Linux to NonSSD drive but I prefer SSD for /var directory and I didn't create swap space because there is high memory. I know physical memory +...
3
votes
1answer
772 views

mmap /dev/random

Why is it that I cannot mmap /dev/random or /dev/urandom on Linux? I get errno 19 which is ENODEV. When I try the same code with /dev/zero it works. int fd = open(path, O_RDONLY); assert (...
4
votes
3answers
12k views

How does mmap'ing /dev/mem work despite being from unprivileged mode?

As far as my understanding goes, User space programs run in the unprivileged mode, and thus do not have direct access to memory or I/O. Then how exactly can we directly access memory or I/O locations ...
3
votes
0answers
575 views

Using shared memory through tmpfs and NUMA on x86_64/Linux

Here's what I'd like to do: put a file on tmpfs but ensure that all of it (all pages of it) ends up on the same NUMA node. First, I thought this would be trivial: I'd just mmap() a tmpfs file with ...
7
votes
1answer
1k views

File backed, locked shared memory and disk interaction

Varnish, a HTTP accelerator, uses a ~80MB file backed SHM log that is mlock()ed into memory. The Varnish docs recommend to store the file on tmpfs to avoid unnecessary disk access. However if the ...
3
votes
1answer
101 views

Grsecurity subject mode x

I am conducting some research on Grsecurity on Hardened Gentoo, see http://en.wikibooks.org/wiki/Grsecurity. To be more specific, I am trying to find an example where subject mode x makes a difference....
2
votes
1answer
372 views

proc/sys/vm/flush_mmap_pages missing in 3.9 kernel, how to get it?

There was an option in redHat 5.2 called flush_mmap_pages, I could set this to zero and that would turn of writing dirty pages from a mapped memory file (so I can do it myself when I want to). However ...
1
vote
1answer
1k views

Ext4 exhibits unexpected write latency variance vs. ext2

I have a latency sensitive application running on an embedded system, and I'm seeing some discrepancy between writing to a ext4 partition and an ext2 partition on the same physical device. ...
5
votes
1answer
2k views

Behavior of mmap'd memory on memory pressure

I have a large tar file (60GB) containing image files. I'm using mmap() on this entire file to read in these images, which are accessed randomly. I'm using mmap() for the following reasons: Thread ...
9
votes
3answers
2k views

Memory size for kernel mmap operation

I'm interested in the way Linux mmaps files into the main memory (in my context its for executing, but I guess the mmap process is the same for writing and reading as well) and which size it uses. So ...
5
votes
1answer
5k views

Hung system call

So I'm working with a custom kernel module that I'm writing a python front end for. The kernel module works, and it adds a framebuffer device file to /dev/fb1. I can read and write to it fine. I've ...