Questions tagged [mmap]

All about using memory mapped files. Questions on programming should be asked on Stack Overflow SE.

Filter by
Sorted by
Tagged with
0 votes
0 answers
26 views

How to read watchdog registers on x86 Linux?

I want to read the Intel iTCO watchdog registers on my Intel Lynx Point system. I found the watchdog here: [ 5598.341020] iTCO_wdt iTCO_wdt.1.auto: Found a Lynx Point TCO device (Version=2, TCOBASE=...
defoe's user avatar
  • 153
0 votes
1 answer
28 views

Can mmap be used to create a file which references memory subset of another file?

I'm interested in writing a program that can create two files, second file would be a "view" of first file and if modified, the first file would also be modified. Is this possible to do with ...
trickingLethargy's user avatar
0 votes
0 answers
25 views

Searching a 1T block device for a specific byte sequence at a specified offset

I'm performing data recovery after an accident with dd. In the longer term, I'll need to use some recovery tools to try and repair the file system In the meantime, there's an image on the system that ...
user587941's user avatar
0 votes
1 answer
38 views

How to measure mmap I/O latency?

I have an application which appears to be slowing/blocking at the same time there's a lot of disk I/O going on, so I suspect it's I/O operations within the application which are blocking. I can't ...
phemmer's user avatar
  • 71.3k
0 votes
0 answers
122 views

Linux HugeTLB: What is the advantage of the filesystem approach?

Moved Post Notice I just moved this question (with slight modifications) from a StackOverflow question (which I have deleted, since cross-posting is strongly discouraged), which has not been answered ...
Lukas Barth's user avatar
0 votes
1 answer
50 views

Mapping Segment of Guest RAM to host file, in PPC QEMU

My desire is conceptually simple, I have a file (really a PCIe resource file from /sys/bus/pci/device/.... but that isn't too relevant) on the host that I want to make available somewhere in guest ...
Seth Robertson's user avatar
1 vote
1 answer
71 views

Partial fsyncs when writing to block device

I'm writing my own data store directly on top of a block device. To ensure durability I want to sync to disk. But here's the thing: I want to sync only part of it. I'm keeping a journal for crash ...
Jille Timmermans's user avatar
1 vote
1 answer
994 views

Is it possible for two processes to use the same shared-memory without resorting to a file to obtain it, be it a memory-mapped file or /dev/shm file?

I'm curious because today the only way I know how to give two different processes the same shared-memory is through a memory-mapped file, in other words, both processes open the same memory-mapped ...
ThreadFrank's user avatar
0 votes
1 answer
35 views

What Linux distros support mmap_shared?

I found a man page for a function called mmap_shared on die.net: https://linux.die.net/man/3/mmap_shared It looks like a convenience wrapper function around mmap, however it doesn't seem to be ...
Wolfy's user avatar
  • 135
1 vote
0 answers
55 views

Impossible to use memory allocated to buffers with transparent huge pages (process killed)

I have 1TB of RAM, 900GB of which I need to allocate and use in a process (I have complete control on the hardware and I'm working on bare metal). I allocate 900GB of memory using mmap() (private, ...
seba's user avatar
  • 251
0 votes
0 answers
84 views

Is it possible ro remap huge pages (e.g. 2 MiB) into smaller ones (4 KiB)?

In my application I have a huge chunk of memory allocated using mmap. Depending on runtime configuration options this chunk can be allocated using huge pages (i.e. with the MAP_HUGETLB flag). This ...
user3234005's user avatar
0 votes
1 answer
245 views

How do I open a mmap of size of a TB

I need to open a huge memory map. The file is one terabyte. I however am getting an errno: ENOMEM 12 Cannot allocate memory. I don't get what is holding me up. Requesting the RLIMIT_AS results in the ...
Tarick Welling's user avatar
0 votes
0 answers
109 views

How mmap works with EFS?

We are using vector search engine (qdrant) which is mounted on EFS with mmap configuration. Recently we started seeing few errors which made us to realise that using database mounted on EFS and using ...
Swastik's user avatar
1 vote
0 answers
44 views

Can kernel's flushing/syncing of mmaped file interrupt a write() call?

I know there is (shamefully) no way to prevent kernel from flushing dirty pages of files mmap-ed with MAP_SHARED, so to take the control back, I wanted to map a file with MAP_PRIVATE, then when I want ...
donaastor's user avatar
  • 214
3 votes
1 answer
4k views

Does mmap allow creating a mapping that is much larger than the amount of physical memory?

Obviously, mmap is limited by the size of the largest available block of virtual address space. But let's assume we have a 64-bit system, where this is almost unlimited for most purposes. Let's also ...
Brian Bi's user avatar
  • 572
1 vote
1 answer
94 views

Is there a way to open a file and prevent flushing updates to hardware unless I command it to?

I know for kernel functions open(), mmap() and msync() and what they do. But no matter what combination of flags I attempt to use, the kernel will either still flush updates using some of its own time ...
donaastor's user avatar
  • 214
2 votes
0 answers
132 views

Question about transfer data between specific physical memory and NVME

I'm trying to transfer data from specific reserved physical memory to my NVME, IIUC, there are 3 steps: In the kernel driver, the reserved-memory (specified physical address and size in dts) is ...
Chen Li's user avatar
  • 367
0 votes
1 answer
145 views

Why Disk stats show many read operations when I measure NVME squance write with fio and mmap as ioengine

Here is my fio configure and report: # cat fio-write.fio [global] name=fio-seq-writes filename=test rw=write bs=1M direct=0 numjobs=1 [file1] size=1G ioengine=mmap iodepth=1 # fio --version fio-3.30 ...
Chen Li's user avatar
  • 367
1 vote
0 answers
232 views

EXT4 journal=data and mmap

Does enabling data=journal for ext4 make any difference when using mmap() to update a file? The ext4 man page says: journal All data is committed into the journal prior to being written into the main ...
lol's user avatar
  • 150
0 votes
1 answer
353 views

In what segment does mmap allocate memory

I thought first that it was the heap, but it seems to allocate memory in a different place. radare2 tags it as folowing: 0x00007fb07dacd000 - 0x00007fb07dace000 - usr 4K s rw- unk2 unk2 Since it ...
Aramya's user avatar
  • 37
0 votes
1 answer
67 views

Is the memory usage of typical software exaggerated?

Suppose multiple processes are using the same shared library (such as Gtk). Presumably, they mmap the library file, and physically, the RAM is shared? However, the size of the library gets added to ...
MWB's user avatar
  • 123
1 vote
0 answers
178 views

Howto enable Nginx memory caching with mmap()

I am a newbie nginx admin. I used nginx-quic to get features like HTTP/3 over QUIC. However, HTTP/3 perfoms 2x to 3x slower than HTTP/2, so I am trying to optimize the experimental server. (The test ...
mt42's user avatar
  • 133
2 votes
2 answers
688 views

Write in /dev/mem without using mmap

It is possible to write on /dev/mem without using mmap? I'm enabling pull-up resistors on a Raspberry Pi inside an LKM and the function void *mmap (caddr_t addr, size_t len, int prot, int flags, int ...
Roger Miranda Perez's user avatar
1 vote
1 answer
2k views

Do docker containers share RAM for files memory mapped from the same layer but a different image?

I'm not 100% certain about whether this is a U&L question or a SO question. On balance I'm posting it on U&L as it's OS related. Background As far as I know, Linux will load shared libraries (...
Philip Couling's user avatar
1 vote
1 answer
960 views

mmap(): Is it possible to prevent writing back to file with MAP_SHARED flag?

As I understand, 'MAP_SHARED' flag in mmap() shares any changes made by a process to the memory map immediately with other processes and eventually writes the changes back to the file. Is it possible ...
user487627's user avatar
1 vote
0 answers
137 views

linux enable large page management

I am doing some experiments. Some huge pages (2MB) are used in the experiment, so that the 21-bit page offset can remain unchanged when performing virtual address translation. I found some methods on ...
Yujie's user avatar
  • 183
1 vote
0 answers
87 views

Is there a way to tell how much memory is file-backed on Linux process?

I'm looking to see how much of my memory is file-backed on a process. Unfortunately proc/pid/smap_rollup shows stats for how much memory is being used, but it doesn't seem to differentiate between ...
HardcoreHenry's user avatar
-1 votes
1 answer
752 views

What is the path of data transfer when using memory mapped file?

Is there any difference between the data transfer paths using read()/write() and using mmap() on a file? What does "kernel" mean in https://stackoverflow.com/a/41419353? mmap doesn't ...
Tim's user avatar
  • 100k
0 votes
1 answer
1k views

Do anonymous memory mapping and shared memory allocate space from physical memory only?

In Linux, when using mmap() for anonymous memory mapping, or using malloc(), do they allocate "space" from only physical memory, or either physical memory or swap or their combination? (I ...
Tim's user avatar
  • 100k
3 votes
1 answer
1k views

mmap a file vs mmap in malloc

I'm kind of confused by mmap. Well, I know that when we malloc a big size of memory, we will invoke the function mmap, which will allocate an area in memory. In this case, mmap just allocate some ...
Yves's user avatar
  • 3,221
2 votes
2 answers
3k views

What is the behaviour of a file-backed memory map when reading from or writing to an address larger than the length of the file?

I'm trying to figure out whether it would be undefined behaviour to open a memory map to a file with a very large length. My use-case is that I want to be able to see new data when the file is ...
Jack Fransham's user avatar
2 votes
1 answer
1k views

Mapping guest RAM to file in qemu

We're emulating a Cortex M3 cpu and would like to pass some parameters to the guest during run-time. The simplest idea seems to be to write directly to some memory area. I tried simply adding -mem-...
Benjamin Lindqvist's user avatar
0 votes
0 answers
197 views

SIGBUS immediately after successful mremap for shared memory

I am using POSIX shared memory. Initially I map four shared memory buffers. One of them exceeds its allocated size so I call mremap: void * remap_shm(void *old_address, size_t old_size, size_t ...
RTC222's user avatar
  • 125
6 votes
1 answer
1k views

Is it possible to disable unnecessary disk writes to mmap files on linux?

I would like to know if there is a way to prevent Linux from periodically syncing mmap'd files to disk, while still allowing the OS to write back when physical memory gets tight. I am writing ...
Peter Fletcher's user avatar
2 votes
1 answer
1k views

How to revoke write permissions on a shared memory object s.t. subsequent writes to aleady mapped pages by other processes will fail?

Is there any way to revoke write permissions to shared memory by the process who created that shared object, s.t. any other process who has mapped the shared memory to its virtual space with write ...
Kristianmitk's user avatar
1 vote
1 answer
403 views

Memory map to process for a large code segment

I know that pmap of a process shows how the memory mapping is done. For example, the first lines of pmap output shows the memory mapping to the text segment of the process' executable. Assume I have a ...
Franc's user avatar
  • 249
1 vote
0 answers
262 views

php 7.3 mmap munmap too much calls\time on Ubuntu 18+

Good day everyone. Here i have problem with after migrating highload backend php server to new one. ( Dedicated ) . For this point i started comparing ubuntu 16 and 20 ( 18 the same ) . I bought 2 ...
TheFrozen's user avatar
2 votes
2 answers
905 views

What is lazy swap reservation?

I am reading the Linux Progamming Interface. 49.9 MAP_NORESERVE and Swap Space Overcommitting Some applications create large (usually private anonymous) mappings, but use only a small part ...
Rick's user avatar
  • 1,147
1 vote
0 answers
3k views

Mmap failed with operation not permitted

I've compiled application on Linux, but I got this error wiringPiSetup: mmap (GPIO) failed: Operation not permitted After that: I've added iomem=relaxed to grub cat /proc/cmdline BOOT_IMAGE=/...
unknown's user avatar
  • 11
2 votes
1 answer
718 views

User-space swap on mmap()-ped files

Imagine a system with very small ram (nearly embedded). It is enough - for nearly all tasks, but with an exception. There is a task X, which requires a huge amount of RAM what the machine has not. ...
peterh's user avatar
  • 9,681
2 votes
1 answer
2k views

Mmaping tremendously large files

I have a very large disk drive (2TB), but not very much RAM (8GB). I'd like to be able to run some big data experiments on a large file (~200GB) that exists on my disk's filesystem. I understand that ...
user308485's user avatar
0 votes
0 answers
51 views

Discrepancy between linux man page and implementation of mmap

I am trying to compile a program that requires the mmap(2) to support the flag MAP_SYNC, but my compiler reports that no such thing exists. Now, I manually verified the header file for mmap and found ...
lol's user avatar
  • 150
2 votes
1 answer
2k views

How do I create a temporary file in memory?

I need to create some (small) temporary files in my zsh scripts, and I want them to live on RAM so as to avoid disk IO. What are the idiomatic ways of doing this? I want it to work on Linux and macOS, ...
HappyFace's user avatar
  • 1,543
1 vote
1 answer
1k views

Weird major page fault number when reading sequentially / randomly in mmap region

I'm following this answer, trying to generate some major page faults with mmap: #include <fcntl.h> #include <stdio.h> #include <sys/mman.h> #include <sys/stat.h> int main(...
zingdle's user avatar
  • 175
4 votes
1 answer
3k views

mmap: effect of other processes writing to a file previously mapped read-only

I am trying to understand what happens when a file, which has been mapped into memory by the mmap system call, is subsequently written to by other processes. I have mmaped memory with PROT_READ ...
user001's user avatar
  • 3,638
0 votes
1 answer
434 views

mmap - map to address above 2^48

I understand that the used address space of 64-bit PCs is [0,2^48), but can I use mmap to map a file to an address above 248? I wrote the following code but found out the mapped address was still ...
z.h.'s user avatar
  • 944
1 vote
1 answer
299 views

unlink/rm target of an open symlink

I've come across an issue where processes that load symlinked .so files crash if I update the .so while the processes are still running. I have long-running processes that load shared libraries (.so)...
Marshall Versteeg's user avatar
7 votes
1 answer
984 views

ld.so.cache and libc.so.6 memory-mapped for every call?

Playing with strace, it appears to me that ld.so.cache and libc.so.6 are opened and mapped to memory for almost every process. At least those processes that I experimented with. Doesn't this mean ...
smolloy's user avatar
  • 173
3 votes
0 answers
676 views

Is there a standard for the Linux user-space memory map?

I'm contemplating a project where I would use a memory mapped file that is shared between multiple processes, and in order to be able to use pointer values in the mapped region in all those processes*,...
Dolda2000's user avatar
  • 315
27 votes
2 answers
28k views

How does memory mapping a file have significant performance increases over the standard I/O system calls?

Operating System Concepts says Consider a sequential read of a file on disk using the standard system calls open(), read(), and write(). Each file access requires a system call and disk access. ...
Tim's user avatar
  • 100k