Questions tagged [virtual-memory]
The virtual-memory tag has no usage guidance.
393
questions
4
votes
1
answer
246
views
Why does my system heavily use swap when there are 86 GB of RAM still usable?
This is not so much a performance problem as it is a desire to understand what is happening and how it works. I have a system with lots of resources, including 128 GB of RAM. What I have discovered (...
- 4,354
0
votes
1
answer
33
views
Heap memory allocation in Linux
I'm confused about whether the memory allocated by Linux when a process requests 'x' amount of heap is actually contiguous physically or not?
Here's my understanding till now:
The unit of memory ...
- 165
0
votes
0
answers
52
views
Linux HugeTLB: What is the advantage of the filesystem approach?
Moved Post Notice
I just moved this question (with slight modifications) from a StackOverflow question (which I have deleted, since cross-posting is strongly discouraged), which has not been answered ...
- 181
5
votes
2
answers
754
views
Who decides the physical address for a particular virtual address, kernel or MMU?
I am trying to understand how the physical addresses are allocated when running an OS.
My question is when the kernel allocates some memory (lets say using kmalloc), who decides what physical memory ...
- 75
0
votes
0
answers
45
views
Will the implementation of mremap free physical pages?
Recently, I am reading the memory management part of linux kernel code (the version I am reading is 4.10). I find that the implementation of mremap is strange.
In short, sys_mremap() will call ...
- 1
0
votes
0
answers
77
views
Use only 2MB pages
I would like to use a linux version in which only 2MB pages are used (i.e., no libhugetlbfs and transparent huge pages).
From other similar questions (e.g., Linux use huge pages only), I understood ...
- 1
0
votes
1
answer
31
views
Do page tables used to store kernel stack pointer when context switch happen in kernel mode of the process?
I have two questions;
Suppose a user space application/process is running in kernel mode. I understand if a context switch is happened now, the kernel stack pointer of that process is stored in the ...
- 229
1
vote
0
answers
60
views
Is it possible to maintain performance of "preferred" applications when certain resources are limited?
I work as a web developer, and due to the tools I generally have running (not to mention web browsers) I generally have high RAM usage at all times. Most of the time, my system handles this just fine. ...
1
vote
1
answer
123
views
How to create a private swap file for a specific process or cgroup in Linux?
I'm currently attempting to generate a swap file in Linux that is exclusive to a particular process or cgroup. After going through the LWN article which discusses the patches that enable this ...
- 11
0
votes
0
answers
65
views
How does memory mapping File works in OS?
In OS concepts, it states that:
memory mapping a file, allows a part of the virtual address space to be logically associated with the file
my concept is:
Usually when a process reads a file on disk ...
0
votes
1
answer
629
views
Trying to understand memory-mapping with C++ in Linux environment
I've been tasked at worked to explore memory mapping to see if we can utilize it.
I'm trying to wrap my head around the concept and how to write the code. I've been trying out the code in the ...
0
votes
1
answer
37
views
How CPU knows/enforcing segment/page access violation?
x86 IA-32
In my understanding accessing memory from user space program does not involve operating system.. Otherwise it would be slow.
So if paging enabled or not CPU should know somehow that this ...
- 3
0
votes
0
answers
32
views
Swap usage relative to memory usage makes no sense
I research memory and swap usage for a company that develops it's own flavor of linux. I gathered usage data from over 25000 machines, and can't find any correlation between the two values.
On some ...
0
votes
0
answers
60
views
Exam question - 64-Bit Virtual memory - Proper address for pointer variable
In a mock exam there is the following question:
Which of the following addresses is a proper memory address for a
pointer variable on an x86_64 processor?
0x7ffffff3328
0x0
0x8043844
0x555555ab3b
...
0
votes
1
answer
198
views
How do I open a mmap of size of a TB
I need to open a huge memory map. The file is one terabyte.
I however am getting an errno: ENOMEM 12 Cannot allocate memory. I don't get what is holding me up. Requesting the RLIMIT_AS results in the ...
- 123
0
votes
0
answers
86
views
How mmap works with EFS?
We are using vector search engine (qdrant) which is mounted on EFS with mmap configuration.
Recently we started seeing few errors which made us to realise that using database mounted on EFS and using ...
- 3
0
votes
1
answer
79
views
Why does hibernation require a swap space to be roughly equal in size to total system RAM when we have file mapping and demand paging?
It is my understanding that pages in memory containing things like program text and shared libraries that have associated real world files will actually map to that associated file instead of swap (i....
- 1
1
vote
1
answer
24
views
How to Get Total Count of Non-Swappable Pages in RAM?
I am trying to debug some ML code that requires pinned physical memory, and in order to help with this, I would like to figure out how much of physical RAM consists of pages that can't be swapped out (...
- 11
1
vote
1
answer
205
views
mapping device file benefits
I wanted to ask what are the benefits of mapping a device file using mmap.
There is a driver which implements file operations API for that device file.
I'm asking since device files are not stored on ...
- 189
0
votes
0
answers
118
views
How do you map a pointer location to /proc/self/{map,mem}?
I am trying to write a light weight test to determine if sensitive parameters are getting zero'd in active memory after use. Here is what I thought would work:
Get the pointer location of the ...
- 158
0
votes
0
answers
624
views
Creating a "virtual/ fake" usb drive on linux filesystem
Reference post: Create virtual USB drive
Problem statement:
I would like to create/ convert and area on my (Ubuntu 20.04) filesystem such that application's think a USB is connected to my machine, and ...
- 1
0
votes
1
answer
339
views
How to dump the stack and heap of a running application to determine if zeroization is occuring?
I am trying to ensure that after the use of sensitive parameters, a program zero-es them from all active memory (stack and heap). The simple test I was hoping to perform was to dump the active memory ...
- 158
0
votes
0
answers
92
views
What is the complexity of mremap?
In case when I allocate an (anonymous) array of n consecutive virtual pages in one call of mmap() and access them all so that they are all mapped to physical RAM (or swap), what is the time complexity ...
- 214
0
votes
1
answer
123
views
.got.plt not loaded into memory
I have an aarch64 process (cat) running on an Android (8). I notice this section in /proc/<PID/maps:
746f308000-746f3de000 r-xp 00000000 103:00 5402 /system/lib64/libc++.so
746f3de000-...
- 697
1
vote
1
answer
164
views
LOAD segment not loaded
While examining the /proc/<PID>/maps file for an aarch64 process running on an Android (8), I notice this section:
726048e000-7260564000 r-xp 00000000 103:00 5402 /system/lib64/libc++.so
...
- 697
1
vote
0
answers
28
views
What happens if a running executable can no longer load its code/data pages?
This is under the assumption that the binary is large enough that not all pages were loaded into memory at the start of execution. What would happen if the underlying file system suddenly disappears (...
- 121
3
votes
1
answer
625
views
Can a Linux Swap Partition Be Too Big?
Can a Linux swap partition be too big?
I'm pretty certain the answer is, "no" but I haven't found any resources on-point, so thought I'd ask.
In contrast, the main Windows swap file, ...
- 281
3
votes
1
answer
100
views
.text section not readable
I was looking at the /proc/<pid>/maps file for an aarch64 process running on an Android (10). This section caught my interest:
72582d2000-72582f6000 --xp 00012000 07:20 93 /apex/com....
- 697
0
votes
1
answer
488
views
How does MMU find out what address it should translate?
When we try to read/write some data to/from a specific address we must know where the first page directory is stored and the address we want translate.
As for the first page directory (as far as I ...
- 133
-1
votes
1
answer
453
views
How to access virtual address via terminal in Ubuntu?
i'm trying to read & write to register by using the busybox devmem command.
I printed its base address from the driver's source code and received the following address:
0xffffff800bdd0000
I ...
0
votes
0
answers
43
views
In Linux what is the difference between swapping and paging? [duplicate]
I am trying to figure out what the difference is between paging and swapping on linux. My question mainly relates to the output vmstat gives you. Example:
$ vmstat -s
8022500 K total memory
...
- 101
0
votes
0
answers
126
views
Addressing in linux
I'm trying to understand addressing and memory. the book Embedded Linux primer says
Notice that the process called hello thinks it is executing somewhere
in high RAM just above the 256MB boundary (...
- 1
0
votes
0
answers
30
views
My swap is getting full and I don't know what's happening
RANGE SIZE STATE REMOVABLE BLOCK
0x0000000000000000-0x0000000087ffffff 2.1G online yes 0-16
0x0000000100000000-0x0000000477ffffff 13.9G online yes 32-...
1
vote
1
answer
129
views
Linux memory allocation for user's process (buddy allocator)
I'm studying memory allocation in Linux and I have a question about the buddy allocator. Reading the documentation available on kernel.org I've learn that the buddy allocator is responsible for the ...
- 121
2
votes
0
answers
47
views
Overlapping address ranges in /proc/.../maps
On some Android devices (possibly only on Android 10 and newer), looking at /proc/self/maps, I observed entries like the following two lines:
79cfd4d000-79cfff5000 --xp 0012e000 fd:00 286 /apex/com....
- 133
1
vote
0
answers
47
views
Is the stack of a forked process shared with its parent? [duplicate]
Is the stack of a forked process shared with its parent?
If so, does this happen via shared copy-on-write pages?
- 1,493
3
votes
0
answers
44
views
How do I undo copying a file that is ended up in cache?
I have copied a large file to USB drive.
The file appeared to be copied almost instantly, but copying operation stalled at the very end. /proc/meminfo is showing large, gigabyte Dirty: entry which is ...
- 5,558
0
votes
1
answer
102
views
Is vmalloc() allocate bytes of memory which is virtually contiguous maps to memory area from different physical pages?
vmalloc(size) allocates a memory of size long which is virtually contiguous but the physical mapping would not be contiguous. Does that mean the the virtually allocated size long memory actually lies ...
- 229
4
votes
1
answer
3k
views
Cgroup and process memory statistics mismatch
I am trying to set up some monitoring to see when a service is using too much memory. The memory usage can be read from two places:
the /proc/<pid>/status for the pid, or
the /sys/fs/cgroup/<...
- 641
1
vote
2
answers
236
views
Spike in number of page faults with make -j`nproc`
I am running a benchmark to figure out the number of jobs I should allow GNU Make to use in order to have optimal compile time. To do so, I am compiling Glibc with make -j<N> with N an integer ...
0
votes
1
answer
918
views
Dumping virtual memory addresses of an executable from /proc/pid/maps
I want to dump the virtual memory addresses of a running executable.
This is the command I'm running: ./executable & cat /proc/$(pidof executable)/maps > maps.dump, and this is the error I get: ...
- 1
0
votes
1
answer
2k
views
When should I alter overcommit_memory and what should I take into consideration when doing so?
I'm having a PC freeze issue that I can't seem to figure out. I have three identical PCs. They are each custom builds with i7 and 64GB of RAM. The OS drives are 512GB nvme drives. They each run ...
- 253
9
votes
1
answer
1k
views
Does linux kernel use virtual memory (for its data)?
Does linux kernel make use of virtual memory for its data structures (page tables, descriptors, etc.)? More specifically:
Are kernel space addresses translated in the MMU (pagetable walking)?
Could ...
- 101
2
votes
0
answers
504
views
How to check if the memory is actually used by the OS in an Embedded Linux system?
This question needs additional clarifications.
Background: I am working on an embedded system (Xilinx zynq ultrascale+) which consists of an ARM processor + FPGA. The memory map (if I were to use the ...
- 149
0
votes
1
answer
435
views
Why can't user process READ memory in kernel address space?
It seems fairly obvious why user processes can't write or modify data on kernel address space. But I can't get my head around why they can't even read the data.
I know that a segmentation trap would ...
1
vote
0
answers
70
views
vmtouch-like utility for file metadata/VFS cache?
vmtouch can query cache information about a file's content and load (or even lock) that into cache.
Is there a utility which do the same with a file's metadata like dentries and inode information?
- 584
0
votes
1
answer
225
views
Can I use virtual memory for ARC ZFS write cache?
I want to use virtual memory for ARC ZFS write cache, because it is troublesome for me to increase physical memory but I want fast writing to a DB in a ZFS.
$ free -h
total used ...
- 2,146
1
vote
1
answer
373
views
How does access_process_vm() work in linux?
AFAIK, the ptrace syscall uses access_process_vm to read data from other process. However, different process' address space are isolated, how is that achieved?
- 115
0
votes
1
answer
405
views
How is linux kernel's fix-mapping implemented?
In some configurations, the top 1G space of each process' memory belongs to the kernel. And in the 1G space, the low 896M part is direct mapped to physical address, called fix-mapping area.
But I'm ...
- 115
0
votes
1
answer
720
views
Do CPUs that support 5-Level Papging implement a 57-bit long address bus? [closed]
Despite having a data bus size of 64 bit, the address bus size of modern AMD64-compatible CPUs is/was 48 bit for some time which allows using 48-bit long virtual memory addresses with a maximum of ...
- 466