Questions tagged [virtual-memory]
The virtual-memory tag has no usage guidance.
365
questions
0
votes
0
answers
16
views
In what order does kernel reclaim memory?
As far as my knowledge goes, when Kernel requires more memory for allocation for the processes it does a few things before entering OOM state.
Such as freeing SlabReclaimable, Dropping annonymous page ...
- 101
1
vote
0
answers
38
views
Is the stack of a forked process shared with its parent? [duplicate]
Is the stack of a forked process shared with its parent?
If so, does this happen via shared copy-on-write pages?
- 1,287
3
votes
0
answers
18
views
How do I undo copying a file that is ended up in cache?
I have copied a large file to USB drive.
The file appeared to be copied almost instantly, but copying operation stalled at the very end. /proc/meminfo is showing large, gigabyte Dirty: entry which is ...
- 5,126
0
votes
1
answer
14
views
Is vmalloc() allocate bytes of memory which is virtually contiguous maps to memory area from different physical pages?
vmalloc(size) allocates a memory of size long which is virtually contiguous but the physical mapping would not be contiguous. Does that mean the the virtually allocated size long memory actually lies ...
- 113
0
votes
0
answers
13
views
Consequence of an empty directory table entry
When a page directory entry is empty because the memory chunk associated to it is not allocated, would its emplacement contain junk data? And if so, can an access to a non-allocated address result in ...
- 27
2
votes
1
answer
115
views
Cgroup and process memory statistics mismatch
I am trying to set up some monitoring to see when a service is using too much memory. The memory usage can be read from two places:
the /proc/<pid>/status for the pid, or
the /sys/fs/cgroup/<...
- 562
1
vote
2
answers
128
views
Spike in number of page faults with make -j`nproc`
I am running a benchmark to figure out the number of jobs I should allow GNU Make to use in order to have optimal compile time. To do so, I am compiling Glibc with make -j<N> with N an integer ...
0
votes
1
answer
116
views
Dumping virtual memory addresses of an executable from /proc/pid/maps
I want to dump the virtual memory addresses of a running executable.
This is the command I'm running: ./executable & cat /proc/$(pidof executable)/maps > maps.dump, and this is the error I get: ...
0
votes
1
answer
86
views
When should I alter overcommit_memory and what should I take into consideration when doing so?
I'm having a PC freeze issue that I can't seem to figure out. I have three identical PCs. They are each custom builds with i7 and 64GB of RAM. The OS drives are 512GB nvme drives. They each run ...
- 253
9
votes
1
answer
1k
views
Does linux kernel use virtual memory (for its data)?
Does linux kernel make use of virtual memory for its data structures (page tables, descriptors, etc.)? More specifically:
Are kernel space addresses translated in the MMU (pagetable walking)?
Could ...
- 101
2
votes
0
answers
105
views
How to check if the memory is actually used by the OS in an Embedded Linux system?
This question needs additional clarifications.
Background: I am working on an embedded system (Xilinx zynq ultrascale+) which consists of an ARM processor + FPGA. The memory map (if I were to use the ...
- 149
0
votes
1
answer
139
views
Why can't user process READ memory in kernel address space?
It seems fairly obvious why user processes can't write or modify data on kernel address space. But I can't get my head around why they can't even read the data.
I know that a segmentation trap would ...
0
votes
0
answers
15
views
vmtouch-like utility for file metadata/VFS cache?
vmtouch can query cache information about a file's content and load (or even lock) that into cache.
Is there a utility which do the same with a file's metadata like dentries and inode information?
- 333
0
votes
1
answer
73
views
Can I use virtual memory for ARC ZFS write cache?
I want to use virtual memory for ARC ZFS write cache, because it is troublesome for me to increase physical memory but I want fast writing to a DB in a ZFS.
$ free -h
total used ...
- 2,036
0
votes
0
answers
71
views
pmap memory usage explanation
I'm a little confused about the results of this memory usage I'm seeing when using pmap (as a result of digging into the Apache2Buddy.pl script).
I'm trying to determine the correct values for ...
user495725
1
vote
1
answer
105
views
How does access_process_vm() work in linux?
AFAIK, the ptrace syscall uses access_process_vm to read data from other process. However, different process' address space are isolated, how is that achieved?
- 115
0
votes
0
answers
101
views
Which parts of a (docker) container are loaded into memory?
I was wondering how containers and RAM/swap are related.
Please correct me where I am wrong in my assumptions below.
All container-related functionality that the kernel offers is always loaded into ...
- 1,844
0
votes
1
answer
44
views
How is linux kernel's fix-mapping implemented?
In some configurations, the top 1G space of each process' memory belongs to the kernel. And in the 1G space, the low 896M part is direct mapped to physical address, called fix-mapping area.
But I'm ...
- 115
0
votes
1
answer
67
views
Do CPUs that support 5-Level Papging implement a 57-bit long address bus? [closed]
Despite having a data bus size of 64 bit, the address bus size of modern AMD64-compatible CPUs is/was 48 bit for some time which allows using 48-bit long virtual memory addresses with a maximum of ...
- 466
0
votes
0
answers
58
views
How to increase vmalloc or virtual memory or VAS on Android kernel
According to the FAQ here: https://support.huaweicloud.com/intl/en-us/ug-exagear-kunpengdevps/kunpengexagear_06_0045.html
Q7: What is the virtual address range of ExaGear?
A: The length of the ...
- 1
0
votes
0
answers
31
views
does socket buffer occupy process memory address?
As we know, we can use setsocketopt api to alter the socket's buffer size.
Since the socket fd is created by user process, does it occupy process memory address, like the top 1G kernal space in each
...
- 115
0
votes
2
answers
214
views
How does VA to PA translation in a 4-level page table just take 4 memory accesses
I am learning page table management, and I learned that VA to PA translation takes 4 memory accesses in a 4-level page table (considering TLB miss and miss in page walk cache).
But, as Linux uses the ...
- 41
1
vote
2
answers
73
views
Does swap out of memory pages belonging to a process pauses or stop the process itself?
I have a real-time networking application that is extremely sensitive to any kind of "blocking" and needs to be extremely responsive most of the time with perfect latency.
The app (...
- 1,492
-1
votes
3
answers
52
views
How to configure cache memory to automatically sync on program close?
Currently I have an issue with programs not de-allocating their stack upon close. It appears to be a natural behaviour of Linux page caching.
When I open my desktop, it runs at ~300-500MB. After ...
- 111
0
votes
0
answers
24
views
Mystery memory accumulating instead of swapping after a program consumes all 16GB available to it in its cgroup
I'm trying to evaluate the memory usage of some software X in a memory-restricted setting. All that's really necessary to know about X for the purposes of this problem is that its memory consumption ...
- 101
0
votes
1
answer
39
views
Why does Virtual Memory exist as a metric?
Why do tools expose the value of virtual memory? What value does it have?
RSS is a simple to understand and correlates directly to requirements/availability of physical RAM.
Virtual memory does not ...
- 103
4
votes
4
answers
812
views
more swapping on RHEL8 than RHEL7?
we've been having a Linux application for quite some time (our journey started a decade ago with RHEL 4). We were recently running our app on RHEL 7.9 and are now moving to RHEL 8.4.
We've always ...
0
votes
1
answer
67
views
Understanding how internal fragmentation occurs in systems using only paging with huge page size
The excerpt below is from the OS text by Galvin et. al.
When we use a paging scheme, we have no external fragmentation: any free frame can be allocated to a process that needs it. However, we may ...
- 111
0
votes
1
answer
109
views
How to use SLAB memory allocation instead of SLUB allocation in Linux kernel?
As I understand it, both SLAB and SLUB are still integrated into the kernel in stable release 5.12. Is there a way to enable SLAB allocation to run on the system instead of SLUB allocation? Or is ...
- 153
0
votes
0
answers
20
views
How do the Contiguous allocation, Segmentation, and Paging allocation schemes deal with the situation of expanding process memory?
How do the Contiguous allocation, Segmentation, and Paging allocation schemes deal with the situation of expanding process memory? For example there may be a growth in the data structures that a ...
0
votes
0
answers
33
views
How to think about swap space allocation?
Q. How do you think about how much swap space to allocate?
Does Linux (Debian 10 in this case) need enough swap space to swap out some VM pages plus more to swap in other pages, and thus the RAM + ...
- 101
0
votes
0
answers
40
views
OpenBSD 6.9 - ENOMEM error on execve (execvp)
I've compiled a program on OpenBSD 6.9 and when I execute the program, I get an enomem error related specifically to execvp which the man pages say is a font-end for execve.
The execve man page ...
1
vote
0
answers
62
views
linux enable large page management
I am doing some experiments. Some huge pages (2MB) are used in the experiment, so that the 21-bit page offset can remain unchanged when performing virtual address translation.
I found some methods on ...
- 173
0
votes
1
answer
49
views
Linux report that lowmem region is more than physical memory available?
I'm running Ubuntu 20.04 64-bit version. I'm starting to learn about kernel programming and I'm now studying kernel VAS. when running a process called procmap by kaiwan on github which is supposed to ...
- 105
1
vote
0
answers
46
views
slub_min_objects : How comes 0 can stand as a valid / default value?
In the comments of the source code (mm/slub.c) of my linux_5.4 kernel, I can read :
In order to reach satisfactory performance we must ensure that a minimum
number of objects is in one slab. Otherwise ...
- 2,151
1
vote
0
answers
51
views
Why oom may still happen even when overcommit_memory is set to 2?
2: always check, never overcommit
In mode 2 (available since Linux 2.6), the total
virtual address space that can be allocated
(CommitLimit in /proc/meminfo) is calculated as
...
- 267
0
votes
1
answer
385
views
How can my PCI device driver remap PCI memory to userspace?
I am trying to implement a PCI device driver for a virtual PCI device on QEMU. The device defines a BAR region as RAM, and the driver can do ioremap() this region and access it without any issues. The ...
0
votes
1
answer
84
views
Two processes mapped in the same address space
I have noticed that when I open two processes with GDB for debugging, /proc/PID/maps indicate the same address regions for the two :
cat /proc/4170/maps
555555554000-555555555000 r--p 00000000 08:01 ...
- 1
2
votes
0
answers
104
views
/sys/kernel/slab/ contents :at-0000016 :t-0000040 :dt-0001024 etc. meaning
While examining some memory leak I found a strange for me contents of /sys/kernel/slab/ directory. It contains a list of directories:
:at-0000016 :t-0000040 :t-0000832
:at-0000032 :t-0000048 :t-...
- 605
2
votes
0
answers
45
views
Kernel refusing to free up disk cache
I have ~20GB of RAM and ~4GB of swap on my machine. Around 8GB of memory is used by applications, the rest is used for disk space by the kernel. Over time, the swap fills up completely and then ...
- 163
1
vote
1
answer
848
views
Make a temp virtual RAM disk from some RAM + some SSD
I'm in an odd situation where I have plenty of RAM sitting around (200gb extra) and ALMOST enough SSD to do a read/write intensive process. Is there any way to say "Dear System, please create a ...
- 153
3
votes
0
answers
60
views
Calculating Free Memory using kstat vs vmstat vs mdb
In attempting to calculate the free memory under Solaris 10, I noticed I get different results using vmstat versus kstat or mdb.
The basic commands:
/usr/bin/kstat -p unix::system_pages:freemem | cut -...
- 131
1
vote
2
answers
238
views
Is releasing memory pages by a process possible in principle?
Disclaimer: I do have a diploma in CS but this topic was never explicitly explained to me. I am just stiching pieces of what I know into a whole. Please correct me if I get something wrong.
Suppose ...
- 443
2
votes
0
answers
108
views
Memory usage out-of-control with shell command, but not with shell script
[Linux Mint 20.1, 16GB RAM, 16GB swap]
I recently converted my system - root and backup partitions - to BTRFS. I've been experimenting with archiving some of root's snapshots to my backup drive, but ...
- 2,554
2
votes
1
answer
93
views
Why my python process cannot access all the memory?
I have 62 GB RAM in my Linux server and my Python code is trying to load 20 GB file in the memory. However, it is crashing and throwing MemoryError.
I am not sure why it would be like this?
I know I ...
- 131
1
vote
1
answer
42
views
Linux use huge pages only
I have a x64 Linux system. The page size reported by getconf is 4 k:
$ getconf PAGESIZE
4096
I want the kernel to use only large pages (2 M or 4 M) for all memory allocations. I've calculated that I ...
- 193
0
votes
2
answers
122
views
Virtual Address Space [closed]
I have started to learn about Virtual Address Space (VAS) and I have few questions:
How much of VAS is created for each process depending on the architecture (32-bit and 64-bit)?
Is VAS for each ...
- 101
0
votes
1
answer
1k
views
How to get the mapping between physical and virtual memory page addresses in Linux/Amd64?
I would like to know, which physical pages are being mapped by induvidual processes.
/proc/<pid>/maps shows the virtual address mapping. I think it would be a good start, if I could somehow ...
- 8,374
1
vote
1
answer
4k
views
How to display current vm.map_max_map_count value?
I tried looking into /proc/sys/vm/max_map_count file and it says 'empty' I have added a line vm.max_map_count=16777216 in etc/sysctl.config and want to see if changes are applied.
I have certain ...
- 27
0
votes
1
answer
27
views
System freezes when paging. Any way to warn in advace?
I often use my laptop with a lot of tabs open, plus some other memory heavy applications such as Pycharm. It happens maybe once a day when I am working that I open a new tab and the system freezes as ...
- 199