Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [cache]

cache is a component that stores data so future requests for that data can be served faster; the data stored in a cache might be the results of an earlier computation, or the duplicates of data stored elsewhere.

1
vote
0answers
31 views

What prevents page migration?

OpenSUSE 42.3, Kernel 4.4.175-89-default Running memory bandwidth intensive applications I realised the following behaviour: The application uses ~55% of the physical memory of a NUMA system with 2 ...
2
votes
1answer
44 views

How I can preconfigure the write mode of drive cache?

journalctl -p err gives [sdX] Assuming drive cache: write through for each of my hard drives, where X is from a to f. These are normal hard drives, Ubuntu runs under VMWare. How I can preconfigure the ...
2
votes
1answer
94 views

Does a tmpfs (/dev/shm) use Linux Page Cache? If it does, then how?

AFAIK, Linux has a page cache to improve performance (example, if you open a file, linux caches that file in RAM), then if file is requested again and it's cached, the OS avoids read the file from ...
10
votes
1answer
563 views

Is “Cached” memory de-facto free?

When running cat /proc/meminfo, you get these 3 values at the top: MemTotal: 6291456 kB MemFree: 4038976 kB Cached: 1477948 kB As far as I know, the "Cached" value is disk ...
0
votes
0answers
22 views

Sizing of Samba Linux Server

Do you know any performance of sizing recommendations about Samba 4.7.1 ? For few informations : 2vCPUs / 1 Core 8GB of RAM 140 simultaneous Windows users / 250 at most / 20 NFS volumes mounted via ...
3
votes
2answers
100 views

Linux GUI becomes very unresponsive when doing heavy disk I/O - what to tune?

I have a device /dev/mydisk that is based on a stack of functionality: a LUKS-encrypted, software RAID-1. From time to time, I do a backup of /dev/mydisk contents to an external USB disk, which is ...
1
vote
0answers
83 views

Is it necessary to explicitly flush the HDD on-disk write caches?

Abstract Sometimes, the Linux kernel is not aware of on-drive write caches of external USB storage devices. Is it necessary in such situations to explicitly flush those caches prior detaching these ...
2
votes
0answers
16 views

Find the size of all relevant caches when doing I/O

So, I'm looking at the stats from iostat, which updates every few seconds, and I see that I/O to device happens in bulks, say, around 40M/s, once every half a minute or so. I'm debugging an ...
4
votes
1answer
509 views

What is good block size for file cache on Linux?

The device block size is usually 512 bytes while the filesystem block size is often 4096 bytes. Why are they different? Why are 512B and 4KB good choices for device and filesystem block sizes? What ...
1
vote
2answers
32 views

Detecting downloaded (and not installed) packages with Synaptic / apt / dpkg?

Is there any way to detect downloaded (in cache) and not installed packages using Synaptic, apt or dpkg? Do I have to create a specific filter for that? If so how do I do that? Where could I learn ...
1
vote
1answer
22 views

Are files always cached in $BLOCKSIZE increments?

Given a 4KB block size, when a 1KB file is read from disk and cached in main memory, is the entire 4KB block that contains the file cached? Similarly, if opening a 100KB file, seeking 50KB in and ...
0
votes
0answers
39 views

Cache behavior in dd command

I am performing some dd outputs and running in parallel vmstat: With no direct write (i.e. via cache) $ dd if=/dev/urandom of=somefile.txt bs=1M count=200 200+0 records in 200+0 records out ...
0
votes
1answer
27 views

How to measure the usage of disk cache?

Load Average itself is not very informative about the reasons of overload, so I want an additional characteristic: the percent of disk read operations that are done from cache. To measure it, I ...
0
votes
0answers
29 views

Finding usage of “buff/cache” (as mentioned in free)?

When I run free, I see free -h total used free shared buff/cache available Mem: 15Gi 1.7Gi 4.3Gi 149Mi 9.5Gi ...
2
votes
0answers
124 views

Disable writeback cache throttling - tuning vm.dirty_ratio

I have a workload with extremely high write burst rates for short periods of times. The target disks are rather slow, but I have plenty of RAM and very tolerant to instantaneous data loss. I've ...
0
votes
1answer
26 views

Is the list of packages available to install with `pkg` stored locally on FreeBSD? If so, how to update that cache?

When looking for a package to install on FreeBSD 11.2, such as the zsh shell: pkg search zsh …is the searching done against a local cache of packages information, or run over the internet against ...
1
vote
0answers
117 views

Determine the specific benefit of Writeback Throttling (CONFIG_WBT)

[PATCH 0/8] Throttled background buffered writeback v7 Since the dawn of time, our background buffered writeback has sucked. When we do background buffered writeback, it should have little ...
2
votes
1answer
109 views

Writeback cache (`dirty`) seems to be limited below the expected threshold where throttling starts. What is it being limited by?

In a previous question and answer, I showed an experiment about dirty_ratio. Writeback cache (`dirty`) seems to be limited to even less than dirty_background_ratio. What is it being limited by? How ...
1
vote
0answers
34 views

How to force the kernel to always maintain at least some page cache?

Is there any way to force the kernel to always maintain at least some page cache for writeback caching purposes, even when faced with memory pressure? I have a rather special use case, where swapping ...
0
votes
2answers
99 views

In /proc/meminfo, does Dirty include or exclude Writeback?

Dirty %lu - Memory which is waiting to get written back to the disk. Writeback %lu - Memory which is actively being written back to the disk. -- man proc Does Dirty include Writeback? Or ...
2
votes
1answer
244 views

Writeback cache (`dirty`) seems to be limited to even less than dirty_background_ratio. What is it being limited by? How is this limit calculated?

I have been testing Linux 4.18.16-200.fc28.x86_64. My system has 7.7G total RAM, according to free -h. I have default values for the vm.dirty* sysctl's. dirty_background_ratio is 10, and ...
8
votes
1answer
330 views

Why were “USB-stick stall” problems reported in 2013? Why wasn't this problem solved by the existing “No-I/O dirty throttling” code?

The pernicious USB-stick stall problem - LWN.net, November 2013. Artem S. Tashkinov recently encountered a problem that will be familiar to at least some LWN readers. Plug a slow storage device (...
3
votes
1answer
52 views

Page cache behavior guarantees and log files

Are there any guarantees as to when and in what order dirty pages are written out to disk? In the specific case of log structured files (files that are only ever appended to), is it possible for dirty ...
0
votes
1answer
93 views

Calling sync without root privileges

I can call sync without root privileges. Does it flush all buffers to disk? Is it the same as calling it with root privileges like sudo sync?
0
votes
0answers
20 views

Set minimal amount of disk read cache

I have slow HDD and 5.5 GB of RAM, and thus I'm using zram and no any disk swap. According to my observations, the lack of disk cache has much worse impact on performance than active swapping. While ...
3
votes
0answers
126 views

How does Linux make sure to reclaim useless “buffers”, from caching the ext4 journal writes, before anything else (e.g. before swapping at all)

My laptop tends to run with about 256MB counted as "Buffers" - in /proc/meminfo and free -w -h - out of about 8GB of RAM. I'm interested in memory usage. I sometimes get into trouble using too much. ...
3
votes
0answers
110 views

Where else could memory be used other than process, cache and tmpfs? [duplicate]

We have a server that runs a huge server process to cache some data. Sometimes the process crashes, and the memory is not freed unless the machine is rebooted. How can I free this stuck memory ...
1
vote
0answers
33 views

Cache Memory Leak (or looks like one) slows down a program

I have a strange problem that looks like a Memory Cache Leak (not a memory leak) leading to a slow down of a program. Let me set the stage first. Reproducibly (using ganglia to monitor), on a cluster ...
1
vote
1answer
148 views

How to run a web page with cache refresh using chromium-browser and command line?

I can start a web page with command line like that cromium-browser http://some-page.com. How can I start a web page with command line and with hard refresh cache option (equivalent of CTRL-F5 if I ...
-1
votes
1answer
301 views

/proc/sys/vm/drop_caches values

I want to know what values are possible for drop_caches, and what the meaning of that values are. For example: echo 3 > /proc/sys/vm/drop_caches What does number 3 mean? Are the numbers, 1, 2 or ...
0
votes
0answers
24 views

Deactivating disk cache for specific device or partition? (play 2160p video off SD card without sacrificing RAM space other programs need)

I would like to deactivate the disk cache for /dev/mmcblk0p1 in order to preserve more RAM for other system parts while playing 2160p video off /dev/mmcblk0p1. 2160p videos ravenously wipe the RAM ...
1
vote
0answers
31 views

Distributed Block Device Cache for Linux

Is there an open source software available for Linux that implements a new block device based on an underlying block device (DRBD or shared SAN) and a distributed memory cache ? What I'm looking for ...
-2
votes
1answer
32 views

What is the relation between a line in cpu cache and a page in main memory (and block in disk)?

Computer Systems: A Programmer's Perspective (2nd Edition) [Randal E. Bryant, David R. O'Hallaron] says A block is a fixed-sized packet of information that moves back and forth between a cache and ...
3
votes
1answer
239 views

Is “writeback throttling” a solution to the “USB-stick stall problem”?

The pernicious USB-stick stall problem - LWN.net, 2013. Plug a slow storage device (a USB stick, say, or a media player) into a Linux machine and write a lot of data to it. The entire system ...
0
votes
1answer
29 views

Is buffer used only for writing or can it be used for both reading and writing?

I am reading about caching and buffering from Operating System Concepts, and here is a copy of it. Is it correct that cache is used only for reading not for writing? Is buffer used only for writing ...
2
votes
1answer
178 views

Block device cache v.s. a filesystem

Block devices provide buffering. This means that write() on a block device can return success, before the kernel has written the data to the device. A program can wait for all the buffered writes by ...
3
votes
1answer
212 views

lvmcache and ext4 data=journal => overkill?

We are running lvmcache (24TB raid5 hdd data, 64GB raid1 ssd cache+meta) and we are using external journal with data=journal for the ext4 fs on that raid5 and have the journal on the same ssd raid1 (...
0
votes
1answer
20 views

Thunderbird overloads my HDD in large operations, what to do?

If I do operations in Thunderbird with many (thousands of) mails, I hear a crashing voice from my HDDs. It sounds really not beautiful. It happens only if I do something with a lot of mails (for ...
0
votes
1answer
151 views

is it safe to delete ~/home/{User}/.cache/ [duplicate]

IF it's cache it is probably meant to be cleared eventually, but can I just manually delete the whole .cache folder? It's eating over 10 gigabytes on a 64 GB partition. I think it is mostly unity ...
1
vote
1answer
94 views

free -m: free size reduced because of cache/buff

I'm using the function rosbag of ROS. rosbag can record lots of data from other ROS nodes. In a word, it will generate a huge file, such as a file of 200MB. After generating such a file, I found that ...
2
votes
0answers
29 views

How to exclude programs from writing to RAM block cache? (e.g. media player)

When playing back huge video files, the media player ravenously consumes disk cache and thus kicks out blocks cached by other programs. Not to confuse with LinuxAteMyRam.com. I would like the media ...
0
votes
1answer
287 views

sync option in /etc/fstab

From the man pages of mount command sync All I/O to the filesystem should be done synchronously. In the case of media with a limited number of write cycles (e.g. some flash drives), sync ...
1
vote
0answers
31 views

Is there a tool for searching keywords super fast in many pdfs files?

I have a bunch of technical books, and I have been using pdfgrep for a while, but it takes substantial amount of time for searching all. can somebody recommend me of a cli tool for searching in pdf ...
-1
votes
0answers
24 views

RAM not reallocatable

I have problems reallocating RAM after I've been working for some time. The problem occurs after I started/stopped various services and reached a certain limit (~16 GB out of 32 in total). I checked ...
0
votes
1answer
21 views

How do I limit the maximum RAM for Linux VPS?

I rent a shared VPS that runs Debian 9. I am alloted 4GB RAM (4096MB), but sometimes my processes go over 4GB. This is largely due to the kernel caching a lot of my stuff, so I'd like to know if I ...
1
vote
0answers
186 views

How can I find out what is making my SLAB Unreclaimable Memory Grow without bound

My SLAB unreclaimable memory (SUnreclaim) grows without bounds and this appears to be the reason why my system eventually runs out of RAM and starts trying to swap until it dies. Here's a graph of my ...
2
votes
0answers
491 views

Cannot clear buffer/cache and next to nothing running [closed]

What is using all my cache and why can't I clear it? I'm not trying to optimise performance or tune the system, the best description is that this is an exercise in understanding and learning and I'm ...
2
votes
1answer
539 views

How to ensure that a shared library will have its memory pages shared by several processes?

Let me first describe what I want to do, followed by what I manage to do, and finally my issue. Goal: implementing flush+flush cache attack in C I am trying to implement in C the flush+flush cache ...
1
vote
1answer
45 views

POSTing large files into Apache causes swap

The setting that I have is a compute with only 2Gb of free memory. When I upload large images (3.5 Gb) through the Horizon (which uses apache2) the image fills the cache and starts to swap. I ...
0
votes
1answer
4k views

Ping 'Name or service not known' how can I clear cache

For several days now, after several reboots I still am faced with the issue of my CentOS 7 machine not being able to ping my domain. [root@ip126 ~]# ping sub.mydomain.eu ping: sub.mydomain.eu: Name ...