6

I want to check out what files are loaded in buffer. Just so it's clear, the buffers & cache I'm referring to are what show up when you run the free -m command:

$ free -m
             total       used       free     shared    buffers     cached
Mem:          7800       7671        128          0        291        724
-/+ buffers/cache:       6655       1144
Swap:         5823        613       5210
5

Take a look at linux-ftools. This suite of tools is specifically designed to analyze the buffers and cache. It includes the following tools:

fincore

$ fincore [options] files...

  --pages=false      Do not print pages
  --summarize        When comparing multiple files, print a summary report
  --only-cached      Only print stats for files that are actually in cache.

root@xxxxxx:/var/lib/mysql/blogindex# fincore --pages=false --summarize --only-cached * 
stats for CLUSTER_LOG_2010_05_21.MYI: file size=93840384 , total pages=22910 , cached pages=1 , cached size=4096, cached perc=0.004365 
stats for CLUSTER_LOG_2010_05_22.MYI: file size=417792 , total pages=102 , cached pages=1 , cached size=4096, cached perc=0.980392 
stats for CLUSTER_LOG_2010_05_23.MYI: file size=826368 , total pages=201 , cached pages=1 , cached size=4096, cached perc=0.497512 
stats for CLUSTER_LOG_2010_05_24.MYI: file size=192512 , total pages=47 , cached pages=1 , cached size=4096, cached perc=2.127660 
...

NOTE: In the example output above, any files in the directory, /var/lib/mysql/blogindex, that are being cached, are displayed. In this case there are several files named CLUSTER_LOG_*.MYI.

fadvise

SYNTAX: filename mode [offset] [,length]
Where mode can be:

  POSIX_FADV_NORMAL       No further special treatment.  
  POSIX_FADV_RANDOM       Expect random page references.  
  POSIX_FADV_SEQUENTIAL   Expect sequential page references.  
  POSIX_FADV_WILLNEED     Will need these pages.  
  POSIX_FADV_DONTNEED     Dont need these pages.  
  POSIX_FADV_NOREUSE      Data will be accessed once.  

Allows an application to to tell the kernel how it expects to use a file handle,
so that the kernel can choose appropriate read-ahead and caching techniques for
access to the corresponding file. This is similar to the POSIX version of the
madvise system call, but for file access instead of memory access. The
sys_fadvise64() function is obsolete and corresponds to a broken glibc API,
sys_fadvise64_64() is the fixed version. The following are the values for the
advice parameter:

FADV_NORMAL

No special treatment.

FADV_RANDOM

Expect page references in random order.

FADV_SEQUENTIAL

Expect page references in sequential order.

FADV_WILLNEED

Expect access in the near future.

FADV_DONTNEED

Do not expect access in the near future. Subsequent access of pages in this
range will succeed, but will result either in reloading of the memory contents
from the underlying mapped file or zero-fill-in-demand pages for mappings
without an underlying file.

FADV_NOREUSE

Access data only once.

fallocate

SYNTAX: fallocate file length

fallocate() allows the caller to directly manipulate the allocated disk space
for the file referred to by fd for the byte range starting at offset and
continuing for len bytes.

The mode argument determines the operation to be performed on the given
range. Currently only one flag is supported for mode:

FALLOC_FL_KEEP_SIZE

This flag allocates and initializes to zero the disk space within the range
specified by offset and len. After a successful call, subsequent writes into
this range are guaranteed not to fail because of lack of disk
space. Preallocating zeroed blocks beyond the end of the file is useful for
optimizing append workloads. Preallocating blocks does not change the file size
(as reported by stat(2)) even if it is less than offset+len.

If FALLOC_FL_KEEP_SIZE flag is not specified in mode, the default behavior is
almost same as when this flag is specified. The only difference is that on
success, the file size will be changed if offset + len is greater than the file
size. This default behavior closely resembles the behavior of the
posix_fallocate(3) library function, and is intended as a method of optimally
implementing that function.

Because allocation is done in block size chunks, fallocate() may allocate a
larger range than that which was specified.

Emptying the buffers cache

If you ever want to empty it you can use this chain of commands.

$ free && sync && echo 3 > /proc/sys/vm/drop_caches && free

             total       used       free     shared    buffers     cached
Mem:       1018916     980832      38084          0      46924     355764
-/+ buffers/cache:     578144     440772
Swap:      2064376        128    2064248
             total       used       free     shared    buffers     cached
Mem:       1018916     685008     333908          0        224     108252
-/+ buffers/cache:     576532     442384
Swap:      2064376        128    2064248

You can signal the Linux Kernel to drop various aspects of cached items by changing the numeric argument to the above command.

NOTE: clean up memory of unnecessary things (Kernerl 2.6.16 or newer). Always make sure to run sync first to flush useful things out to disk!!!

  • To free pagecache:

    $ echo 1 > /proc/sys/vm/drop_caches
    
  • To free dentries and inodes:

    $ echo 2 > /proc/sys/vm/drop_caches
    
  • To free pagecache, dentries and inodes:

    $ echo 3 > /proc/sys/vm/drop_caches
    

The above are meant to be run as root. If you're trying to do them using sudo then you'll need to change the syntax slightly to something like these:

$ sudo sh -c 'echo 1 >/proc/sys/vm/drop_caches'
$ sudo sh -c 'echo 2 >/proc/sys/vm/drop_caches'
$ sudo sh -c 'echo 3 >/proc/sys/vm/drop_caches'

Alternatives methods to the above:

# alternative #1
$ sudo tee /proc/sys/vm/drop_caches <<<1
# alternative #2
$ echo "echo 1 > /proc/sys/vm/drop_caches" | sudo sh

Why the change in syntax? The /bin/echo program is running as root, because of sudo, but the shell that's redirecting echo's output to the root-only file is still running as you. Your current shell does the redirection before sudo starts.

References

  • Regarding the sudo calls, why use such a complex method rather than sudo sh -c 'echo 1 >/proc/sys/vm/drop_caches' or sudo tee /proc/sys/vm/drop_caches <<<1? – Gilles Aug 23 '13 at 22:41
  • @Gilles - never thought of those, thanks I'll add those instead of mine. – slm Aug 23 '13 at 22:42
  • @Gilles - fixed! – slm Aug 23 '13 at 22:46
  • 1
    The official documentation for the /proc/sys/vm/drop_caches mechanism is in kernel.org/doc/Documentation/sysctl/vm.txt – tanius Feb 16 '17 at 20:25
  • @tanius - thanks I included that in the answer. Note that these docs are for a specific version of the Kernel (2.6.29). Not sure how frequently it gets updated but adjust as necessary to whatever version of the Kernel you're using. – slm Feb 17 '17 at 1:26
2

You can also try vmtouch

vmtouch opens every file provided on the command line and maps it into virtual memory        
with mmap(2). The mappings are opened read-only. It recursively crawls any directories 
and does the same to all files it finds within them.

With no options, vmtouch will not read from (touch) any memory pages. It will only use 
mincore(2) to determine how many pages of each file are actually resident in memory. 
Before exiting, it will print a summary of the total pages encountered and how many were 
resident.

Using this, you can print out what is present in the memory similar to fincore. In addition, you can also evict a file, permanently map a file into the cache etc.

-1
echo 2 | sudo tee -a /proc/sys/vm/drop_caches

perfect command up

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.