Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

After adding more memory to my Linux machine I was tempted to use the cache more agressively.

The thing is: I boot up my machine, go to bathroom, come back and log in. It takes like half a minute to log in. Then I start firefox and again it takes some time. When I log out and back in, all these things happen within seconds, because the cache is warm then. So I thought: how can I teach my machine to put all these files into the cache while I am in the bathroom.

I looked into preload and found the results somewhat disappointing. The net cause of this seems to be that it only tracks mmap()ed files and not files which are read by means of read().

I suspect this is because mmap()ed files are easier to observe. When an application mmap()s a file it stays in memory for some time, whereas open()-read()-close() could all happen within a fraction of a second (correct me if I'm wrong).

There does not seem to be a simple way to get notified when a file is read from. There is inotifiy, but it requires registering directories and IIRC correctly, is not meant for monitoring entire filesystems.

Ideally I would like to have a device in /dev (like a tty) from which I can read the filenames from which some application reads from.

(1) Is something like this available ?

(2) Are there reasons why this is difficult ?

If I had information about read file accresses I could easily warm the cache during boot.

share|improve this question
1  
I don't understand what you're trying to do. Once a program calls read, the data ends up in the cache automatically. Are you monitoring common usage patterns? Are you designing a predictive load system? The audit subsystem (search for auditctl) might help: you can put a watch on read. –  Gilles Aug 27 '13 at 22:09
    
Yes, I want to preload data into the cache before the real read() occurs. –  Martin Drautzburg Aug 28 '13 at 11:10
    
So you want to make predictions on what applications will read next. Where are you getting the predictions from? Are you looking for a tool to make them? By the way, when an application calls read to get a chunk of file, the kernel does sometimes read ahead the next few chunks of the file already. –  Gilles Aug 28 '13 at 11:19
    
The thing is: I boot up my machine, got to bathroom, come back and log in. It takes like half a minute to log in. Then I start firefox and again it takes some time. When I restart the display manager, all these things happen within seconds. So I thought: how can I teach my machine to put all these files into the cache while I am in the bathroom. I just need my usage pattern as "prediction", and to get that I wanted a listing of all files which are read() from during login and shortly afterwards. –  Martin Drautzburg Aug 29 '13 at 19:14
    
Ah, now we're getting somewhere. You should edit your question, because that last comment really clarifies what you're after. Look up prelinking and preloading. –  Gilles Aug 29 '13 at 19:45
add comment

1 Answer 1

Regarding the caching of read system calls:

The read system call can be applied on a big number of socket types. So there is no caching logic for the read itself. However, if you do IO operations on a file system, the file system itself might support caching techniques, but this is depending on your file system.

This is different to mmap, because mmap is specifically made for mapping big files into memory and operating on them. If you do this, it might be worth to consider using mmap instead of read.


Regarding the second part of your post, where you say that you would like some way to get the list of files that a defined application or process is reading from:

I don't know how you could get this kind of data in a device file in /dev. But by using System Tap it should be easy to create a kernel module which prints you all the arguments for read or open system calls that a defined process is doing into dmesg. Then you could add something like a predefined prefix to this output, so if you do dmesg | grep <prefix> you would get all the output from this kernel module in the same way as you describe you would like it from a /dev device.

Please note that this should not be used in production, but only for debugging or other non-production purposes.

This is a nice guide on System Tap basics, after learning it you should be able to create a System Tap script that only requires the name of an executable or the PID of a process as input. This could then print the arguments to all read or open system calls that come from a certain process or application into your dmesg:

https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/5/html/SystemTap_Beginners_Guide/

share|improve this answer
    
THanks for the link. Just to clarify: I want to warm my cache, so I am not interested in file accesses by a particular process, but by all processes. –  Martin Drautzburg Aug 27 '13 at 17:33
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.