I am executing some benchmark using a DBMS and virtual machines. And I am using this command

free && sync && echo 3 > /proc/sys/vm/drop_caches && free

on each Guest, and on each Host, before executing a query, to minimize the effect of cache/buffering on my execution time. If the data is cached/buffered the execution time for a query may vary depending on what query was executed before, because a part of the file used to answer the query might be in memory.

I saw this post: How do you empty the buffers and cache on a Linux system?

But for me it's a little tricky to execute sync/drop caches after each query. I need to ask for every VM and for every physical server to sync/drop caches. I could write a script, and use SSH to do this, but I wonder if there's a way to keep the OS from buffering/caching the files because it would be easier for me.

I am using Ubuntu 12.04 on all the VMs (using KVM) and servers.

  • It doesn't really make sense to completely disable caching. This would mean that every instruction that executes in a program would have to be fetched from disk, instead of loading the program into memory in whole blocks. – Barmar Oct 15 '14 at 18:15
  • Is there a way to avoid caching for files used by a specific process? All I want is to garantee that the queries will cause the data to be fetched from the disk, and not from memory. – hlustosa Oct 15 '14 at 18:24
  • So if the program reads from the file a byte at a time, each read should go to disk instead of using the block that was read for the previous read? I suspect that's not what you want, either. You just want the cache to be flushed before the program starts. – Barmar Oct 15 '14 at 18:26
  • Note that anything you do regarding the VM's cache probably won't affect the host's cache. – Barmar Oct 15 '14 at 18:27
  • 2
    If sync fixes the problem, then they aren't due to caching, but to buffering. – Gilles 'SO- stop being evil' Oct 15 '14 at 23:16

Use something like https://code.google.com/p/linux-ftools/ to mark all the files used by the database system as FADV_DONTNEED?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.