1

I have a several GB directory that is rarely accessed, but when it is I'd like to be able to read the files in it quickly instead of having to wait. Since the files are rarely accessed they will not be in the file system cache, but I'd like to suggest that Linux preloads them in the file system cache before I go to access them.

Some may argue that I will need to wait for it to load them the first time and so I'm not saving any time -- suffice it to say that in my particular use case I can automate this so it is not a matter of saving system time, it is a matter of saving a user's time (as they don't need to wait if the files are cached).

I also realize that the file system cache is not a guarantee without using something like tmpfs, it is a suggestion to the OS, and I'm fine with this.

So it seems like the obvious method would be to do:

tar -cf - /mydirectory > /dev/null

However when I do this, the command returns immediately, even for a several GB directory. It appears that bash is seeing that the output is being directed to /dev/null and optimizing the entire command away, so tar is not actually even reading from the directory.

What is a simple command to make the OS read all the files in a directory?

  • 1
    is the directory several GB or the files in it several GB. after about 1/2 megabyte directory size i notice slowness. – Jasen Jun 26 '16 at 3:22
  • There are subdirectories within it and the files in those total several GB. When I run the tar command I can tell that there is virtually no disk access. – Nick Jun 26 '16 at 12:39
  • you can use tar ...|cat >/dev/null to stop tar optimising. – meuh Jun 26 '16 at 12:43
  • 1
    There is also quite nice comprehensive answer to this question. – JustMe Jun 26 '16 at 15:19
1

It isn't bash doing this optimization, it's GNU tar. It recognizes when its output is /dev/null and doesn't read the contents of the files.

Either arrange for the output not to be /dev/null:

tar -cf - /mydirectory > /dev/zero

Or use a different tool, such as

find . | cpio -o >/dev/null
find . -type f -exec cat {} + >/dev/null

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.