I recently learned about the convenient "shared memory" file system at /dev/shm. I wanted to see if I could use this to speed up a sometimes-disk-bound program that writes to and reads from a directory, so I ran a little experiment on an EC2 instance (c4.8xlarge running Ubuntu 16.04):

$ time yes asdfjkl | head -1000000000 > /mnt/fake.txt
real    0m21.381s
$ time yes asdfjkl | head -1000000000 > /dev/shm/fake.txt
real    0m20.266s
$ time yes asdfjkl | head -1000000000 > /dev/null
real    0m14.334s

The EC2 instance seems to have comparable write throughput to /dev/shm/ as it does to an EBS drive, which was surprising. htop indicates that the machine isn't using swap space in order to write to /dev/shm. The noticeably faster write to /dev/null in the third case indicates that I'm probably not bound by some other factor (e.g. CPU via the implementation of yes) in the first two cases.

I ran the same experiment on my personal computer -- enough memory that /dev/shm can hold 7.5 GB of asdfjkl\n by default, I can dig up more hardware details if anyone thinks they'll matter -- also running Ubuntu 16.04:

$ time yes asdfjkl | head -1000000000 > /mnt/fake.txt
real    0m36.520s
$ time yes asdfjkl | head -1000000000 > /dev/shm/fake.txt
real    0m12.516s
$ time yes asdfjkl | head -1000000000 > /dev/null
real    0m11.252s

This is much closer to what I expected. Read throughput (writing to /dev/null) on both machines and from both file systems is roughly proportional to write throughput in the corresponding case.

Two other observations, which I don't know quite how to interpret:

  • On the EC2 instance, htop indicates memory usage comparable to the size of /dev/shm/fake.txt once it's written, while on my desktop, it does not.
  • On the EC2 instance, concurrent disk write congestion appears to slow down a write to shared memory by an amount comparable to how much the writes to disk were slowed down, while on my desktop, it does not.
  • 2
    You really need to ask Amazon. They won't tell you, but probably nobody else can. – Michael Homer Jan 24 '18 at 5:36
  • 2
    You would improve the question by adding details of the swap usage for both systems to it. – JdeBP Jan 24 '18 at 6:29
  • 1
    /dev/zero might (?) be a better data source. Interesting question. – Michael - sqlbot Jan 24 '18 at 13:11
  • @JdeBP neither machine is using swap, clarified in the question. Thanks – James Jan 24 '18 at 13:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.