I have an s3 share that's mounted via [s3fs-fuse][1] and would like to run some speed tests to compare throughput on DreamHost's DreamObjects vs Amazon S3. Everything is mounted and working just fine (s3fs testbucket ~/mnt/test -o passwd_file=/path/to/passwd-s3fs -o url=http://objects-us-west-1.dream.io), but traditional tests like dd and hdparm just don't work out.

Any recommendations on running a successful speed test on a block storage device in such a scenario?

  • What will you be using it for? What metrics are you interested in other than price per gb (both storage and xfer?) – ivanivan Apr 11 '17 at 3:21
  • Primarily I'm attempting to decide whether or not DreamObjects is sufficiently fast from a Linode server at a particular location for utilization as a block storage device for the moment and / or other purposes. I'm comparing a few locations in addition to this and so I need ascertain firstly the throughput and secondly the latency. – ylluminate Apr 11 '17 at 4:59
  • Neither S3 nor DreamObjects is block storage. They are object stores, so of course dd and hdparm are completely irrelevant. Additionally, s3fs emulates a filesystem, not a block storage device. – Michael - sqlbot Apr 11 '17 at 9:54
  • The point is to measure it as though it is block storage. Sure, it's going to have the same performance profile whatsoever, but they should be similarly testable. – ylluminate Apr 11 '17 at 14:12

You cannot generally use object store to emulate a block device. There's s3backer which can work but there are caveats.

Using dd with s3fs should work though. I've wrapped a set of benchmarks comparing s3fs/goofys/riofs as a docker container: https://github.com/kahing/goofys/#runbenchmark . Should be simple to modify it to run on a different object store.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.