Can a user level program do page level or block level I/O on SSDs?

I've looked at disk devices, but am unsure if they provide this as they work only if the partition has a file system.


I want to write a high performance key value store for SSDs and so I need some way to do low level access (including read, write and erase).

I know that my approach must be kernel level, but before that I want to test it out in user-space (avoiding the complexity of learning kernel level programming).

  • 1
    Can you elaborate this a bit more? What exactly are you trying to accomplish? – slm Jun 2 '14 at 8:14
  • @slm i hope this helps! – user1065734 Jun 2 '14 at 8:20
  • A bit, but what did you look at when you stated "disk devices" to determine if they could do this? – slm Jun 2 '14 at 8:24
  • Couple of stackoverflow posts such as: stackoverflow.com/questions/10362111/… stackoverflow.com/questions/7289453/… All of these provide information only on hdd rather than ssds. – user1065734 Jun 2 '14 at 8:26
  • 1
    OK that makes sense now, thanks for clarifying. – slm Jun 2 '14 at 9:06

You can do low-level disk I/O on any type of storage via the block device, something like /dev/sda (for a whole disk) or /dev/sda1 (for a partition) under Linux. This completely bypasses the filesystem.

If you implement your own key-value store, I absolutely guarantee that what you'll come up with will be significantly slower and buggier than filesystems and databases written by professionals. An efficient storage mechanism needs to take account things like caching, concurrent writes, resilience to power failures, etc. This is very hard!

| improve this answer | |

I don't think optimizing I/O at that level may be accomplished by an user program (let alone the optimization mechanics needed to do so). So my approach would be to optimize the process right in the application by implementing something like a queue that flushes its content to the desired output once it has surpassed an established data threshold. A pseudo-code might look like this:


M[100]=new M[100]

function saveObj(obj) {
    if (M.size > MAX_OBJS-1) {
        M = new M[100]


while (true) {
    saveObj( new Obj )

As you can see, it would have a buffer of 100 objects. As soon as the 101st object is tried to be saved, it writes down to disk the other 100 and clears the buffer to leave space for another 100 objects. Of course you can implement more complex techniques, such as performing the write on another thread and locking the array so other objects are not added until it is done writing the objects to disk and clearing the buffer. Or something like that.

| improve this answer | |
  • This type of optimization wont be good enough for my applications. What would be a better approach if say , i am even able to do it in kernel level? – user1065734 Jun 2 '14 at 11:59
  • Talking hypothetically, I think your best bet is to reduce the real I/O operations as much as possible. Something like the cache they already use but that only flushes when it's completely full. You'd then have to write your own kernel module (or tweak an existent one), or even create your own filesystem that perfectly fits the needs of your application. – arielnmz Jun 2 '14 at 17:14

I already pointed out in a different question of yours why you must abstain from a kernel-level approach.

Before you engage in such an endeavor, few points should be clarified:

"High Performance" is not an one-size-fits-all property.

Optimization should be performed for specific cases and only when you have spotted the main bottleneck.

You should ask yourself the following questions:

  • Have I assessed the current mainstream implementations of key-value storage systems? If not, why not?
  • If I did, why they are not fit for my use case? Have I performed extensive benchmarking and testing? Have I traced the main bottleneck? Can I fix it in the current state-of-the-art implementations? If not, why do I think I can fix it in my own implementation?
  • What are my exact performance requirements? Have I defined "performance" and have I found ways to measure it? High-performance during storage operations? High performance during retrieval operations? High-performance under high load due to the large number of client connections?

Once you have a clear picture of what exactly you want to achieve and once you rejected the current state-of-the-art software, only then you should start exploring potential implementation strategies.

Kernel is the last place you want to touch. Especially if you have no prior kernel development experience. Most of the kernel subsystems are highly optimized through processes that took years of testing and development by highly skilled engineers.

My advice would be to consider optimization through a combination of pre-forking, smart caching and delayed writes. It would be a good idea to familiarize yourself with popular caching algorithms, load balancing approaches and have a look into how things work under the hood of modern filesystems (like readahead, Write policies, LRU) -maybe these are not directly related to your problem, but it helps to know how people solved performance issues in similar domains. Of course, this is not meant as an advice to re-implement these features in your application since they are already implemented better by the filesystem itself -in most cases, this will harm the performance of your application instead of enhancing it.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.