I am writing a device mapper target and am having a issue. While running an i/o test, the number of i/o inflight to my target gets very high, presumably from the filesystem paging out the cache. At times, I see tens of thousands of i/o inflight to my target, which makes it unresponsive to other commands on it. Is there a way to limit the amount of i/o submitted against it at one time? Request based targets have a busy callback which seems like it would do the trick, but there are limits to the type of devices those can map, which is a problem for me. Most of the existing bio based targets set up a congested function, but that doesn't seem to solve the problem. Returning DM_MAPIO_REQUEUE or completing the i/o with EBUSY just get converted into EIO in most scenarios and cause operations to fail.
After playing around with the existing targets, as well as some test code, it looks like high inflight i/o is associated with targets which put i/o on a workqueue for processing, while immediately submitting the i/o to the backing device limits this. If this is true, I'm unable to find the mechanism by which the limiting occurs.
Is there a way to slow the i/o inflow? Or if not, are there best practices to keeping to device responsive? Or a recommendation for the order in which they should be handled?