Occasionally, something on my system starts hammering the disk. I would like to log data that would let me know about I/O usage per process, then I could go back to it later and see what keeps doing this. How should I go about solving this problem? Bonus points for solving this for arbitrary block devices.
I would give
iotop a try (Linux only). It is pretty good at giving you the processes with high I/O transfers, but
iotop is more for displaying the current status than logging.
sar is always a good tool for performance aspects.