1

I'm currently working on a Linux-based server with high disk I/O demands, and I'm looking for advanced strategies to fine-tune the system for optimal performance. While I've implemented basic optimizations, I'm seeking insights into deeper, more specialized techniques.

Specifically, I'd like to know:

  • What are some advanced kernel-level configurations and tuning parameters that can significantly improve disk I/O performance on Linux? How can I leverage features like the I/O scheduler, file system tuning, and block device settings?

  • Are there any specialized file systems or storage solutions that excel in high I/O environments, and what considerations should be taken when choosing or configuring them?

  • How can I effectively monitor and analyze disk I/O performance to identify bottlenecks or areas for improvement? Are there specific tools or utilities that provide deep insights into I/O operations?

  • What are some advanced caching and buffering strategies that can be implemented to reduce disk I/O latency and increase throughput, especially for frequently accessed data?

  • Are there any best practices for optimizing I/O-bound applications or workloads, such as databases or high-throughput web servers, to make the most of available disk resources?

1 Answer 1

0

In Redhat, there is tuned. And from RHEL 7.9 a tuned-adm list shows

Available profiles:
- atomic-guest                - Optimize virtual guests based on the Atomic variant
- atomic-host                 - Optimize bare metal systems running the Atomic variant
- balanced                    - General non-specialized tuned profile
- cpu-partitioning            - Optimize for CPU partitioning
- default                     - Legacy default tuned profile
- desktop                     - Optimize for the desktop use-case
- desktop-powersave           - Optmize for the desktop use-case with power saving
- enterprise-storage          - Legacy profile for RHEL6, for RHEL7, please use throughput-performance profile
- hpc-compute                 - Optimize for HPC compute workloads
- laptop-ac-powersave         - Optimize for laptop with power savings
- laptop-battery-powersave    - Optimize laptop profile with more aggressive power saving
- latency-performance         - Optimize for deterministic performance at the cost of increased power consumption
- mssql                       - Optimize for MS SQL Server
- network-latency             - Optimize for deterministic performance at the cost of increased power consumption, focused on low latency network performance
- network-throughput          - Optimize for streaming network throughput, generally only necessary on older CPUs or 40G+ networks
- oracle                      - Optimize for Oracle RDBMS
- powersave                   - Optimize for low power consumption
- server-powersave            - Optimize for server power savings
- spindown-disk               - Optimize for power saving by spinning-down rotational disks
- throughput-performance      - Broadly applicable tuning that provides excellent performance across a variety of common server workloads
- virtual-guest               - Optimize for running inside a virtual guest
- virtual-host                - Optimize for running KVM guests

I would research into tuned into what they are doing for profiles such as enterprise storage along with any of the other ones that seem relevant. You would not want to have such things done by the spindown-disk profile happening... I would think... or any kind of power saving stuff.

Unless you have I/O happening that exceeds RAM capacity of the system, 768gb being the norm of 4 socket high end servers and ~128gb of not-too-expensive workstations, the optimal way to do it would be creating a tmpfs mounted as a /data folder so that you are reading/writing directly from [DDR4 or DDR5] RAM, and then manage that volatile data location accordingly. Would be a matter of doing in /etc/fstab something like tmpfs /data tmpfs defaults,size=512G 0 0. I wouldn't call it an advanced buffering strategy, but making use of RAM exclusively instead of disk at SATA/SAS speeds of 6 Gbit/s or 12 Gbit/s will be orders of magnitude faster, as well as something I don't think you have to tune linux for other than simply doing an appropriate power profile like latency-performane instead of powersave.

I also think, a lot of what you are asking, is answered more by choosing optimal hardware to run on more so than tuning linux. The Linux operating system will inherently disk cache and do exactly what I mentioned above with tmpfs in that it will operate as much as it can in RAM before having to write/flush to actual disk. Which is simply saying if you want better performance, increase your RAM - if you are running only 8 or 16gb on a pc then update it to 64gb or higher; likewise with servers if it's < 128gb then max it to 768gb or better, along with fastest RAM DIMM timings supported.

1
  • Thank you for your answer.
    – MRashad
    Sep 29, 2023 at 14:35

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .