We are working with a huge Hadoop cluster.
We have 736 datanode machines and each node has 16 cores × 2 threads.
On some machines we saw CPU load average (98-128 for 5 min).
After deep investigation, we found:
- no HW problem
- no disk problem
- no network problem
- no infra problem
What we still have not checked is about how to tune the Linux kernel parameters.
What are the parameters or any kernel parameters that can help the machines in order to get good CPU working with most CPU LOW load average?