63

We are installing SAP HANA in a RAID machine. As part of the installation step, it is mentioned that,

 To disable the usage of transparent hugepages set the kernel settings 
 at runtime with echo never > /sys/kernel/mm/transparent_hugepage/enabled 

So instead of runtime, if I wanted to make this a permanent change, should I add the above line inside /proc/vmstat file?

11 Answers 11

80

To make options such as this permanent you'll typically add them to the file /etc/sysctl.conf. You can see a full list of the options available using this command:

$ sysctl -a

Example

$ sudo sysctl -a | head -5
kernel.sched_child_runs_first = 0
kernel.sched_min_granularity_ns = 6000000
kernel.sched_latency_ns = 18000000
kernel.sched_wakeup_granularity_ns = 3000000
kernel.sched_shares_ratelimit = 750000

You can look for hugepage in the output like so:

$ sudo sysctl -a | grep hugepage
vm.nr_hugepages = 0
vm.nr_hugepages_mempolicy = 0
vm.hugepages_treat_as_movable = 0
vm.nr_overcommit_hugepages = 0

It's not there?

However looking through the output I did not see transparent_hugepage. Googling a bit more I did come across this Oracle page which discusses this very topic. The page is titled: Configuring HugePages for Oracle on Linux (x86-64).

Specifically on that page they mention how to disable the hugepage feature.

excerpt

The preferred method to disable Transparent HugePages is to add "transparent_hugepage=never" to the kernel boot line in the "/etc/grub.conf" file.

   title Oracle Linux Server (2.6.39-400.24.1.el6uek.x86_64)
            root (hd0,0)
            kernel /vmlinuz-2.6.39-400.24.1.el6uek.x86_64 ro root=/dev/mapper/vg_ol6112-lv_root rd_NO_LUKS  KEYBOARDTYPE=pc KEYTABLE=uk
    LANG=en_US.UTF-8 rd_NO_MD SYSFONT=latarcyrheb-sun16  rd_NO_DM rd_LVM_LV=vg_ol6112/lv_swap rd_LVM_LV=vg_ol6112/lv_root rhgb quiet numa=off
    transparent_hugepage=never
            initrd /initramfs-2.6.39-400.24.1.el6uek.x86_64.img

The server must be rebooted for this to take effect.

Alternatively you can add the command to your /etc/rc.local file.

if test -f /sys/kernel/mm/transparent_hugepage/enabled; then
   echo never > /sys/kernel/mm/transparent_hugepage/enabled
fi
if test -f /sys/kernel/mm/transparent_hugepage/defrag; then
   echo never > /sys/kernel/mm/transparent_hugepage/defrag
fi

I think I would go with the 2nd option, since the first will be at risk of getting unset when you upgrade from one kernel to the next.

You can confirm that it worked with the following command after rebooting:

$ cat /sys/kernel/mm/transparent_hugepage/enabled
always madvise [never]
  • 7
    You can make the first option survive kernel updates by adding transparent_hugepage=never to the GRUB_CMDLINE_LINUX_DEFAULT option in /etc/default/grub on most distributions. – Rwky Apr 20 '14 at 9:52
  • 2
    If the output of cat /sys/kernel/mm/transparent_hugepage/enabled is [always] madvise never, then the status is always or that it is enabled (note the [] brackets around always) – ub3rst4r Oct 30 '14 at 5:08
  • 4
    Don't confuse hugepages and transparent hugepages. The latter can cause many issues, mainly high CPU usage while constantly trying to defragment memory and convert normal 4kB pages into huge 2MB pages. – Marki555 May 20 '15 at 16:26
  • 2
    @Rwky -- when tweaking as per your suggestion - I'd emphasize the next action after saving the file is to execute sudo update-grub to get the new settings "written in stone". +1 for pointing the grub file line. – Faron Jul 13 '15 at 17:25
  • 3
    An update for those on EC2, cloudimg adds the file /etc/default/grub.d/50-cloudimg-settings.cfg which overrides the settings in /etc/default/grub adding a file /etc/default/grub.d/99-transparent-hugepage.cfg with the content GRUB_CMDLINE_LINUX_DEFAULT="$GRUB_CMDLINE_LINUX_DEFAULT transparent_hugepage=never" will solve this. – Rwky Sep 19 '17 at 9:53
13

I just wanted to add to this question as I was trying to disable transparent hugepages on CentOS v6 in order to enable TokuDB for MariaDB. I added the script mentioned by @slm to /etc/rc.local and it disabled transparent hugepages. However, because of the way startup scripts work in Linux, /etc/rc.local is executed after all the services are started. Therefore, transparent huge pages was being disabled after MariaDB was already started and the TokuDB engine wouldn't initialize. The only other way to disable transparent hugepages is by adding transparent_hugepage=never to the kernel parameter.

I noticed @Rwky's comment You can make the first option survive kernel updates by adding transparent_hugepage=never to the GRUB_CMDLINE_LINUX_DEFAULT option in /etc/default/grub on most distributions. and found out that CentOS doesn't support the /etc/default/grub file and was worried about transparent_hugepage=never disappearing from the kernel parameters when it is updated. But not to worry, CentOS is setup to keep any changes made to the kernel parameters in grub so when it is updated they are kept.

To also add, the proper way to modify the kernel parameters for grub is with grubby. I created this simple script to add transparent_hugepage=never to each kernel with grubby:

#!/bin/sh

if [[ $EUID -ne 0 ]]; then
   echo "This script must be run as root" 1>&2
   exit 1
fi

for KERNEL in /boot/vmlinuz-*; do
    grubby --update-kernel="$KERNEL" --args='transparent_hugepage=never'
done
  • For an option such as this I would still probably try and add it to /etc/sysctl.conf since that will be consistently applied to any Kernel that may ever be installed on the box. – slm Oct 30 '14 at 11:01
  • I just wanted to add that if you modify the <code>/etc/default/grub</code> file, you need to run grub-mkconfig as 'root' to generate the actual configuration file for grub. – seattlegaucho Dec 23 '14 at 0:45
  • 3
    For an EC2 ubuntu instance, I have to modify the /etc/default/grub.d/50-cloudimg-settings.cfg file instead of the /etc/default/grub to make it work. – zhengyue Apr 2 '15 at 2:53
  • I am using CentOS 6.6. I have modified /etc/rc.local but it is not working for me. Can you please explain the root case to me. – Sandeep Singh Jun 13 '15 at 3:29
  • @s.singh As I stated /etc/rc.local is executed after all the services are started so it needs to be disabled at the kernel level – ub3rst4r Jun 13 '15 at 7:18
6

Here's an implementation using puppet:

exec { "disable_transparent_hugepage_enabled":
  command => "/bin/echo never > /sys/kernel/mm/transparent_hugepage/enabled",
  unless  => "/bin/grep -c '\[never\]' /sys/kernel/mm/transparent_hugepage/enabled 2>/dev/null",
}

exec { "disable_transparent_hugepage_defrag":
  command => "/bin/echo never > /sys/kernel/mm/transparent_hugepage/defrag",
  unless  => "/bin/grep -c '\[never\]' /sys/kernel/mm/transparent_hugepage/defrag 2>/dev/null",
}
6

All of the above didn't work for me on an EC2 Ubuntu 16.04, but this did:

sudo apt install hugepages
sudo hugeadm --thp-never
4

Since the kernel line transparent_hugepage=never only disables half of what I need (both, for annoying mongodb failing/logs), that I didn't persist through systemd startup script but now have: echo never | sudo tee /sys/kernel/mm/transparent_hugepage/enabled. That works in either systemctl boot script (when properly configured one in /etc/systemd/system) or straight from the cli as is.

  • Could you please expand your instructions to also include the "properly configured script" and steps how to setup everything? Official MongoDB instructions docs.mongodb.org/manual/tutorial/transparent-huge-pages show the old way, while now that systemd is more common, it would be nice to have this in an systemd way. – ssasa Jan 13 '16 at 10:51
3

In case of Redis, it also emits a warning which suggests to disable THP. But as noted in the bug report, on many distros /etc/rc.local is executed after services and it has no effect on them until they restart. Also note that in virtualised environments (e.g. Digitalocean) you can't control GRUB settings.

The solution in such case is use dedicated init script to disable transparent huge pages as this page suggests, by settings X-Start-Before. For example, Debian init script for Redis:

#!/bin/bash
### BEGIN INIT INFO
# Provides:          disable-thp
# Required-Start:    $local_fs
# Required-Stop:
# X-Start-Before:    redis-server
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Disable THP
# Description:       disables Transparent Huge Pages (THP) on boot
### END INIT INFO

case $1 in
start)
  if [ -d /sys/kernel/mm/transparent_hugepage ]; then
    echo 'never' > /sys/kernel/mm/transparent_hugepage/enabled
    echo 'never' > /sys/kernel/mm/transparent_hugepage/defrag
  else
    return 0
  fi
;;
esac 
  • Exactly what I was looking for. – nelaaro May 9 '17 at 10:59
2

Thanks to github & PyYoshi
I found this example for systemd

Create the file

sudo vim /etc/systemd/system/disable-transparent-huge-pages.service

Put this into the service file

[Unit]
Description=Disable Transparent Huge Pages

[Service]
Type=oneshot
ExecStart=/usr/bin/sh -c "/usr/bin/echo "never" | tee /sys/kernel/mm/transparent_hugepage/enabled"
ExecStart=/usr/bin/sh -c "/usr/bin/echo "never" | tee /sys/kernel/mm/transparent_hugepage/defrag"

[Install]
WantedBy=multi-user.target

For debian/ubuntu users

[Unit]
Description=Disable Transparent Huge Pages

[Service]
Type=oneshot
ExecStart=/bin/sh -c "/usr/bin/echo "never" | tee /sys/kernel/mm/transparent_hugepage/enabled"
ExecStart=/bin/sh -c "/usr/bin/echo "never" | tee /sys/kernel/mm/transparent_hugepage/defrag"

[Install]
WantedBy=multi-user.target

Then enable the service

systemctl enable disable-transparent-huge-pages
systemctl start disable-transparent-huge-pages
systemctl status disable-transparent-huge-pages
1

If you meet with the problem

-bash: /sys/kernel/mm/transparent_hugepage/khugepaged/defrag: Permission denied

even with sudo, try the commands below:

sudo /bin/bash -c 'echo never > /sys/kernel/mm/transparent_hugepage/enabled'
sudo /bin/bash -c 'echo never > /sys/kernel/mm/transparent_hugepage/defrag'
  • Hi, welcome on the Unix SE! Note, it would be much better if you would also explain, what your commands are doing. – peterh says reinstate Monica Jul 26 '18 at 11:08
1

Here is a quick hack for Ansible (I don't want to go managing a template for /etc/rc.local):

- name: Disable Transparent Huge Pages at boot
  lineinfile:
    dest: /etc/rc.local
    line: "if [ -d /sys/kernel/mm/transparent_hugepage ]; then echo 'never' > /sys/kernel/mm/transparent_hugepage/enabled ; echo 'never' > /sys/kernel/mm/transparent_hugepage/defrag ; fi"
  register: transparent_hugepage
- name: Disable disabled rc.local
  lineinfile:
    dest: /etc/rc.local
    line: 'exit 0'
    state: absent
- name: Disable Transparent Huge Pages at run time 1/2
  command: echo never > /sys/kernel/mm/transparent_hugepage/defrag
  when: transparent_hugepage|changed
- name: Disable Transparent Huge Pages at run time 2/2
  command: echo never > /sys/kernel/mm/transparent_hugepage/enabled
  when: transparent_hugepage|changed
0

In SLES11 SP3 with YAST and GRUB bootloader we have to add transparent_hugepage=never in [ YAST-bootloader-edit- line with optional kernel parameter ]. NOTE: This tool changes the file /boot/grub/menu.lst.

Only after making this change and rebooting resulted in in disabling THP.

0

Here's another puppet solution in the Forge: https://forge.puppetlabs.com/ramseymcgrath/remove_hugepage/

Just use:

include remove_hugepage

in your puppet node definition.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.