How do I compile my Gentoo kernel for AMD IOMMU?

Question

I have an AMD A4-5300 CPU with a motherboard that supports IOMMU and I am trying to enable IOMMU on my Gentoo Hardened 4.3.3-r4 kernel with the intention of using a graphics card on a KVM VM. According to this guide here, I'm supposed to recompile my kernel with a few provided options ("Support for DMA Remapping Devices", "Enable DMA Remapping Devices" and "PCI Stub driver"). However, "Support for DMA Remapping Devices" and "Enable DMA Remapping Devices" are absent from my kernel configuration.

I then searched the configuration for "remap" and tried enabling a number of options, which were CONFIG_IRQ_REMAP, CONFIG_DMAR_TABLE, CONFIG_IOMMU_SUPPORT, CONFIG_SWIOTLB and CONFIG_PCI_IOV. Then I recompiled my kernel with these options enabled, started my system with amd_iommu=fullflush iommu=force on the kernel command line, and according to the aforementioned guide I checked if IOMMU was running on my system by looking for "AMD-Vi" on my kernel messages:

dmesg | grep AMD-Vi

Nothing shows up on my kernel log about AMD-Vi.

In addition, when I try to start a VM on virt-manager with an attached PCI device, I get a message that says "Error starting domain: unsupported configuration: host doesn't support passthrough of host PCI devices".

Pretty much everything I find is for Fedora, Ubuntu and Debian where IOMMU apparently Just Works out of the box. I don't believe it's a matter of my motherboard and CPU not supporting it, because my BIOS has an option for it, and because I remember being able a few years ago to run a VirtualBox VM with IOMMU on Debian (though I didn't follow up on it because I never thought at the moment that I required two graphic cards to use a GPU on a VM through IOMMU).

Here's what I have already tried but didn't work:

1. Using the standard kernel instead of hardened.
2. Enabling Device Drivers >> IOMMU Hardware Support, AMD IOMMU support, Support for Intel IOMMU using DMA Remapping Devices, Support for Interrupt Remapping and Bus options (PCI etc.) >> PCI Stub.
3. Using the latest kernel, currently just upgraded to 4.4.8-hardened-r1.

This is the command line I'm using to start my kernel, with each parameter on a separate line for convenience:

root=/dev/ram0 
crypt_root=/dev/disk/by-uuid/c1ce6456-4c6a-4a64-8027-5ec4fdc0f344 
dolvm 
real_root=/dev/mapper/cryptgentoo-raiz 
rootfstype=jfs 
real_init=/sbin/init 
root_keydev=/dev/disk/by-partuuid/086194de-e476-4abb-82da-9744e6fb0257 
root_key=luks-key.gpg 
keymap=la 
amd_iommu=on 
iommu=on

Am I still missing a kernel configuration option?

---

Update: Out of absent-mindedness I mistakenly issued ps -ef | grep -i iommu, and look what I just found...

rockshooter ~ # ps -ef | grep iommu
root        66     2  0 02:57 ?        00:00:00 [amd_iommu_v2]

If I see a kernel thread, I guess there might be a chance that I can get IOMMU running, it's just that my kernel isn't saying anything about it. It's just a wild guess... but I'm going to go ahead and try using it.

Answer 1

IOMMU Main Options

Try Device Drivers >> IOMMU Hardware Support:

AMD IOMMU

Then enable AMD IOMMU support.

DMA Remapping

Enable Support for Intel IOMMU using DMA Remapping Devices

Optional: IRQ Remapping

Enable Support for Interrupt Remapping

Optional: Enable Version 2 Support

AMD IOMMU Version 2 driver

---

PCI Option

Bus options (PCI etc.) >> PCI Stub

---

Never grep the config file, always use menuconfig and the / key to search for the required options. There are three issues with the guide you linked:

1. It's tailored for Fedora/Redhat users.
2. Because of #1, the kernel version is different, therefore the options are in a different place.
3. Gentoo's kernel is newer than the kernel used in the guide.

---

Update

My motherboard is too old to support PCI Passthrough, but I wanted you to see that the options are now properly enabled.Using the exact options I instructed you to use:

IOMMU Main Options

AGP: Checking aperture...
[    0.000000] AGP: No AGP bridge found
[    0.000000] AGP: Node 0: aperture [bus addr 0xc4000000-0xc5ffffff] (32MB)
[    0.000000] Aperture pointing to e820 RAM. Ignoring.
[    0.000000] AGP: Your BIOS doesn't leave an aperture memory hole
[    0.000000] AGP: Please enable the IOMMU option in the BIOS setup
[    0.000000] AGP: This costs you 64MB of RAM
[    0.000000] AGP: Mapping aperture over RAM [mem 0xc4000000-0xc7ffffff] (65536KB)
...
[    0.959270] PCI-DMA: Disabling AGP.
[    0.959337] PCI-DMA: aperture base @ c4000000 size 65536 KB
[    0.959338] PCI-DMA: using GART IOMMU.
[    0.959340] PCI-DMA: Reserving 64MB of IOMMU area in the AGP aperture
...
[    1.375277] AMD IOMMUv2 driver by Joerg Roedel <[email protected]>
[    1.375277] AMD IOMMUv2 functionality not available on this system

Answer 2

Finally, after much wailing, complaining and head bashing, I finally figured out what was going on.

As it turns out, IOMMU was working all along, it's just that the kernel logs weren't showing it. But the IOMMU driver was there, it was working, and I serendipiously found out when I mistakenly typed ps -ef instead of dmesg like this:

rockshooter ~ # ps -ef | grep -i iommu
root        66     2  0 04:19 ?        00:00:00 [amd_iommu_v2]  

When I saw that kernel thread running I thought, maybe my IOMMU is actually working? So I performed the same configuration steps I did previously for Debian, and one hour later, lo and behold, I was on my virtual machine hooked up to my graphics card playing some games.

Please save this thread for future reference, because it took me some good 3 weeks to figure this out. I still gave the other poster some well-deserved +50 reputation for bothering with setting up and testing this issue.