1

I have an AMD A4-5300 CPU with a motherboard that supports IOMMU and I am trying to enable IOMMU on my Gentoo Hardened 4.3.3-r4 kernel with the intention of using a graphics card on a KVM VM. According to this guide here, I'm supposed to recompile my kernel with a few provided options ("Support for DMA Remapping Devices", "Enable DMA Remapping Devices" and "PCI Stub driver"). However, "Support for DMA Remapping Devices" and "Enable DMA Remapping Devices" are absent from my kernel configuration.

I then searched the configuration for "remap" and tried enabling a number of options, which were CONFIG_IRQ_REMAP, CONFIG_DMAR_TABLE, CONFIG_IOMMU_SUPPORT, CONFIG_SWIOTLB and CONFIG_PCI_IOV. Then I recompiled my kernel with these options enabled, started my system with amd_iommu=fullflush iommu=force on the kernel command line, and according to the aforementioned guide I checked if IOMMU was running on my system by looking for "AMD-Vi" on my kernel messages:

dmesg | grep AMD-Vi

Nothing shows up on my kernel log about AMD-Vi.

In addition, when I try to start a VM on virt-manager with an attached PCI device, I get a message that says "Error starting domain: unsupported configuration: host doesn't support passthrough of host PCI devices".

Pretty much everything I find is for Fedora, Ubuntu and Debian where IOMMU apparently Just Works out of the box. I don't believe it's a matter of my motherboard and CPU not supporting it, because my BIOS has an option for it, and because I remember being able a few years ago to run a VirtualBox VM with IOMMU on Debian (though I didn't follow up on it because I never thought at the moment that I required two graphic cards to use a GPU on a VM through IOMMU).

Here's what I have already tried but didn't work:

  1. Using the standard kernel instead of hardened.
  2. Enabling Device Drivers >> IOMMU Hardware Support, AMD IOMMU support, Support for Intel IOMMU using DMA Remapping Devices, Support for Interrupt Remapping and Bus options (PCI etc.) >> PCI Stub.
  3. Using the latest kernel, currently just upgraded to 4.4.8-hardened-r1.

This is the command line I'm using to start my kernel, with each parameter on a separate line for convenience:

root=/dev/ram0 
crypt_root=/dev/disk/by-uuid/c1ce6456-4c6a-4a64-8027-5ec4fdc0f344 
dolvm 
real_root=/dev/mapper/cryptgentoo-raiz 
rootfstype=jfs 
real_init=/sbin/init 
root_keydev=/dev/disk/by-partuuid/086194de-e476-4abb-82da-9744e6fb0257 
root_key=luks-key.gpg 
keymap=la 
amd_iommu=on 
iommu=on

Am I still missing a kernel configuration option?


Update: Out of absent-mindedness I mistakenly issued ps -ef | grep -i iommu, and look what I just found...

rockshooter ~ # ps -ef | grep iommu
root        66     2  0 02:57 ?        00:00:00 [amd_iommu_v2]

If I see a kernel thread, I guess there might be a chance that I can get IOMMU running, it's just that my kernel isn't saying anything about it. It's just a wild guess... but I'm going to go ahead and try using it.

1
+50

IOMMU Main Options

Try Device Drivers >> IOMMU Hardware Support:

IOMMU Options

AMD IOMMU

Then enable AMD IOMMU support.

DMA Remapping

Enable Support for Intel IOMMU using DMA Remapping Devices

Optional: IRQ Remapping

Enable Support for Interrupt Remapping

Optional: Enable Version 2 Support

AMD IOMMU Version 2 driver


PCI Option

Bus Options

Bus options (PCI etc.) >> PCI Stub


Never grep the config file, always use menuconfig and the / key to search for the required options. There are three issues with the guide you linked:

  1. It's tailored for Fedora/Redhat users.
  2. Because of #1, the kernel version is different, therefore the options are in a different place.
  3. Gentoo's kernel is newer than the kernel used in the guide.

Update

My motherboard is too old to support PCI Passthrough, but I wanted you to see that the options are now properly enabled.Using the exact options I instructed you to use:

IOMMU Main Options

AGP: Checking aperture...
[    0.000000] AGP: No AGP bridge found
[    0.000000] AGP: Node 0: aperture [bus addr 0xc4000000-0xc5ffffff] (32MB)
[    0.000000] Aperture pointing to e820 RAM. Ignoring.
[    0.000000] AGP: Your BIOS doesn't leave an aperture memory hole
[    0.000000] AGP: Please enable the IOMMU option in the BIOS setup
[    0.000000] AGP: This costs you 64MB of RAM
[    0.000000] AGP: Mapping aperture over RAM [mem 0xc4000000-0xc7ffffff] (65536KB)
...
[    0.959270] PCI-DMA: Disabling AGP.
[    0.959337] PCI-DMA: aperture base @ c4000000 size 65536 KB
[    0.959338] PCI-DMA: using GART IOMMU.
[    0.959340] PCI-DMA: Reserving 64MB of IOMMU area in the AGP aperture
...
[    1.375277] AMD IOMMUv2 driver by Joerg Roedel <jroedel@suse.de>
[    1.375277] AMD IOMMUv2 functionality not available on this system
  • I just tried enabling these options. It didn't work. I also tried with the standard kernel and it didn't work, then I tried upgrading to kernel 4.4.8-hardened-r1 and it didn't work. – RAKK Jun 30 '16 at 6:03
  • What errors did you recurve when trying these? Did you remember to add the kernel to your boot loader configuration? Have you rebooted? – eyoung100 Jun 30 '16 at 6:14
  • Yep, I have rebooted. I don't get any error message, I just don't get any kernel log message saying AMD-Vi was picked up. – RAKK Jun 30 '16 at 6:21
  • To give yourself an idea, on the same computer where I have Gentoo I managed to get PCI passthrough working on Debian Stretch. I even used the steps I took to piece together this guide here: wiki.installgentoo.com/index.php/PCI_passthrough – RAKK Jun 30 '16 at 6:22
  • Let me follow that guide myself tomorrow. I'll report back with my findings. If I get the same results as you, we'll solve this together. – eyoung100 Jun 30 '16 at 6:31
0

Finally, after much wailing, complaining and head bashing, I finally figured out what was going on.

As it turns out, IOMMU was working all along, it's just that the kernel logs weren't showing it. But the IOMMU driver was there, it was working, and I serendipiously found out when I mistakenly typed ps -ef instead of dmesg like this:

rockshooter ~ # ps -ef | grep -i iommu
root        66     2  0 04:19 ?        00:00:00 [amd_iommu_v2]  

When I saw that kernel thread running I thought, maybe my IOMMU is actually working? So I performed the same configuration steps I did previously for Debian, and one hour later, lo and behold, I was on my virtual machine hooked up to my graphics card playing some games.

Please save this thread for future reference, because it took me some good 3 weeks to figure this out. I still gave the other poster some well-deserved +50 reputation for bothering with setting up and testing this issue.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.