I'm working with PCIe Gen 3 cards and from time to time they seem to fall back to PCIe 1 or 2 speeds (according to lspci and also observed by the throughput).

When rebooting/power cycling the machine the speed goes back to the full PCIe Gen 3 speed in most cases.

Is there a less intrusive way to force a renegotiation of the PCI link speed (trying to bring it back to PCI Gen 3) on e.g. RHEL6 ?

  • 2
    In windows I've seen it change automatically, it's a function of pci-e bus power-saving. serverfault.com/questions/226319/what-does-pcie-aspm-do seems to have a nice explanation – Marcin Jul 5 '12 at 14:07
  • thanks for this information ! Unfortunately, in my case the speed reduction happens on only some of the hosts and it does not go back to full speed when we start using the device again... – Andre Holzner Jul 5 '12 at 15:16
  • 1
    Could this be controlled by ACPI? - maybe update or install any ACPI related packages for your distribution. Alternatively maybe you can disable the feature in the BIOS. – LawrenceC Aug 15 '12 at 20:59

You can check your PCIe energy policy on this file:

# cat /sys/module/pcie_aspm/parameters/policy

Since Gen3 are pretty straightforward on power management through ASPM(Active-State Power Management ), this could be the root cause of the issue on your bus: The throughput is low so, the modules reduces the speed but it forgets to increase it again when needed(if needed). You could enforce on grub to avoid using the "powersave" or the "default" policy by disabling aspm with the following parameter:


Test this on just one kernel appending this option at /boot/grub/grub.conf on the "kernel" line of your default boot linux. Example of grub config extracted from the Red Hat docs:

title Red Hat Enterprise Linux Server (2.6.18-2.el5PAE)         
root (hd0,0)         
kernel /boot/vmlinuz-2.6.18-2.el5PAE ro root=LABEL=/1 rhgb quiet pcie_aspm=off      
initrd /boot/initrd-2.6.18-2.el5PAE.img
  • 1
    thanks for the answer ! In fact, in the end it turned out that the PCI card had a problem with data transmission on the PCI bus and the manufacturer fixed it with a firmware upgrade. – Andre Holzner Apr 4 '13 at 13:40
  • 1
    We had simmilar problems with a HBA at our work(brocade), and it was a firmware problem too :) – user34720 Apr 4 '13 at 14:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.