17

I have a PCIe device that only works correctly when the computer is fully powered off then on again. Issuing a simple reboot or reboot -p command does not appear to cycle the power to the PCIe card, which causes it not to work after the reboot.

Is there a way to, from the OS, cycle power to a device in a PCIe slot? I can find it in /sys/bus/pci/devices/0000*/, but I can't figure out how to properly reset the board. Toggling power seems to be the only way.

Barring that, can I change a setting somewhere that will cause a full power cycle on a reboot command?

I'm running Ubuntu 12.10 by the way.

  • Did you try reboot -f? This is similar to pressing the power button of the CPU. – ktan Apr 23 '15 at 7:59
  • Two years ago the OP pointed out that a soft reboot didn't work. Your reboot -f is still a soft reboot. – roaima Apr 23 '15 at 10:18
16

Potential Method #1

I think you can do it with these commands:

disable

echo 0 > /sys/bus/pci/slots/$NUMBER/power

enable

echo 1 > /sys/bus/pci/slots/$NUMBER/power

Where $NUMBER is the number of the PCI slot.

lspci -vv may help to identify the device. This is not very well documented...

Potential Method #2

I came across this thread on U&L, similar issue: there are some answers to that question that say you can reset with this command:

echo "1" > /sys/bus/pci/devices/$NUMBER/reset

However, I would read the answers there! There are conditions around doing it this way! Specifically I would read this answer!

Potential Method #3

There is a Unix command, setpci, that may give you a method for resetting a device in the PCI bus.

I did not see any specific examples with this command so you'll have to google for examples and look through the man page. I would tread lightly with this command until you're confident in it's use. From what I've read about it, it's manipulating the hardware directly and so there are always risks in doing it yourself vs. using a tool that is exposing this type of functionality!

  • 1
    Nothing shows up in slots, even though I have multiple cards plugged in. I do have a power directory in /sys/bus/pci/devices/$NUMBER/. But nothing seems to warrant setting 0 or 1 – zachd1_618 Apr 27 '13 at 23:30
  • 1
    I came across this thread on U&L, similar issue: there are some answers to that Q that say you can reset with this: echo "1" > /sys/bus/pci/devices/$NUMBER/reset. Read that Q though, there are conditions around doing it that way! – slm Apr 27 '13 at 23:42
  • Thanks for the link. I've tried that however and it doesn't seem to do anything. Specifically, the device doesn't power cycle and the system still knows it is there. (When the card is on and plugged in, there are devices in /dev that I can watch). They don't disappear when I echo "1" > .... – zachd1_618 Apr 28 '13 at 0:06
  • 1
    Are you unloading the kernel modules for that card prior to the power cycling? I think you have to do that as well. – slm Apr 28 '13 at 0:33
  • 1
    I think I'll check the kernel source code to see if toggling power actually puts it into D3. – forest Aug 7 '18 at 0:04
5

remove and rescan will allow the kernel to cycler-power the PCI device without reboot:

echo "1" > /sys/bus/pci/devices/DDDD\:BB\:DD.F//remove
sleep 1
echo "1" > /sys/bus/pci/rescan

where DDDD.BB.DD.F = Domain:Bus:Device.Function

  • echo "1" > /sys/bus/pci/rescan is worked for me in lenovo g560 mini pci slot. I plugged in USB 3.0 minipci card. System is Ubuntu 16.04 x64 – kodmanyagha Mar 19 '18 at 12:26
3

Resets in PCI express are a bit complex. There are two main types of resets - conventional reset, and function-level reset. There are also two types of conventional resets, fundamental resets and non-fundamental resets. See the PCI express specification for all of the details.

A 'cold reset' is a fundamental reset that takes place after power is applied to a PCIe device. There appears to be no standard way of triggering a cold reset, save for turning the system off and back on again. On my machines, the /sys/bus/pci/slots directory is empty.

A 'warm reset' is a fundamental reset that is triggered without disconnecting power from the device. There appears to be no standard way of triggering a warm reset.

A 'hot reset' is a conventional reset that is triggered across a PCI express link. A hot reset is triggered either when a link is forced into electrical idle or by sending TS1 and TS2 ordered sets with the hot reset bit set. Software can initiate a hot reset by setting and then clearing the secondary bus reset bit in the bridge control register in the PCI configuration space of the bridge port upstream of the device.

A 'function-level reset' (FLR) is a reset that affects only a single function of a PCI express device. It must not reset the entire PCIe device. Implementing function-level resets is not required by the PCIe specification. A function-level reset is initiated by setting the initiate function-level reset bit in the function's device control register in the PCI express capability structure in the PCI configuration space.

Linux exposes the function-level reset functionality in the form of /sys/bus/pci/devices/$dev/reset. Writing a 1 to this file will initiate a function-level reset on the corresponding function. Note that this only affects that specific function of the device, not the whole device, and devices are not required to implement function-level resets as per the PCIe specification.

I am not aware of any 'nice' method for triggering a hot reset (there is no sysfs entry for that). However, it is possible to use setpci to do so:

#!/bin/bash

dev=$1

if [ -z "$dev" ]; then
    echo "Error: no device specified"
    exit 1
fi

if [ ! -e "/sys/bus/pci/devices/$dev" ]; then
    dev="0000:$dev"
fi

if [ ! -e "/sys/bus/pci/devices/$dev" ]; then
    echo "Error: device $dev not found"
    exit 1
fi

port=$(basename $(dirname $(readlink "/sys/bus/pci/devices/$dev")))

if [ ! -e "/sys/bus/pci/devices/$port" ]; then
    echo "Error: device $port not found"
    exit 1
fi

echo "Removing $dev..."

echo 1 > "/sys/bus/pci/devices/$dev/remove"

echo "Performing hot reset of port $port..."

bc=$(setpci -s $port BRIDGE_CONTROL)

echo "Bridge control:" $bc

setpci -s $port BRIDGE_CONTROL=$(printf "%04x" $(("0x$bc" | 0x40)))
sleep 0.01
setpci -s $port BRIDGE_CONTROL=$bc
sleep 0.5

echo "Rescanning bus..."

echo 1 > "/sys/bus/pci/devices/$port/rescan"

Ensure that all attached drivers are unloaded before running this script. This script will attempt to remove the PCIe device, then command the upstream switch port to issue a hot reset, then attempt to rescan the PCIe bus. This script has also only been tested on devices with a single function, so it may need some reworking for devices with multiple functions.

  • This script worked for my AMD RX480. Context: PCI passthrough to a Win10 guest, then shutting down or rebooting the guest. Starting the guest again (without using this script) would hang if the GPU was still attached. Running this script in between fixed the issue – 小太郎 Oct 20 '18 at 7:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.