I want to test an NVMe SSD that is connected to a PCIe slot of my motherboard. The test procedure is a specific algorithm that writes workloads to the SSD, while the SSD is exposed to radiation (e.g., neutrons).
I am running Fedora 22, with kernel 4.4.6.
My current software successfully works with SATA SSD. Since the SSD can become unresponsive due to radiation, it's sometimes mandatory to power cycle it in order to resume operations. It is made possible with an externally controlled power supply.
Now, I would like to port my software to test NVMe SSD PCIe. I have modified a PCIe extender to externally apply voltage to the SSD; the derived power lines (+12V and 3.3V) are isolated from the PCIe connector power lines. With this setup, the SSD is well recognized – and works – when booting with the external power supply on.
Removing the device and re-scanning the PCI bus works as long as the NVMe SSD is powered on, namely:
echo 1 > /sys/bus/pci/devices/0000\:01\:00.0/remove
followed by:
echo 1 > /sys/bus/pci/rescan
works. However, if I power-off and then power-on the device after removing it, the PCI bus rescan
does not work (and no message appears in dmesg
).
If I "brutally" power off the SSD (with my controlled power supply) without removing the SSD under sysfs
, I would get the following:
[ 192.688934] nvme 0000:01:00.0: Failed status: ffffffff, reset controller
[ 192.689274] Trying to free nonexistent resource <000000000000e000-000000000000e0ff>
[ 192.699900] nvme 0000:01:00.0: Refused to change power state, currently in D3
[ 192.699946] Trying to free nonexistent resource <000000000000e000-000000000000e0ff>
[ 192.699953] nvme 0000:01:00.0: Device failed to resume
And obviously, rescanning the PCI bus does nothing.
Question: what would be necessary to achieve the power-cycling of the SSD without rebooting my test station? From similar threads, I understand that this problem is not trivial, so I would be content with a wide range of solutions (or hints), including:
- Adding kernel boot parameters
- Use of
setpci
commands (hints?) - Use of extra logic, e.g., wire modifications on the PCIe extender to "fool" the PCIe bus
- Modifications in the kernel sources (hints?)
rtcwake -m mem -s 5
, I can suspend for 5 seconds, and the voltage is indeed 0V on my SSD (I checked with a voltmeter). I'm using vnc to connect to the PC testing the SSD and it even turns out I do no loose the session (it just freezes for the 5 seconds). Thanks again for all the nice inputs!