2

I have two Intel SSDSC2CW120A3 SSDs in a SuperMicro X9SCL/X9SCM set for software RAID-1 on CentOS 7:

Linux hostname.local 3.10.0-957.5.1.el7.x86_64 #1 SMP Fri Feb 1 14:54:57 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

In dmesg I keep seeing "hard resetting link" on both ata1 and ata2, most of the time I (or my monitoring) don't notice any downtime but sometimes the server freezes completely and I'll have to do a power reset (Can not SSH to it anymore), according to the output of dmesg the reset happens pretty often:

$ dmesg | grep "hard resetting link"
[161507.540860] ata1: hard resetting link
[161751.123732] ata2: hard resetting link
[161798.132697] ata2: hard resetting link
[161879.126542] ata2: hard resetting link
[161939.134102] ata2: hard resetting link
[162536.225103] ata1: hard resetting link
[164738.176816] ata1: hard resetting link

More output from dmesg:

[229999.873718] ata1.00: failed command: WRITE FPDMA QUEUED
[229999.879043] ata1.00: cmd 61/08:f0:28:12:d5/00:00:00:00:00/40 tag 30 ncq 4096 out
     res 40/00:00:00:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
[229999.894050] ata1.00: status: { DRDY }
[229999.897815] ata1: hard resetting link
[230000.206411] ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[230000.223165] ata1.00: ACPI cmd ef/10:06:00:00:00:00 (SET FEATURES) succeeded
[230000.223179] ata1.00: ACPI cmd f5/00:00:00:00:00:00 (SECURITY FREEZE LOCK) filtered out
[230000.231187] ata1.00: ACPI cmd b1/c1:00:00:00:00:00 (DEVICE CONFIGURATION OVERLAY) filtered out
[230000.253132] ata1.00: ACPI cmd ef/10:06:00:00:00:00 (SET FEATURES) succeeded
[230000.253137] ata1.00: ACPI cmd f5/00:00:00:00:00:00 (SECURITY FREEZE LOCK) filtered out
[230000.261148] ata1.00: ACPI cmd b1/c1:00:00:00:00:00 (DEVICE CONFIGURATION OVERLAY) filtered out
[230000.273568] ata1.00: configured for UDMA/133
[230000.277980] ata1: EH complete

I've checked the SATA cables and they seem alright, unplugged them and plugged them in again, smartctl reports quite some uncorrectable errors but other than that nothing really suspicious. Also there's no kernel updates available.

Before I start replacing either the drives or the board I'm wondering if there is there anything else I can check? I'm trying to figure out whether this issue is hard or software related.

TIA

1

Before I start replacing either the drives or the board I'm wondering if there is there anything else I can check?

CentOS 7.6 on my home pc, asrock lga1156. Had NVidia graphics drivers installed support gtx970, did a kernel update... as you many know the nvidia install does a kernel mod which is lost [unless you use dkms]. As such, after my kernel update I had problems which manifested as things like

ACHI does not support sleep

{repeated} usb1-2 reset high speed device number 2 using xhci_ncd

reinstalled nvidia graphics drivers and problem solved.

if you don't have Nvidia drivers don't overlook other software that installs in a similar manner installing a kernel module.

1

"I've checked the SATA cables and they seem alright, unplugged them and plugged them in again"

I had the same problem at home that got solved only by changing those sata cables. Same as you, they seemed alright physically, but there was something wrong inside of them. Do not judge a cable just by looking at it. And those ata errors are annoying and they can pretty much come from ANYWHERE, even from your PSU. Example:

... I first thought it was a problem of the SATA controller or cables so I switched the drives around, yet the same drive was still affected. I then assumed it was related to the drive which got me sidetracked for a while, till I routinely decided to reset the BIOS. Doing so I randomly had a look at the supply voltages, the 12V rail had dropped to around 10.2V... ...It really was an annoying experience and the symptoms were highly misleading. After all the drive was fine, and it was just the manifestation of a completely different problem. From now on I will routinely check lm-sensors for voltages as well....

With your specific scenario, i would do the troubleshooting at the following way.

First things first(because, mdraid):

  • Check inside your BIOS if your disk controller is a FakeRAID, and see if raid mode is enabled on your setup. Change it to Sata AHCI and reinstall your system. It is pretty common knowledge (and even Red Hat has a document about) that FakeRAID's raid mode can mess up with mdraid setups. This could not solve the problem but it will avoid near future headaches.

After sata check:

  • Replace sata cables. They are pretty much inexpensive these days, unless supermicro uses some kind of "proprietary" connector at the motherboard side.
  • Update your SSD and disk controller firmwares. Your problem (failed command: WRITE FPDMA QUEUED, Emask 0x4 (timeout)) seems highly related with this, but with different hardware: [solved] "failed command: WRITE FPDMA QUEUED" ATA errors
  • Borrow a spare PSU from a friend, taking care with the specification of it, and try to replace it.
  • Contact supermicro and try a support case focused on your motherboard since you have done all the basics.
  • After all that, replace the disks.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.