4

I'm running rsync to backup a remote machine to a USB hard drive on an ARM SBC and sometimes rsync just stops with "read error from input device (I/O error)". I believe the issue is related to UAS + USB 3.0 + rsync causing high I/O load, because of uas_eh_device_reset_handler on /var/log/messages:

sd 0:0:0:0: [sda] tag#1 data cmplt err -32 uas-tag 2 inflight: 
sd 0:0:0:0: [sda] tag#1 CDB: opcode=0x28 28 00 38 80 0a 68 00 00 a0 00
sd 0:0:0:0: [sda] tag#0 data cmplt err -32 uas-tag 1 inflight: CMD 
sd 0:0:0:0: [sda] tag#0 CDB: opcode=0x2a 2a 00 57 50 28 78 00 03 00 00
sd 0:0:0:0: [sda] tag#1 uas_eh_abort_handler 0 uas-tag 2 inflight: CMD 
sd 0:0:0:0: [sda] tag#1 CDB: opcode=0x28 28 00 38 80 0a 68 00 00 a0 00
sd 0:0:0:0: [sda] tag#2 uas_eh_abort_handler 0 uas-tag 3 inflight: CMD 
sd 0:0:0:0: [sda] tag#2 CDB: opcode=0x2a 2a 00 19 47 7f 20 00 00 90 00
sd 0:0:0:0: [sda] tag#0 uas_eh_abort_handler 0 uas-tag 1 inflight: CMD 
sd 0:0:0:0: [sda] tag#0 CDB: opcode=0x2a 2a 00 57 50 28 78 00 03 00 00
scsi host0: uas_eh_device_reset_handler start
usb 5-1: reset high-speed USB device number 2 using ehci-platform
scsi host0: uas_eh_device_reset_handler success
sd 0:0:0:0: [sda] tag#0 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=0x08
sd 0:0:0:0: [sda] tag#0 Sense Key : 0x2 [current] 
sd 0:0:0:0: [sda] tag#0 ASC=0x3a ASCQ=0x0 
sd 0:0:0:0: [sda] tag#0 CDB: opcode=0x2a 2a 00 57 50 28 78 00 03 00 00
sd 0:0:0:0: [sda] tag#1 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=0x08
sd 0:0:0:0: [sda] tag#1 Sense Key : 0x2 [current] 
sd 0:0:0:0: [sda] tag#1 ASC=0x3a ASCQ=0x0 
sd 0:0:0:0: [sda] tag#1 CDB: opcode=0x2a 2a 00 19 47 7f 20 00 00 90 00
EXT4-fs warning (device sda1): ext4_end_bio:323: I/O error 10 writing to inode 13001563 (offset 0 size 73728 starting block 53014518)

This SBC doesn't have a USB 3 port, however it still loads the hard drive with UAS. According to this, UAS is broken on some HD enclosure chips. The solution provided is to disable UAS, however:

1- If I blacklist UAS completely with blacklist uas into /etc/modprobe.d/blacklist-uas.conf I get:

lsusb -t
    /:  Bus 05.Port 1: Dev 1, Class=root_hub, Driver=ehci-platform/1p, 480M
        |__ Port 1: Dev 2, If 0, Class=Mass Storage, Driver=, 480M

Looking at Class=Mass Storage, Driver=, 480M => seems like the system doesn't load any other way to deal with the drive.

2- If I just try to disable UAS for a specific USB device, like the post recommended, it still loads with UAS:

echo options usb-storage quirks=174c:55aa:u | tee /etc/modprobe.d/blacklist-uas.conf
update-initramfs -u
reboot

(...)

dmesg | grep sda
[    2.488105] sd 0:0:0:0: [sda] 2930277168 512-byte logical blocks: (1.50 TB/1.36 TiB)
[    2.488584] sd 0:0:0:0: [sda] Write Protect is off
[    2.488592] sd 0:0:0:0: [sda] Mode Sense: 43 00 00 00
[    2.489335] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[    2.539288]  sda: sda1
[    2.543875] sd 0:0:0:0: [sda] Attached SCSI disk
[    6.898109] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,data=ordered

lsusb | grep ASMedia
Bus 005 Device 002: ID 174c:55aa ASMedia Technology Inc. ASM1051E SATA 6Gb/s bridge, ASM1053E SATA 6Gb/s bridge, ASM1153 SATA 3Gb/s bridge

lsusb -t
/:  Bus 05.Port 1: Dev 1, Class=root_hub, Driver=ehci-platform/1p, 480M
    |__ Port 1: Dev 2, If 0, Class=Mass Storage, Driver=uas, 480M

What am I doing wrong? Is it possible to disable UAS and make the system still use the HD any other way? Why does options usb-storage quirks=174c:55aa:u doesn't disable UAS as it should?

Thank you.

Some notes:

  • OS: Debian GNU/Linux 9.4 (stretch) kernel 4.14.18-sunxi64 from armbian
  • SBC: NanoPi NEO2
  • The option doesn't actually apply to the uas module: it applies to the usb_storage module. Is that a stock Debian kernel? 1/ are you sure usb_storage is a module and not built-in? 2/ built-in = you have to adapt & move the option to the kernel boot 3/ if a module: if the module is loaded at the initramfs phase, reboot is not enough. You have to rebuild the initramfs. usually with update-initramfs -u -k $(uname -r). Some ARM bootloaders after this require an other step (mkimage) – A.B May 4 '18 at 6:34
  • I red rebuild the image using update-initramfs -u. The kernel is 4.14.18-sunxi64 from armbian. usb_storage doesn't show under lsmod so I guess it is build in. How should I "move the option to the kernel boot"? Thank you. – TCB13 May 4 '18 at 11:47
  • @A.B from what I've read here I can possible try echo "174c:55aa:u" >/sys/module/usb_storage/parameters/quirks but how permanent is this? Will it survive a reboot? If no, where should I put it? – TCB13 May 4 '18 at 12:01
4

With the precious help from @A.B I managed to fix this. As he said, my kernel (probably every armbian SBC kernel) doesn't have usb_storage loaded as a module, it is built-in.

In this case, we need to change the boot options that are visible under /proc/cmdline:

root=UUID=b58.... rootfstype=ext4 console=tty1 console=ttyS0,115200 panic=10 consoleblank=0 loglevel=1 ubootpart=096d26e5-01 usb-storage.quirks=0x2537:0x1066:u,0x2537:0x1068:u   cgroup_enable=memory swapaccount=1

At the end there is usb-storage.quirks=0x2537:0x1066:u,0x2537:0x1068:u already set. We can't edit this file directly, in armbian this options are stored on the file /boot/armbianEnv.txt:

verbosity=1
console=both
overlay_prefix=sun50i-h5
overlays=usbhost1 usbhost2
rootdev=UUID=b58048d3-ca7b-4ea6-9812-95d403fddadd
rootfstype=ext4
usbstoragequirks=0x2537:0x1066:u,0x2537:0x1068:u

So I just added my device in the last line as ,174c:55aa:u, making it:

usbstoragequirks=0x2537:0x1066:u,0x2537:0x1068:u,174c:55aa:u

Just in case I re-run update-initramfs -u and after a reboot the USB HD now uses only usb-store instead of uas:

lsusb -t
/:  Bus 05.Port 1: Dev 1, Class=root_hub, Driver=ehci-platform/1p, 480M
    |__ Port 1: Dev 2, If 0, Class=Mass Storage, Driver=usb-storage, 480M

As you can see here, uas is now properly blacklisted for the device:

dmesg | grep "usb 5-1"
[    2.308569] usb 5-1: new high-speed USB device number 2 using ehci-platform
[    2.467087] usb 5-1: New USB device found, idVendor=174c, idProduct=55aa
[    2.467106] usb 5-1: New USB device strings: Mfr=2, Product=3, SerialNumber=1
[    2.467117] usb 5-1: Product: ASM1153E
[    2.467127] usb 5-1: Manufacturer: Inateck
[    2.467137] usb 5-1: SerialNumber: 12345678910E
[    2.468297] usb 5-1: UAS is blacklisted for this device, using usb-storage instead
2

I had a similar problem on Debian 9 and my answer turned out to be quite similar but not exactly the same:

(as root:)

    echo "options usb-storage quirks=4971:8017:u" >> /etc/modprobe.d/usb-storage-quirks.conf
    update-initramfs -u

The 4971:8017 device id is a "SimpleTech" based Rosewill RX307-PU3-35B USB-3 disk enclosure advertised as supporting UASB. With some drives, at least, it has to be reset frequently, generating many alarming messages in syslog like this:

Nov 16 13:12:10 guy kernel: [226970.895835] sd 8:0:0:0: [sda] tag#8 uas_eh_abort_handler 0 uas-tag 1 inflight: CMD IN
Nov 16 13:12:10 guy kernel: [226970.895840] sd 8:0:0:0: [sda] tag#8 CDB: ATA command pass through(16) 85 08 0e 00 d0 00 01 00 00 00 4f 00 c2 00 b0 00
Nov 16 13:12:10 guy kernel: [226970.919935] scsi host8: uas_eh_device_reset_handler start
Nov 16 13:12:10 guy smartd[1079]: Device: /dev/sda [SAT], failed to read SMART Attribute Data
Nov 16 13:12:10 guy kernel: [226971.217025] scsi host8: uas_eh_device_reset_handler success
Nov 16 13:13:11 guy kernel: [227032.337409] sd 8:0:0:0: [sda] tag#8 uas_eh_abort_handler 0 uas-tag 1 inflight: CMD IN
Nov 16 13:13:11 guy kernel: [227032.337412] sd 8:0:0:0: [sda] tag#8 CDB: ATA command pass through(16) 85 08 0e 00 d5 00 01 00 06 00 4f 00 c2 00 b0 00
Nov 16 13:13:11 guy kernel: [227032.361403] scsi host8: uas_eh_device_reset_handler start
Nov 16 13:13:12 guy smartd[1079]: Device: /dev/sda [SAT], Read SMART Self Test Log Failed
Nov 16 13:13:12 guy kernel: [227032.654494] scsi host8: uas_eh_device_reset_handler success

Weirdly, at least for bulk data transfers, it appears to be actually faster now than it was with the uas driver!

  • You solution doesn't usually work on SBCs because in most of them you can't use modprobe.d to fix this kind of issues. So, in short, we're using the exact same solution but applying it in different places because the underlaying kernel/hardware is different. :) – TCB13 May 3 at 11:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.