9

I'm running rsync to backup a remote machine to a USB hard drive on an ARM SBC and sometimes rsync just stops with "read error from input device (I/O error)". I believe the issue is related to UAS + USB 3.0 + rsync causing high I/O load, because of uas_eh_device_reset_handler on /var/log/messages:

sd 0:0:0:0: [sda] tag#1 data cmplt err -32 uas-tag 2 inflight: 
sd 0:0:0:0: [sda] tag#1 CDB: opcode=0x28 28 00 38 80 0a 68 00 00 a0 00
sd 0:0:0:0: [sda] tag#0 data cmplt err -32 uas-tag 1 inflight: CMD 
sd 0:0:0:0: [sda] tag#0 CDB: opcode=0x2a 2a 00 57 50 28 78 00 03 00 00
sd 0:0:0:0: [sda] tag#1 uas_eh_abort_handler 0 uas-tag 2 inflight: CMD 
sd 0:0:0:0: [sda] tag#1 CDB: opcode=0x28 28 00 38 80 0a 68 00 00 a0 00
sd 0:0:0:0: [sda] tag#2 uas_eh_abort_handler 0 uas-tag 3 inflight: CMD 
sd 0:0:0:0: [sda] tag#2 CDB: opcode=0x2a 2a 00 19 47 7f 20 00 00 90 00
sd 0:0:0:0: [sda] tag#0 uas_eh_abort_handler 0 uas-tag 1 inflight: CMD 
sd 0:0:0:0: [sda] tag#0 CDB: opcode=0x2a 2a 00 57 50 28 78 00 03 00 00
scsi host0: uas_eh_device_reset_handler start
usb 5-1: reset high-speed USB device number 2 using ehci-platform
scsi host0: uas_eh_device_reset_handler success
sd 0:0:0:0: [sda] tag#0 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=0x08
sd 0:0:0:0: [sda] tag#0 Sense Key : 0x2 [current] 
sd 0:0:0:0: [sda] tag#0 ASC=0x3a ASCQ=0x0 
sd 0:0:0:0: [sda] tag#0 CDB: opcode=0x2a 2a 00 57 50 28 78 00 03 00 00
sd 0:0:0:0: [sda] tag#1 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=0x08
sd 0:0:0:0: [sda] tag#1 Sense Key : 0x2 [current] 
sd 0:0:0:0: [sda] tag#1 ASC=0x3a ASCQ=0x0 
sd 0:0:0:0: [sda] tag#1 CDB: opcode=0x2a 2a 00 19 47 7f 20 00 00 90 00
EXT4-fs warning (device sda1): ext4_end_bio:323: I/O error 10 writing to inode 13001563 (offset 0 size 73728 starting block 53014518)

This SBC doesn't have a USB 3 port, however it still loads the hard drive with UAS. According to this, UAS is broken on some HD enclosure chips. The solution provided is to disable UAS, however:

1- If I blacklist UAS completely with blacklist uas into /etc/modprobe.d/blacklist-uas.conf I get:

lsusb -t
    /:  Bus 05.Port 1: Dev 1, Class=root_hub, Driver=ehci-platform/1p, 480M
        |__ Port 1: Dev 2, If 0, Class=Mass Storage, Driver=, 480M

Looking at Class=Mass Storage, Driver=, 480M => seems like the system doesn't load any other way to deal with the drive.

2- If I just try to disable UAS for a specific USB device, like the post recommended, it still loads with UAS:

echo options usb-storage quirks=174c:55aa:u | tee /etc/modprobe.d/blacklist-uas.conf
update-initramfs -u
reboot

(...)

dmesg | grep sda
[    2.488105] sd 0:0:0:0: [sda] 2930277168 512-byte logical blocks: (1.50 TB/1.36 TiB)
[    2.488584] sd 0:0:0:0: [sda] Write Protect is off
[    2.488592] sd 0:0:0:0: [sda] Mode Sense: 43 00 00 00
[    2.489335] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[    2.539288]  sda: sda1
[    2.543875] sd 0:0:0:0: [sda] Attached SCSI disk
[    6.898109] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,data=ordered

lsusb | grep ASMedia
Bus 005 Device 002: ID 174c:55aa ASMedia Technology Inc. ASM1051E SATA 6Gb/s bridge, ASM1053E SATA 6Gb/s bridge, ASM1153 SATA 3Gb/s bridge

lsusb -t
/:  Bus 05.Port 1: Dev 1, Class=root_hub, Driver=ehci-platform/1p, 480M
    |__ Port 1: Dev 2, If 0, Class=Mass Storage, Driver=uas, 480M

What am I doing wrong? Is it possible to disable UAS and make the system still use the HD any other way? Why does options usb-storage quirks=174c:55aa:u doesn't disable UAS as it should?

Thank you.

Some notes:

  • OS: Debian GNU/Linux 9.4 (stretch) kernel 4.14.18-sunxi64 from armbian
  • SBC: NanoPi NEO2
Improve this question
3
  • The option doesn't actually apply to the uas module: it applies to the usb_storage module. Is that a stock Debian kernel? 1/ are you sure usb_storage is a module and not built-in? 2/ built-in = you have to adapt & move the option to the kernel boot 3/ if a module: if the module is loaded at the initramfs phase, reboot is not enough. You have to rebuild the initramfs. usually with update-initramfs -u -k $(uname -r). Some ARM bootloaders after this require an other step (mkimage)
    – A.B
    May 4, 2018 at 6:34
  • I red rebuild the image using update-initramfs -u. The kernel is 4.14.18-sunxi64 from armbian. usb_storage doesn't show under lsmod so I guess it is build in. How should I "move the option to the kernel boot"? Thank you.
    – TCB13
    May 4, 2018 at 11:47
  • 1
    @A.B from what I've read here I can possible try echo "174c:55aa:u" >/sys/module/usb_storage/parameters/quirks but how permanent is this? Will it survive a reboot? If no, where should I put it?
    – TCB13
    May 4, 2018 at 12:01

3 Answers 3

Reset to default
7

With the precious help from @A.B I managed to fix this. As he said, my kernel (probably every armbian SBC kernel) doesn't have usb_storage loaded as a module, it is built-in.

In this case, we need to change the boot options that are visible under /proc/cmdline:

root=UUID=b58.... rootfstype=ext4 console=tty1 console=ttyS0,115200 panic=10 consoleblank=0 loglevel=1 ubootpart=096d26e5-01 usb-storage.quirks=0x2537:0x1066:u,0x2537:0x1068:u   cgroup_enable=memory swapaccount=1

At the end there is usb-storage.quirks=0x2537:0x1066:u,0x2537:0x1068:u already set. We can't edit this file directly, in armbian this options are stored on the file /boot/armbianEnv.txt:

verbosity=1
console=both
overlay_prefix=sun50i-h5
overlays=usbhost1 usbhost2
rootdev=UUID=b58048d3-ca7b-4ea6-9812-95d403fddadd
rootfstype=ext4
usbstoragequirks=0x2537:0x1066:u,0x2537:0x1068:u

So I just added my device in the last line as ,174c:55aa:u, making it:

usbstoragequirks=0x2537:0x1066:u,0x2537:0x1068:u,174c:55aa:u

Just in case I re-run update-initramfs -u and after a reboot the USB HD now uses only usb-store instead of uas:

lsusb -t
/:  Bus 05.Port 1: Dev 1, Class=root_hub, Driver=ehci-platform/1p, 480M
    |__ Port 1: Dev 2, If 0, Class=Mass Storage, Driver=usb-storage, 480M

As you can see here, uas is now properly blacklisted for the device:

dmesg | grep "usb 5-1"
[    2.308569] usb 5-1: new high-speed USB device number 2 using ehci-platform
[    2.467087] usb 5-1: New USB device found, idVendor=174c, idProduct=55aa
[    2.467106] usb 5-1: New USB device strings: Mfr=2, Product=3, SerialNumber=1
[    2.467117] usb 5-1: Product: ASM1153E
[    2.467127] usb 5-1: Manufacturer: Inateck
[    2.467137] usb 5-1: SerialNumber: 12345678910E
[    2.468297] usb 5-1: UAS is blacklisted for this device, using usb-storage instead
Improve this answer
3
  • 2
    Quick update: after +24h running rsync and some other I/O intensive tasks on the USB HD everything is working fine so far.
    – TCB13
    May 6, 2018 at 1:28
  • 1
    BTW the documentation for the "usb-storage.quirks" option can be found in github.com/torvalds/linux/blob/master/Documentation/admin-guide/…
    – plugwash
    May 2, 2019 at 15:29
  • worked also for me on a nanoPi r4S + same USB adapter. Note i don't explain myself why my first attempt failed with ,0x0174c:0x55aa:u instead of ,174c:55aa:u, despite other quicks in the line already present are of the form 0xsomething:
    – user313492
    Dec 30, 2021 at 9:59
4

I had a similar problem on Debian 9 and my answer turned out to be quite similar but not exactly the same:

(as root:)

    echo "options usb-storage quirks=4971:8017:u" >> /etc/modprobe.d/usb-storage-quirks.conf
    update-initramfs -u

The 4971:8017 device id is a "SimpleTech" based Rosewill RX307-PU3-35B USB-3 disk enclosure advertised as supporting UASB. With some drives, at least, it has to be reset frequently, generating many alarming messages in syslog like this:

Nov 16 13:12:10 guy kernel: [226970.895835] sd 8:0:0:0: [sda] tag#8 uas_eh_abort_handler 0 uas-tag 1 inflight: CMD IN
Nov 16 13:12:10 guy kernel: [226970.895840] sd 8:0:0:0: [sda] tag#8 CDB: ATA command pass through(16) 85 08 0e 00 d0 00 01 00 00 00 4f 00 c2 00 b0 00
Nov 16 13:12:10 guy kernel: [226970.919935] scsi host8: uas_eh_device_reset_handler start
Nov 16 13:12:10 guy smartd[1079]: Device: /dev/sda [SAT], failed to read SMART Attribute Data
Nov 16 13:12:10 guy kernel: [226971.217025] scsi host8: uas_eh_device_reset_handler success
Nov 16 13:13:11 guy kernel: [227032.337409] sd 8:0:0:0: [sda] tag#8 uas_eh_abort_handler 0 uas-tag 1 inflight: CMD IN
Nov 16 13:13:11 guy kernel: [227032.337412] sd 8:0:0:0: [sda] tag#8 CDB: ATA command pass through(16) 85 08 0e 00 d5 00 01 00 06 00 4f 00 c2 00 b0 00
Nov 16 13:13:11 guy kernel: [227032.361403] scsi host8: uas_eh_device_reset_handler start
Nov 16 13:13:12 guy smartd[1079]: Device: /dev/sda [SAT], Read SMART Self Test Log Failed
Nov 16 13:13:12 guy kernel: [227032.654494] scsi host8: uas_eh_device_reset_handler success

Weirdly, at least for bulk data transfers, it appears to be actually faster now than it was with the uas driver!

Improve this answer
2
  • You solution doesn't usually work on SBCs because in most of them you can't use modprobe.d to fix this kind of issues. So, in short, we're using the exact same solution but applying it in different places because the underlaying kernel/hardware is different. :)
    – TCB13
    May 3, 2019 at 11:25
  • This exact fix worked for me on Debian 10 as well. Thanks.
    – Chris Moore
    Apr 2, 2021 at 2:06
2

In my case the IO problems were caused by insufficient power supply from my computer to USB SATA3 adapter. The log messages looked the same as here.

Playing with uas parameters did not help. Buying a powered USB3 HUB did resolve the issue.

Improve this answer
1
  • Oh, wow - not only was this my problem (on a Sager laptop with USB2 ports), but my device (174c:1153) doesn't work correctly if it's not in UAS mode, so the quirk (174c:1153:u) also caused aborts! Thanks, Jan.
    – Bill McGonigle
    Mar 18 at 15:16

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .