2

I'm making a bash script that copies files from one USB drive to other multiple drives.

One oldschool machine running Xubuntu dedicated for the job detects 30 devices with no problem. It has USB 2.0 ports only.

While another modern machine running Linux Mint doesn't detect more than 16 of these drives at a time. It has USB 3.0 ports only. After I connect more than 16 drives, the new ones stop to show up in lsblk.

What could be the cause?

EDIT:

As fduff pointed out, I checked dmesg after connecting the 17th USB drive, this is what it shows:

[  531.519845] usb 3-9.3.2.6: new high-speed USB device number 46 using xhci_hcd
[  531.974582] usb 3-9.3.2.6: New USB device found, idVendor=abcd, idProduct=1234
[  531.974585] usb 3-9.3.2.6: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[  531.974586] usb 3-9.3.2.6: Product: UDisk
[  531.974587] usb 3-9.3.2.6: Manufacturer: General
[  531.974588] usb 3-9.3.2.6: SerialNumber: Љ
[  531.975337] usb 3-9.3.2.6: Not enough host controller resources for new device state.
[  531.975340] usb 3-9.3.2.6: can't set config #1, error -12

This makes it look like a hardware limitation, but the number of allowed USB devices is not constant.

Now for example I can't connect more than 15 USB drives (the 16th gets error -12), before I was also able once to connect about 20 devices, and they got all discovered properly.

I tried using a different USB port to spread the usage across multiple USB controllers, but no luck.

My hardware specs:

MB: ASRock Z97 Extreme6
CPU: Intel Core i5-4690K
GPU: MSI GeForce GTX 960 2GB
RAM: HyperX Fury Black 8GB [2x4GB 1600MHz DDR3 CL10 DIMM]
PSU: Chieftec GPS-500A8 [500W]
Display: 2x AOC I2276VWM [IPS, 1920x1080]
HDD: WD Blue 3TB

I'm also using a 28-port Manhattan USB hub: https://www.amazon.com/Manhattan-Port-USB-Hub-161718/dp/B0074024XU

A low-end dying PC running Xubuntu from LiveCD (on a USB drive) was able to detect 29 devices (30 counting the LiveCD drive) and perform file transfers - even though Windows XP that it had installed on HDD at some point failed to detect the 28-port hub at all. That PC now is being looked at by a repair service, because it died (I bet on PSU or MB). Still - it worked, and the shiny new one isn't.

  • 2
    Have you checked the output of dmesg once you insert the 16th and 17th USB devices? It could provide some pointers. – fduff Mar 10 '17 at 11:46
  • I didn't. Updating my question now. – unfa Mar 10 '17 at 13:13
4

This could be hardware related issue. I've spotted this on the Intel forum:

For 8th series motherboard the maximum end points is 96. Each USB device can support multiple end points and how many end points can support vary by device. Once it reaches to the max endpoints, you will get a pop-message; the limitation is not based on number of devices supported but the end points.

and also

It is not the number of devices that is significant, but rather the number of endpoints that those devices use. The fact that you are experiencing the issue on USB 3.0 ports implies the use of a xHCI controller, as opposed to a USB 2.0 eHCI controller. There is a lower limit on Intel's xHCI controller than the eHCI controller. For the xHCI controller it is 96 endpoints. It sounds like you are hitting this endpoint limit. You could use something like Microsoft's USB utility 'USBVIEW' to show you how many endpoints each USB device is using.

There is nothing you can do about it, apart from trying to distribute your devices across multiple USB controllers. This, of course, depends on whether your board has multiple USB controllers, and whether you can figure out which physical USB connectors are routed to which USB controller (again USBVIEW would be useful).

Have a closer at the forum thread, it's quite interesting.

You can use the following command to find out how many endpoints are listed:

lsusb -v | grep bEndpointAddress | wc -l

  • With 14 + 1 USB drives connected (and a mouse, a keyboard and two hubs plugged in) the command returned 64 endpoints. I plugged in one more and it counted 66 endpoints. I plugged in 13 more USB drives but it got only up to 76 endpoints, while the drives were already not detected by lsblk, but still reported as 2 endpoints per device. Maybe the 2 endpoints per device is for separate write and read? – unfa Mar 10 '17 at 13:43
0

This issue comes from the underlying USB hardware controller which cannot handle that much devices. Linux source code shows the xHC returned a "resource error" code when trying to configure the interface. Chapter 4.4.6 of the xHCI standard explains this case:

The Resources Required variable is compared to the Resources Available variable, if the result indicates an oversubscription of resources by the command (i.e. Resources Available - Resources Required is less than 0), then the command shall be unsuccessful and a Resource Error Completion Code shall be returned in the Command Completion Event. Refer to section 4.14.1.1 for more information on xHC resources.

My workaround is to release xHCI resources by removing every USB device I don't need, such as internal USB devices (e.g. bluetooth, wifi, etc.) and USB 3.0 hubs with only USB 2.0 devices attached to them.

Here are the steps:

  1. Run lsusb to see if there is anything unused.
  2. Remove your unused devices using sysfs echo 1 > /sys/<path to device>/remove. Have a look at dmesg and see how it removes the device and every sub-device. Which means that you can remove an entire USB tree by using this command on the root device.
  3. You should now be able to correctly plug more devices (cf. dmesg logs).

To make this persistent when rebooting, add udev rules to remove the devices:

SUBSYSTEM=="usb" <your conditions to match unused devices> RUN="sh -c 'echo 1 > /sys$DEVPATH/remove'"

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.