72

I have a really strange situation here. My PC works fine, at least in most cases, but there's one thing that I can't deal with. When I try to copy a file from my pendrive, everything is ok -- I got 16-19M/s , it works pretty well. But when I try to copy something to the same pendrive, my PC freezes. The mouse pointer stops moving for a sec or two, then it moves a little bit and it stops again. When something is playing, for example, in Amarok, the sound acts like a machine gun. The speed jumps from 500K/s to 15M/s, average 8M/s. This occurs only when I'm copying something to a pendrive. When the process of copying is done, everything backs to normal.

I tried everything -- other pendrive, a different USB port on front panel or those ports from back, I even changed the USB pins on motherboard (front panel), but no matter where I put my USB stick, it's always the same. I tried different filesystem -- fat32, ext4. I have no problem with the device on Windows, on my laptop. It has to be my PC or something in my system. I have no idea what to look for. I'm using Debian testing with standalone Openbox. My PC is kind of old -- Pentium D 3GHz, 1GiB of RAM, 1,5TB WD Green disk. If you have something that would help me to solve this issue, I'd be glad to hear that.

I don't know what else info I should provide, but if you need something, just ask, I'll update this post as soon as possible.

I tried to reproduce this problem on ubuntu 13.04 live cd. I mounted my encrypted partition + encrypted swap and connected my pendrive to a usb port. Next I tried to start some apps, and now I have ~820MiB in RAM and about 400MiB in SWAP. There's no problem with copying, no freezing at all, everything is as it should be. So, it looks like it's a fault of the system, but where exactly? What would cause such a weird behavior?

Improve this question
6
  • When I encountered something similar to this it was because I had hard drive issues on my laptop. The disk had some bad areas and every time I was trying to read anything from those areas it would freeze until it was done. Just an idea to look into. Maybe you have bad sectors where you're trying to read from.
    – slybloty
    Jan 3, 2014 at 16:22
  • My hdd is ok, at least smart say so (after full scan).
    – Mikhail Morfikov
    Jan 3, 2014 at 16:26
  • Try reducing the IO priority of the copying process, e.g. ionice -c3 cp something.tgz /media/pendrive. This will put the newly spawned cp process in the third (=lowest) priority class "idle".
    – n.st
    Jan 3, 2014 at 18:00
  • I tried this, but it has no effect.
    – Mikhail Morfikov
    Jan 3, 2014 at 18:22
  • @MikhailMorfikov FYI, this issue was addressed in linux 4.9. Still haven't tested the fix myself. YMMV.
    – Seamus Connor
    Jan 24, 2017 at 20:27

3 Answers 3

Reset to default
107

Are you using a 64-bit version of Linux with a lot of memory? In that case the problem could be that Linux can lock for minutes on big writes on slow devices like for example SD cards or USB sticks. It's a known bug that should be fixed in newer kernels.

See http://lwn.net/Articles/572911/

Workaround: as root issue:

echo $((16*1024*1024)) > /proc/sys/vm/dirty_background_bytes
echo $((48*1024*1024)) > /proc/sys/vm/dirty_bytes

I have added it to my /etc/rc.local file in my 64bit machines.

TANSTAAFL; this change can (and probably will) reduce your throughput to these devices --- it's a compromise between latency and speed. To get back to the previous behavior you can 

echo 0 > /proc/sys/vm/dirty_background_bytes
echo 0 > /proc/sys/vm/dirty_bytes

...which are the default values, meaning that the writeback behavior will be controlled by the parameters dirty_ratio and dirty_background_ratio.

Note for the not-so-expert-with-linux people: the files in /proc are pseudofiles --- just communication channels between the kernel and user space. Never use an editor to change or look at them; get instead a shell prompt --- for example, with sudo -i (Ubuntu flavors) or su root and use echo and cat).

Update 2016/04/18 it seems that, after all, the problem is still here. You can look at it at LWN.net, in this article about writeback queues.

Improve this answer
17
  • 4
    I have 64bit but only 1GiB of RAM, and I have to tell you this solution works! I've just tested it, and after setting the two parameters, there's no freezing anymore. :)
    – Mikhail Morfikov
    Jan 3, 2014 at 19:10
  • 1
    In my 14.04 installation, uname -a returns 3.13.0-32-generic, so yes. But I have not checked if the patch for the issue was finally integrated in the kernel or not. I have a 16GB machine and it seems to work ok without the workaround, although I have to say that I didn't try with especially slow devices.
    – Rmano
    Aug 7, 2014 at 15:56
  • 1
    @IonicăBizău --- that is a pseudo-file, do not edit it with vim ever. Get a root shell (with sudo -i) and use the aforementioned commands.
    – Rmano
    Oct 5, 2014 at 17:52
  • 1
    @Rmano It worked! However, I did edit it with VIM. Thanks!
    – Ionică Bizău
    Oct 5, 2014 at 18:49
  • 3
    I'm using ubuntu 16.04 on a brand new laptop (with 16 GB of RAM). I was really angry at this issue. Your solution worked like a charm! Maybe you could add that this could still be needed with kernel 4.8.0-45 .
    – LGenzelis
    Apr 14, 2017 at 18:58
2

hardware vs. software

I've run into strange problem similar to this with USB thumbdrives, and in my research it's almost always either a driver issue or the specific hardware within the PC/Motherboard.

I know this because I've got several systems that are identical hardware, and on one, I can do this operation without issue, while on another the problem shows up.

What to do?

You options are really limited here. About the only things you can do are make sure you have the latest BIOS/firmware installed on your system, and make sure you have the latest versions of your disto's packages.

Beyond that all I can suggest is making sure that you avoid this situation by not attempting to copy files while another copy is in progress.

If you have the type of personality where things like this irk you, you could try another live distro of Linux and repeat the steps that lead to your problem. This would just eliminate whether it's a distro specific issue or a hardware issue as I've described above. It would be a small consolation, but I always like to know things rather than bury my head in the sand, and not.

Anything else?

If you're truly obsessive you could try running the application that you're doing the copy with through strace in the hopes of catching the system in whatever system call is freezing. You should be able to do this from the command line as well.

Example

$ strace -o cp1.log cp -r /path/to/dir1 /path/to/usb/.

Then while that's running start another one.

$ strace -o cp2.log cp -r /path/to/dir2 /path/to/usb/.

The system will hopefully freeze during this operation and maybe you'll get lucky and find some smoke in either of those log files.

Improve this answer
5
  • I always use the only one instance of file copying. I have BIOS updated (2008), and there's no newer version since then. I think it's not the BIOS. My debian distro is also updated to the testing branch. I tried using strace and it starred freezing almost instantly, so I waited a few sec and killed the process. I got 1Mb log, but I can't read it, I don't know what to look for. You can check it here pastebin.com/u29RvqgC -- it's not the full log (limited to 500Kb), but there were only similar lines to those at the end. I will try to reproduce this issue with ubuntu live cd.
    – Mikhail Morfikov
    Jan 3, 2014 at 17:08
  • I updated the question as to live cd testing.
    – Mikhail Morfikov
    Jan 3, 2014 at 17:51
  • @MikhailMorfikov - I think you're pretty much at the end of what you can expect to do. Your hardware is pretty old (2008) and there's really not much else you can do beyond what I've outlined above.
    – slm ♦
    Jan 3, 2014 at 18:03
  • But even older pcs are able to copy files without problems.
    – Mikhail Morfikov
    Jan 3, 2014 at 18:20
  • 1
    @MikhailMorfikov - Age isn't the only factor, but the likely hood of getting any updates to firmware or updates to software for old hardware is low, is what I meant.
    – slm ♦
    Jan 3, 2014 at 18:22
2

The reason can be write amplification, as the system tries to write in chunks that are smaller, than erase block (doing read/mod/write) + block misalignment.

To check yours current setting do:

cat /sys/block/sd**X**/device/max_sectors

You can tune hall rules for those devices:

Change value of USB "max_sectors" for an entire family of devices

In this case I had replaced max_sectors for all devices, that used default of 240 (USB storage) to 32K sectors or 2K sectors.

On my system (Mageia 4, 3.14.24 core i7) I had to do this due terribly slow write speeds (2MB/sec) on Kingston DT101 G2 16GB: 

vi /usr/lib/udev/rules.d/81-udisks_maxsect.rules

and add:

SUBSYSTEMS=="scsi", ATTR{max_sectors}=="240", ATTR{max_sectors}="32678"

And the dd write speed went up 3x times. mc cp probably 10-20x up (after I had started first partition @8192'th sector and reformatted with 64k aligned clusters): 

fdisk -u /dev/sdh # make DOS compat off if on
mkfs.vfat /dev/sdh1 -n KINGSTON16G -s 128 **-R 4592*** and use *fsck.vfat -v /dev/sdh1

to check alignment (check [data start sector] should be multiple of 128 (cluster size)). Adjust number of the reserved sectors (-R) if needed.

Default max_sectors (240) seem to cause high write amplification on some of the cheap new drives. But be very careful with such high setting, the similar effect is achieved at 2048 sectors (probably 1M erase blocks:

SUBSYSTEMS=="scsi", ATTR{max_sectors}=="240", ATTR{max_sectors}="2048"

Test all yours old USB devices, that they still work well. Use vendor/model attributes in the rules files to be more specific.

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .