4

I't trying to move around 4.5 million files (size ranges from 100 - 1000 bytes) from one partition to other. The total size of the folder is ~2.4 GB

First I tried to zip it and move the zipped file to the new location. It is able to paste only ~800k files and shows "out of space" error.

Next I tried the mv command and it also resulted in same condition.

Using rsync also resulted in the same error with only ~800k files being moved.

I checked the disk free status and it is way under the limit. ( The new partition has ~700 GB free space and the required space is ~2.4 GB).

I checked the free inode for that partition it is the same. It is using only ~800k out of the maximum possible 191 M inodes. ( I had actually formatted the partition with 'mkfs.ext4 -T small /dev/sdb3' )

I have no idea of what is going wrong here. Everytime it is only able to copy or move ~800k files only.

Improve this question
  • 1
    Are all the files in the same directory? You might have to split them up. What are the filesystems involved? – DopeGhoti Jan 10 '18 at 17:07
  • Yes, all of them are in the same directory. The filesystem is ext4 in both the source and destination partition. Why do i need to split them? What difference does it make to split them in directories? – deathholes Jan 10 '18 at 17:59
  • I think DopeGhoti means it can be a max number of separate files on each directory restrictions, therefore I think this limit is higher than that. – francois P Jan 10 '18 at 18:27
  • 2
    I found the error was due to hash collisions in the ext4 filesystem. Did a 'tune2fs -O "^dir_index" ' on the partition and everything is good to go albeit a little slow. – deathholes Jan 12 '18 at 16:48
  • 1
    @deathholes Note that If you solved the problem yourself you can still answer your own question. – sakisk Jan 12 '18 at 17:42
2

I have found the reason for the error (found it on a different forum). The error was due to the hashing algorithm used by ext4 which is enabled by "dir_index" parameter. There were too many hashing collisions for me so I disabled it by the following command:

tune2fs -O "^dir_index" /dev/sdb3

The downside is that my partition is slower than before due to no indexing.

For more information on the problem : ext4: Mysterious “No space left on device”-errors

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.