7

I't trying to move around 4.5 million files (size ranges from 100 - 1000 bytes) from one partition to other. The total size of the folder is ~2.4 GB

First I tried to zip it and move the zipped file to the new location. It is able to paste only ~800k files and shows "out of space" error.

Next I tried the mv command and it also resulted in same condition.

Using rsync also resulted in the same error with only ~800k files being moved.

I checked the disk free status and it is way under the limit. ( The new partition has ~700 GB free space and the required space is ~2.4 GB).

I checked the free inode for that partition it is the same. It is using only ~800k out of the maximum possible 191 M inodes. ( I had actually formatted the partition with 'mkfs.ext4 -T small /dev/sdb3' )

I have no idea of what is going wrong here. Everytime it is only able to copy or move ~800k files only.

7
  • 1
    Are all the files in the same directory? You might have to split them up. What are the filesystems involved?
    – DopeGhoti
    Commented Jan 10, 2018 at 17:07
  • Yes, all of them are in the same directory. The filesystem is ext4 in both the source and destination partition. Why do i need to split them? What difference does it make to split them in directories?
    – deathholes
    Commented Jan 10, 2018 at 17:59
  • I think DopeGhoti means it can be a max number of separate files on each directory restrictions, therefore I think this limit is higher than that.
    – francois P
    Commented Jan 10, 2018 at 18:27
  • 2
    I found the error was due to hash collisions in the ext4 filesystem. Did a 'tune2fs -O "^dir_index" ' on the partition and everything is good to go albeit a little slow.
    – deathholes
    Commented Jan 12, 2018 at 16:48
  • 1
    @deathholes Note that If you solved the problem yourself you can still answer your own question.
    – sakisk
    Commented Jan 12, 2018 at 17:42

1 Answer 1

5

I have found the reason for the error (found it on a different forum). The error was due to the hashing algorithm used by ext4 which is enabled by "dir_index" parameter. There were too many hashing collisions for me so I disabled it by the following command:

tune2fs -O "^dir_index" /dev/sdb3

The downside is that my partition is slower than before due to no indexing.

For more information on the problem : ext4: Mysterious “No space left on device”-errors

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .