2

Fragmentation seems to create a lot of unnecessary seeks when traversing a directory tree on a HDD:

# stat -c %F 00 01 02
directory
directory
directory

# filefrag -v 00 01 02
Filesystem type is: ef53
File size of 00 is 12288 (3 blocks of 4096 bytes)
 ext:     logical_offset:        physical_offset: length:   expected: flags:
   0:        0..       0:  428351942.. 428351942:      1:            
   1:        1..       2:  428352760.. 428352761:      2:  428351943: last,eof
00: 2 extents found
File size of 01 is 12288 (3 blocks of 4096 bytes)
 ext:     logical_offset:        physical_offset: length:   expected: flags:
   0:        0..       0:  428351771.. 428351771:      1:            
   1:        1..       2:  428891667.. 428891668:      2:  428351772: last,eof
01: 2 extents found
File size of 02 is 12288 (3 blocks of 4096 bytes)
 ext:     logical_offset:        physical_offset: length:   expected: flags:
   0:        0..       0:  428351795.. 428351795:      1:            
   1:        1..       2:  428352705.. 428352706:      2:  428351796: last,eof
02: 2 extents found

e4defrag isn't able to defrag them

# e4defrag -v 00
ext4 defragmentation for directory(00)
[1/116] "00"
    File is not regular file        [ NG ]

So how do I defragment a directory? Not its contents, but the directory itself. The directories are in use, so it should be done atomically, just like defragmenting regular files does not interfere with their use.

6
  • @roaima The title is my question: [How to] atomically defragment ext4 directory files? – the8472 Mar 12 '17 at 5:21
  • @roaima 1. I said "directory file", referring to the directory object itself (everything is a file, just like block devices are files!) 2. atomically is a word. but "online defragmentation" might work too in this context. 3. I already edited my question. – the8472 Mar 12 '17 at 9:19
  • "Your example is of a directory." - I intentionally highlighted that with the stat command. So I am quite aware of that. – the8472 Mar 12 '17 at 9:19
  • You don't, at least not online. Offline, e2fsck has an option to defrag directories, and then there is the old e2defrag. Of course, having only two fragments isn't going to affect performance in any meaningful way. – psusi Mar 17 '17 at 13:25
  • @psusi running fsck -D asked to optimize some extent trees (Inode 41947219 extent tree (at level 1) could be shorter. Fix? yes), but it didn't rewrite the directories in question to a single contiguous range. – the8472 Mar 19 '17 at 21:35
2

Since there does not seem to be any online defragmentation tool for directory indices and even the offline defragmenters don't seem to help I had to resort to rebuilding the directory tree recursively.

I've written a small tool (defrag-dirs) for that purpose. Alas, that approach requires the application using the directory tree to be taken down during defragmentation, which can take a considerable amount of time when dealing with millions of files.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.