1

I have an ext4 file system on my work computer and I came across a directory that is taking up >1 mb of space. There are 7 entries in the directory. The directory hasn't ever contained more than 20 entries.

What is going on here? The directory hasn't given me any problems... yet. Should I be worried?

As far as I know I haven't set up any ACL's or extents on the volume. When I created it I used the default block size of 4096.

=[3 dc@work ~/documents/work/foo ]= $ ls -lat -r
[...]
drwxr-xr-x  2 dchurch users    4096 Dec 23 11:53 _old/
drwxr-xr-x  5 dchurch users    4096 Dec 30 17:43 ./
drwxr-xr-x  3 dchurch users 1032192 Jan  2 08:51 account_import/
drwxr-xr-x 14 dchurch users    4096 Jan  2 08:54 ../
=[3 dc@work ~/documents/work/foo ]= $ ls -la account_import/
total 5500
drwxr-xr-x 3 dchurch users 1032192 Jan  2 08:51 ./
drwxr-xr-x 5 dchurch users    4096 Dec 30 17:43 ../
drwxr-xr-x 2 dchurch users    4096 Jan  2 08:54 _first/
-rw-r--r-- 1 dchurch users  804852 Dec 17 15:37 ca.csv
-rw-r--r-- 1 dchurch users    3495 Dec 17 16:14 slurp.php
-rw-r--r-- 1 dchurch users    3707 Dec 17 16:13 slurp.php~
-rw-r--r-- 1 dchurch users 3769577 Dec 17 15:37 us.csv
=[3 dc@work ~/documents/work/foo ]= $ stat account_import
  File: 'account_import'
  Size: 1032192         Blocks: 2024       IO Block: 4096   directory
Device: 804h/2052d      Inode: 4458105     Links: 3
Access: (0755/drwxr-xr-x)  Uid: ( 1000/ dc)   Gid: (  100/   users)
Access: 2015-01-05 04:40:11.629350078 -0600
Modify: 2015-01-02 08:51:06.024793685 -0600
Change: 2015-01-02 08:51:06.024793685 -0600
 Birth: -
  • 2
    I think the size pretty much proves that at one point it did contain a bunch of files. – derobert Jan 5 '15 at 16:34
  • @derobert The directory would have to have contained ~40,000 files to make it grow to that size. I think it's more likely that something is amiss. – amphetamachine Jan 5 '15 at 16:43
  • I did some digging and apparently dirty directories are a known issue on ext* file systems. More reason to use xfs. – amphetamachine Jan 5 '15 at 17:02
  • 1
    mkdir foobar; mv account_import/* foobar; rmdir account_import; mv foobar account_import – goldilocks Jan 5 '15 at 17:49
1

4096B is only the initial size of the directory, equal to one sector. If however the amount of data to be stored (metadata, which includes file names) grows, so will the size of the directory, without it ever going down again to reduce problems with disk fragmentation:

 $ mkdir testdir
 $ cd testdir
 $ ls -lhd .
   drwxrwxr-x 2 me me 4.0K Jan  5 18:06 .
 $ for ((i=0; i<1000; i++)); do touch this_is_a_long_file_name_$i; done 
 $ ls -lhd .
   drwxrwxr-x 2 me me 64K Jan  5 18:08 .
 $ rm this*
 $ ls -lhd .
   drwxrwxr-x 2 me me 64K Jan  5 18:09 .

EDIT: this is the same experiment for reiserfs:

 $ mkdir testdir
 $ cd testdir
 $ ls -lhd .
   drwxr-xr-x 2 me me 48 gen  5 18:54 .
 $  for ((i=0; i<1000; i++)); do touch this_is_a_really_long_file_name_$i; done
 $ !ls
   ls -lhd .
   drwxr-xr-x 2 me me 55K gen  5 18:55 .
 $ rm this*
 $ !ls
   ls -lhd .
   drwxr-xr-x 2 me me 48 gen  5 18:55 .

And this is for xfs:

 $ mkdir testdir
 $ cd testdir
 $ ls -lhd .
   drwxr-xr-x 2 me me 6 gen  5 19:00 .
 $ for ((i=0; i<1000; i++)); do touch this_is_a_really_long_file_name_$i; done
 $ ls -lhd .
   drwxr-xr-x 2 me me 48K gen  5 19:01 .
 $ rm this*
 $ !ls
   ls -lhd .
   drwxr-xr-x 2 me me 6 gen  5 19:01 .

It is obvious that neither reiserfs nor xfs* care as much about disk fragmentation, since the directory size, upon deletion of the numerous files, returns to its pristine value.

  • I'm wondering how XFS and ReiserFS handle this. They evidently truncate their directories when entries are deleted. – amphetamachine Jan 5 '15 at 17:19
  • @amphetamachine See my edit. – MariusMatutiae Jan 5 '15 at 18:05
  • It is obvious that neither reiserfs nor xfs care as much about disk fragmentation - That's a pretty arrogant thing to say; there's no reason to think those filesystems don't avoid fragmentation in a way that allows directory truncation. – amphetamachine Jan 7 '15 at 22:01
  • @amphetamachine I agree with you, I did not mean to be arrogant, only to stress that this does not appear to be a concern for either fs. – MariusMatutiae Jan 8 '15 at 7:26
  • Also, how terrible a practice would it be if you created a 10GB file, then truncated it to 2GB, only to have du report the size as 10GB of unreclaimable space? ext4 does not do this so by your logic it doesn't care about disk fragmentation either, or at least only selectively so. – amphetamachine Jan 8 '15 at 22:55

Not the answer you're looking for? Browse other questions tagged or ask your own question.