14

I want to check which directories take most disk space, quickly.

I tried du -sh subdir but it took more than 20 seconds on bigger directories.

I'm not sure how to display the size of all the subdirectories in the home directory at once with this method, but I'm afraid that it might take minutes...

Is there a fast way to do this?

I don't need to display the size of files, just directories.

1
  • 1
    To find out interactively what takes the most space, you could use ncdu
    – jfs
    Sep 3, 2016 at 8:38

9 Answers 9

18

Sample directory

$ ls -aF
./  ../  .asd/  folder1/  folder2/  list  t1  t2  xyz/

To find sizes only for folders, excluding hidden folders:

$ find -type d -name '[!.]*' -exec du -sh {} + 
4.0K    ./folder1
4.0K    ./folder2
8.0K    ./xyz

If you need a total at the end as well:

$ find -type d -name '[!.]*' -exec du -ch {} + 
4.0K    ./folder1
4.0K    ./folder2
8.0K    ./xyz
16K total

To sort the results:

$ find -type d -name '[!.]*' -exec du -sh {} + | sort -h
4.0K    ./folder1
4.0K    ./folder2
8.0K    ./xyz

To reverse the sorting order:

$ find -type d -name '[!.]*' -exec du -sh {} + | sort -hr
8.0K    ./xyz
4.0K    ./folder2
4.0K    ./folder1

If you need with hidden directories as well, remove -name '[!.]*' from find command. I don't know any other command to find size of folders that is faster than du. Use df for file system disk space usage

Use find -maxdepth 1 -type d -name '[!.]*' -exec du -sh {} + to avoid sub-folders showing up

7
  • 1
    this is a good answer and can be easily adapted to show only folders from a certain user by doing -user username Jan 25, 2018 at 4:43
  • Why would be $ find -type d -name '[!.]*' -exec du -sh {} + fast than just du -sh subdir? - isn't it the same command in the end?
    – Framester
    Jun 7, 2018 at 13:17
  • 1
    @Framester sorry if that's your take away from the answer.. I've only illustrated use of find+du as OP asked display the size of all the subdirectories ... I mention in the answer that I do not know if there's a command that's faster than du.. use of find is to restrict what kind of files/directories you want to give to du
    – Sundeep
    Jun 7, 2018 at 13:26
  • 1
    +1 for sort -h -- that's a great flag to know
    – SpinUp
    Nov 8, 2018 at 18:47
  • This still would not make du any faster. A directory that takes 20 seconds to run du on would still take 20 seconds to process.
    – Kusalananda
    Feb 14, 2021 at 8:28
6

Use package ncdu. you can install it by sudo apt-get install ncdu.
on server you can use it with options ncdu -q -x (Quiet Mode and Omit mounted directories).

Ncdu vs du/df

The interface of ncdu is built using ncurses and is interactive. Ncdu is different from df or du. Ncdu just does the one task of reporting the space used by a directory and drill down. On the other hand the df command reports space used by different storage devices.

So ncdu is a powerful tool to monitor, check and analyse disk space usage on your linux system

1
  • 3
    This looks like it may be useful, but is it faster than du?
    – SpinUp
    Nov 8, 2018 at 19:00
4

All folders occupy the same amount of space, namely 4096 bytes. You don't want to know the size of the folders, but the size of what's in them. And this demands counting, which in turn demands time.

du counts directory sizes by default. So to get the sizes for it either:

cd && du

Or

du ~

See man du for more options.

1
  • 4
    I don't understand you answer. He said he used du to check directory size, but it was too slow. He asked an advice, a faster method. And your answer: use du ! Are you sure ? Is it really faster than du ?
    – Tele
    Dec 28, 2020 at 7:50
2

Use the following command, it will display quickly the top 10 directories according to the size occupied in the system:

du -hsx /* | sort -rh | head -10

Output e.g.

[root@x ~]# du -hxs /* | sort -rh | head -10

10G     /mnt
5.4G    /usr
1.5G    /var
418M    /lib
274M    /opt
224M    /root
55M     /boot
36M     /home
30M     /lib64
16M     /sbin
2
  • I hope you understood and observed the difference between the following commands: du -hxs /* | sort -rh | head -10 and du -sh subdir Note: The output is self explanatory Feb 15, 2021 at 9:49
  • Ok, so there's two things: Your code does not ignore files. It also does not speed up the processing of any single directory. What it does do is to run du over all directories at once rather than doing the du manually on each. As a consequence of this, if there are hard links, these would only be counted once by your method, but would be counted twice by the method used by the user in the question. I would have expected you to say something about this and how the question possibly was not well specified.
    – Kusalananda
    Feb 15, 2021 at 10:06
1

As others have said, it is just a slow process. We have multi user machines and the worst is when a disk runs low and several people start du at the same time. So, we use a cached view of the disk usage (we run it twice a day). There are lots out there, but my faviourite from a visual perspective is duc

http://duc.zevv.nl/

Indexes are generated from cron jobs like below - this one specifies 6 levels of hierarchy in the index

30 5,12 * * * /usr/local/bin/duc index -x -m 6 -d /mydisk/ducdb_1.4.sqlite /mydisk/users >/dev/null 2>&1

Then we use the Web front end (duc cgi) to view the index. It is very slick and aesthetically great. Many thanks to the authors Ico Doornekamp & John Stoffel

0

You can list the top 10 folders by size using this command below.

find $(pwd) -type d -print0 | xargs -0 du | sort -n | tail -10 | cut -f2 | xargs -I{} du -sh {}

check size of folders via terminal Linux

1
  • I might be looking at this wrong but if I get this right I must say I have my doubts that invoking du twice is faster than invoking du once...
    – confetti
    Dec 26, 2020 at 4:20
0

If running du on a particular directory takes 20 seconds, then that is what it will take to run du on that directory. You can't really do it quicker.

What you can do quicker is to eliminate the manual labor, i.e. remove the need to run du manually on each individual directory and to compare the sizes.

You can do this in a few different ways, depending on how you want to count the sizes. If your subdirectories of your home directory contains hard links (files that have multiple names in the file hierarchy), and you want to count these once per name, you can do this with the -l ("ell", not "one") option to GNU du. With standard du, multiple hard links would be counted separately only if encountered in separate du runs.

You also have to decide whether you want to count the apparent ("logical") size, i.e. the number of bytes that a program reading the file from start to finish would read, or the disk usage ("physical") size, i.e. the number of bytes actually used for storing the file on disk. These two sizes may be different from each other if a file is sparse, i.e. if it contains holes. Unallocated disk images or swap files are typically sparse. GNU du can give you the "logical" sizes using --apparent-size or --bytes, but standard du can't.

Assuming that you are only interested in actual disk usage of each directory, individually, i.e. that you want to count hard links multiple times only if they are located in different subdirectories under $HOME, and that you want to count the actual, "physical", size of any sparse files, then you may loop over the directories in your home directory and run du on each one like so:

for dirpath in "$HOME"/*/; do
    du -k -s -- "$dirpath"
done

The pattern "$HOME"/*/ will expand to a list of pathnames of (visible) directories beneath your home directory, and/or to visible symbolic links to these.

This will give you the sizes in kilobytes. Many du implementations can show you the "human readable", approximate, sizes if you use -h in place of -k (see man du).

You may sort the output of this loop numerically, with the largest directory appearing first, like so:

for dirpath in "$HOME"/*/; do
    du -k -s -- "$dirpath"
done | sort -n -r

If you used -h in place of -k with du, you will need to have a sort that understands "human readable" units. Some sort implementations do this with their -h option (see man sort).

0

You may try this: This solution assumes a lots:

  • that your ultimate goal is to quickly determine the biggest directory (NOT counting its subdirs) to for exemple find quickly where you should do some cleaning up to free space (an often needed thing)
  • that the filenames do not contain spaces/tabs/newlines/etc
  • that the size of a file is always in the 7th field when outputted by find ... -ls :

And it is restricted to using basic versions of : find, awk (not the GNU versions, which would help by proposing better ways to display the filename and its size)

find ./ -type f -ls | awk ' 
    { pathname=$0; sub("[^/]*$","",pathname); size[pathname]+=$7 } 
END { for(dirname in size){print size[dirname],dirname }
    }' | sort -n

This will output directly the biggest directories at the end, and each size displayed is only "this directories content, not counting subdirs", which will help you to aim directly at the biggest occupancies. (du also adding subdirs can say "A/" is the biggest but it is in fact "A/B/C" (or even anotherdir/E/F/) that is the biggest without counting any subdirs...

Of course in the real world: your files may contain spaces, etc, and this will require lots of tweaking. One possibility is then to instead use GNU find, as with it you can only output: size_of_file name of file here, and thus add field $1, and the rest of the line is used to compute the dirpath. But there will always be special cases not covered [until someone else proposes a better version]

-1

A simple command is ls | xargs du -sh

1
  • The user said "I don't need to display the size of files, just directories", but your solution does not care about this. Also, the main issue is the speed at which du operates, and you provide no reasoning why your code would be faster than what the user is already doing. Also note that this may fail if a filename in the current directory contains whitespaces, something that is common on some Unix systems.
    – Kusalananda
    Feb 14, 2021 at 8:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.