Tag Info

Hot answers tagged

57

Adding up numbers is easy. The problem is, there are many different numbers to add. How much disk space does a file use? The basic idea is that a file containing n bytes uses n bytes of disk space, plus a bit for some control information: the file's metadata (permissions, timestamps, etc.), and a bit of overhead for the information that the system needs to ...


39

You can use du -sh * | sort -h. This tells sort that the input is the human-readable format. This feature was added recently to GNU Core Utilities 7.5 in Aug 2009, so many distributions do not yet have it.


35

The block size of the file system must be 4 kB. When data is written to a file that is contained in a file system the operating system must allocate blocks of storage to contain the data that will be written to the file. Typically, when a file system is created the storage contained in that file system is segmented into blocks of a fixed size. This ...


33

Saving space for important root processes (and possible rescue actions) is one reason. But there's another. Ext3 is pretty good at avoiding filesystem fragmentation, but once you get above about 95% full, that behavior falls off the cliff, and suddenly filesystem performance becomes a mess. So leaving 5% reserved gives you a buffer against this. Ext4 ...


29

du == Disk Usage. It walks through directory tree and counts the sum size of all files therein. It may not output exact information due to the possibility of unreadable files, hardlinks in directory tree, etc. It will show information about the specific directory requested. Think, "How much disk space is being used by these files?" df == Disk Free. Looks at ...


28

Take a look at ionice. From man ionice: This program sets or gets the io scheduling class and priority for a program. If no arguments or just -p is given, ionice will query the current io scheduling class and priority for that process. To run du with the "idle" I/O class, which is the lowest priority available, you can do something like this: ionice ...


24

Symbolic links do take room, of course, but just the room it takes to store the name and target plus a few bytes for other metadata. The space taken by a symbolic link does not depend on the space taken by the target (after all, the target is not even required to exist). Plain du reports the space taken by a directory tree on the disk. du -L reports the ...


20

check with lsof if there are files held open, space will not be freed until they are closed sudo /usr/sbin/lsof | grep deleted will tell you which deleted files are still held open


19

If you want a command-line tool, I prefer ncdu, an ncurses version of du. It scans the disk (or a given folder) and then shows the top-level space usages; you can select a given directory to get the corresponding summary for that directory, and go back without needing to reanalyze: If you're ok with a GUI program, Filelight is the closest thing to ...


19

If you allow others to log on to your system, via ssh, for example, having these 5% blocks reserved ensures external users cannot fill the disk. Even if you don't allow others to log in to your system, the reserved blocks prevents programs not running as root from filling your disk.


18

Based on your issues in installing ncdu my recommendation would be to use du and sort on together. For instance: du /home | sort -rn (will search all files/directories under /home and sort them by largest to smallest. du -h /home | sort -rh (same but will show it in MB/KB/etc) - Note this requires coreutils 7.5 or newer (sort --version to check) You can ...


18

It will happen if you have sparse files: $ mkdir test; cd test $ truncate -s 1000000000 file-with-zeroes $ ls -l total 0 -rw-r--r-- 1 gim gim 1000000000 03-08 22:18 file-with-zeroes A sparse file is a file which has not been populated with filesystem blocks (or only partially). When you read a non-populated zone of a sparse file you will obtain zeros. ...


17

The --inodes option to df will tell you how many inodes are reserved for use. For example: $ df --inodes / /home Filesystem Inodes IUsed IFree IUse% Mounted on /dev/sda1 3981312 641704 3339608 17% / /dev/sda8 30588928 332207 30256721 2% /home $ sudo find / -xdev -print | wc -l 642070 $ sudo find /home -print | wc ...


16

If you can't kill your application, you can truncate instead of deleting the log file to reclaim the space. If the file was not open in append mode (with O_APPEND), then the file will appear as big as before the next time the application writes to it (though with the leading part sparse and looking as if it contained NUL bytes), but the space will have been ...


16

tar -c data_dir | wc -c without compression or tar -cz data_dir | wc -c with gzip compression or tar -cj data_dir | wc -c with bzip2 compression will print the size of the archive that would be created in bytes, without writing to disk. You can then compare that to the amount of free space on your target device. You can check the size of the data ...


14

find -maxdepth 1 -type d | while read -r dir; do printf "%s:\t" "$dir"; find "$dir" -type f | wc -l; done Thanks to Gilles and xenoterracide for safety/compatability fixes. The first part: find -maxdepth 1 -type d will return a list of all directories in the current working directory. This is piped to... The second part: while read -r dir; do begins a ...


13

Try using the -k flag to count 1K blocks intead of using human-readable. Then, you have a common unit and can easily do a numeric sort. du -ck | sort -n You don't explictly require human units, but if you did, then there are a bunch of ways to do it. Many seem to use the 1K block technique above, and then make a second call to du. ...


12

Your compressed tar file is smaller than its contents. ls prints file sizes in bytes by default. du -k prints file sizes in kilobytes. To make ls print file sizes in kilobytes, use the -k flag.


11

There are programs like Bootchart that can be used to show what programs you ran during startup - you can probably keep it going after boot to see what's been invoked during a session. A better solution may be to use remastering tools. There are remastering tools for Fedora, Ubuntu, and others; you can use these to customize a distribution. You might want ...


11

Do the “Used” and “Available” columns of df output add up to the figure in the total column? (The output of df reflects the data from the underlying statvfs system call, so you'll find the same numbers in any other application.) If not (which is probably the case), that's because by default, on an ext2/ext3/ext4 filesystem, 5% of the space is reserved to ...


11

I usually use the -exec utility. Like this: find . -type f -exec du -a {} + I tried it both on bash and ksh with GNU find. I never tried AIX, but I'm sure your version of find has some -exec syntax. The following snippet sorts the list, largest first: find . -type f -exec du -a {} + | sort -n -r | less


11

use lsof to find the deleted, but open, file still consuming space lsof | grep deleted | grep etilqs_1IlrBRwsveCCxId chrome 3446 user 128u REG 253,2 16400 2364626 /var/tmp/etilqs_1IlrBRwsveCCxId (deleted) find the entry in /proc//fd/ that cooresponds to the filehandle ls -l ...


11

You fail to mention your operating system, but on linux, this works: $ df /path/to/some/file/or/directory Filesystem 1k-blks Used Avail Cap Mounted /dev/harddisk_partition 8388348 5187768 3200580 62% /home/username ^^^^^^^^^^^^^^^^^^


11

All file systems have a cluster or block size, or the smallest amount of disk space that can be allocated to hold a file. Even if the actual file size is smaller than the cluster/block size, it will still consume one cluster, or 4K on your file system. The cluster size depends on the file system, and the file system options. If it contains zero bytes, as ...


10

Let's see. The device size is 1,465,138,583½ kB = 1,500,301,909,504 B. The filesystem consists of 366,284,288 blocks of 4096 B each, which is 1,500,300,443,648 B. I don't know what the remaining 1,465,856 B (1.4 MB) are used for (additional copies of the superblock? I know there are a few kB of space at the beginning for the bootloader.). The filesystem ...


10

Ok, that's a weird one… not enough space to remove a file! This turns out to be a relatively common issue with ZFS, though it could potentially arise on any filesystem that has snapshots. The explanation is that the file you're trying to delete still exists on a snapshot. So when you delete it, the contents keep existing (in the snapshot only); and the ...


10

Only for tree 1.6 and above You might want to look at: man tree --du For each directory report its size as the accumulation of sizes of all its files and sub-directories (and their files, and so on). The total amount of used space is also given in the final report (like the 'du -c' command.) This option requires ...


10

You really should use something like md5sum or sha1sum to check integrity. If you really want to use the size use ls -l or du -b. The du utility normally only shows the disk usage of the file, i.e. how much of the file system is used by it. This value totally depends on the backing file system and other factors like sparse files. Example: $ truncate -s ...


9

What you are seeing when you rerun a du command is the effect of disk buffering. Once you read a block its disk buffer is kept in the buffer cache until that block is needed. For du you need to read the directory and the inode for each file in the directory. The du results are not cached in this case, but can be derived with far less disk IO. While it ...


9

I realize this is going to sound both simplistic and absurd, but if you have control over the apps in question (maybe in a test environment) you could mount ONLY that directory on a partition of its own, then iostat, etc. would tell you only about it, and nothing else on that spot. If there are physical drives involved you could fake it up with a loopback ...



Only top voted, non community-wiki answers of a minimum length are eligible