76

Is there a way to tell the kernel to give back the free disk space now? Like a write to something in /proc/ ? Using Ubuntu 11.10 with ext4.

This is probably an old and very repeated theme. After hitting 0 space only noticed when my editor couldn't save source code files I have open, which to my horror now have 0 byte size in the folder listing, I went on a deleting spree.

I deleted 100's of MB of large files both from user and from root, and did some hardlinking too.

Just before I did apt-get clean there was over 900MB in /var/cache/apt/archives, now there is only 108KB:

# du
108 /var/cache/apt/archives

An hour later still no free space and cannot save my precious files opened in the editor, but notice the disparity below:

# sync; df
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/sda4             13915072  13304004         0 100% /

Any suggestions? I shut off some services/processes but not sure how to check who might be actively eating disk space.

More info

# dumpe2fs  /dev/sda4
Filesystem state:         clean
Errors behavior:          Continue
Filesystem OS type:       Linux
Inode count:              884736
Block count:              3534300
Reserved block count:     176715
Free blocks:              422679
Free inodes:              520239
First block:              0
Block size:               4096
Fragment size:            4096
Improve this question
3
  • 4
    The file system frees the space immediately. However, the root-reserved blocks feature of ext[234] and how the kernel keeps open files reserved may give the appearance of lost space.
    – hhaamu
    Commented Mar 14, 2012 at 13:34
  • If you have several filesystems (patitions), freeing up space in one won't do any good in the other one.
    – vonbrand
    Commented Feb 4, 2014 at 19:48
  • Why was I able to fill the partition before the "reserved" 5Go blocks reclaim themselves?
    – Psddp
    Commented Sep 30, 2014 at 15:29

7 Answers 7

Reset to default
124

Check with lsof to see if there are files held open. Space will not be freed until they are closed.

sudo /usr/sbin/lsof | grep deleted

will tell you which deleted files are still held open.

Improve this answer
4
  • Good. Showed me some mysqld locks in /tmp, but many apport-gt uses of extinct files in /var/lib/apt/lists/partial/ that apparently have been accumulating. So I might killall apport-gt but will investigate it first.
    – Marcos
    Commented Mar 14, 2012 at 11:29
  • 1
    Marking as closest answer, although never really "gave back" the space immediately after closing file handles/processes using them. Looking for other kernel/proc/fs-based approaches.
    – Marcos
    Commented Nov 8, 2012 at 22:00
  • 19
    You can also use lsof +L1 (select open files that have been unlinked).
    – Martin Fido
    Commented May 24, 2013 at 7:07
  • The information in this answer is correct, but the problem the OP was having was likely not caused by this but by root reserved space (which an other answer addresses).
    – marcelm
    Commented Sep 22, 2017 at 9:44
40

Use lsof to find the deleted, but open, file still consuming space:

lsof | grep deleted | grep etilqs_1IlrBRwsveCCxId
chrome     3446       user  128u      REG              253,2              16400       2364626 /var/tmp/etilqs_1IlrBRwsveCCxId (deleted)

Find the entry in /proc/<pid>/fd/ that cooresponds to the filehandle:

ls -l /proc/3446/fd/etilqs_1IlrBRwsveCCxId
lrwx------. 1 user unix 64 Feb 11 15:31 128 -> /var/tmp/etilqs_1IlrBRwsveCCxId (deleted)

Now, just cat /dev/null into the fd:

cat /dev/null > /proc/3446/fd/128

Note that the inode is still open, but now it's 0 length

chrome     3446       user  128u      REG              253,2         0    2364626 /var/tmp/etilqs_1IlrBRwsveCCxId (deleted)
Improve this answer
2
  • 6
    Superfluous use of cat to truncate. In Bourne shell, just > /proc/3446/fd/128 will do.
    – 200_success
    Commented Apr 5, 2014 at 5:37
  • 3
    Do NOT do this if your program is actually expected to re-read any portion of the file in the future which may or may not be available in the page cache.
    – Michael Galaxy
    Commented Aug 8, 2016 at 16:13
14

df will not show space reserved for root (even when run as root):

# df -h
Filesystem            Size  Used Avail Use% Mounted on
...
/dev/optvol           625G  607G     0 100% /opt
...

How to change "reserved block percentage"

  1. Reduce reserved space to 4%

    # tune2fs -m4 /dev/sda4

df -h now showed 45M free.

  1. Saved my files quickly

  2. Put it back to 5%

    # tune2fs -m5 /dev/sda4

Improve this answer
4
  • 2
    The for root reserved space is nowadays nearly always too big. You can reduce it to some few percent. df displays the for normal user usable space. As apt runs as root, the reserved space is only useful to protect against fill-ups caused by non-root users (=normal users and services that have their own user).
    – jofel
    Commented Mar 14, 2012 at 11:22
  • I agree; an mkfs these days should reserve eg. 5% or 300MB, whichever is less. Just re-tuned some of my servers to 2% and freed GBs back!
    – Marcos
    Commented Mar 14, 2012 at 15:38
  • 4
    @jofel, no, it isn't. Any time you go over 90% utilization, you start getting lots of fragmentation. You need to free up some more space, not run closer to 100% utilization.
    – psusi
    Commented Mar 14, 2012 at 15:44
  • @psusi You are true, thanks for your comment. But the opportunity to use (temporary) nearly all available space as normal user could be really practical and with ext4, the things are not that bad anymore, see unix.stackexchange.com/a/7965/15241
    – jofel
    Commented Mar 14, 2012 at 17:20
9

In Ubuntu, if you deleted files using your trash bin, your files were more than likely not be completely removed.

Even after emptying your trash your files will remain in ~/.local/share/Trash/expunged until after a reboot and maybe even longer.

I haven't found a good reason for this, but if I run out of space, I always manually rm the expunged trash files.

Improve this answer
2
  • 1
    Good point. Even though I'm one of those who live and die by the command line, and rarely use graphical file manager. Haven't yet noticed that expunged folder as a hiding place though--an Empty Trash click had always been final and returned my disk space when needed.
    – Marcos
    Commented Mar 14, 2012 at 17:53
  • This happened for me when some of the files inside the Trash were owned by root instead of the normal user. As I deleted them from Trash, they ended up inside the expunged folder and I had to manually sudo rm them.
    – shivams
    Commented Sep 19, 2020 at 8:07
7 
sudo lsof | grep "(deleted)$" | sed -re 's/^\S+\s+(\S+)\s+\S+\s+([0-9]+).*/\1\/fd\/\2/' | while read file; do sudo bash -c ": > /proc/$file"; done

Explanation:
Grep lsof output to extract only deleted files. Sed extract the process id and filedescriptor id from each line, and create a string in format {pid}/fd/{fid}. While loop and output nothing to each file, setting them to empty.

Improve this answer
2
  • 4
    I got an Error " syntax error near unexpected token `(' "
    – Majid Golshadi
    Commented Nov 8, 2017 at 7:11
  • I got the syntax error, but the script worked!
    – hpaknia
    Commented Sep 23, 2021 at 14:31
5

I wonder if sync is of any help here — but it shouldn't be, as IIRC in most ("many"?) systems, filesystems are synced each 30 s.

I'd check the kernel log (so dmesg) to find if anything nasty is going on and run lsof to see if any big, deleted file is still open (actually, I think deleted files will be marked as so in the lsof output).

Two reasons (one of these pointed in the question you link) that may cause deleted files to release no space are

  • files that weren't actually deleted: you deleted a file which is hardlinked somewhere else (more precisely, you unlink()ed a file with more than one link)
  • files that are still open: open files are bookkept using, well, files, inodes themselves, not directory entries, if you delete the entry, the inode will remain there as long as it is still open.

But I don't know of a specific reason why that might happen with so many files...

Improve this answer
1
  • sync never helped. As for logs, it's an Ubuntu system so it's pretty buggy, so yeah they're typically noisy. apport has be deploying often because every nightly apt-get update crashes, though /var/crash has only 77MB. Also noticed atd has flooded /var/log/syslog with repeating lines like atd[8892]: File a0015c0152ab76 is in wrong format - aborting probably since the few files in /var/spool/cron/atspool were all 0 sized, making the problem circular of course
    – Marcos
    Commented Mar 14, 2012 at 11:41
1

CentOS 6.3 also does the not-actually-emptying-trash-can-when-you-empty-trash-can thing. I couldn't find a way to reclaim the space until I just ran rm -rf ~/.local/share/Trash/expunged/. Caused much head-scratching.

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .