0

I'm using Ubuntu Linux 16. I'm running out of disk space ...

[rails@server ~]$ df -h /
Filesystem      Size  Used Avail Use% Mounted on
/dev/simfs       30G   29G  1.5G  96% /

but what I don't understand is when I get the total size of the "/" directory, it is showing way less than 29G ...

[rails@server ~]$ sudo du -sh /
du: cannot access ‘/proc/883/task/883/fd/4’: No such file or directory
du: cannot access ‘/proc/883/task/883/fdinfo/4’: No such file or directory
du: cannot access ‘/proc/883/fd/4’: No such file or directory
du: cannot access ‘/proc/883/fdinfo/4’: No such file or directory
4.5G    /

What's going on here? More importantly, how do I figure out where all this mystery disk space is getting used?

Edit:

Output as requested by GracefulRestart ...

[rails@server ~]$ sudo lsof -nP 2>&1 | grep 'deleted'
postmaste  443      postgres  mem       REG              144,4           147159237 (deleted)/dev/zero (stat: No such file or directory)
postmaste  443      postgres  mem       REG              144,4                   0 (deleted)/VE20428-SYSV0052e2c1 (stat: No such file or directory)
postmaste  469      postgres  mem       REG              144,4           147159237 (deleted)/dev/zero (stat: No such file or directory)
postmaste  469      postgres  mem       REG              144,4                   0 (deleted)/VE20428-SYSV0052e2c1 (stat: No such file or directory)
postmaste  470      postgres  mem       REG              144,4           147159237 (deleted)/dev/zero (stat: No such file or directory)
postmaste  470      postgres  mem       REG              144,4                   0 (deleted)/VE20428-SYSV0052e2c1 (stat: No such file or directory)
postmaste  471      postgres  mem       REG              144,4           147159237 (deleted)/dev/zero (stat: No such file or directory)
postmaste  471      postgres  mem       REG              144,4                   0 (deleted)/VE20428-SYSV0052e2c1 (stat: No such file or directory)
postmaste  472      postgres  mem       REG              144,4           147159237 (deleted)/dev/zero (stat: No such file or directory)
postmaste  472      postgres  mem       REG              144,4                   0 (deleted)/VE20428-SYSV0052e2c1 (stat: No such file or directory)
httpd     1130          root  mem       REG              144,4           147485837 (deleted)/dev/zero (stat: No such file or directory)
httpd     1130          root  mem       REG              144,4               65537 (deleted)/VE20428-SYSV0176748a (stat: No such file or directory)
httpd     1131        apache  mem       REG              144,4           147485837 (deleted)/dev/zero (stat: No such file or directory)
httpd     1131        apache  mem       REG              144,4               65537 (deleted)/VE20428-SYSV0176748a (stat: No such file or directory)
httpd     1131 1133   apache  mem       REG              144,4           147485837 (deleted)/dev/zero (stat: No such file or directory)
httpd     1131 1133   apache  mem       REG              144,4               65537 (deleted)/VE20428-SYSV0176748a (stat: No such file or directory)
httpd     1131 1134   apache  mem       REG              144,4           147485837 (deleted)/dev/zero (stat: No such file or directory)
httpd     1131 1134   apache  mem       REG              144,4               65537 (deleted)/VE20428-SYSV0176748a (stat: No such file or directory)
httpd     1131 1135   apache  mem       REG              144,4           147485837 (deleted)/dev/zero (stat: No such file or directory)
httpd     1131 1135   apache  mem       REG              144,4               65537 (deleted)/VE20428-SYSV0176748a (stat: No such file or directory)
httpd     1131 1136   apache  mem       REG              144,4           147485837 (deleted)/dev/zero (stat: No such file or directory)
httpd     1131 1136   apache  mem       REG              144,4               65537 (deleted)/VE20428-SYSV0176748a (stat: No such file or directory)
httpd     1131 1137   apache  mem       REG              144,4           147485837 (deleted)/dev/zero (stat: No such file or directory)
httpd     1131 1137   apache  mem       REG              144,4               65537 (deleted)/VE20428-SYSV0176748a (stat: No such file or directory)
httpd     1131 1138   apache  mem       REG              144,4           147485837 (deleted)/dev/zero (stat: No such file or directory)
httpd     1131 1138   apache  mem       REG              144,4               65537 (deleted)/VE20428-SYSV0176748a (stat: No such file or directory)
httpd     1131 1139   apache  mem       REG              144,4           147485837 (deleted)/dev/zero (stat: No such file or directory)
httpd     1131 1139   apache  mem       REG              144,4               65537 (deleted)/VE20428-SYSV0176748a (stat: No such file or directory)
httpd     1131 1140   apache  mem       REG              144,4           147485837 (deleted)/dev/zero (stat: No such file or directory)
httpd     1131 1140   apache  mem       REG              144,4               65537 (deleted)/VE20428-SYSV0176748a (stat: No such file or directory)
httpd     1131 1141   apache  mem       REG              144,4           147485837 (deleted)/dev/zero (stat: No such file or directory)
httpd     1131 1141   apache  mem       REG              144,4               65537 (deleted)/VE20428-SYSV0176748a (stat: No such file or directory)
httpd     1131 1142   apache  mem       REG              144,4           147485837 (deleted)/dev/zero (stat: No such file or directory)
httpd     1131 1142   apache  mem       REG              144,4               65537 (deleted)/VE20428-SYSV0176748a (stat: No such file or directory)
httpd     1131 1143   apache  mem       REG              144,4           147485837 (deleted)/dev/zero (stat: No such file or directory)
httpd     1131 1143   apache  mem       REG              144,4               65537 (deleted)/VE20428-SYSV0176748a (stat: No such file or directory)
httpd     1131 1144   apache  mem       REG              144,4           147485837 (deleted)/dev/zero (stat: No such file or directory)
httpd     1131 1144   apache  mem       REG              144,4               65537 (deleted)/VE20428-SYSV0176748a (stat: No such file or directory)
httpd     1131 1145   apache  mem       REG              144,4           147485837 (deleted)/dev/zero (stat: No such file or directory)
httpd     1131 1145   apache  mem       REG              144,4               65537 (deleted)/VE20428-SYSV0176748a (stat: No such file or directory)
httpd     1131 1146   apache  mem       REG              144,4           147485837 (deleted)/dev/zero (stat: No such file or directory)
httpd     1131 1146   apache  mem       REG              144,4               65537 (deleted)/VE20428-SYSV0176748a (stat: No such file or directory)
httpd     1131 1147   apache  mem       REG              144,4           147485837 (deleted)/dev/zero (stat: No such file or directory)
httpd     1131 1147   apache  mem       REG              144,4               65537 (deleted)/VE20428-SYSV0176748a (stat: No such file or directory)
httpd     1131 1148   apache  mem       REG              144,4           147485837 (deleted)/dev/zero (stat: No such file or directory)
httpd     1131 1148   apache  mem       REG              144,4               65537 (deleted)/VE20428-SYSV0176748a (stat: No such file or directory)
httpd     1131 1149   apache  mem       REG              144,4           147485837 (deleted)/dev/zero (stat: No such file or directory)
httpd     1131 1149   apache  mem       REG              144,4               65537 (deleted)/VE20428-SYSV0176748a (stat: No such file or directory)
httpd     1132        apache  mem       REG              144,4           147485837 (deleted)/dev/zero (stat: No such file or directory)
httpd     1132        apache  mem       REG              144,4               65537 (deleted)/VE20428-SYSV0176748a (stat: No such file or directory)
httpd     1158        apache  mem       REG              144,4           147485837 (deleted)/dev/zero (stat: No such file or directory)
httpd     1158        apache  mem       REG              144,4               65537 (deleted)/VE20428-SYSV0176748a (stat: No such file or directory)
postmaste 1270      postgres  mem       REG              144,4           147159237 (deleted)/dev/zero (stat: No such file or directory)
postmaste 1270      postgres  mem       REG              144,4                   0 (deleted)/VE20428-SYSV0052e2c1 (stat: No such file or directory)

Edit 2: Here's the output from the second command ...

[rails@server ~]$ sudo du -h --max-depth=1 /; ls -plah / | grep -v /
214M    /opt
4.0K    /media
12K /boot
6.7M    /run
4.0K    /lost+found
du: cannot access ‘/proc/3478/task/3478/fd/4’: No such file or directory
du: cannot access ‘/proc/3478/task/3478/fdinfo/4’: No such file or directory
du: cannot access ‘/proc/3478/fd/4’: No such file or directory
du: cannot access ‘/proc/3478/fdinfo/4’: No such file or directory
0   /proc
4.0K    /dev
4.0K    /.cpt_hardlink_dir_a920e4ddc233afddc9fb53d26c392319
114M    /tmp
0   /sys
14M /etc
3.6M    /root
396M    /home
12K /.rbenv
4.0K    /srv
1.9G    /usr
1.9G    /var
8.0K    /mnt
4.5G    /
total 64K
-rw-r--r--  1 root root    0 May  5  2018 .autorelabel
-rw-r--r--  1 root root    0 Nov  8 17:14 reboot
-rw-r--r--  1 root root    0 Nov  8 17:15 .vzfifo
  • 1
    try with a bind mount unix.stackexchange.com/a/358331/30851 or check from live cd – frostschutz Nov 8 at 21:53
  • That's a useful utilty, but sadly it only confirmed what I already new -- the top usage directory within "/" is "/var" at 1.9 GB. So still uncertain what the issue is. – Dave Nov 8 at 22:09
  • It is possible for deleted files to hold disk space on a disk. See if you get any output for lsof -nP 2>&1 | grep 'deleted'. The column before the path should be a file size. – GracefulRestart Nov 8 at 22:13
  • @GracefulRestart, that produced a whole bunch of stuff, which I included as an edit to my question. However, I'm a little clueless as to what it means and what to do next. For instance, is "(deleted)/dev/zero" a file path that I can delete? – Dave Nov 8 at 22:34
  • According to that output, postmaster (PostgreSQL) and httpd (Apache) are the processes holding those deleted files open. Restarting those processes should release those holds. Unfortunately, it does not look like that would equal the ~25GB of missing space... What does the output of du -h --max-depth=1 /; ls -plah / | grep -v / show? Any large files at the root (/)? – GracefulRestart Nov 8 at 23:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.