-2

EC2 instance running centos 7.0 hard disk space is consumed slowly, which is flushed after reboot.

Content : Base OS, Cloudwatch Agent, CodeDeploy agent, nginx, Anti Virus MacFee, Java application (log file is under control).

Screen shot 1 before reboot : Disk space utilized 75%

As per this screenshot we are finding top 10 most space consuming files and /proc is on top.

enter image description here

Screen shot 2 After reboot : Disk space utilized 9%

used files are flushed, not able to identify if the issue is caused by application or other components.

enter image description here

closed as unclear what you're asking by Rui F Ribeiro, Anthon, Anthony Geoghegan, Jeff Schaller, GAD3R Jul 28 '17 at 12:00

Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • 4
    /proc doesn't live on disk, it's merely a representation of processes currently running. Unsure what you're asking. – steve Jul 28 '17 at 8:07
  • 5
    Welcome to Unix & Linux! Please don't post images of text. Instead, paste the text directly into your question and use the formatting tools. – terdon Jul 28 '17 at 8:11
  • "we are finding top 10 most space consuming files and /proc is on top" No it is not, and this could hardly be the case, since /proc is not even a real filesystem. It seems you barely looked at the output of the command you ran -- do you read where it says du: cannot access ./proc ... ? – dr01 Jul 28 '17 at 9:22
  • There's not really any question in here, and as several folks have pointed out, you badly misinterpreted the output. But if you want to know what uses your disk space and what to do about it, I suggest you use something like du -x --max-depth=1 / before and after a boot, and then before the next boot use a similar command in the directories that shrink the most - if it isn't painfully obvious what uses the space. – Henrik Jul 28 '17 at 11:30
  • do you really care what the largest file is? or are you trying to find what's using up all the space? if the latter, then start by running du -sch against all real-filesystem directories in / (e.g. with something like find / -mindepth 1 -maxdepth 1 -type d \( -fstype ext4 -o -fstype xfs \) -exec du -sch {} + | sort -h. That will give you the top level directories sorted by size. then do the same for each of the largest directories (e.g. /var, /opt) to find which of their subdirectories use the most space. and keep drilling down until you find the culprit. – cas Jul 29 '17 at 1:55
8

You’re misreading the output: the lines about /proc are error messages because du can’t read some of the files find is asking it to use; they don’t indicate that files in /proc are among the top-ten biggest files on your system. In any case, as steve mentioned, /proc is its own, virtual, file system, and doesn’t occupy any space on your drive.

To avoid listing irrelevant files, you can tell find not to cross file system boundaries using the -xdev option.

The files which appear to be giving you trouble are /var/log/awslogs.log and /var/log/awslogs.1: your first command lists them as the two biggest files on your system, and they don’t appear in the output of your second command (or rather, they’re smaller — I see awslogs.log.1 further down the list).

0

This is what happens when you pipeline stdout and don't suppress stderr.

Because sort needs to have the entire input buffer before it can do its job (obviously) it doesn't actually start passing output on to head before find is finished searching.

However, since you left stderr unsuppressed, du reports errors as to files that stopped existing between the time find saw them and du was started, and these error messages get reported the moment they're raised (because stderr isn't being buffered).

If you run the find command without the sort and head pipeline you'll see what's happening.

Not the answer you're looking for? Browse other questions tagged or ask your own question.