8

If I want to see all relevant log files of my apache2 server at once, I use

tail -f /var/kunden/logs/*log /var/kunden/logs/*log /var/log/apache2/*log |grep -v robots|grep -v favicon

But since those are too many files by now, I would like to encrease that limit.

How can I increase it for one ssh session? And How could I increase it globally systemwide?

I can see the open files limit is 1024 on my machine:

ulimit -n
1024
  • Exceeding the file limit is often a sign of a problem. As already noted, you may have a problem with log rotation. (Rotated log may drop off your command.) Outside /var/log/apache2 there shouldn't be many (any) logs related to your Apache server. Based on your grep pattern, you likely want to restrict your access to access logs. – BillThor Aug 4 '13 at 13:52
  • thanks for the hint, but my logrotaion works fine. I have so many logs, cause all 1025 domains on my server have their own log file ;) – rubo77 Aug 4 '13 at 13:58
  • 1
    Your patterns look like they will also pick up error logs as well as access logs. This should be a minor issue if you have a single error log. If you leave this running in a shell, you may want to use -F instead of -f so logs get reopened when rotated. – BillThor Aug 4 '13 at 15:16
15

It is Important to know that there are two kinds of limits:

  • A hard limit is configurable by root only. This is the highest possible value (limit) for the soft limit.
  • A soft limit can be set by an ordinary user. This is the actual limit in effect.

Solution for a single session

In the shell set the soft limit:

ulimit -Sn 2048

This example will raise the actual limit to 2048 but the command will succeed only if the hard limit (check: ulimit -Hn) is the same or higher. If you need higher values, raise the hard limit using one of the methods below. The limits are set per process and they are inherited by newly spawned processes, so anything you run after this command in the same shell will have the new limits.

Changing hard limit in a single session

This is not easy because only root can change a hard limit and after switching to root you have to switch back to the original user. Here is the solution with sudo:

sudo sh -c "ulimit -Hn 9000 ; exec su \"$USER\""

System-wide solution

In Debian and many other systems using pam_limits you can set the system-wide limits in /etc/security/limits.conf and in files in /etc/security/limits.d. The conf file contains description. Example lines:

@webadmins       hard     nofile     16384
@webadmins       soft     nofile      8192

This will set the hard limit and default soft limit for users in group webadmins after login.

Other limits

The hard limit value is limited by global limit of open file descriptors value in /proc/sys/fs/file-max which is pretty high by default in modern Linux distributions. This value is limited by NR_OPEN value used during kernel compilation.

Is there not a better solution?

Maybe you could check if all the *log files you feed to tail -f are really active files which need to be monitored. It is possible that some of them are already closed for logging and you can just open a smaller number of files.

  • ulimit -Sn 4096 -bash: ulimit: open files: Kann die Grenze nicht ändern: Das Argument ist ungültig -- means the argument is not valid – rubo77 Aug 4 '13 at 4:44
  • @rubo77: I added information about the hard limit which is most probably the cause of the problem. – pabouk Aug 4 '13 at 4:47
  • @rubo77: It is also possible to set the hard limit in a shell session but is it tricky as you have to su to root and su back to the user: su -c "ulimit -Hn 6000 ; su $USER" – pabouk Aug 4 '13 at 4:55
  • OK, so for a temporary change of the limit I will use ulimit -Hn 6000; ulimit -Sn 6000 as root then – rubo77 Aug 4 '13 at 13:02
  • 1
    @amenthes I guess you mean the configuration files in /etc/limits.*. These limits are applied whenever the PAM module pam_limits.so is called. Normally it is at start of a session. For example in Ubuntu 14.04 it is in these tools / commands: cron, login (text console login), lightdm (the GUI login), su. You can set where is pam_limits used in /etc/pam.*. See for example: faqs.org/docs/securing/chap5sec44.html – pabouk Sep 23 '14 at 8:46
0

I had the PHP Warning in my Apache error.log:

failed to open stream: Too many open files in ...

So I found out, apache sets this value individually while starting (on my Ubuntu 14.04). It is configured in /etc/apache2/envvars. It says:

## If you need a higher file descriptor limit, uncomment and adjust the
## following line (default is 8192):
#APACHE_ULIMIT_MAX_FILES='ulimit -n 65536'

so I had to adjust the third line.

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.