What is the difference between hard and soft limits in ulimit?
For number of open files, I have a soft limit of 1024 and a hard limit of 10240. It is possible to run programs opening more than 1024 files. What is the soft limit for?
A hard limit can only be raised by root (any process can lower it). So it is useful for security: a non-root process cannot overstep a hard limit. But it's inconvenient in that a non-root process can't have a lower limit than its children.
A soft limit can be changed by the process at any time. So it's convenient as long as processes cooperate, but no good for security.
A typical use case for soft limits is to disable core dumps (
ulimit -Sc 0) while keeping the option of enabling them for a specific process you're debugging (
(ulimit -Sc unlimited; myprocess)).
ulimit shell command is a wrapper around the
setrlimit system call, so that's where you'll find the definitive documentation.
Note that some systems may not implement all limits. Specifically, some systems don't support per-process limits on file descriptors (Linux does); if yours doesn't, the shell command may be a no-op.
The hard limit is for the security purpose. For a non-root user, he can only decrease the hard limit from the currently set hard limit; he cannot increase it. Increasing the hard limit can be done only by root user (or maybe with sudo privilege, not sure about that). What a non-root user can do is choose a limit (called soft limit) which can be in the range [0, hard limit] for its processes. Its the soft limit which is seen and taken in consideration by the processes.