Hot answers tagged

117

No but you should close all active sessions windows. They still remember the old values. In other words, log out and back in. Every remote new session or a local secure shell take effect of the limits changes.


90

The reason is that the operating system needs memory to manage each open file, and memory is a limited resource - especially on embedded systems. As root user you can change the maximum of the open files count per process (via ulimit -n) and per system (e.g. echo 800000 > /proc/sys/fs/file-max).


67

ulimit is made for this. You can setup defaults for ulimit on a per user or a per group basis in /etc/security/limits.conf ulimit -v KBYTES sets max virtual memory size. I don't think you can give a max amount of swap. It's just a limit on the amount of virtual memory the user can use. So you limits.conf would have the line (to a maximum of 4G of ...


66

Please note that lsof | wc -l sums up a lot of duplicated entries (forked processes can share file handles etc). That number could be much higher than the limit set in /proc/sys/fs/file-max. To get the current number of open files from the Linux kernel's point of view, do this: cat /proc/sys/fs/file-nr Example: This server has 40096 out of max 65536 open ...


49

iptables -A INPUT -p tcp --syn --dport 80 -m connlimit --connlimit-above 15 --connlimit-mask 32 -j REJECT --reject-with tcp-reset This will reject connections above 15 from one source IP. iptables -A INPUT -m state --state RELATED,ESTABLISHED -m limit --limit 150/second --limit-burst 160 -j ACCEPT In this 160 new connections (packets really) are ...


49

The problem is caused by the TasksMax systemd attribute. It was introduced in systemd 228 and makes use of the cgroups pid subsystem, which was introduced in the linux kernel 4.3. A task limit of 512 is thus enabled in systemd if kernel 4.3 or newer is running. The feature is announced here and was introduced in this pull request and the default values were ...


38

Apply the changes directly to a running process if you have prlimit installed (comes with util-linux-2.21) prlimit --pid <pid> --<limit>=<soft>:<hard> for example prlimit --pid 12345 --nofile=1024:2048 Refer here


34

That is certainly not trivial task that can't be done in userspace. Fortunately, it is possible to do on Linux, using cgroup mechanizm and its blkio controller. Setting up cgroup is somehow distribution specific as it may already be mounted or even used somewhere. Here's general idea, however (assuming you have proper kernel configuration): mount -t tmpfs ...


32

Limits are process-specific. ulimit is a shell bult-in and can only change limits for this shell and the processes started from this shell. sudo ulimit doesn't make any sense. Even if it won't fail it would only influence the processes started by sudo at the same time, and there aren't any. In order to raise your limit above the hard limit you have to ...


30

To temporarily set the open files limit for the user you are currently logged in under (e.g. 'root'):You can also use the ulimit command to change the values in your current shell. However, hard limits can only be adjusted downwards unless you're root. Example: # ulimit -a core file size (blocks, -c) 0 data seg size (kbytes, -d) ...


29

A process can change its limits via the setrlimit(2) system call. When you run ulimit -n you should see a number. That's the current limit on number of open file descriptors (which includes files, sockets, pipes, etc) for the process. The ulimit command executed the getrlimit(2) system call to find out what the current value is. Here's the key point: a ...


28

Take a look at ionice. From man ionice: This program sets or gets the io scheduling class and priority for a program. If no arguments or just -p is given, ionice will query the current io scheduling class and priority for that process. To run du with the "idle" I/O class, which is the lowest priority available, you can do something like this: ionice -c ...


27

nice / renice nice is a great tool for 'one off' tweaks to a system. nice COMMAND cpulimit cpulimit if you need to run a CPU intensive job and having free CPU time is essential for the responsiveness of a system. cpulimit -l 50 -- COMMAND cgroups cgroups apply limits to a set of processes, rather than to just one cgcreate -g cpu:/cpulimited cgset -r ...


27

What internally limits these things? Long answer ZFS's limits are based on fixed-size integers because that's the fastest way to do arithmetic in a computer. The alternative is called arbitrary-precision arithmetic, but it's inherently slow. This is why arbitrary-precision arithmetic is an add-on library in most programming languages, not the default way ...


24

Your limitation does not stem indeed from the filesystem; or from package versions I think. Your 2GB limit is coming from you using a 32-bit version of your OS. The option to increase the file would be installing a 64-bit version if the hardware supports it. See Large file support Traditionally, many operating systems and their underlying file system ...


23

Have a look at trickle a userspace bandwidth shaper. Just start your shell with trickle and specify the speed, e.g.: trickle -d 100 zsh which tries to limit the download speed to 100KB/s for all programs launched inside this shell. As trickle uses LD_PRELOAD this won't work with static linked programs but this isn't a problem for most programs.


23

ionice from the util-linux does something similar to what you want. It doesn't set absolute IO limits, it sets IO priority and 'niceness' - similar to what nice does for a process' CPU priority. From the man page: ionice - set or get process I/O scheduling class and priority DESCRIPTION This program sets or gets the I/O scheduling class and priority ...


23

Most of the values¹ in limits.conf are limits that can be set with the ulimit shell command or the setrlimit system call. They are properties of a process. The limits apply independently for each process. In particular, each process can have up to nofile open files. There is no limit to the number of open files cumulated by the processes of a user. The ...


23

You can throttle a pipe with pv -qL (or cstream -t provides similar functionality) tar -cf - . | pv -q -L 8192 | tar -C /your/usb -xvf - -q removes stderr progress reporting. The -L limit is in bytes. More about the --rate-limit/-L flag from the man pv: -L RATE, --rate-limit RATE Limit the transfer to a maximum of RATE bytes per second. A ...


22

You can use pv to throttle the bandwidth of a pipe. Since your use case is strongly IO-bound, the added CPU overhead of going through a pipe shouldn't be noticeable, and you don't need to do any CPU throttling. tar cf - mydata | pv -L 1m >/media/MYDISK/backup.tar


22

To change the limits of a running process, you may use the utility command prlimit. prlimit --pid 12345 --nofile=1024:1024 What that does internally is to call setrlimit(2). The man page of prlimit should contain some useful invocation examples. Source: https://sig-io.nl/posts/run-time-editing-of-limits-in-linux/


20

While it can be an abuse for memory, it isn't for CPU: when a CPU is idle, a running process (by "running", I mean that the process isn't waiting for I/O or something else) will take 100% CPU time by default. And there's no reason to enforce a limit. Now, you can set up priorities thanks to nice. If you want them to apply to all processes for a given user, ...


18

I think it's largely for historical reasons. A Unix file descriptor is a small int value, returned by functions like open and creat, and passed to read, write, close, and so forth. At least in early versions of Unix, a file descriptor was simply an index into a fixed-size per-process array of structures, where each structure contains information about an ...


17

ulimit is a shell built-in, not an external command. It needs to be built in because it acts on the shell process itself, like cd: the limits, like the current directory, are a property of that particular process. sudo bash -c 'ulimit -n 4096' would work, but it would change the limit for the bash process invoked by sudo only, which would not help you. ...


17

Here we see evidence of a problem: tail: inotify resources exhausted By default, Linux only allocates 8192 watches for inotify, which is ridiculously low. And when it runs out, the error is also No space left on device, which may be confusing if you aren't explicitly looking for this issue. Raise this value with the appropriate sysctl: fs.inotify....


16

On Linux (3.5 at least), it's hardcoded to 40 (see follow_link() in fs/namei.c), and note that it's the number of links followed when resolving all the components of a path, you can only change it by recompiling the kernel. $ ln -s . 0 $ n=0; repeat 50 ln -s $((n++)) $n $ ls -LdF 39 39/ $ ls -LdF 40 ls: cannot access 40: Too many levels of symbolic links $ ...


16

When logging in using SSH, you use a pseudo-terminal (a pty) allocated to the SSH daemon, not a real one (a tty). Pseudo-terminals are created and destroyed as needed. You can find the number of ptys allowed to be allocated at one time at /proc/sys/kernel/pty/max, and this value can be modified using the kernel.pty.max sysctl variable. Assuming that no other ...


16

Limits are inherited from a parent process to its child processes. Processes running as root can change limits arbitrarily; other processes cannot increase hard limits. Thus the hard limits set by the login process affect all the processes in a session. If you change /etc/security/limits.conf, this will affect all new sessions, and processes in these new ...


Only top voted, non community-wiki answers of a minimum length are eligible