2

We are using an Amazon EC2 instance running Ubuntu 14.04 and we are stuck with a couple questions.

What is the maximum number of open files I can have based on the CPU configuration that we are currently running?

Is there a calculation for this? Or an algorithm?

Our CPU Config (Dual CPUs):

processor   : 0 
vendor_id   : GenuineIntel
cpu family  : 6
model       : 62
model name  : Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
stepping    : 4
microcode   : 0x415
cpu MHz     : 2494.046
cache size  : 25600 KB
physical id : 0
siblings    : 2
core id     : 0
cpu cores   : 1
apicid      : 0
initial apicid  : 0
fpu     : yes
fpu_exception   : yes
cpuid level : 13
wp      : yes
flags       : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx rdtscp lm constant_tsc rep_good nopl xtopology eagerfpu pni pclmulqdq ssse3 cx16 pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm xsaveopt fsgsbase smep erms
bogomips    : 4988.09
clflush size    : 64
cache_alignment : 64
address sizes   : 46 bits physical, 48 bits virtual
power management:

For clarity I am not asking how to set these limits, rather how to calculate the maximum limit for our setup.

1

I'm pretty sure the limit you'll hit is that you'll run out of RAM for the kernel data structures involved. It's possible CPU architecture might matter (e.g., because the data structures are a different size, or have to be aligned differently), but which particular x86_64 CPU you have shouldn't matter.

A 32-bit platform might limit you to 231-1 (i.e., MAXINT), but you'd surely run out of RAM first—a file descriptor takes up more than two bytes of RAM. (Of course, you're on a 64-bit platform—and 263-1 is an absurd number of files to have open)

I confess to not knowing how to calculate the number—and it surely depends on the type of file descriptor; kernel memory usage is surely different if it's a TCP socket, open local file (might even differ per filesystem), pipe, etc.

Honestly, I'd suggest you fire up a temporary EC2 instance (with the same machine type), set the limit to an absurdly high value, and test.

4
  • You can improve this answer with information from unix.stackexchange.com/questions/104929 for starters. Similar information exists for Linux. – JdeBP Dec 8 '16 at 22:19
  • @JdeBP How so? The other question is about OpenBSD and configuring limits, whereas this one is about Ubuntu (Linux) and OP superficially states he's not asking how to configure them. Maybe I'm missing something there quickly looking from my phone, but I don't see anything relevant. – derobert Dec 8 '16 at 22:22
  • You "confess to not knowing how to calculate the number", which is what the questioner actually wants to know. The other answers show where such calculations are made and the form that they take, as well as indicating the form in which one can find their results visible to applications mode code. As I said, one can hunt up similar stuff for Linux. – JdeBP Dec 8 '16 at 22:32
  • @JdeBP That answer talks about how the default limit is calculated. That's a different number, I don't know how to calculate how many files you could open (with the administrative limit set to infinity) before the kernel runs out of RAM. That's going to depend on the exact type of file, other memory already used on the system, etc. – derobert Dec 8 '16 at 22:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.