12

I'm interested in theoretical limits, perhaps with examples of systems having huge numbers of CPU's.

  • 1
    How many can it handle? or how many can it handle before you lose some benefit? also what kernel? I suspect that this answer changes somewhat for a computer patched to run a supercomputer. I seem to recall reading about a single instance using 4096 processors... – xenoterracide Dec 3 '10 at 10:43
  • what patch set, the normal patchset can't handle 4096 processors but linux has been patched to do it. (IIRC some answers seem to suggest it can) – xenoterracide Dec 3 '10 at 12:33
  • @xeno I think the fact that there's even a patchset that handles 4096 processors is stuff that should be mentioned on the Answer. – Tshepang Dec 3 '10 at 13:42
  • I don't recall much more than that, otherwise I'd give an answer, I also her performance gains pass 16? cores were limited... and that certain parts of the kernel needed a rewrite that had already begun. but really I don't have any citations and am not 100% that's why I'm not answering. – xenoterracide Dec 3 '10 at 14:41
  • xkcd.com/619 – balki Aug 25 '17 at 20:45
14

At least 2048 in practice. As a concrete example, SGI sells its UV system, which can use 256 sockets (2,048 cores) and 16TB of shared memory, all running under a single kernel. I know that there are at least a few systems that have been sold in this configuration.

According to SGI:

Altix UV runs completely unmodified Linux, including standard distributions from both Novell and Red Hat.

10

this is what Launchpad has to say about Ubuntu, so I guess it applies to others:

1.Intel x86:
Maximum CPUs: 32 (including logical CPUs)
Maximum memory: 64GB
Maximum filesize: 8TB
Maximum filesystem size (ext3) 16TB
Maximum per-process virtual address space: 4GB

2.AMD64/EM64T:
Maximum CPUs: 64
Maximum memory: 128GB
Maximum filesize: 8TB
Maximum filesystem size (ext3): 16TB
Maximum per-process virtual address space: N/A

These are standard max limitations whereas Linux cluster systems can scale up to 1024 CPU's.

That is 32 or 64 CPUs for x86 and x86_64 respectively.

Redhat says the same, but in a management-friendly table. Redhat EL6 can do 32 for x86, or 128 or 4096 CPUs cores for x86_64.

  • 3
    arch/x86/Kconfig says these CONFIG_NR_CPUS limits can be raised if CONFIG_MAXSMP is enabled. – ephemient Dec 3 '10 at 18:30
4

The x86_64 Linux kernel can handle a maximum of 4096 Processor threads in a single system image. This means that with hyper threading enabled, the maximum number of processor cores is 2048. Yes there is computers with more than 2048 processor cores; but these runs as clusters where several Linux kernels cooperate, connected with a high speed interconnect, typically an Infiniband fabric.

from the most current kernel 3.13, in ~/arch/x86/Kconfig :

config NR_CPUS

    ---help---
      This allows you to specify the maximum number of CPUs which this
      kernel will support.  If CPUMASK_OFFSTACK is enabled, the maximum
      supported value is 4096, otherwise the maximum value is 512.  The
      minimum value which makes sense is 2.

      This is purely to save memory - each supported CPU adds
      approximately eight kilobytes to the kernel image.

Update: On newer kernels this is architecture specific - for example on 4.15 x86_64 allows you to set NR_CPUS to 8192 in the right circumstances whereas 32 bit arm stops at 32.

2

This baby runs 10,368!

  • I know this is an old post, but you're linking to a cluster computer. The question is about running a single kernel instance. – frodeborli Jul 31 '15 at 0:22
1

Threads are subjective to the multitasking model and thread management scheme. The Gdt of intel based systems is used in linux if I remember rightly. The idea is it has a possibility 8192 threads at max size. This assuming that the system is using the gdt to manage the threads. On 32 bit machines task switching is managed and the on 32 and 64 bit machines interrupt vectors need to have gdt entries. Not sure how arm does it but the same articulation has to be achieved. Task switching concepts iterate the GDT in tasking models.

If you break out of the gdt scheme you assumably can reach what you have memory for when you have, for each thread, a page stack frame, page code base for the thread and page of heap space. You cannot assume you have a page of code or heap, which is the random variables. Generally there are two stack frames for each thread, one maintained by the thread and one maintained by the linux kernel. You add virtual memory concepts of swap space and the model gets blown out of the water but it is about thread priority..

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.