Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I know that there is a hardware capacity of 2^48 bits, and I know that there isn't a lot of native limiting of app memory constraints in general by the kernel, but is there an upper bound of the memory that an app can consume in general besides 2^48 because that's the number of channels available on the DIMM sockets?

For reference, Redmond limits their products to ~2TB on most server products.

share|improve this question
3  
On what OS and what architecture? I'd expect a theoretical limit at 8EB (2^63) on most OSes, but this is entirely dependent on the unix variant, and possibly on versions or kernel compilation options. The architecture is going to be a limiting factor in practice (you've got to fit the MMU table), unless you use huge pages (which should let you reach the theoretical maximum on some OS/architecture combinations). Of course, the real limiting factor will probably be your budget. –  Gilles Sep 27 '11 at 0:42
    
The only architecture I can think of is either x86_64 or AMD64. I know that Microsoft has a specific max size on their processes, and I thought the 32 bit Unix and Linux variants had a max process size below 2^32 (especially since PAE allows for more than 2^32 addressing) –  jcolebrand Sep 27 '11 at 14:39
    
Linux does allow a 4G/4G split, which gives the process and kernel space each almost a full 4GB, with a bit of the space reserved to help switch between the two. I'm not aware that any Unices allow this. –  Ignacio Vazquez-Abrams Sep 28 '11 at 1:35
    
FWIW there was a question on Database Administrators that asked about running a 500GB in memory database, so that was the impetus to this Q (cc @Gilles @Ignacio) –  jcolebrand Sep 28 '11 at 1:39
    
@jcolebrand Link? This is well above my pay grade, but my first reflex would be to arrange to split the database over about 100 consumer-grade PCs (the Google way). If you want 500GB in the same machine (about 10 times what you can get on current PC-style servers), the machine will come with a working OS anyway, probably Linux or perhaps AIX (or Solaris?) (also: you have a dedicated pro working for you). –  Gilles Sep 28 '11 at 7:16
show 1 more comment

2 Answers

up vote 4 down vote accepted

By definition (at least, by a common definition), a 64-bit process can only see 264 bytes of memory. In practice, most operating systems limit each process to a little less than the available address space, so that the kernel can manipulate 64-value that can be pointers to kernel memory, process memory or device memory. A 4EB or 8EB limit is common.

Well before that, you'll probably hit an architecture-dependent limit. In particular, a process's memory map has to fit in the machine's MMU, and hardware tends to have only as many bits as necessary. MMUs come with up to four levels of page tables, and Linux (amongst others) can support all four: a PGD (page global directory) listing PUDs (page upper directories) listing PMDs (page mid-level directories) listing PTEs (page table entries). Since each level of indirection is one more step to take on most pointer dereferences, systems . It is common for the size of a page to be 4kB (21242 addressable bytes, however most 64-bit architectures can be configured for larger page tables, so it is hard to give any kind of general answer to the question of how high you can go.

Note that the per-process address limit is independent of how much physical memory there can be in the machine. For example, the number of channels available on the DIMM sockets in a PC are a theoretical limit to how much RAM you can have in that PC (although the width of the CPU memory bus is usually the limiting factor). It has no bearing on how much memory or how much address space a process can consume (the memory could be swapped out, or shared between several nodes of a NUMA multiprocessor machine).

share|improve this answer
add comment

The theoretical maximum is limited by the memory segmentation used on the CPU, e.g. X86-64 uses an absolute mechanism (the segments start at 0 and have a capacity of 264), but the OS itself can enforce artificial limits on the process size.

share|improve this answer
    
define "OS" and how that's different from the kernel. Do any of the current kernels limit this behavior on 64 bit systems? –  jcolebrand Sep 27 '11 at 0:41
    
The "OS" is the kernel plus any modules and libraries required to support operation of the machine. I am not aware of any *nices that enforce a process limit, but I do know that some enforce a physical limit. –  Ignacio Vazquez-Abrams Sep 27 '11 at 0:56
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.