2

I am using the super computers in the network provided by Compute Canada and in the documentation page I see the following:

enter image description here

I am quite curious - what is the concept of memory per core here? I thought all cores should share the same memory normally? Does it mean that, if I have a job that takes 16GB memory space, and the memory per core is only 8GB, I need at least two cores (i.e. multi-processing) to accomplish it?

  • 1
    They most likely have some sort of VM provisioning system. When you run a job they spin up a VM for it to run on and said VM needs CPU/memory specifications, they have set them at 4 or 8GB of memory per vCPU (core). – jesse_b Jul 7 '18 at 19:24
  • So should I care about the memory per core or just the memory per node is sufficient? (i.e. if there are 32 cores, 32 cores * 8GB_per_core = 256 GB RAM per node. If I have a job that requires 100 GB RAM, will it go through?) – Jinhua Wang Jul 7 '18 at 19:29
  • I don't know for sure but I believe so. In my experience most cloud computing platforms provide packages with similar metrics, but ultimately it is simply a virtual machine with the given specs. – jesse_b Jul 7 '18 at 19:43
  • 1
    Ask them. Could be simple language that RAM is given as in 1 CPU 4GB, 2 CPUs 8GB... Only them can clear that out. – Rui F Ribeiro Jul 7 '18 at 19:49
  • 2
    I'm voting to close this question as off-topic because it's about a particular service, isn't likely to be useful to the public at large, and should probably be directed at the administrators of said service. – ilkkachu Jul 7 '18 at 22:01
2

If you search for your question via Google like this - 'Compute Canada memory per core' you'll be directed to the glossary of terms for Compute Canada. On that page they define it like this:

Memory per core: The amount of memory (RAM) per CPU core. If a compute node has 2 CPUs, each having 6 cores and 24GB (gigabytes) of installed RAM, then this compute node will have 2GB of memory per core.

Memory per node: The total amount of installed RAM in a compute node.

I'd also direct you to this page titled: Allocations and resource scheduling. They cover in excruciating detail how they handle the billing/scheduling of jobs that are RAM vs. core heavy.

A core equivalent is a bundle made up of a single core and some amount of associated memory. In other words, a core equivalent is a core plus the amount of memory considered to be associated with each core on a given system.

Cedar and Graham are considered to provide 4GB per core, since this corresponds to the most common node type in those clusters, making a core equivalent on these systems a core-memory bundle of 4GB per core. Niagara is considered to provide 4.8GB of memory per core, make a core equivalent on it a core-memory bundle of 4.8GB per core. Jobs are charged in terms of core equivalent usage at the rate of 4 or 4.8 GB per core, as explained above. See Figure 1.

So I do not believe this has anything to do with NUMA in the traditional sense. It's more the case that the Canada cluster management group has arbitrarily decided what a "core equivalent" is with respect to the different compute clusters they provide.

Their Graham + Cedar clusters provide 4GB/core, whereas Niagara provides 4.8GB/core.

The concept would appear to be completely a logical segmentation at the job/scheduling level of their compute cluster.

| improve this answer | |
  • So ... Does it mean that I need 50 GB per core to run a job that takes 50 GB memory? – Jinhua Wang Jul 8 '18 at 21:23
  • Or rather, can one core use another core's RAM? – Jinhua Wang Jul 8 '18 at 21:23
  • It means that if you have a job with a specific RAM requirement, your request will get however "core equivalents" are required to provide that amount, and you'll be "charged" in usage accordingly. – slm Jul 8 '18 at 21:52
  • So I will be charged with idle core as well (because i used their RAM)? – Jinhua Wang Jul 9 '18 at 10:00
  • That was my interpretation of the compute cluster material. – slm Jul 9 '18 at 11:14
2

What's you're looking for is NUMA repartition see the wikipedia page for that.

numa wiki schematic

it is harware bus design optimised for faster access between core & memory but also allows core to address memory of any another core (this is just slower in that case)

| improve this answer | |

Not the answer you're looking for? Browse other questions tagged or ask your own question.