As part of my current research, I've been doing tests where I disable/re-enable cores by echoing 0 or 1 to /sys/devices/system/cpu/cpuN/online to simulate vertical scaling.

With Kernel v4.4.0 (Ubuntu 16.04 LTS), this works perfectly on multiple Xeon E3 machines. Recently, I had the chance to test dual-socket EPYC 7281 machines, also on Ubuntu 16.04, with v4.4.0. Disabling cores on these systems worked just fine. When re-enabling them, however, they were shown as online by lscpu and htop, but no work was ever scheduled on them again until I rebooted the machine, i.e. when I stressed the machine, htop showed that only cores that had not been disabled and re-enabled before were being used.

In other words: On a Xeon machine, i first disable all cores but core #0, then re-enable them all again. Afterwards, everything runs normally as it did before disabling cores. On the EPYC machines, cores that had been disabled were never used again, even after re-enabling them and even though they were shown as being available by htop and lscpu.

I can no longer access the EPYC machines to test this further myself (only had them for 2 weeks and wasn't allowed to upgrade to newer kernel), but we're thinking about buying new hardware and I need to find out whether Threadripper/EPYC (Zen in general) can generally not disable/re-enable cores via the kernel, or whether this was just some weird isolated problem.

My question(s) is (are):

  • Could cores not coming online properly be a problem of the older 4.4. kernel, so would this issue be fixed if I were to upgrade to a newer kernel? Unfortunately, I can't test this, as I no longer have access to the EPYC machines (otherwise, this post would not be here :)). I would, however, have full control over our new hardware. So if this issue is fixed by a newer kernel, I'd be happy.

  • If not due to the kernel version, why would EPYCs behave in this way, and what could I do to remedy this?

I tried searching for answers, but so far haven't found anything conclusive, other than the fact that apparently there have been lots of patches for Zen-based CPUs since kernel v4.10 and later.

I looked at this link, which gives a few details about how the process works for Intel CPUs, and even links to the actual code responsible for disabling cores. Unfortunately, that didn't help a lot, at least I didn't understand enough to answer my questions.

This didn't help either, as I need to vertically scale cores during runtime, without rebooting.

I also found some references to a new power management system in newer kernels, but that went way over my head :-/

Or maybe instead of digging and finding the root of the problem, it would be enough if anyone who currently has a Threadripper or EPYC system with a recent kernel could maybe run a quick test by disabling/re-enabling cores and then loading them to check whether they are properly being utilized?

I would really appreciate any insight and help with this problem!

I'll be here to give further details if necessary, I don't know what else could be needed to explain this behavior. Also, is this the right exchange to ask this question, or would you recommend any other means of asking for help?

closed as off-topic by Jeff Schaller, forcefsck, jimmij, Timothy Martin, telcoM Apr 30 at 18:49

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "Questions describing a problem that can't be reproduced and seemingly went away on its own (or went away when a typo was fixed) are off-topic as they are unlikely to help future readers." – Jeff Schaller, forcefsck, jimmij, Timothy Martin, telcoM
If this question can be reworded to fit the rules in the help center, please edit the question.

up vote 1 down vote accepted

Ok, just for posterity, I've figured out the answer experimentally.

A colleague of mine did a quick test on a fully loaded EPYC 7281 machine with kernel v4.15, by disabling 31/32 cores and re-enabling them again. All disabled cores were correctly utilized again, so the issue I had earlier seems to be gone.

It is unclear which version includes this fix or whether there were any other changed factors that might impact this result, but for now I'm rather certain that having a newer kernel lets one disable/enable cores on Zen-based CPUs without a problem.

Not the answer you're looking for? Browse other questions tagged or ask your own question.