How to detect if isolcpus is activated and on which cpus, when for example you connect for the first time on a server. Conditions:

not spawning any process to see where it will be migrated.

The use case is that isolcpus=1-7 on a 6 cores i7, seems to not activate isolcpus at boot, and i would like to know if its possible from /proc/, /sys or any kernel internals which can be read in userspace, to provide a clear status of activation of isolcpus and which cpu are concerned. Or even read active setting of the scheduler which is the first concerned by isolcpus.

Consider the uptime is so big, that dmesg is no more displaying boot log to detect any error at startup. Basic answer like "look at kernel cmd line" will not be accepted :)

  • 1
    Can you use pidstat -C isolcpus. pidstat is from the sysstat package. – Timothy Pulliam Jan 9 '17 at 15:46
  • 1
    May be im dumb, but i don't see any link between pidstat and isolcpus. Can you elaborate a little more ? – netmonk Jan 9 '17 at 15:53
  • You say you need to know what CPUs the command is running on. I don't know what isolcpus is, but pidstat can tell you what CPU a process is running on if you pass the -C process_name flag. For example pidstat -C top produces the following. 10:56:52 AM PID %usr %system %guest %CPU CPU Command 10:56:52 AM 3457 0.00 0.00 0.00 0.00 6 top – Timothy Pulliam Jan 9 '17 at 15:57
  • 2
    Ok thank you for your reply. isolcpus is a kernel parameter wich forces the scheduler to not migrate any process by itself if they are isolated. For exemple on a 8 cpu servers, isolcpus=1-7 in the kernel command line, will force all spawned process forked by init and any kind of command in shell to run only on CPU0. To run a task on an isolated cpu, you need to launch it with taskset for exemple. So in regard of what i am asking, it seems your answer is out of topic. Thank anyway for trying to answer me – netmonk Jan 9 '17 at 15:58
up vote 5 down vote accepted

What you look for should be found inside

/sys/devices/system/cpu/isolated

and the reverse in

/sys/devices/system/cpu/possible

From drivers/base/cpu.c we see that the source displayed is the kernel variable cpu_isolated_map:

static ssize_t print_cpus_isolated(struct device *dev,
    n = scnprintf(buf, len, "%*pbl\n", cpumask_pr_args(cpu_isolated_map));
...
static DEVICE_ATTR(isolated, 0444, print_cpus_isolated, NULL);

and cpu_isolated_map is exactly what gets set by kernel/sched/core.c at boot:

/* Setup the mask of cpus configured for isolated domains */
static int __init isolated_cpu_setup(char *str)
{
    int ret;

    alloc_bootmem_cpumask_var(&cpu_isolated_map);
    ret = cpulist_parse(str, cpu_isolated_map);
    if (ret) {
            pr_err("sched: Error, all isolcpus= values must be between 0 and %d\n", nr_cpu_ids);
            return 0;
    }
    return 1;
}

But as you observed, someone could have modified the affinity of processes, including daemon-spawned ones, cron, systemd and so on. If that happens, new processes will be spawned inheriting the modified affinity mask, not the one set by isolcpus.

So the above will give you isolcpus as you requested, but that might still not be helpful.

Supposing that you find out that isolcpus has been issued, but has not "taken", this unwanted behaviour could be derived by some process realizing that it is bound to only CPU=0, believing it is in monoprocessor mode by mistake, and helpfully attempting to "set things right" by resetting the affinity mask. If that was the case, you might try and isolate CPUS 0-5 instead of 1-6, and see whether this happens to work.

  • Tested on Centos 7.x, note that /sys/devices/system/cpu/possible doesn't appear to be the 'reverse' depending on how you interpret 'reverse'. E.g. if .../cpu/isolated returns 2,4 because they had been isolated, .../cpu/possible would return 0-191. – bgura Jan 29 at 22:54

One of the easier ways to detect if isolcpus is consulting proc to see which parameters were passed to the kernel in runtime.

For that, you would use:

$cat /proc/cmdline 
BOOT_IMAGE=/boot/vmlinuz-4.8.0-1-amd64 root=/dev/sda1 ro isolcpus=2,3 quiet

As you can see, in this particular example isolcpus=2,3was passed as an argument to the running kernel.

You can also use taskset pointed to PID 1. As PID 1 is the standard PID for the first task launched by the kernel, we can take as a pretty good indication that it will reflect whether we have isolcpus working. As in:

$taskset -cp 1
pid 1's current affinity list: 0,1

Comparing with the lscpu command in the same server:

$lscpu | grep CPU.s
CPU(s):                4
On-line CPU(s) list:   0-3
NUMA node0 CPU(s):     0-3

As it can be seen, lscpu is showing 4 CPU/cores, while taskset is only showing 0,1, so this shows isolcpus is working here.

Have a look to at: How to ensure exclusive CPU availability for a running process?

  • Thank you sir for you answer, but i won't accept it. i specified that answer talking about the kernel command line won't be accepted. i obviously know about /proc/cmdline which is the exact copy of the content of grub.conf file. This is not what i asked ! Thank you anyway ! – netmonk Jan 9 '17 at 22:23
  • I have added to the answer. – Rui F Ribeiro Jan 10 '17 at 19:26
  • 1
    What is someone previously modified the affinity of processus with taskset ? what if someone sets init affinity to cpu 0 only, does it gives me any information about activation or not of isol cpu ? – netmonk Jan 13 '17 at 14:00

You can check Cpus_allowed and Cpus_allowed_list for current shell process to see what cpus were reserved

cat /proc/$$/status|tail -6

for eg

Cpus_allowed_list:      0-1, 3-5

means that the cpu=2 was reserved by isolcpus on a 6 cpus server

  • 1
    Actually... no. It shows what CPUs are in the affinity mask inherited by the current shell process. If it was, say, ssh, and one had manually limited the affinity of the parent sshd to CPU 1, you would see always CPU 1, whether isolcpus had "taken" or not. The $$ branch gives you the process inherited values, not the original system ones. – LSerni Jan 18 '17 at 18:07

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.