Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I have a Java thread pinned at my cpu core. It is busy spinning and I see through top that it is running fine:

%Cpu0  :  0.0 us,  0.0 sy,  0.0 ni,100.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu1  :  0.0 us,  0.0 sy,  0.0 ni,100.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu2  :100.0 us,  0.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu3  :  0.0 us,  0.0 sy,  0.0 ni,100.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st

I traced down the LWP number of the Java thread, with the steps below:

First the JVM PID:

$ ps -ef | grep -i java | grep -v grep
1000      9062  1531 99 22:41 pts/0    00:16:42 java -cp target/mylib-all.jar -Dbind=2 com.examples.PinnedThread

Then the LWP ID:

$ ps -eLo pid,lwp,nlwp | grep 9062
 9062  9062   20
 9062  9063   20
 9062  9064   20
 9062  9065   20
 9062  9066   20
 9062  9067   20
 9062  9068   20
 9062  9069   20
 9062  9070   20
 9062  9071   20
 9062  9072   20
 9062  9073   20
 9062  9074   20
 9062  9075   20
 9062  9076   20
 9062  9077   20
 9062  9078   20
 9062  9079   20
 9062  9080   20
 9062  9105   20

It is one of those in the second column above, so I use jstack to find out:

$ jstack 9062

"PinnedThread" prio=10 tid=0x00007fcf9817b800 nid=0x2378 runnable [0x00007fcf7e1ed000]
   java.lang.Thread.State: RUNNABLE

0x2378 is 9080 in DEC, so I found the LWP for my thread, it is 9080.

Now I want to monitor my thread context switches, so I do:

$ watch -n.5 grep ctxt /proc/9080/status
Every 0.5s: grep ctxt /proc/9080/status

voluntary_ctxt_switches:        9
nonvoluntary_ctxt_switches:     120743

Problem is: nonvoluntary_ctxt_switches increments like a counter, non-stop and fast. I am sure my thread is not doing hundreds of context switches per second. What is going on here? Is that something particular about LWP and that nonvoluntary_ctxt_switches ?

Interesting enough, if I check the status of the main process PID (JVM), I get much quieter numbers:

$ cat /proc/9062/status | grep ctxt
voluntary_ctxt_switches:    3
nonvoluntary_ctxt_switches: 2

And they never change! Can anyone shed a light here? I am pinning the thread with sched_setaffinity and it looks from top that this part is working very well but 100 nonvoluntary_ctxt_switches per second doesn't make any sense.

I also tried to use pidstat and it gives me 84 nonvoluntary_ctxt_switches per second. :(

pidstat -w -I -t -p 9062 3 | grep 9080
12:21:07 AM         -     9080      0.00     84.33  |__java
12:21:10 AM         -     9080      0.00     84.00  |__java
12:21:13 AM         -     9080      0.00     84.33  |__java

Here is what my thread is doing, in other words, nothing, just busy spinning and incrementing a counter:

    Thread t = new Thread(new Runnable() {

        long count = 0;

        @Override
        public void run() {

            Affinity.bind();

            try {

                while(true) {
                    count = (count == Long.MAX_VALUE ? 0 : count + 1);
                }

            } finally {
                Affinity.unbind();
            }
        }
    }, "PinnedThread");

Below is my /proc/interrupts. I turned off irqbalance, so all hardware interrupts should go to CPU0. For CPU2 I have:

  • 23 is moving very slowly (IO-APIC-fasteoi ehci_hcd:usb2)
  • NMI is moving one per second (Non-maskable interrupts)
  • LOC is moving very fast (Local timer interrupts)
  • PMI is moving one per second (Performance monitoring interrupts)
$ watch -n.5 cat /proc/interrupts
           CPU0       CPU1       CPU2       CPU3       CPU4       CPU5       CPU6       CPU7
  0:         43          0          0          0          0          0          0          0   IO-APIC-edge      timer
  1:          3          0          0          0          0          0          0          0   IO-APIC-edge      i8042
  8:          1          0          0          0          0          0          0          0   IO-APIC-edge      rtc0
  9:          0          0          0          0          0          0          0          0   IO-APIC-fasteoi   acpi
 12:          4          0          0          0          0          0          0          0   IO-APIC-edge      i8042
 16:      67936        557       1530        275       6474       2516      16305       4790   IO-APIC-fasteoi   ehci_hcd:usb1, nouveau
 17:         59          2          0          0          6          1          0          0   IO-APIC-fasteoi   snd_hda_intel
 23:    3109042     102263      57885       9186     196294     119229     716776     184137   IO-APIC-fasteoi   ehci_hcd:usb2
 40:          0          0          0          0          0          0          0          0   PCI-MSI-edge      PCIe PME
 41:          0          0          0          0          0          0          0          0   PCI-MSI-edge      PCIe PME
 42:          0          0          0          0          0          0          0          0   PCI-MSI-edge      PCIe PME
 43:          0          0          0          0          0          0          0          0   PCI-MSI-edge      PCIe PME
 44:          0          0          0          0          0          0          0          0   PCI-MSI-edge      PCIe PME
 45:          0          0          0          0          0          0          0          0   PCI-MSI-edge      PCIe PME
 46:      16996        781        242         46       5245       1435       3424        932   PCI-MSI-edge      ahci
 47:          0          0          0          0          0          0          0          0   PCI-MSI-edge      ahci
 48:          0          0          0          0          0          0          0          0   PCI-MSI-edge      xhci_hcd
 49:          0          0          0          0          0          0          0          0   PCI-MSI-edge      xhci_hcd
 50:          0          0          0          0          0          0          0          0   PCI-MSI-edge      xhci_hcd
 51:          0          0          0          0          0          0          0          0   PCI-MSI-edge      xhci_hcd
 52:          0          0          0          0          0          0          0          0   PCI-MSI-edge      xhci_hcd
 53:          0          0          0          0          0          0          0          0   PCI-MSI-edge      xhci_hcd
 54:          0          0          0          0          0          0          0          0   PCI-MSI-edge      xhci_hcd
 55:          0          0          0          0          0          0          0          0   PCI-MSI-edge      xhci_hcd
 56:          0          0          0          0          0          0          0          0   PCI-MSI-edge      xhci_hcd
 57:         13          0          0          0          0          0          0          0   PCI-MSI-edge      mei
 58:        300         31          0          0         21          0          0          0   PCI-MSI-edge      snd_hda_intel
NMI:       2966       2642      10738       3122       1490       1251        858        136   Non-maskable interrupts
LOC:    1549246     742527    2500092     745512     465500     434371     451682     117591   Local timer interrupts
SPU:          0          0          0          0          0          0          0          0   Spurious interrupts
PMI:       2966       2642      10738       3122       1490       1251        858        136   Performance monitoring interrupts
IWI:          0          0          0          0          0          0          0          0   IRQ work interrupts
RTR:          7          0          0          0          0          0          0          0   APIC ICR read retries
RES:     153708      47896         15         39       4188       4699          1          1   Rescheduling interrupts
CAL:        299        345        390        391        346        213        391        386   Function call interrupts
TLB:      49114      50157        826        292      44726      44832         86          4   TLB shootdowns
TRM:          0          0          0          0          0          0          0          0   Thermal event interrupts
THR:          0          0          0          0          0          0          0          0   Threshold APIC interrupts
MCE:          0          0          0          0          0          0          0          0   Machine check exceptions
MCP:         99         99         99         99         99         99         99         99   Machine check polls
ERR:          0
MIS:          0
share|improve this question

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.