4

Every now and then (every few days) I notice that a process is using 100% CPU. The process is avrdude started by the Arduino IDE which, under certain circumstances that I haven't been able to reproduce just sits there at 100% CPU as shown in top.

Possibly the circumstances are that an upload to the Arduino board commences, and the board is disconnected during the process.

I have 8 cores in the processor, so it isn't immediately obvious that one of them is maxed out. In fact, it only becomes noticeable if it happens a few times in a row, and then I have maybe 3 cores at 100% CPU.

Is there a way of having some background task check for this (say, every 15 minutes), and then alert me in some way (eg. some pop-up dialog)? I am using Ubuntu 14.04 LTS.


Thanks to MelBurslan for his answer, but I'm stumped as to why it isn't fully working. My current script is this:

cpupercentthreshold=2
pstring=""
top -b -n 1 | sed -e "1,7d" | while read line; do
cpuutil=$(echo ${line} | awk '{print $9}' | cut -d"." -f 1)
procname=$(echo ${line} | awk '{print $12}' )
if [ ${cpuutil} -ge ${cpupercentthreshold} ]
then
  echo ${cpuutil}
  pstring=${pstring}${procname}" "
  echo pstring is currently ${pstring}
fi
done
echo pstring is ${pstring}
if [ -n "${pstring}" ]
then
  zenity --title="Warning!" --question --text="These processes are above CPU threshold limit ${pstring}" --ok-label="OK"
fi

I dropped the threshold down for testing. However as you can see it collects the individual processes OK, but the final test (to display the dialog box) fails because pstring has suddenly become empty for reasons I can't see:

13
pstring is currently VirtualBox
6
pstring is currently VirtualBox Xorg
6
pstring is currently VirtualBox Xorg compiz
6
pstring is currently VirtualBox Xorg compiz ibus-engin+
6
pstring is currently VirtualBox Xorg compiz ibus-engin+ top
pstring is
3
  • I suggest looking for existing server monitoring software. It'll probably be easier than rolling your own. – Gilles 'SO- stop being evil' Feb 16 '16 at 23:02
  • Actually I second, Gilles' suggestion. These simple script based monitoring tools are like band aid on a gushing wound. Should not be used as a long term solution. – MelBurslan Feb 16 '16 at 23:06
  • You have a pipeline stuff | while ... otherstuff ... done. Pipeline segments including the last at least for bash are run in subshells, so side-effects in one do not affect the parent shell. In bash (and ksh) you can fix this with process substitution, but in this case one awk can do all of it: top -b -n1 | awk -vthresh=2 'NR>7 && $9>thresh { p=p $12 " " } END { print p }' or even ...| awk -vthresh=2 -vORS=" " 'NR>7 && $9>thresh'; echo – dave_thompson_085 Feb 17 '16 at 4:28
1

After reading the answer by MelBurslan and various comments, I decided to try (inspired by their suggestions) to do a version in Lua. This was done in Lua 5.1.5 - I'm not sure if it will work with the latest Lua.

The general idea is to use Lua's popen (open a pipe) to execute top and then process the resulting data using a regular expression (or pattern, as it is called in Lua). Matching lines (which would be most of them) are then considered for crossing the threshold percentage. If they do, they are added to a table.

If the table is not empty, then zenity is called to display a message to the user. A few "gotchas" I found during development:

  • I added a timeout of 60 seconds to zenity so that, if you were not at the PC at the time, you didn't fill the screen with warning dialogs.
  • I added --display=:0.0 so that a display screen was found when running under cron.
  • I simplified the test for "every 15 minutes" in the crontab, like this:

    */15 * * * * /home/nick/check_cpu_usage.lua
    
  • The regular expression captures everything from top in case you want to do other tests (eg. using too much memory).

I think this would be faster than firing off lots of processes and subshells. It seems to work OK. Test by reducing the threshold (eg. to 5) and change the crontab entry to check every minutes.


check_cpu_usage.lua

#! /usr/local/bin/lua

THRESHOLD = 90  -- percent

-- pipe output of top through a file "f"
f = assert (io.popen ("top -b -n 1 -w 512"))
t = { }

-- check each line
for line in f:lines() do

  -- match top output, eg.
  --   PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
  -- 30734 nick      20   0 6233848 3.833g 3.731g S   8.6 12.2   3:11.75 VirtualBox

  local pid, user, priority, nice, virt, res, shr, 
        status, cpu, mem, time, command =
    string.match (line,
      "^%s*(%d+)%s+(%a+)%s+(%-?%d+)%s+(%-?%d+)" ..
--         pid      user   priority    nice
      "%s+([%d.]+[g]?)%s+([%d.]+[g]?)%s+([%d.]+[g]?)%s+([DRSTZ])%s+(%d+%.%d+)%s+(%d+%.%d+)" ..
--        virtual          res           shr             status       %cpu        %mem
      "%s+([0-9:.]+)%s+(.*)$")
--         time       command

  -- if a match (first few lines won't) check for CPU threshold
  if pid then
    cpu = tonumber (cpu)
    if cpu >= THRESHOLD then
      table.insert (t, string.format ("%s (%.1f%%)", command, cpu))
    end -- if
  end -- if

end -- for loop

f:close()

-- if any over the limit, alert us
if #t > 0 then
  os.execute ('zenity --title="CPU usage warning!" --info ' ..
              '--text="These processes are using more than ' ..
              THRESHOLD .. '% CPU:\n' ..
              table.concat (t, ", ") ..
              '" --ok-label="OK" ' ..
              '--timeout=60 ' ..   -- close dialog after one minute in case we aren't around
              '--display=:0.0 '  -- ensure visible when running under cron
              )
end -- if
1
  • Worked for me after changing the shebang to:#! /usr/bin/env lua – srking May 11 '16 at 21:50
0

create a simple script like this

cpupercentthreshold=75
pstring=""
top -b -n 1 | sed -e "1,7d" | while read line; do
cpuutil=$(echo ${line} | awk '{print $9}' | cut -d"." -f 1)
if [ ${cpuutil} -ge ${cpupercentthreshold} ]
then
  pstring=${pstring}${procname}" "
fi
done
if [ -z "${pstring}" ]
then
  echo "Everything looks good $(date)" >>mylogfile #if you want to keep track 
else
  zenity --title="Warning!" --question --text="These processes are above CPU threshold limit ${pstring}" --ok-label="OK"
fi

then run command crontab -e and insert a line like this:

0,15,30,45 * * * * /path/to/my/checker/script 

save and exit. Then execute

chmod 755 /path/to/my/checker/script

I am not familiar with zenity as I have not used a graphical display on a linux server for a while and never needed to use it. So, if it fails for some reason, look for help on man zenity. It is the replacement for the legacy executable xdialog, as I heard.

8
  • It's failing on the line if [ ${cpuutil} -ge ${cpupercentthreshold} ] with 0.0: integer expression expected - how do you compare floats in bash? – Nick Gammon Feb 16 '16 at 22:10
  • Or strip the period would probably be OK. I don't care about parts of a percent. – Nick Gammon Feb 16 '16 at 22:14
  • edited the answer, to disregard whatever is after the decimal point. – MelBurslan Feb 16 '16 at 22:16
  • No dialog is appearing because ${procname} is empty, but I'll try to work that one out. I feel an idiot because I am familiar with regular expressions etc., but not bash. – Nick Gammon Feb 16 '16 at 22:25
  • Please see amended question - everything works except the dialog box. – Nick Gammon Feb 16 '16 at 22:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.