3

My top info... 

top - 16:02:16 up 461 days, 20:32,  1 user,  load average: 4.12, 4.10, 4.22
Tasks: 273 total,   5 running, 267 sleeping,   0 stopped,   1 zombie
Cpu0  :  0.7%us,  0.3%sy, 99.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu1  :  0.0%us,  0.3%sy, 99.7%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu2  :  0.3%us,  0.3%sy, 99.3%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu3  :  0.3%us,  0.3%sy, 99.3%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:  16463796k total, 14791704k used,  1672092k free,   242224k buffers
Swap:  8388604k total,  1076164k used,  7312440k free, 11015344k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 7520 username  39  19 90992 1880 1864 R 99.6  0.0   4563:36 ftp
 7670 username  39  19 90992 1512 1496 R 98.9  0.0   8562:36 ftp
 9926 username  39  19 90992 1532 1516 R 98.9  0.0   8548:47 ftp
14414 username  39  19 90992 1528 1512 R 98.9  0.0   8662:33 ftp
30987 username2 20   0 4796m 803m  11m S  1.0  5.0   9:06.95 java
 2060 username2 20   0 3608m 1.2g 2480 S  0.3  7.8   6025:29 java

CentOS release 6.9 (Final)
LSB_VERSION=base-4.0-amd64:base-4.0-noarch:core-4.0-amd64:core-4.0-noarch:graphics-4.0-amd64:graphics-4.0-noarch:printing-4.0-amd64:printing-4.0-noarch

If I toggle idle processes off in top the only processes showing are ftp and top itself.

All the other non-ftp processes are less than one percent %CPU.

tcpdump and strace for the ftp processes don't show activity.

It looks like this means the 4 cpu's are each using a small amount of %us userspace time to the four ftp processes which are niced so it shows at 99 %ni, meaning of the little userspace time used it was almost all niced. There is no idle time %id because those ftp processes are doing something.

Am I reading these number correctly?

Are the ftp processes busy with a Linux empty task?

Improve this question
3
  • 2
    of course a process which simply eats up cpu cycles with a busy loop will show little or no any activity in strace: strace only prints signals and system calls. Example: strace sh -c 'while :; do :; done' will push one of your cores to 100% without strace showing it doing anything.
    – user313992
    Commented Dec 27, 2019 at 1:28
  • 2
    Your ftp command is probably buggy, and it either hit a bump and went into a trip by itself, or someone exploited it as a denial-of-service attack. Any ftp command should be i/o bound, it has no case for transforming so much electrical energy into heat.
    – user313992
    Commented Dec 27, 2019 at 1:35
  • @LinuxSecurityFreak sorry! will add tags next time
    – Michael Curtis
    Commented Dec 27, 2019 at 15:01

2 Answers 2

Reset to default
3

This is not an answer to your specific issue with ftp but a more general answer.

strace is short for system trace. It only traces system calls and signals.

So this will eat a CPU thread, but strace will show nothing, because it does not make system calls:

perl -e 'while(1){}'

This is not the normal way to wait. Normally you use sleep or blocking on I/O both of which take no CPU time.

Improve this answer
2
  • Thanks for clarifying strace/system calls & signals. So is the point that if a process is waiting it eats a thread and that will show up at %CPU 99? Does that mean the CPU did nothing else but handle that process's thread? 99% seems like it would mean "worked a lot" to the layman, but in this case it really means all the CPU time was devoted to a process that essentially did nothing but eat up time. Is that correct?
    – Michael Curtis
    Commented Dec 27, 2019 at 15:10
  • 2
    @LinuxSecurityFreak But this fundamentally does address the question of "what 99% of CPU on a process which appears idle in strace" means. Let's not discourage perfectly valid contributions by valued contributors :-)
    – Chris Down
    Commented Dec 27, 2019 at 21:12
2

The ftp processes are probably multithreaded, and you are only seeing the details of the first thread.

If you have a similar version of top to the one on my machine, then pressing H will toggle the threads display, and you will see additional details appear of the threads.

My guess is that the strace output is showing the ftp processes are blocked on a futex system call (there are other possibilities including read and nanosleep) as it waits for worker threads. I would also guess that these processes have nothing to do with file transfer protocol, but are instead doing crypto coin mining although that is much less certain.

The ftp processes are not busy with a Linux empty task, for any reasonable interpretation of what that means.

Improve this answer
5
  • Icarus my thought, too.
    – Chris Davies
    Commented Dec 27, 2019 at 9:42
  • FWIW, I used H and I don't see it is multithreaded. I compared with another process to see multiple threads just to confirm how threads will appear using H
    – Michael Curtis
    Commented Dec 27, 2019 at 14:56
  • OK, if it is not multithreaded then as others have said it is in a loop not making (m)any system calls and is therefore probably just hitting a bug. Is /proc/7520/exe a symlink to /usr/bin/ftp or something else (the number is the process id of one of the things burning cpu)? If so does stat /usr/bin/ftp show the file has a recent change time?
    – icarus
    Commented Dec 27, 2019 at 15:19
  • yes, there is a symlink to /usr/bin/ftp, but it has only a recent access date 12/23, modify and change are years old.
    – Michael Curtis
    Commented Dec 27, 2019 at 17:04
  • 2
    In that case the chances are reasonable that the binary has not been changed (it could have been, and the ctime and mtime reset), and my guess that it was a crypto miner is just wrong. Without debug symbols for the ftp binary you can't do much more. You can attach a debugger to one of the running processes and inspect the stack (use the 'where' command), let it continue for a bit, interrupt it and inspect the stack again. If most of the stack is common then it is probably in a loop. perf record can help automate some of this. If it was my system I would kill the 4 processes and move on.
    – icarus
    Commented Dec 27, 2019 at 18:20

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .