(In)significance of [... ] and tty root processes

Question

A colleague of mine assigned me to perform a root-cause analysis of an enterprise security server sometime ago. We knew the problem resided in one of the processes run by root but did not know which one.

I first began my research by issuing the command:

ps -aux | grep -v grep | grep root

This returned a list of several dozen processes. My colleague then helped me optimize my search with this command:

ps -aux | grep -v grep | grep root | grep -Ev '[[]'| grep -v tty

This new command reduced the list of processes returned significantly, and we soon found the problem. The latter two parts of the command filter out

1) processes surrounded by [ ], such as [ksoftirqd/0] 2) processes running via teletype terminal

ps(1) tells me that processes with arguments that cannot be located are placed in square brackets (not sure what this means), and tty is just a predecessor of the modern bash terminal. However, I still do not understand their significance (or should I say insignificance) as to why we could've just eliminated them from the more "fruitful" root processes.

Would anyone be able to shed some light here?

Thank you

Answer 1

"arguments that cannot be located" typically means that the process has no command line arguments because it's not a normal user process but one started directly by the kernel. When you're looking for a process that's doing something wrong, it's usually not one of these kernel processes, so your colleague suggested filtering those out.

tty is not "just a predecessor of the modern bash terminal". It has nothing to do with bash or shells at all, and it is not a predecessor of anything. The tty listed by ps is the process' controlling terminal, if it has one. Processes with a controlling tty were most likely started by some user as part of an interactive session (whether it was a session started on the local console, a local serial port, an SSH session, or screen, or something else doesn't matter: they're all terminals). Processes without a controlling tty are typically daemons and system services. Your colleague was suggesting that the problem was more likely to be in a daemon or system service.

As for the rest of the command line:

ps -aux | grep root

I don't know what operating system you're using, but versions of ps with aux-style arguments are generally of BSD heritage. I am not familiar with BSD's ps, but I suspect there is an option to list only processes belonging to a particular user, instead of listing every process and then filtering out all the ones not owned by root afterwards. If you were using Linux for example it would be:

ps -fu root

Answer 2

Frankly it sounds a bit silly.

grep -Ev "[[]"

excludes processes having the opening square bracket in their command. While these are often kernel processes, even a regular user space program can have that character present on the command line.

grep -Ev "tty"

is the same in pale blue. It excludes processes having the string "tty" somewhere on their line of output of ps. This includes processes that have a terminal device open (but not processes than have a pseudo-terminal open(!)), so it sounds like restricting the output to daemonised tasks.

However, as with the above case, it will also exclude processes that appear in output of ps as:

pid ?    R+  310:03 /home/pwned_user/bin/rootkit_tty[_rofl crack root-password

That said, you are quite lucky you found anything and you can't really be sure you've found everything. Considering your question has "enterprise security server" at the beginning brings it yet one more level up (or down, if you like).

Answer 3

As a general rule, you never want to do ps aux | grep -v grep, that's what pgrep is for:

pgrep -u root

The command above will list all processes started by root. You do need ps if you want to 1) exclude those whose name is enclosed with [] and 2) restrict to those not listed as running in ttys. You can avoid some of the problems mentioned in the other answers by specifying where exactly each string shouldn't match (as opposed to using grep -v which will discard a line if it matches anywhere):

ps aux | awk '$1=="root" && $7 !~/^tty[0-9]$/ && $NF!~/^\[.*\]$/' 
              ----------    ----------------     ---------------
                  |               |                    |
                  |               |                    |-> If the last field 
                  |               |                        is within [ ]  
                  |               |---------------> If the 7th field starts
                  |                                 with tty and ends in a digit 
                  |----------------------------> If the 1st field is 'root'