While debugging an related issue, I noticed that pgrep was returning a PID for seemingly arbitrary command-line patterns, e.g.:
$ sudo pgrep -f "asdf"
13017
$ sudo pgrep -f ";lkj"
13023
$ sudo pgrep -f "qwer"
13035
$ sudo pgrep -f "poiu"
13046
$ sudo pgrep -f "blahblahblah"
14038
$ sudo pgrep -f "$(pwgen 16 1)"
14219
The same command without sudo returned nothing (as expected):
$ pgrep -f blahblahblah
I tried to pipe the PID to ps in order to see what the command was, but that didn't work:
$ sudo pgrep -f blahblahblah | xargs ps -f -p
UID PID PPID C STIME TTY TIME CMD
It looks as though the process terminates too quickly. Then I tried using ps and grep, but that didn't work either (i.e. there were no results):
$ sudo ps -e -f | grep [a]sdf
$ sudo ps -e -o command | grep asdf
grep asdf
I also noticed that if I reran the command quickly enough then it seemed as though the PID was steadily climbing:
$ for i in $(seq 1 10); do sudo pgrep -f $(pwgen 4 1); done
14072
14075
14078
14081
14084
14087
14090
14093
14096
14099
$ for i in $(seq 1 10); do sudo pgrep -f blahblahblah; done
13071
13073
13075
13077
13079
13081
13083
13085
13087
13089
As a sanity check I tried using find and grep to search the proc directory:
$ sudo find /proc/ -regex '/proc/[0-9]+/cmdline' -exec grep adsfasdf {} \;
Binary file /proc/14113/cmdline matches
Binary file /proc/14114/cmdline matches
$ sudo find /proc/ -regex '/proc/[0-9]+/cmdline' -exec grep adsfasdf {} \;
Binary file /proc/14735/cmdline matches
Binary file /proc/14736/cmdline matches
Again it seems that the PID is climbing and that the cmdline matches arbitrary strings.
I tried this out on both CentOS 6.7 and on Ubuntu 12.04 with the same results. When I tried similar experiments on my Mac the tests came back negative - no mystery processes.
What's going on here?