I have a Linux server (Red Hat 2.6 kernel) which had a spike in load due to an Rsync process which seems to have forked many times, but the process was run by rsyncuser@notty I didnt see any cron job scheduled for that time with an rsync that is run under that name, and I assume if there were it would have a tty. How do I track down what the actual command which caused these processes was?
Starting with the assumption is probably too restrictive. You should generate a list of possible causes of the spike and try to invalidate them based on whether their prerequisites are met and/or resulting effects are available or not.
Without knowing your exact setup there seem to be at least the following possibilities that might have caused them
the cause was local, in which case you should search for entries in the log files. You did not see a cron job scheduled, not in the Logs? Or did you look at
rsyncuser's crontab? Or also at the daily/hourly etc cron jobs.
the cause was external. To check if that is possible, check if the that user happens to run the rsync daemon (if you have it, via /the init mechanism). Or if someone could remotely login to the account and try and start copying files (which should also have left traces in the log files).
I would probably also investigate whether the
rsyncuser account has a password and/or ssh public keys to see if someone could login to that account. And if anybody with root access could have started these.