0

I have a Linux server (Red Hat 2.6 kernel) which had a spike in load due to an Rsync process which seems to have forked many times, but the process was run by rsyncuser@notty I didnt see any cron job scheduled for that time with an rsync that is run under that name, and I assume if there were it would have a tty. How do I track down what the actual command which caused these processes was?

5
  • Do you have the commandline arguments with whch rsync did run? Looked at log files? If rsyncd running with owner rsyncuser? Does rsyncuser have login credentials? It is a bit of a broad question without some extra details
    – Anthon
    Commented Feb 18, 2014 at 5:33
  • I searched for rsyncuser in logs and saw nothing really useful. I did su -l rsyncuser and checked the crontab to find nothing. Nothing in root's crontab using su to execute command as that user. The ps output showed no rsync arguments that I could see other than the process was rsync itself, no visible switches. I am not even sure if this server was the src or dest for the rsync. Commented Feb 18, 2014 at 14:56
  • That is why in my answer I indicated the cause might be external. Does the rsyncuser have a ~/.ssh and authorisation files? Is an rsyncd running?
    – Anthon
    Commented Feb 18, 2014 at 14:58
  • No rsyncd, but there are 2 authorized hosts in rsyncusers auth hosts file. Commented Feb 18, 2014 at 17:00
  • Maybe you track who inserted those and when, or look at the remote machines if something triggered this.
    – Anthon
    Commented Feb 18, 2014 at 17:15

2 Answers 2

1

Starting with the assumption is probably too restrictive. You should generate a list of possible causes of the spike and try to invalidate them based on whether their prerequisites are met and/or resulting effects are available or not.

Without knowing your exact setup there seem to be at least the following possibilities that might have caused them

  1. the cause was local, in which case you should search for entries in the log files. You did not see a cron job scheduled, not in the Logs? Or did you look at rsyncuser's crontab? Or also at the daily/hourly etc cron jobs. root's cron.

  2. the cause was external. To check if that is possible, check if the that user happens to run the rsync daemon (if you have it, via /the init mechanism). Or if someone could remotely login to the account and try and start copying files (which should also have left traces in the log files).

I would probably also investigate whether the rsyncuser account has a password and/or ssh public keys to see if someone could login to that account. And if anybody with root access could have started these.

-1

OK you have a problem Step 1 WEBSITE: they have access logs and Error logs did a hacker activate a request that directly impacted the database Backend ? Not a WEBSITE: run wireshark to observe traffic related to rsync On 1 site I noticed that the time server was a little more than 15 ms out. (because some "duffus" decided to remove the hardware that ran the time service 1 system at a time until the last time server was decom'd. The database system's thought they were out of sync and that mandated a resend for backups and each system backed up to ALL the other databases - because they backed up each other.

           I once saw a system backup fail because the MTU was too big
           for the network. a Solaris system with internal connections
           using 9k packet size tried to do backups to an outside backup
           and it was so slow, eventually they made the MTU less than
           512 bytes (512 -32) I think was the solution. The network in
           question was a link out to a service organisation that swore
           there would be no problems with backups. Apparently that was
           just in the 1 case where the outside client had (512 - 32) as
           the MTU value. Basically the main issue was the internal 
           Manager - never checked things out prior to making the 
           contract and paying heaps for a slow service.

There are other issues, to consider but ... there are many other issues.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .