Recently, my linux server sends a very large file (10mb-10G/s outside at certain times, but I can't figure out what is the cause. Here is the screen shot

enter image description here

i use clamav to scan all files and got the results

19975  /usr/sbin/lsof: Linux.Trojan.Agent FOUND
19988  /usr/sbin/ss: Linux.Trojan.Agent FOUND
20076  /usr/bin/bsd-port/getty: Linux.Trojan.Agent FOUND
20095  /usr/bin/.sshd: Linux.Trojan.Agent FOUND
103568  /bin/ps: Linux.Trojan.Agent FOUND
103575  /bin/netstat: Linux.Trojan.Agent FOUND
103580  /opt/apache-tomcat-8.0.23/bin/.Rape: Unix.Trojan.Elknot-1 FOUND
8781  /tmp/udp25111: Linux.Trojan.Agent FOUND

It seems I got a backdoor trojan and am occupied by hacker as a ddos device. I saw this article that describes exactly the same thing I encounter http://news.drweb.com/?i=5801&c=5&lng=en&p=0

How can I remove the Trojan, and retrieve the inflected utilities ?

There are general incident response procedures you really should follow:

  • Disconnect
  • Assess
  • Change security information
  • Fix
  • Normalize
  • Analyze

Disconnect:

As long as you're network connected, those trojans may continue to reach out to C&C servers and come up with new badness.


Assess:

Figure out what's happening. Maybe other systems in your network are also affected.


Change security information:

You MUST assume that any security information you had on this server was compromised. This means any passwords, certificates, tokens, etc that live on the server should be changed, revoked, or otherwise become untrusted.


Fix:

It's often not worth removing trojans. In many cases, modern malware is extremely resilient to removal attempts. Rebuild the system or restore from backups. If restoring from backups, make sure to scan as soon as the restore is complete - roll back even farther if necessary.

I'm not immediately able to find much information on removal of the two trojans identified in your scan results. That's a little worrisome to me. I strongly recommend building from scratch over removing the trojan.


Normalize:

Reconnect to the network, make sure everything is working as it was before this happened.


Analyze:

How did you get the trojan? Is someone doing something wrong? Can you harden firewall rules, security policies, update software, etc to improve your posture going forward?

I would recommend trashing the server hardware and rebuilding from scratch.

The better the hack the less of a chance you have of actually removing the malware. Some bad actors can compromise your server's firmware. Unless you can isolate the point at which you actually got compromised the malware could easily be in all of your backups as well unless you have verified images that are pre-intrusion.

Servers are cheap, just get new hardware and lock that box down better.

  • 1
    That's not a financially feasible approach, especially when the firmware can simply be re-flashed from a vendor firmware compared against the md5. Also, what if it's a server from a blade chassis? Individual blades can easily be worth $10k+ for just a base model. – stevieb Aug 19 '15 at 19:41
  • @stevieb The fallout from getting hacked is expensive. Servers are cheap in comparison. A company can easily lose / spend $100K + just dealing with the fallout from having to let their customers know "we lost all your data" or even worse losing trade secrets. I think Sony / Target / {insert company name} here would jump at the chance to drop $10K to ensure that their compromised system will be replaced with something that is free of malware. – datUser Aug 19 '15 at 19:53

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.