25

I'm running a Ubuntu based web server (Apache, MySQL) on a 512MB VPS. This is more than sufficient for the website it is running (small forum).

As I wanted to add some protection against viruses I installed ClamAV and use it to scan uploaded files as part of the upload handling script (PHP).

I'm running the clamav-daemon service so the definitions don't have to be loaded every time a file is scanned. One downside to this practise seems to be the "huge" amount of memory used by clamav-daemon service: >200 MB. This already resulted in the service being forced to stop and the uploads being rejected.

I can simply upgrade the memory of the VPS to 1024MB, but I want to know if there is a way to reduce the memory usage of ClamAV by e.g. not loading unwanted definitions.

14

ClamAV holds the search strings using the classic string (Boyer Moore) and regular expression (Aho Corasick) algorithms. Being algorithms from the 1970s they are extemely memory efficient.

The problem is the huge number of virus signatures. This leads to the algorithms' datastructures growing quite large.

You can't send those datastructures to swap, as there are no parts of the algorithms' datastructures accessed less often than other parts. If you do force pages of them to swap disk, then they'll be referenced moments later and just swap straight back in. (Technically we say "the random access of the datastructure forces the entire datastructure to be in the process's working set of memory".)

The datastructures are needed if you are scanning from the command line or scanning from a daemon.

You can't use just a portion of the virus signatures, as you don't get to choose which viruses you will be sent, and thus can't tell which signatures you will need.

Here's the memory used on a 32-bit machine running Debian Wheezy and it's clamd.

# ps_mem.py 
 Private  +   Shared  =  RAM used   Program
281.7 MiB + 422.5 KiB = 282.1 MiB   clamd

Edit: I see someone suggests setting the resident set size. If this succeeds then having a resident set size less than the working set size will lead to the process thrashing to and from swap. This will lower the entire system performance substantially. In any case the Linux manual page for setrlimit(RLIMIT_RSS, ...) says that setting the resident set size is no longer supported and never had any effect on processes which chose not to call madvise(MADV_WILLNEED, ...).

1

I've hit a similar problem running clamd on a small home NAS box with just 512MB. From a survey of questions across the net it seems that there isn't any way to reduce the memory usage. The database of nasty things just keeps getting bigger and bigger.

It is possible to configure clamav to run in non-daemon mode by installing "clamav" rather than "clamav-daemon". This may allow you to have more memory most of the time. When you scan uploads it's always going to need a large chunk of RAM.

  • 1
    The main downside on this approach is that it will take more time to do the scan. Every time a file is uploaded, the database needs to be loaded before the scan can be done. Loading of the db takes a lot of valuable minutes if you want to have a quick response for the user. Also, when multiple uploads are processed at the same time you will have multiple threads loading the database resulting in even greater RAM usage. Hence the usage of the daemon. My solution was to upgrade my VPS service and pay an extra 5 USD/month for the extra RAM. I can live with this cost increase :) – Niels R. Apr 28 '14 at 7:55
1

Edit 1:
This answer is not verified and it might not work. It also does not answer to how to reduce memeory usage, but to how to limitate memory usage, which is a bit different.


You can edit the clamAV init script (int /etc/init.d/ to add the command ulimit -m amountofram.
It will limitate the possibility of clamAV and you will probably swap which will probably slow down your whole system.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.