Identifying running programs that are using the old version of a library I just replaced

Question

After installing updates to address CVE-2014-0160 (the OpenSSL Heartbleed bug), I had to take care to restart anything that might be using libssl -- many services, such as Apache and my VPN software, still had the old vulnerable libssl loaded up, and my package manager made no attempt to rectify this.

This got me thinking: After I update a shared library, how can I reliably find out which running programs currently have an old version of the library linked in? I am sure there must be a way to interrogate running processes either at the linker level or at the file descriptors level to determine whether the instance of a given shared library they have loaded is the same as the one currently on disk.

Answer 1

I found two ways to do this:

1. Debian-specific, lists most deleted/replaced files held by processes (with the exception of certain files known to be transient, e.g. stuff in /tmp): The debian-goodies package contains checkrestart, which accomplishes something like what I've described by scraping the output of lsof to find open files that are gone or replaced on disk. It identifies the processes in question and (if possible) the package to which they belong and any init script that can be used to restart them. The -v option will identify the files concerned.

2. Generic, manual, allows specifying the file you're worried about: You can look at the output of lsof to identify open file handles to deleted or replaced files. In the output of lsof -nnP, such a file appears to be identified by DEL in the fourth column. You can do something like lsof -nnP | grep DEL.*libssl.so to look for stale handles to a particular library (OpenSSL, in this case). This is probably highly dependent on the specific version of lsof you use and the behavior of your package manager, so proceed with caution.

   pluto      3592       root  DEL       REG      202,0               98831 /lib/i386-linux-gnu/libssl.so.1.0.0
   pluto      3604       root  DEL       REG      202,0               98831 /lib/i386-linux-gnu/libssl.so.1.0.0
   

Answer 2

A quick and dirty way on Linux (thanks to Lekensteyn):

grep '/usr/lib/libssl1.*(deleted)' /proc/*/maps

For precise parsing, you can call lsof with the -F option to get parsable output. Include the f field to filter on deleted files (fDEL) and the n field to get the path to the file. Note that the snippet below chokes on file names containing newlines.

lsof -F pfn | awk '
    /^p/ {pid=substr($0,2)}
    /^fDEL$/ {getline; if (/n\/usr\/lib\/libssl1\.0\.1.*(deleted)$/) print pid}
'