recovering deleted file held open by apache?

Question

Suppose an apache log file gets deleted but it's held open by apache; then this is what I am doing:

pid=$(lsof | grep text.txt | awk '/deleted/ {print $2}')
fd=$(lsof | grep text.txt | awk '/deleted/ {print $4}' | grep -oE "[[:digit:]]{1,}")

cp /proc/$pid/fd/$fd directorytobecopied/testfile.txt

This is what I am doing to recover the file and put it back where it was. Is there any simpler way of doing this because above code doesn't look good. Furthermore how do I know from where the file was deleted(directorytobecopied) so that I don't have to manually ask somebody where the file was located originialy and put it back there.

Answer 1

If a file has been deleted but is still open, that means the file still exists in the filesystem (it has an inode) but has a hard link count of 0. Since there is no link to the file, you cannot open it by name. There is no facility to open a file by inode either.

There is no way to discover the file through its filesystem, and especially no way to look for the file in the directory where it last was. The directory entry is gone. All that remains is the file itself. You can get to the file with a filesystem debugger, but that requires root permissions and is hard to use and error-prone.

Linux exposes open files through special symbolic links under /proc. These links are called /proc/12345/fd/42 where 12345 is the PID of a process and 42 is the number of a file descriptor in that process. A program running as the same user as that process can access the file (the read/write/execute permissions are the same you had as when the file was deleted).

The name under which the file was opened is still visible in the target of the symbolic link: if the file was /var/log/apache/foo.log, then the target of the link is /var/log/apache/foo.log (deleted). (If the file was renamed after it was opened, the target of the symlink may reflect the renaming.)

Thus you can recover the content of an open deleted file given the PID of a process that has it open and the descriptor that it's opened on like this:

recover_open_deleted_file () {
  old_name=$(readlink "$1")
  case "$old_name" in
    *' (deleted)')
      old_name=${old_name%' (deleted)'}
      if [ -e "$old_name" ]; then
        new_name=$(TMPDIR=${old_name%/*} mktemp)
        echo "$oldname has been replaced, recovering content to $new_name"
      else
        new_name="$old_name"
      fi
      cat <"$1" >"$new_name";;
    *) echo "File is not deleted, doing nothing";;
  esac
}
recover_open_deleted_file "/proc/$pid/fd/$fd"

If you only know the process ID but not the descriptor, you can recover all files with

for x in /proc/$pid/fd/*; do
  recover_open_deleted_file "$x"
done

If you don't know the process ID either, you can search among all processes:

for x in /proc/[1-9]*/fd/*; do
  case $(readlink "$x") in
    /var/log/apache/*) recover_open_deleted_file "$x";;
  esac
done

You can also obtain this list by parsing the output of lsof, but it isn't simpler nor more reliable nor more portable (this is Linux-specific anyhow).