Restrict the commands that can be invoked by the key
If an SSH key is going to be used by any kind of automated or unattended task, you should restrict what commands it is able to execute on a remote machine, no matter what decision you make about how and where to store the key.
Use something like this in
command="/usr/sbin/the-backup-daemon" ssh-rsa AAAAAAAAAAAABBBBBXXXXXX.....
That way, at least the key should not be able to, as you say, wreak havoc. It can only access what it's supposed to access, etc... It can most likely still do damage, but it should have less than full access to the remote system.
You can also restrict the IP addresses that are allowed to connect using that key and disable a bunch of other SSH features like port forwarding for connections where that key is used:
from="10.1.2.3",no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty,command="/usr/sbin/the-backup-daemon" ssh-rsa AAAAAAAAAAAAXXXXXX.....
All that has to go on a single line in
Protecting the key
It depends on what your threat model is.
If you're worried about the key being stolen while "cold", for example, having the computer where it is saved physically stolen, then you won't want to save it without a passphrase in that location.
You could start a kind of background SSH agent manually after the server boots, add the key to that agent, and record the agent's
$SSH_AUTH_SOCK for future use by the cron job, but honestly that sounds like more trouble than it's worth. You might as well just store the unencrypted key in a
tmpfs filesystem and have the cron job access it from there. Either way the key lives in memory only (if you have no swap or encrypted swap). Of course you should
chmod the file so only the target user can access it.
Then again, if you're worried about that, you've probably already set up this computer with an encrypted root filesystem and swap (e.g. luks) so you may not need to worry about it as such.
If you're worried about the key being stolen while "hot" (loaded in memory) then there's not much you can do about that. If the cron job can access it, then so can something else that has managed to gain the same access. It's that, or give up the convenience of unattended job execution.
In conclusion, you should treat a backup server as a very privileged system since it will, by necessity, be given read-only access to the complete filesystems of all the computers it backs up. Your backup server should not be accessible from the Internet, for example.