I'm using ssh-agent for caching private keys; it's working well before connecting remote PC its not asking for a PASSPHRASE but its asking remote user password. Ex:
host$ sudo ssh-agent bash
sudo ssh localhost2.localdomain
after that its not asking passphrase its asking only remote user password.