Your issue seems to be that you're starting a new invocation of the agent on each shell initialization (probably in your
.bashrc). This is not necessary, so you should find those ssh-agent calls and remove them.
On OSX, there is usually one ssh-agent started at boot time for your desktop session. For example:
myhost:~)-> ps -ef | grep ssh-agent
501 986 1 0 10:18AM ?? 0:00.14 /usr/bin/ssh-agent -l
501 2126 736 0 5:30PM ttys007 0:00.00 grep ssh-agent
In order to access it, you need to find out where it's listening socket is, usually in
<blah> is some random alpha-numeric identifier. There are likely to be several, but you want the one that has the 'Listeners' socket in it).
Once you find it's socket, you can connect to it, using the standard SSH environment variable,
SSH_AUTH_SOCK. For example:
myhost:~)-> export SSH_AUTH_SOCK=/tmp/com.apple.launchd.1h2j3k4l5/Listeners
myhost:~)-> ssh-add ~/.ssh/id_rsa
Enter passphrase for ~/.ssh/id_rsa: <typing_here>
Identity added: ~/.ssh/id_rsa (~/.ssh/id_rsa)
Once this is exported to the shell, one can, at this point, use the
ssh-add command to provide his or her key passphrase and add a key to the agent. You can also see other options you might want to explore in the manual pages for the ssh-agent.