Basically, it's the last argument to the previous command.
!$ is the "end" of the previous command. Consider the following
example: We start by looking for a word in a file:
grep -i joe /some/long/directory/structure/user-lists/list-15
if joe is in that userlist, we want to remove him from it. We can either fire up vi with that long directory tree as the argument, or as simply as
vi !$ Which
bash expands to:
(source; handy guide, by the way)