We’re rewarding the question askers & reputations are being recalculated! Read more.
5 of 10 Better explanation of 'histverify'

man bash, search for section Special Parameters. For _ you'll find:

expands to the last argument to the previous command, after expansion.

So in a command line context the difference is the expansion. For example if I have a variable FOO set to 'bar' and I execute echo $FOO then $_ on the next line will result in bar while !$ will result in $FOO

This behavior won't necessarily be clear by default but becomes obvious if you use shopt -s histverify. With this setting when you use the history expansion feature Bash will not submit the expanded command immediately but display it and wait for you to hit enter again (you may modify it first if required). So we would see something like this after FOO=bar:

$ echo $FOO
bar
$ ls !$
$ ls $FOO  

Then separately...

$ echo $FOO
bar
$ ls $_
$ ls bar

Note that $_ has different meanings in other contexts:

At shell startup, set to the absolute pathname used to invoke the shell or shell script being executed as passed in the environment or argument list.

...and...

Also set to the full pathname used to invoke each command executed and placed in the environment exported to that command.

...and...

When checking mail, this parameter holds the name of the mail file currently being checked.