As well as the answer given by Daniel, you can also simply turn off history expansion altogether if you don't use it with:
set +o histexpand
(the latter also working in
histexpand is an alternative name for the
banghist option for bash compatibility).
In (t)csh where history expansion originates, you'd disable it by assigning the empty string to the
$histchars variable with:
set histchars = ''
That also works in bash and zsh, the two shells that copied that feature, though the syntax is:
bash (contrary to (t)csh or zsh) does not ignore a
$histchars variable found in the environment, so beware of the possible consequences if you export that variable.
"foo!"not invoke history expansion, but