Daimrod's answer (currently top ranked) doesn't work for me, because EasyPG (epa) isn't an auto-mode, so modifying auto-mode-alist is wrong, at least as of emacs 24.3.1.
Instead, you have to modify the epa-file-name-regexp value to add a pattern to match *.asc files.
(setq epa-file-name-regexp "\\.gpg\\(~\\|\\.~[0-9]+~\\)?\\'\\|\\.asc")
Personally I think all the extra syntax required to match emacs backup files is both unnecessary for almost all purposes as well as unconducive to easy understanding and maintenance, so I left that out of the ".asc" pattern. Add it if you prefer it verbatim as it appears after the ".gpg" pattern.
Note if you do this after emacs has started up and enabled epa, you have to run the epa-file-name-regexp-update function to pick up a changed value. That's unnecessary if you add this code to your .emacs startup file (nor do you need to explicitly enable epa).