I don't have an issue with how pacman handles caching 99% of the time. However, occasionally there is a package which is both large and also not necessary to cache.
For example, I have
wesnoth which is 418 MB in the cache. 4 versions of this are cached for a total of ~2 GB. Fun as wesnoth is, it's not really critical for my system's operation and if a new release breaks it, so be it, I'll just wait until a fix comes out rather than rolling back. It makes no sense to waste all that space to cache it.
Would it be bad to simply go into /var/cache/pacman/pkg and delete the wesnoth files manually?