The following command will list all packages which have ever been removed (or purged), as far back as
apt's history allows:
zgrep -E '^(Remove:|Purge)' /var/log/apt/history.log*
This does not distinguish automatic removals from explicit removals, but with a little work you should be able to reconstruct that information.
If you always use the command line, then
zgrep -E ^Commandline /var/log/apt/history.log*
will show you all the commands you issued, which will include explicit removals. You could filter it to only list
purge, but that won't include other forms of removals.
If you remove packages using
dpkg, that won't appear in
apt's logs; you'd need to look at
/var/log/dpkg.log and search for