I noticed it when installing gentoo
But I can't figure out of it's possible to get it to work outside of a tty.
I'm using GNOME-Terminal if that matters.
Not really. The library checks if it's running in "xterm" (by checking the
TERM variable), and if so, just assumes that the application knows how to work with the xterm mouse protocol.
With ncurses, gpm is something of a nuisance outside of its use in the Linux console. There are few applications which use it in a terminal emulator.