You need to use multiple clauses in your config file entry but in a very specific way. There is a bug in some setups that cause the generally recommended, and simplest, syntax ( "Match Group FOOGROUP User !username" ) to either cause everyone else in the group to fail to Match or lets them escape their chroot jail.
On Debian Jessie using OpenSSH_6.0p1 Debian-4, OpenSSL 1.0.2d I get the result that everyone else in the group can no longer connect. Others report jail breaks. In both cases a syntax of
Match Group FOOGROUP User *,!username
seems to work without side-effects. Some sort of bug in the parser no doubt.