ControlMaster, and I wasn't actually disconnecting.
ControlMaster is an SSH configuration option that keeps connections open for a certain time, and can multiplex SSH sessions over the same connection (which avoids key exchanges, which are slow). However, if you ^D from a shell, and then re-run
ssh, you've not killed the original connection.
sshd only restarts the listening process: any in-progress session remain alive, so that doesn't restart the connection either.
Apparently launching a new shell doesn't re-check
/etc/passwd for changes.
Solution was just to kill the connection:
ssh <hostname> -O exit, and log in again.