I finally found what was causing the issue. This was due to my router blocking TCP keepalive messages when I connected wirelessly (go figure).
ssh my_server -o TCPKeepAlive=no solved all my problems. Yay!
From the documentation:
Specifies whether the system should send TCP keepalive messages
to the other side. If they are sent, death of the connection or
crash of one of the machines will be properly noticed. However,
this means that connections will die if the route is down tem-
porarily, and some people find it annoying. On the other hand,
if TCP keepalives are not sent, sessions may hang indefinitely on
the server, leaving "ghost" users and consuming server resources.
The default is "yes" (to send TCP keepalive messages), and the
server will notice if the network goes down or the client host
crashes. This avoids infinitely hanging sessions.
To disable TCP keepalive messages, the value should be set to