I have my ssh port changed to XXX. Then I did
ufw limit XXX/tcp
However, when I try to login, after 5/6 failed attempts I get thrown back to prompt, but I can try again. Is that supposed to be the way it works?
If I understand your description correctly, you're typing