I have a small SBC running in my apartment that I'm using for backups from my cloud located elsewhere. Each night, my cloud connects to my SBC via SFTP to create a new backup timestamp of my data.
Since this means I need to expose an SSH port to the internet, I want SSH configuration to be very secure. It will only allow one specific user to log in, only permitting certificate based login and so on. I will also change the port so that I don't have to forward port 22 requests from the internet to my home network.
However, when I'm at home, I want to be able to login into my SBC via SSH using a different account (that is able to sudo) and a password using the regular port 22. I want this to only be allowed from a very specific ip address that is in the private ip address block (192.168.1.10). If I ever need to access my SBC from somewhere else, I can log into my router using a VPN, which will then assign me the correct local IP address inside my home network to connect to my SBC.
Now, I know how to configure these things in sshd_config, but what I don't know is how to have these different policies to be applied to specific ports. I know I can use the Host directive, but afaik this only affects hostnames, and not ports.