Your second code example:
ssh remote-server "print $HOST"
localhost
will be flagged by the great Shellcheck tool with the diagnostic SC2029:
Bash expands all arguments that are not escaped/singlequoted. This means that the problematic code is identical to
ssh host "echo clienthostname"
and will print out the client's hostname, not the server's hostname.
By escaping the $ in $HOSTNAME, it will be transmitted literally and evaluated on the server instead.
By using
ssh host "print \$HOST"
or
ssh host 'print $HOST'
you can prevent your local shell from expanding host and instead have the shell on remote-server
expand it.