According to these docs, sshfs doesn't support --no-recursive, --folder-exclude, maxdepth, or anything in the same vein (i.e. limit what's being mounted).
What you could try, if it's feasible for your use case, is make a selective clone of the remote dir on the remote machine, and mount that instead.
This question started with trying to authenticate sshfs with an OpenPGP smartcard, then was edited to its current form when I thought authentication was solved.
My current problem was using -o SmartcardDevice=/dev/hidraw7, which started because otherwise I was getting a password prompt from the remote host. (Users have passwords disabled and are setup for ...