To my knowledge only more expensive cards made specifically for virtualization support this properly. So be prepared for very strange problems including complete freezes as the card freaks out in a weird state. (Even if you find a way.)
And used to forbid Nvidia even forbids it on other cards. Which is why last time I checked you needed some hacks to hide that. Nividia only recently changed their stance on this:
https://nvidia.custhelp.com/app/answers/detail/a_id/5173/~/geforce-gpu-passthrough-for-windows-virtual-machine-%28beta%29
"Enabled" does not equal "Supported"
I currently have a pass through setup running and have been using it regularly for a few years now. And my experience tells me: You are in for a world of pain. Those other cards cost a lot more. So Nvidia isn't exactly incentivized to support this.
That said, I hope you succeed. I just wanted to tell you: Don't hold your breath you are working against the odds. Be prepared for heavy tinkering. Good luck!
Update:
Mind sharing your setup?
Mainboard and GPU would be most interesting.
modprobe -r vfio-pci && modprobe nvidia
not work?vfio-pci.ids=10de:1c03
on your kernel cmd line), and IIRC you can't change that at run time. Have you considered doing your video encoding in a VM? either your existing VM or maybe a new Linux VM (which will probably be easier to script the transcode than windows)...just don't run both windows and linux vm using the gpu at the same time.