I thought they were completely different. But I just noticed, running
Screen Sharing on MacOS that the window titlebar says "pitosalas's X desktop". I thought Screen Sharing on MacOS is a VNC client and so, I am confused.
Not sure what your exact setup on your Mac and possible the other machine is, but:
On Linux, you run xvnc or a similar program, which is both an X server (for X clients, i.e. your desktop apps, to connect to) and a VNC server (for other VNC clients which want to see your screen to connect to). What it shares is a virtual framebuffer.
On a Mac, there's XQuartz, which is also an X server for Mac applications which need to connect one. It's possible that this includes something similar to xvnc when screen sharing is active.
So whatever your concrete situation is, it's very likely there's an X server involved somewhere, which shares its framebuffer via VNC.
And yes, the VNC protocol and X servers are indeed completely different.