If you start the X server, you will always switch to the VT where it is started. You cannot start X on VT 1, but stay on VT 2, you will switch to VT 1. You can pass the
-novtswitch option which affects what happens when the X server is stopped. With this option, stopping VT is done without the VT switch.
Why is this not possible to start the X server without switching VT?