Q1: do I need to have internet connectivity when trying to set up static networking on a LAN between the VMs?
No you do not require actual network connectivity to do the configuration of the devices, though often times it's easier if you do.
Q2: If I don't need connectivity, do I need to create a separate eth1 adapter on CentOS and a Local Area Connection 2 on Windows?
It typically depends on your virtualization technology (VMware, Virtualbox, KVM, etc.). Each has their own variations on the same general concepts of:
Your VMs can use the host's network interface as a NAT and cannot see other network devices directly, they're only able to talk to the host system.
This works like a private LAN where multiple computers can talk to each other, or to the Internet through a NAT router. Hosts on the other side of the NAT wall can't connect back in through to the hosts on the LAN.
In the VM case, the VM guests can talk to each other or to the outside world through the host NAT, but outside computers cannot connect directly back in to the VM guests.
The VMs can see each other directly on a separate network that's unique to the host system, but otherwise, they're not accessible to the rest of the general network that the host system is a part of.
This is similar to the previous case, except that there is no NAT layer. It works like a purely private LAN, with the VM host enforcing the border.
The VM guests are full members on the network with a MAC address and can be configured either manually or via DHCP with a IP address.
This mode is usually called bridging.