I have two seperate vps boxes with vultr.com (one is debian and the other centos), I would like to set it up so two subdomains go to each one. Possible? For example, box 1 would be box1.mydomain.com and box 2 would be box2.mydomain.com, but I do not want the url to redirect to the IP address of the vps.
I'm assuming you have a webserver on each sub domain.
Sounds like you want to create two A records in your dns, one for each of your sub domains.
Then you want to configure each box's webserver and set the default domain to each sub domain. ie. box1 config webserver to serve box1.domain.com on box2 conig webserver to serve box2.domain.com