Combine the two answers from jfgagne and Rhaul Patil
Routing happens in two directions. Your route to the destination, and the destinations route to you.
If your lookups are coming from real world IP's or the local routing is what you are testing you need to have your local routes set and make the request come from the correct IP address
For example, If you run a lookup on a name server on ISP2, from a source adress of IP1 owned by ISP1. ISP2 will normally route data to IP1 out via WAN links to ISP1 who will route the data to you (unless you have manual routes added at each ISP or are doing something advanced like BGP advertisements)
isp1 - isp2
You should already have more general routes like @jfgagne suggested set for each ISP's network ranges so data for an ISP goes via the quickest route, if not the more specific ones for the name server can be added
ip route add ns.ip.isp.1/32 via gw.ip.isp.1
ip route add ns.ip.isp.2/32 via gw.ip.isp.2
Then you can get
dig to use a source address with
dig @ns.ip.isp.1 -b ur.ip.isp.1 whatever.com
dig @ns.ip.isp.2 -b ur.ip.isp.2 whatever.com
If you are running the lookup from inside two NAT gateways the source IP will be taken care of on the gateway and you don't need to worry about setting the source address of the request. Just set the routing to the gateway you want.
If you want to affect routing specifically for these name servers but not change routing generally you're heading into
iptables and policy routing which is a whole other question.