1

Good day, gentlemen.

I have recently faced a problem when a device connected to the FreeBSD server via StrongSwan can't route outside its subnet. Let me explain in details:

FreeBSD 10.0-RELEASE, kernel compiled with (full config: http://pastebin.com/5PQFXqhx):

options IPSEC
options IPSEC_NAT_T
device crypto
device cryptodev

Forwarding turned on in sysctl.conf:

net.inet.ip.forwarding=1
net.inet6.ip6.forwarding=1

We have two interfaces. The first one is 192.168.1.100, the router forwards ports 500 and 4500 from the Internet to this IP (so we can connect from anywhere to the "sweet home" network). The second one is 10.0.0.1 which is used for Samba and other services.

/etc/rc.conf:

hostname="bsdserver"
ifconfig_igb0="inet 192.168.1.100 netmask 255.255.255.0"
ifconfig_igb1="inet 10.0.0.1 netmask 255.255.255.0"
defaultrouter="192.168.1.1"
gateway_enable="YES"
ipv6_gateway_enable="YES"

Network scheme

ipsec.conf:

config setup
    strictcrlpolicy = no
ca bsdserver
    cacert = /usr/local/etc/ipsec.d/cacerts/bsdserver_CA.der
conn android
auto = add
dpdaction = clear
keyexchange = ikev2
esp=aes-aes256-sha-modp1024,aes256-sha512-modp4096
ike=aes-aes256-sha-modp1024,aes256-sha512-modp4096
mobike = yes
left = 192.168.1.100
leftsubnet = 0.0.0.0/0,::0/0
leftauth = pubkey
leftid = "111.111.111.111" # real "white" ip here
leftcert = /usr/local/etc/ipsec.d/certs/111.111.111.111_cert.der
leftfirewall = yes
leftsourceip = %config4,%config6
right = %any
rightsourceip = 10.0.0.3/32
rightauth = pubkey
rightid = "C=XX, ST=XX, L=XX, O=ORG, OU=UNIT, CN=111.111.111.111, E=mail@host.com"
rightcert = /usr/local/etc/ipsec.d/certs/android_cert.der

This is the output from ipsec start --nofork:

Starting strongSwan 5.1.1 IPsec [starter]...
no netkey IPsec stack detected
no KLIPS IPsec stack detected
no known IPsec stack detected, ignoring!
00[DMN] Starting IKE charon daemon (strongSwan 5.1.1, FreeBSD 10.0-RELEASE, amd64)
00[KNL] unable to set UDP_ENCAP: Invalid argument
00[NET] enabling UDP decapsulation for IPv6 on port 4500 failed
00[CFG] loading ca certificates from '/usr/local/etc/ipsec.d/cacerts'
00[CFG]   loaded ca certificate "C=XX, ST=XX, L=XX, O=ORG, OU=UNIT, CN=111.111.111.111, E=mail@host.com" from '/usr/local/etc/ipsec.d/cacerts/UNIT_CA.der'
00[CFG] loading aa certificates from '/usr/local/etc/ipsec.d/aacerts'
00[CFG] loading ocsp signer certificates from '/usr/local/etc/ipsec.d/ocspcerts'
00[CFG] loading attribute certificates from '/usr/local/etc/ipsec.d/acerts'
00[CFG] loading crls from '/usr/local/etc/ipsec.d/crls'
00[CFG] loading secrets from '/usr/local/etc/ipsec.secrets'
00[CFG]   loaded RSA private key from '/usr/local/etc/ipsec.d/private/111.111.111.111_key.der'
00[LIB] loaded plugins: charon test-vectors curl aes des blowfish rc2 sha1 sha2 md4 md5 random nonce x509 revocation constraints pubkey pkcs1 pkcs7 pkcs8 pkcs12 pgp dnskey sshkey pem openssl fips-prf gmp xcbc cmac hmac attr kernel-pfkey kernel-pfroute resolve socket-default stroke updown eap-identity eap-aka eap-aka-3gpp2 eap-md5 eap-mschapv2 eap-dynamic eap-tls eap-ttls eap-peap xauth-generic xauth-eap whitelist addrblock
00[LIB] unable to load 4 plugin features (4 due to unmet dependencies)
00[JOB] spawning 16 worker threads
charon (49667) started after 220 ms
16[CFG] received stroke: add connection 'android'
16[CFG] adding virtXXl IP address pool 10.0.0.3/32
16[CFG] 'android' has both left- and rightsourceip, but IKE can negotiate one virtXXl IP only, ignoring local virtXXl IP
16[CFG]   loaded certificate "C=XX, ST=XX, L=XX, O=ORG, OU=UNIT, CN=111.111.111.111, E=mail@host.com" from '/usr/local/etc/ipsec.d/certs/111.111.111.111_cert.der'
16[CFG]   loaded certificate "C=XX, ST=XX, L=XX, O=ORG, OU=UNIT, CN=111.111.111.111, E=mail@host.com" from '/usr/local/etc/ipsec.d/certs/android_cert.der'
16[CFG] added configuration 'android'
16[NET] received packet: from 22.222.222.22[16483] to 192.168.1.100[500] (660 bytes)
16[ENC] parsed IKE_SA_INIT request 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) ]
16[IKE] 22.222.222.22 is initiating an IKE_SA
16[IKE] IKE_SA (unnamed)[1] state change: CREATED => CONNECTING
16[IKE] local host is behind NAT, sending keep alives
16[IKE] remote host is behind NAT
16[IKE] sending cert request for "C=XX, ST=XX, L=XX, O=ORG, OU=UNIT, CN=111.111.111.111, E=mail@host.com"
16[ENC] generating IKE_SA_INIT response 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) CERTREQ N(MULT_AUTH) ]
16[NET] sending packet: from 192.168.1.100[500] to 22.222.222.22[16483] (337 bytes)
16[NET] received packet: from 22.222.222.22[16549] to 192.168.1.100[4500] (1964 bytes)
16[ENC] parsed IKE_AUTH request 1 [ IDi CERT N(INIT_CONTACT) CERTREQ AUTH CPRQ(ADDR ADDR6 DNS DNS6) N(ESP_TFC_PAD_N) SA TSi TSr N(MOBIKE_SUP) N(NO_ADD_ADDR) N(MULT_AUTH) N(EAP_ONLY) ]
16[IKE] received cert request for "C=XX, ST=XX, L=XX, O=ORG, OU=UNIT, CN=111.111.111.111, E=mail@host.com"
16[IKE] received end entity cert "C=XX, ST=XX, L=XX, O=ORG, OU=UNIT, CN=111.111.111.111, E=mail@host.com"
16[CFG] looking for peer configs matching 192.168.1.100[%any]...22.222.222.22[C=XX, ST=XX, L=XX, O=ORG, OU=UNIT, CN=111.111.111.111, E=mail@host.com]
16[CFG] selected peer config 'android'
16[CFG]   using trusted ca certificate "C=XX, ST=XX, L=XX, O=ORG, OU=UNIT, CN=111.111.111.111, E=mail@host.com"
16[CFG] checking certificate status of "C=XX, ST=XX, L=XX, O=ORG, OU=UNIT, CN=111.111.111.111, E=mail@host.com"
16[CFG] certificate status is not available
16[CFG]   reached self-signed root ca with a path length of 0
16[CFG]   using trusted certificate "C=XX, ST=XX, L=XX, O=ORG, OU=UNIT, CN=111.111.111.111, E=mail@host.com"
16[IKE] authentication of 'C=XX, ST=XX, L=XX, O=ORG, OU=UNIT, CN=111.111.111.111, E=mail@host.com' with RSA signature successful
16[IKE] processing INTERNAL_IP4_ADDRESS attribute
16[IKE] processing INTERNAL_IP6_ADDRESS attribute
16[IKE] processing INTERNAL_IP4_DNS attribute
16[IKE] processing INTERNAL_IP6_DNS attribute
16[IKE] received ESP_TFC_PADDING_NOT_SUPPORTED, not using ESPv3 TFC padding
16[IKE] peer supports MOBIKE
16[IKE] authentication of '111.111.111.111' (myself) with RSA signature successful
16[IKE] IKE_SA android[1] established between 192.168.1.100[111.111.111.111]...22.222.222.22[C=XX, ST=XX, L=XX, O=ORG, OU=UNIT, CN=111.111.111.111, E=mail@host.com]
16[IKE] IKE_SA android[1] state change: CONNECTING => ESTABLISHED
16[IKE] scheduling reauthentication in 9802s
16[IKE] maximum IKE_SA lifetime 10342s
16[IKE] sending end entity cert "C=XX, ST=XX, L=XX, O=ORG, OU=UNIT, CN=111.111.111.111, E=mail@host.com"
16[IKE] peer requested virtXXl IP %any
16[CFG] assigning new lease to 'C=XX, ST=XX, L=XX, O=ORG, OU=UNIT, CN=111.111.111.111, E=mail@host.com'
16[IKE] assigning virtXXl IP 10.0.0.3 to peer 'C=XX, ST=XX, L=XX, O=ORG, OU=UNIT, CN=111.111.111.111, E=mail@host.com'
16[IKE] peer requested virtXXl IP %any6
16[IKE] no virtXXl IP found for %any6 requested by 'C=XX, ST=XX, L=XX, O=ORG, OU=UNIT, CN=111.111.111.111, E=mail@host.com'
16[IKE] CHILD_SA android{1} established with SPIs c11f2b7e_i f7ac6573_o and TS 0.0.0.0/0 ::/0 === 10.0.0.3/32
16[CHD] updown: /usr/local/libexec/ipsec/_updown: iptables: not found
16[CHD] updown: /usr/local/libexec/ipsec/_updown: iptables: not found
16[ENC] generating IKE_AUTH response 1 [ IDr CERT AUTH CPRP(ADDR) N(ESP_TFC_PAD_N) SA TSi TSr N(AUTH_LFT) N(MOBIKE_SUP) N(ADD_4_ADDR) ]
16[NET] sending packet: from 192.168.1.100[4500] to 22.222.222.22[16549] (1596 bytes)

Looking at the tcpdump output: 10.0.0.3 is an IP that Android client got assigned by StrongSwan.

DNS:

22:21:43.971797 IP 10.0.0.3.55123 > ns1.kyivstar.net.domain: 41221+ A? www.cdn.viber.com. (35)
22:26:00.484696 IP 10.0.0.3.25462 > gi-dns1-kie3.kyivstar.net.domain: 436+ A? mtalk.google.com. (34)
22:25:59.225291 IP 10.0.0.3.53999 > ns1.kyivstar.net.domain: 56585+ A? graph.facebook.com. (36)

OK, pinging "bsdserver" works:

22:26:06.460464 IP 22-222-222-22-gprs.kyivstar.net.16445 > bsdserver.local.sae-urn: UDP-encap: ESP(spi=0xc289c560,seq=0x96), length 132
22:26:06.460749 IP bsdserver.local.sae-urn > 46-211-183-74-gprs.kyivstar.net.16445: UDP-encap: ESP(spi=0xe77a2567,seq=0x1f), length 132

But pinging 8.8.8.8 doesn't:

22:30:51.179070 IP 10.0.0.3 > google-public-dns-a.google.com: ICMP echo request, id 3, seq 6, length 64
22:30:52.098315 IP 22-222-222-22-gprs.kyivstar.net.16445 > bsdserver.local.sae-urn: UDP-encap: ESP(spi=0xc289c560,seq=0xce), length 132
22:30:52.098825 IP 10.0.0.3 > google-public-dns-a.google.com: ICMP echo request, id 3, seq 7, length 64
22:30:52.754209 STP 802.1d, Config, Flags [none], bridge-id 8000.b0:48:7a:c2:f2:92.8001, length 43
22:30:53.078436 IP 22-222-222-22-gprs.kyivstar.net.16445 > bsdserver.local.sae-urn: UDP-encap: ESP(spi=0xc289c560,seq=0xcf), length 132
22:30:53.079070 IP 10.0.0.3 > google-public-dns-a.google.com: ICMP echo request, id 3, seq 8, length 64
22:30:53.358693 IP 22-222-222-22-gprs.kyivstar.net.16445 > bsdserver.local.sae-urn: isakmp-nat-keep-alive
22:30:53.738572 IP 22-222-222-22-gprs.kyivstar.net.16445 > bsdserver.local.sae-urn: UDP-encap: ESP(spi=0xc289c560,seq=0xd0), length 116
22:30:53.739164 IP 10.0.0.3.25984 > gi-dns1-kie3.kyivstar.net.domain: 1638+ A? www.cdn.viber.com. (35)
22:30:54.098740 IP 22-222-222-22-gprs.kyivstar.net.16445 > bsdserver.local.sae-urn: UDP-encap: ESP(spi=0xc289c560,seq=0xd1), length 132
22:30:54.099248 IP 10.0.0.3 > google-public-dns-a.google.com: ICMP echo request, id 3, seq 9, length 64
22:30:54.754083 STP 802.1d, Config, Flags [none], bridge-id 8000.b0:48:7a:c2:f2:92.8001, length 43

And routing table from Android device:

Android route

I feel like problem isn't exactly IPsec related. Any bright ideas about possible solution?

  • Did you already read Forwarding and Split-Tunneling on the strongSwan wiki? – ecdsa May 2 '14 at 7:07
  • @ecdsa, thanks, I did. In FreeBSD 10 there is no net.ipv4.ip_forward=1, but instead forwarding is enabled via net.inet.ip.forwarding=1 net.inet6.ip6.forwarding=1. Or you are pointing out to iptables? Then I have no one because host is FreeBSD :) If you know that PF should be adjusted someway, please let us know. – asv May 2 '14 at 8:22
  • Sure, I was more referring to the sentiment of the page. For instance, that you need some kind of NAT to provide internet access to clients with a virtual IP. – ecdsa May 2 '14 at 16:34
  • @ecdsa, posted the same question on the freebsd.org community forum. Interesting, I can successfully ping -S 10.0.0.1 8.8.8.8 from FreeBSD, but from Android reaching anything further server (192.168.1.100 or 10.0.0.1) is impossible. – asv May 3 '14 at 22:50
  • Update: I have installed squid and can confirm that setting android client to use 10.0.0.1 as proxy works. Without it - no news, packets failed to reach something further than FreeBSD server. – asv May 4 '14 at 20:10
-1

you may have related to the bug: ipfw not matching incoming packets decapsulating ipsec

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.