4

Cloud IAP is a sort of proxy for Google Cloud Platform that lets you connect to compute instances that don't have public IP addresses, without a VPN. You stand up an instance and then you can use the gcloud utility to connect to it by name like so: gcloud compute ssh my-server-01. This handles authenticating you through the proxy and logging you into the target server with your own Google account (using a feature called OS Login).

I figure to make ansible do what the gcloud tool is doing I would need a custom connection plugin.

1
4

I figured out a way to make this work without a connection plugin. Basically you can write a script that wraps the gcloud tool and point the ansible_ssh_executable parameter at this script, which you can define at the inventory level. You do need to make sure the gcp_compute inventory plugin identifies hosts by name, because this is what gcloud compute ssh expects.

Here's the script:

#!/bin/sh
set -o errexit
# Wraps the gcloud utility to enable connecting to instances which are behind
# GCP Cloud IAP. Used by setting the `ansible_ssh_executable` setting for a play
# or inventory. Parses out the relevant information from Ansible's call to the
# script and injects into the right places of the gcloud utility.

arg_string="$@"

grep_hostname_regex='[a-z]*[0-9]\{2\}\(live\|test\)'
sed_hostname_regex='[a-z]*[0-9]{2}(live|test)'

target_host=$(
  echo "$arg_string\c" | grep -o "$grep_hostname_regex"
)

ssh_args=$(
  echo "$arg_string\c" | sed -E "s# ${sed_hostname_regex}.*##"
)

cmd=$(
  echo "$arg_string\c" | sed -E "s#.*${sed_hostname_regex} ##"
)

gcloud compute ssh "$target_host" \
  --command="$cmd" \
  --tunnel-through-iap \
  -- $ssh_args

Note:

  • This is tested on macOS. The sed options might be different on Linux, for example.
  • The "host" regexes will need to fit your naming convention. If you don't have a consistent naming convention that would work like it does for me you'll need to find some other way to parse out the information.
1
  • 1
    Thanks a lot, this was very useful! – hallvors Feb 11 '20 at 11:12
4

After discussion on https://www.reddit.com/r/ansible/comments/e9ve5q/ansible_slow_as_a_hell_with_gcp_iap_any_way_to/ I altered solution to use an SSH connection sharing via socket.

It is two times faster then @mat solution. I put it on our PROD. Here is an implementation that doesn't depend on host name patterns!

The proper solution is to use Bastion/Jump host because gcloud command still spawns Python interpreter that spawns ssh - it is still inefficient!

ansible.cfg:

[ssh_connection]
pipelining = True
ssh_executable = misc/gssh.sh
ssh_args =
transfer_method = piped

[privilege_escalation]
become = True
become_method = sudo

[defaults]
interpreter_python = /usr/bin/python
gathering = False
# Somehow important to enable parallel execution...
strategy = free

gssh.sh:

#!/bin/bash

# ansible/ansible/lib/ansible/plugins/connection/ssh.py
# exec_command(self, cmd, in_data=None, sudoable=True) calls _build_command(self, binary, *other_args) as:
#   args = (ssh_executable, self.host, cmd)
#   cmd = self._build_command(*args)
# So "host" is next to the last, cmd is the last argument of ssh command.

host="${@: -2: 1}"
cmd="${@: -1: 1}"

# ControlMaster=auto & ControlPath=... speedup Ansible execution 2 times.
socket="/tmp/ansible-ssh-${host}-22-iap"

gcloud_args="
--tunnel-through-iap
--zone=europe-west1-b
--quiet
--no-user-output-enabled
--
-C
-o ControlMaster=auto
-o ControlPersist=20
-o PreferredAuthentications=publickey
-o KbdInteractiveAuthentication=no
-o PasswordAuthentication=no
-o ConnectTimeout=20"

exec gcloud compute ssh "$host" $gcloud_args -o ControlPath="$socket" "$cmd"

UPDATE There is response from Google engineer that gcloud aren't supposed to be called in parallel! See "gcloud compute ssh" can't be used in parallel

Experiments were shown that with Ansible fork=5 I almost always hit an error. With fork=2 I've never experienced one.

UPDATE 2 Time passed and as of end of 2020 I can run gcloud compute ssh in parallel (in WSL I did fork = 10) without locking errors.

1
  • 1
    I took a slightly different approach for handling the argument parsing by taking passing all but the last two arguments as --ssh-flag options: gist.github.com/acdha/… I also opened an issue about requiring the kludge to re-specify the zone since there doesn't appear to be a great way to pull that from the Ansible inventory and it's annoying to have to repeat a value the API returns: issuetracker.google.com/u/1/issues/157498448 – Chris Adams May 27 '20 at 0:28
1

Thanks for posting these @gavenkoa and @matt. One suggestion is to add the following so you don't need to hard code the zone.

Snippet:

ZONE=$(gcloud compute instances list --filter="name:${host}" --format='value(zone)')

gcloud_args="
--tunnel-through-iap \
--zone=${ZONE}
2
  • 1
    Cool. One note that extra command might introduce annoying delays. I work with Windows WSL env, it is noticeable here (( Also you can replace gcloud compute instances list with gcloud compute instances describe $host --format='value(zone)'. For me that commands adds 2.6s delay. Ansible calls SSH client for each task / each host. 10 hosts * 10 tasks give 260s of delay. I would keep that info in the inventory or try to load it with docs.ansible.com/ansible/latest/collections/google/cloud/… (service secret key is required in that case) – gavenkoa Dec 19 '20 at 20:00
  • I tried gcloud compute instances describe $host --format='value(zone)' but it fails with Could not detect which major revision of yum is in use, which is required to determine module backend.. – Stefano Pirrello Dec 21 '20 at 14:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.