8

Let's say I have a set of machines (called here the customers' machines) that only a small list of people (called the support staff) is allowed to SSH into, using only one account by machine (the support access account).

The support staff are only supposed to log into the customers' machines using keys. Moreover, the support staff can evolve, so someone who leaves the support staff is not permitted to log in to any customer machine. Therefore, staff people are prohibited from reading the private keys used to log into the customers' machines. Also, it is forbidden to modify the authorized_keys file on the customers' machines.

To realize that configuration, I had the idea to use an SSH proxy that the support staff will log into (with LDAP authentication, but that's another problem) and that contains the private keys.

The question is: How do I allow support staff to use the SSH private key without being able to read it?

I believe that I have to make a daemon running as root on the proxy machine that will accept a user's request and open an SSH session for them, but I have no idea how to do it. Any ideas?

  • Edited the question because "clients machines" can be hard to understand (doesn't mean the opposite of server). – Raspbeguy Sep 3 '15 at 12:02
  • That's an awful lot of bold, and most of it seems extraneous. It's distracting. Do you think in the future you could use a whole lot less bold? Thank you. – D.W. Sep 3 '15 at 21:13
  • I assume that the authorized_keys restriction only refers to dynamic changes (eg. when a new support member joins/leaves), since you need an initial configuration in the customer machine. – Ángel Sep 3 '15 at 22:10
6

I would suggest a couple of options.

  1. Protect the ssh key and require the use of sudo on your support team's side. You could do this transparently with a wrapper. Call the wrapper, say, /usr/local/bin/ssh-support and have it contain something like this (untested):

    #!/bin/bash
    export PATH=/usr/local/bin:/usr/bin:/bin
    export IFS=$' \t\n'
    export SUSER=ssupport
    
    # Restart if not running under sudo
    test "X$1" != 'X-S' && exec sudo -u "$SUSER" /usr/local/bin/ssh-support -S "$@"
    shift
    export SHOME="$(getent passwd "$SUSER" | cut -d: -f6)"
    
    # Extract single argument as hostname LABEL and validate that we have
    # an RSA private key for it. The target username, real hostname, port,
    # etc. can be defined in ~/.ssh/config for the user $SUSER (ssupport)
    label="$1"
    idfile="$SUSER/.ssh/id_rsa_for_$label"
    cgfile="$SUSER/.ssh/config"
    
    ok=true
    [[ "$label" =~ '/' ]] && { echo "Invalid label: $label" >&2; ok=; }
    [[ ! -s "$idfile" ]] && { echo "Missing identity file: $idfile" >&2; ok=; }
    [[ ! -s "$cgfile" ]] && { echo "Missing configuration file: $cgfile" >&2; ok=; }
    
    if test -n "$ok"
    then
        logger -t ssh-support "$SUDO_USER requested ssh to $label"
        exec ssh -i "$idfile" -F "$cgfile" "$label"
    fi
    exit 1
    

    This would require an entry in the sudoers file that permitted users in the support group to use the tool. This command allows them to run the ssh-support tool as the ssupport user - which you must create. It does not confer any root privilege.

    %support ALL = (ssupport) /usr/local/bin/ssh-support
    

    If you are happy that the support users should not need to provider their own password to run the tool (as requested by the sudo invocation within the script itself) you can amend the sudoers definition thus:

    %support ALL = (ssupport) NOPASSWD: /usr/local/bin/ssh-support
    

    Assuming PATH contained /usr/local/bin/ you would then call it with ssh-support clientname. Also assuming you had created the ssupport user as /home/ssupport you would create /home/ssupport/.ssh/id_rsa_clientname and /home/ssupport/.ssh/id_rsa_clientname.pub as the certificate pair, and have a host entry in /home/ssupport/.ssh/config for clientname that defined the user, host, port, etc. for the target machine. You would probably disable X11 forwarding, port forwarding, etc. explicitly. As usual, the /home/ssupport/.ssh directory would need to be protected with permissions 0700.

  2. Give each member of support their own local user account, and have each person use their own private ssh key to access the client's servers. When a person leaves the support group you remove their ssh key from the client's servers. This means that you no longer need to worry about preventing your staff from knowing the private ssh key.

  • The second option isn't valid. As I said there is only one support account on the clients servers and you cannot add public keys to the ssh configuration, that's the rules. In fact, you cannot modify anything about ssh configuration on the clients machines. – Raspbeguy Sep 3 '15 at 11:54
  • 4
    By the way, this answer is a typical example of (ab)using sudo. I wrote an article on the topic (dmitry.khlebnikov.net/2015/07/…), it seems that there is a trend to try to solve anything using sudo. However, this only weakens the security of your systems and doesn't improve it. – galaxy Sep 3 '15 at 12:06
  • 1
    @Raspbeguy of course they won't. The sudoers line limits access to the single command – roaima Sep 3 '15 at 13:36
  • 1
    You pass unchecked options to ssh with "$@" which runs as root! I can think of different ways to ether corrupt system files (use -E to append), forward privileged ports (-L,-R,-D) or simply gain root (-o PermitLocalCommand=yes -o 'LocalCommand=/bin/bash'. The comment from galaxy about abusing sudo is on the spot correct! – nkms Sep 3 '15 at 19:05
  • 1
    @roaima: So anyway, your sudo to root idea has problems, but they're different problems than the ones galaxy's blog post talked about. He's talking about ssh root@somewhere with keys, vs. ssh user@somewhere then sudo. It's actually a really good point. However, the only way it's applicable to this case is that ssh keyowner@localhost ssh_to_client client.example.org is an alternative to sudo -u keyowner ssh_to_client client.example.org. Similar to sudo, SSH can limit the commands a user is allowed to run. We're talking about passwordless sudo to non-root, not Galaxy's use-case. – Peter Cordes Sep 3 '15 at 20:32
11

What you really want to do is to use SSH CA and sign keys used by each support person (they should have their own ssh keys, like passports) and configure your clients' servers to use the TrustedUserCAKeys /etc/ssh/users_ca.pub in the /etc/ssh/sshd_config. This way the server will accept any key signed by the CA key (which you have access to) and you will be able to revoke keys of people who are no longer in support without even touching authorized_keys.

A quick search for "ssh ca" pointed to this tutorial: https://www.digitalocean.com/community/tutorials/how-to-create-an-ssh-ca-to-validate-hosts-and-clients-with-ubuntu (scroll down to "How To Configure User Keys") - although the tutorial mentions Ubuntu it's distribution independent, but you need a fresh version of OpenSSH that supports SSH CA

Another good blog entry on the topic is https://ef.gy/hardening-ssh (scroll down to "SSH certificates").

Pay particular attention that you can sign the key to be valid for a limited time, so they will automatically expire!

  • It would be a good idea if the customers' machines had access to the internet (to update the validity of a signature), which isn't always the case. We connect to these machines via a VPN provided by the customer. – Raspbeguy Sep 3 '15 at 12:14
  • Well, this is as close as you can get without leaking a key with a person who leaves your company. I provided you with the mechanism, you can automate it, e.g. you can have a duty officer who is supporting your HR when people leave company and you can define a procedure on how to revoke keys. – galaxy Sep 3 '15 at 12:19
  • 1
    Moreover, you can sign a key (support person's one) for a limited time - e.g. you can sign it just for 8 hours (for their shift), after 8 hours it will expire and the server won't let them in. – galaxy Sep 3 '15 at 12:20
  • 3
    Regarding your LDAP part - I'm currently working at a company which manages hundreds of instances and we are working on an open source solution to integrate SAML 2.0 and SSH CA. The idea is that a person authenticates using SSO and if authorised gets their keys signed for a defined period of time (e.g. 5 minutes). All of this is quite transparent to the user and is implemented using ~/.ssh/config's ProxyCommand . – galaxy Sep 3 '15 at 12:31
2

There are a couple different answers involving a wrapper script for ssh invoked through sudo or setuid-executable (to some special-purpose non-root account). As nkms says, passing through all args to ssh lets the user do arbitrary things with the ssh keys we're trying to protect. The other extreme some have come up with is to allow only a hostname.

The OP says admins need to be able to upload things.

You could have two different fixed-args-to-ssh wrappers. One for a login shell, another for scp. No user-supplied args other than hostname (and filename to upload).

Or a wrapper script that uses getopt itself to parse very limited options, and plug things into a mostly-fixed ssh command. Ignoring (or erroring) on unrecognized options would be the way to go.

Don't try to filter out "dangerous" ssh options, just write a wrapper that can do two things: interactive login, or upload a file (local and remote filenames). You still need to do some sanitizing there, I guess.

Actually, this is still easy to get wrong. You have to find a way to stop the user from giving the file holding the ssh keys as the local file. So you're still trying to think of everything that needs to be disallowed, instead of starting from not allowing anything. It would be a start to make sure the filenames don't contain any @ or :.

  • The wrapper idea is the solution indeed, but that implies a security breach that needs to be filled. – Raspbeguy Sep 4 '15 at 9:59
  • I'm amazed how people are re-inventing the wheel when there is no need for it. SSH CA functionality was introduced to OpenSSH exactly for the reasons and situation like the original question. My solution didn't use any wrappers and achive exactly what was asked, yet, for some reason @Raspbeguy decided to go the DIY way with questionable security implications. :) – galaxy Sep 4 '15 at 11:49
  • @galaxy: yeah, your SSH CA solution is the best one for sure, unless you have an absolute requirement that you can't make a one-time change to the remote systems to config their sshd to use that. It sounds perfect, and easy to get right. – Peter Cordes Sep 4 '15 at 11:53
  • @galaxy: your solution is really elegant, but not applicable in my case as I told you earlier. I really like the idea, I spoke about it to my boss but it is still impossible. No need to get upset. – Raspbeguy Sep 4 '15 at 15:38
  • 1
    @Raspbeguy, I'm not upset, I just don't understand why your company wants to sacrifice their security and implement something using "duct tape" when there is a proper way to do it. And your comment was that it would be a good solution, but you didn't like that you could not automate the revocation -- and I provided you with the solution to this as well (by signing keys for the shift only). Anyway, you chose your answer, let's move on :) – galaxy Sep 4 '15 at 16:15
1

If you set up an SSH proxy server, you could arrange that your staff users' shell (in /etc/passwd) is not set to a shell such as bash, but instead to a simple script that doesn't allow shell access. Instead, it would ask for a target hostname (read target), then exec ssh "support@$target".

Note that using a proxy like this might make it difficult to use tools such as scp to transfer files to/from the customer machines. This may be a problem or an advantage!

  • This is wrapper-script idea is similar to @roaima's idea. You're giving access to it with ssh, but sudo also works. His answer is suggesting using the local root user, but a non-root user would work, too. See my comments on his answer for discussion of that. (My ssh keyowner@localhost idea is the same as your idea.) – Peter Cordes Sep 3 '15 at 20:40
  • @Peter - quite right: root is absolutely the wrong user for this! – Toby Speight Sep 3 '15 at 21:33
  • Supports guys have to upload things, so this solution may not work. – Raspbeguy Sep 4 '15 at 7:28
  • @Raspbeguy: posted an answer that addresses this. Use in combination with roaima's non-root sudo setup. – Peter Cordes Sep 4 '15 at 9:40
0

Your support people always connecting through that proxy machine, and only from it, so the clients could simply authenticate the proxy machine using HostbasedAuthentication.

Let's say the proxy machine is supporters.pc and you provide support to customer.pc

customer.pc will have HostbasedAuthentication yes in /etc/ssh/ssd_config, supporters.pc listed in /etc/ssh/shosts.equiv and its public key in /etc/ssh/ssh_known_hosts.

When your support staff log into support@supporters.pc, and perform ssh customer.pc, it will spawn ssh-keysign(8) (which needs to be setuid), which will handle the key signing with the file-you-can't-read and provide proof to supporters.pc that the connection is indeed comming from supporters.pc. As customer.pc trusts supporters.pc, your staff member gets logged in as support.

  • This is a terrible solution from the security point of view since if you are granting access to a machine as the whole and you lose accountability of who did what. Machine-to-machine trust should be banned from the modern world where IT security started to emerge as a pretty much hot topic. – galaxy Sep 4 '15 at 11:52
  • @galaxy, from the OP my understanding is that they are already doing basically that. The trust in that machine is in the specification. Note specifically that they are "using only one account by machine (the support access account)" it's possible they are using a shared account in customer.pc, but connect to the server as their own user. Changing the permissions of ssh and ssh-keygen in order to force a sudo -u support ssh … would solve that and also add a log entry. – Ángel Sep 4 '15 at 21:48
0

Use a wrapper binary

For this particular use case (see the important note below), one possibility is to create a user (say support-ssh) specifically for these outgoing SSH connections, then install a small wrapper binary that execs /usr/bin/ssh.

  • Don't copy+chmod the ssh binary itself, because you won't remember to re-copy it every time you apply security updates.
  • And don't use root as the more-privileged user, for reasons I trust are obvious.

This is a functionally-equivalent alternative to using sudo to elevate privileges to that of the support-ssh account with the following trade-offs:

  • This is smaller and leaner than sudo, so there's less risk of configuration error opening up more than you intended.
  • It is your responsibility to code it correctly - only do this if you are extremely careful and (ideally) have experience of security-critical coding.
  • It can be tailored to be more specific than sudo (but the more code you write, the more you need to audit for security).

The wrapper binary should set HOME to the support-ssh user's home directory, so that ssh will pick up the appropriate ssh_config and private key. But the invoking user should not be permitted to read ~support-ssh/.ssh/ or its contents.

The wrapper could be as simple as:

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>

int main(int argc, char **argv)
{
    setenv("HOME", "/home/support-ssh", 1);
    /* allow only a single, non-option argument */
    if (argc!=2 || *argv[1]=='-') {
        fprintf(stderr, "Usage: %s <hostname>", argv[0]);
        exit(EXIT_FAILURE);
    }
    execv("/usr/bin/ssh", argv);
    return EXIT_FAILURE;
}

You might want to be more restrictive, and check that the argument argv[1] is in a set of permitted hostnames. Or less restrictive, and allow (a subset of) option arguments. You might want to completely replace environment variables (but keep important ones such as TERM, LC_*, etc); note that LD_LIBRARY_PATH and LD_PRELOAD are particularly dangerous.

A similar wrapper program could be provided for scp if needed.

A note on applicability

This answer addresses the specific circumstances of the question, where the users are contractually obliged to follow procedures, and there are sanctions (e.g. dismissal) for violating them. It supposes that you want to prevent the employees casually copying private keys around, rather than preventing a determined attacker from obtaining unauthorised access.

I'm taking the view that security is achieved through both technical and non-technical defences, and that the balance achieved here or by using sudo is appropriate for the situation presented.

  • You pass unchecked arguments to ssh running as another user, see my comment on the answer above. I can still read the key... and I would not count on strict argument checks, ssh is not meant to be run under that way. – nkms Sep 3 '15 at 19:12
  • @nkms - I've made the wrapper more restrictive. It can be made more open if required, but you're right to lock things down except as required otherwise, rather than vice versa. – Toby Speight Sep 4 '15 at 10:32
  • @TobySpeight: yup, looks good. I put most of what I said in comments into my own answer anyway. – Peter Cordes Sep 4 '15 at 11:27
  • 1
    It would be better to use sudo than a binary wrapper. Contrary to what you claim, using sudo is a lot less risky than rolling your own. Sudo has safe defaults. How sure are you that you haven't forgotten to remove an environment variable that would influence ssh? – Gilles 'SO- stop being evil' Sep 4 '15 at 21:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.