3

I've got this ssh command that I run on different databases:

ssh -i $8 -l $9 $1 "set -o pipefail set -o StrictHostKeyChecking=no; mysqldump --single-transaction --skip-lock-tables -u $2 -p$3 -P $4 -h $5 $6" | gzip -c > $7;

The variables are arguments to this command.

It works except when the backend of an ip address changes, despite having set -o StrictHostkeyChecking=no, I get this message:

Offending ECDSA key in /home/admin/.ssh/known_hosts:130
  remove with:
  ssh-keygen -f "/home/admin/.ssh/known_hosts" -R "172.30.0.63"
ECDSA host key for 172.30.0.63 has changed and you have requested strict checking.
Host key verification failed.

There is something wrong with the command but I'm not sure where. I've tried a couple of different variations but wihtout success.

5
  • 1
    I really recommend using ansible or similar automation for things like this rather than one-liner scripts that I can guarantee you won't remember what they did or how to modify two years from now when something breaks. Mar 19, 2023 at 9:36
  • Also, the reason you're getting this particular error is because the copy of the host's identity key you have stored locally doesn't match the one the remote host provides. Using the suggested command will clear the offending local key, and the next time you log in you'll be asked to confirm the remote key once, after which the error disappears. Mar 19, 2023 at 10:47
  • It’s more to ensure things get backed up every night then fix a key for a server that moved to a different internal ip address.
    – StevieD
    Mar 21, 2023 at 1:05
  • If that's something that happens on a regular basis, you are doing network topology wrong. Mar 21, 2023 at 5:24
  • AWS assigns new back-end ip addresses to cloned instances. Complain to them.
    – StevieD
    Mar 22, 2023 at 13:27

3 Answers 3

14

The StrictHostKeyChecking=no argument needs to be passed as an option to the ssh command, rather than as an option to the remote shell command. So for example:

ssh -i "$8" -o StrictHostKeyChecking=no -l "$9" "$1"  "
  set -o pipefail; mysqldump --single-transaction --skip-lock-tables -u $2 -p$3 -P $4 -h $5 $6
" | gzip -c > "$7"

(I double-quoted the parameter expansions here because, although you didn't mention which local shell you are using, many shells subject unquoted expansion to word splitting and filename generation.)

Note that this option does not suppress warnings about a possible MiM attack.

1
  • 1
    Addendum to note: Disabling strict host key checking is almost always a worse solution than fixing the actual host keys. Mar 19, 2023 at 10:48
3

If disabling at run time with command line options doesn't work, then disable it in your ~/.ssh/config file. You can create the file if it doesn't exist,remember to give it rw-r-r permissions with chmod 644 ~/.ssh/config or rw- with chmod 600 ~/.ssh/config, and add this:

 Host *
    StrictHostKeyChecking no

check this post too: How to disable strict host key checking in ssh?

4
  • 6
    Note that this will disable StrictHostKeyChecking for all servers this user sshs to. I don't know if this is desired.
    – terdon
    Mar 17, 2023 at 15:02
  • 2
    yes right, he can change the * to a specific ip/domain
    – Z0OM
    Mar 17, 2023 at 15:04
  • Yeah, I'd prefer not to bypass it all the time, inly when doing automated backups. The first comment from @terdon worked, though initially I thought it wasn't because I was still getting a warning.
    – StevieD
    Mar 17, 2023 at 15:07
  • Thanks for reminding me to check the file permissions ;-)
    – jo_
    Mar 11 at 10:54
0

I'm not normally in the business of telling other people how to run their server administration, but...

If you're in a position where you find yourself passing eight variables to a one-liner shell script, you are doing things horribly, horribly wrong.

Let's start by simplifying this.

First, the SSH bit:

I'm going to give you the benefit of the doubt and assume that each of those variables is actually variable, that you have multiple hosts, each of which needs to be accessed as a different user with a different identity file, none of which can be counted on to have a correct host key, and that you do not have the authority to change any of the above.

(I'll spare you a five-page invective-filled rant about consistency and proper security measures; take it as read)

Add the following to ~/.ssh/config on whatever administrator account you intend to use for this:

Host foo
   StrictHostKeyChecking No
   HostName 172.30.0.63
   User foouser
   IdentityFile ~/.ssh/id_foo.ed25519

Host bar
  StrictHostKeyChecking No
  HostName bar.internal.elsewhere
  User baruser
  IdentityFile ~/.ssh/id_bar.ecdsa

Host qux
  StrictHostKeyChecking No
  HostName qux.eternal.universe
  User quxuser
  IdentityFile ~/.ssh/id_qux.rsa

And so on. At this point you can omit $8 and $9 as well as the -o StrictHostKeyChecking No on the command line which, as steeldriver pointed out, was in the wrong place entirely to begin with.

Secondly, the mysqldump part.

This is where automation software like Ansible shines.

First off, you create an ansible inventory as follows:

all:
  dbservers:
    hosts:
      foo:
      bar:
      qux:

Use the same naming convention here as in your ~/.ssh/config file. Use ansible -m ping dbservers to check if you got it right, correct errors as necessary.

Then add the following in host_vars/baz/vars.yaml:

mysql_db_user: 'bazdbuser'
mysql_db_password: 'bazdbpassword'
mysql_db_port: 12345
mysql_db_host: 'baz-db.remote
mysql_db_dbs:
  - 'bazdb1'
  - 'bazdb2'
  - 'bazdb3'
mysql_dump_local_destination: '/backups/baz/'

Then create the following playbook anywhere, let's call it make_backup_dumps.yaml:

---
- hosts: dbservers
  become: false
  tasks:
    - name: Dump requested databases.
      community.mysql.mysql_db:
        login_host: '{{ mysql_db_host }}'
        login_user: '{{ mysql_db_user }}'
        login_port: '{{ mysql_db_port }}'
        skip_lock_tables: true
        state: dump
        target: '/tmp/db_{{ ansible_host }}_{{ item }}.sql.gz'
      loop: {{ mysql_db_dbs }}

    - name: Retrieve remote dumps.
      ansible.builtin.fetch:
        src: '/tmp/db_{{ ansible_host }}_{{ item }}.sql.gz'
        dest: '{{ mysql_dump_local_destination }}'
      loop: {{ mysql_db_dbs }}

Test the script on one host with ansible-playbook --limit foo make_backup_dumps.yaml; if the output is to your liking, run it for all hosts with ansible-playbook make_backup_dumps.yaml.

This may be a bit bulkier than your one-liner, but it's significantly more readable, much easier to maintain, supports multiple database table dumps per host, will run in parallel on each host, and will give you an overview of successes/failures for each step on each host, with optional levels of verbosity so you can troubleshoot.

Whether you use it is up to you, but regardless, for the love of GNU at the very least clear up your host keys because the error you're actually getting is something caused locally.

3
  • I use rex, not ansible.
    – StevieD
    Mar 20, 2023 at 16:59
  • If that SSH one-liner is characteristic of rex scripting, I'm staying far away from it. Mar 20, 2023 at 17:19
  • It’s just a Perl with helper functions to help automate running commands on a remote server. I prefer it to dealing with some other layer of abstraction.
    – StevieD
    Mar 20, 2023 at 17:34

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .