1

I'm trying to run jq command via ssh for this JSON:

{
  "nodes": {
    "app": {
      "nodes": 1,
      "is_manager": true,
      "ip": [
        "0.0.0.0"
      ],
      "cpus": 16,
      "memory": 64
    },
    "data": {
      "nodes": 1,
      "ip": [
        "0.0.0.0"
      ],
      "cpus": 16,
      "memory": 64
    },
    "analysis": {
      "nodes": 1,
      "ip": [
        "0.0.0.0"
      ],
      "cpus": 16,
      "memory": 64
    },
    "elastic_kafka_1": {
      "nodes": 1,
      "ip": [
        "0.0.0.0"
      ],
      "cpus": 16,
      "memory": 64
    },
    "elastic_kafka_2": {
      "nodes": 1,
      "ip": [
        "0.0.0.0"
      ],
      "cpus": 16,
      "memory": 64
    },
    "elastic_kafka_3": {
      "nodes": 1,
      "ip": [
        "0.0.0.0"
      ],
      "cpus": 16,
      "memory": 64
    },
    "master": {
      "nodes": 1,
      "ip": [
        "0.0.0.0"
      ],
      "cpus": 16,
      "memory": 64
    }
  }
}

This is what I'm trying to run:

ssh -o StrictHostKeyChecking=no -i key.pem user@"172.13.1.23"
"jq -Rn --argjson original_doc \"\$(<nodes.json)\" '
  input | split(\"\u0000\") as \$ips
  | \$original_doc
  | .nodes.app.ip = \$ips[0]
  | .nodes.data.ip = \$ips[1]
  | .nodes.analysis.ip = \$ips[2]
  | .nodes.elastic_kafka_1.ip = \$ips[3]
  | .nodes.elastic_kafka_2.ip = \$ips[4]
  | .nodes.elastic_kafka_3.ip = \$ips[5]
  | .nodes.master.ip = \$ips[6]
' < <(printf '%s\0' \"\${GCP_INSTANCES[@]}\") > test.json && mv test.json nodes.json"

And this is an output:

{
  "nodes": {
    "app": {
      "nodes": 1,
      "is_manager": true,
      "ip": "",
      "cpus": 16,
      "memory": 64
    },
    "data": {
      "nodes": 1,
      "ip": "",
      "cpus": 16,
      "memory": 64
    },
    "analysis": {
      "nodes": 1,
      "ip": null,
      "cpus": 16,
      "memory": 64
    },
    "elastic_kafka_1": {
      "nodes": 1,
      "ip": null,
      "cpus": 16,
      "memory": 64
    },
    "elastic_kafka_2": {
      "nodes": 1,
      "ip": null,
      "cpus": 16,
      "memory": 64
    },
    "elastic_kafka_3": {
      "nodes": 1,
      "ip": null,
      "cpus": 16,
      "memory": 64
    },
    "master": {
      "nodes": 1,
      "ip": null,
      "cpus": 16,
      "memory": 64
    }
  }
}

As you can see jq doesn't work properly due to some syntax issue with ssh or something.

I tested this command locally, without ssh, and it works properly.

I think the problem is with printf '%s\0', but couldn't figure out what exatly I'm doing wrong.

  • 2
    getting nested quotes right is difficult at the best of times. have you tried turning those commands into a script, scp-ing the script to the remote host. and then just running it? if you can only upload to a noexec mount, you can still run it with e.g. bash /tmp/myscript instead of /tmp/myscript. – cas Oct 18 at 4:44
2

The easiest way to ensure that all the quoting is done right is to have the shell do it for you, by using declare -f to generate a textual representation of an already-locally-defined function, and declare -p to generate textual representations of any local variables that function needs access to. Thus:

doRemoteWork() {
  jq -Rn --argjson original_doc "$(<nodes.json)" '
    input | split("\u0000") as $ips
    | $original_doc
    | .nodes.app.ip = $ips[0]
    | .nodes.data.ip = $ips[1]
    | .nodes.analysis.ip = $ips[2]
    | .nodes.elastic_kafka_1.ip = $ips[3]
    | .nodes.elastic_kafka_2.ip = $ips[4]
    | .nodes.elastic_kafka_3.ip = $ips[5]
    | .nodes.master.ip = $ips[6]
  ' < <(printf '%s\0' "${GCP_INSTANCES[@]}") >"nodes.json.$$" \
  && mv "nodes.json.$$" nodes.json
}

ssh -o StrictHostKeyChecking=no -i key.pem user@172.13.1.23 \
  "$(declare -p GCP_INSTANCES; declare -f doRemoteWork); doRemoteWork"

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.