15

I would like to print each path and value of a file with included key values line by line. Given the following and , is it also possible to add the values for each line? If not using is there another way? I am thinking of something vaguely similar to for . Also, is there a technical term for what I am trying to do?

$ cat short.json | jq '.'
{
  "Reservations": [
    {
      "Groups": [],
      "Instances": [
        {
          "ImageId": "ami-a",
          "InstanceId": "i-a",
          "InstanceType": "t2.micro",
          "KeyName": "ubuntu"
        }
      ]
    }
  ]
}
$ cat short.json | jq -r '[paths | map(.|tostring) | join(".")]'
[
  "Reservations",
  "Reservations.0",
  "Reservations.0.Groups",
  "Reservations.0.Instances",
  "Reservations.0.Instances.0",
  "Reservations.0.Instances.0.ImageId",
  "Reservations.0.Instances.0.InstanceId",
  "Reservations.0.Instances.0.InstanceType",
  "Reservations.0.Instances.0.KeyName"
]

Example of a single line from the multiline output:
"Reservations.0.Instances.0.ImageId": "ami-a",

Bonus if the output could be formatted to be used in using copy and paste with value easily detached using linux cut:

'.Reservations[].Instances[].ImageId': "ami-a" 
$ cat short.json | jq -r '.Reservations[].Instances[].ImageId'
ami-a
1
  • the same query it possible to resolve with other utilities, not only with jq. For example this is a complete solution jtc based: <short.json jtc -w'<$_:.>a:' -T'{"{$path}":{{}}}' -ll. PS. I'm the developer of the jtc unix utility. PPS. I'm not trying to promote my tool, merely showing an alternative approach to the query of the OP.
    – Dmitry
    Jan 16, 2020 at 10:41

2 Answers 2

16

I'm not super knowledgeable on jq but I found this on the interwebs and I think it will work in your case:

Cut and paste version:

jq -r 'paths(scalars | true) as $p  | [ ( [ $p[] | tostring ] | join(".") ), ( getpath($p) | tojson )] | join(": ")' short.json 

Easy(er) to read version:

jq -r '
paths(scalars | true) as $p
  | [ ( [ $p[] | tostring ] | join(".") )
    , ( getpath($p) | tojson )
    ]
  | join(": ")
' short.json

Result:

Reservations.0.Instances.0.ImageId: "ami-a"
Reservations.0.Instances.0.InstanceId: "i-a"
Reservations.0.Instances.0.InstanceType: "t2.micro"
Reservations.0.Instances.0.KeyName: "ubuntu"

And because I want the bonus points, here's a hacky sed you can tack on to get the output you want:

... | sed "s/^/\'./; s/:/\':/; s/\.0/[]/g"

Which outputs:

'.Reservations[].Instances[].ImageId': "ami-a"
'.Reservations[].Instances[].InstanceId': "i-a"
'.Reservations[].Instances[].InstanceType': "t2.micro"
'.Reservations[].Instances[].KeyName': "ubuntu"
1
1

Using the following jq expression, we ensure that both the key and the value part of the output are valid JSON (encoded and quoted if strings, and unquoted if boolean, numeric or null). I have created a helper function, kv_to_str, to do the formatting of the key-value string, just for readability:

def kv_to_str($k; $v): "\($k): \($v)";

paths(scalars | true) as $p |
    kv_to_str(
        $p | join(".") | tojson;
        getpath($p) | tojson
    )

Testing on your data:

$ jq -r 'def kv_to_str($k;$v): "\($k): \($v)"; paths(scalars|true) as $p | kv_to_str($p|join(".")|tojson; getpath($p)|tojson)' file
"Reservations.0.Instances.0.ImageId": "ami-a"
"Reservations.0.Instances.0.InstanceId": "i-a"
"Reservations.0.Instances.0.InstanceType": "t2.micro"
"Reservations.0.Instances.0.KeyName": "ubuntu"

To get something that you can use as a jq expression to query for each value, we can make a couple of smaller adjustments so that instead of creating keys like "Reservations.0.Instances.0.ImageId" we get .["Reservations"][0]["Instances"][0]["ImageId"]. Note that we leave the zeros in there, which allows us to generate more specific keys for array elements.

def kv_to_str($k; $v): "\($k): \($v)";

paths(scalars | true) as $p |
    kv_to_str(
        "." + ($p | map([.] | tojson) | join(""));
        getpath($p) | tojson
    )

The key is generated by taking each part of the path, putting it in [...], and converting it to JSON (strings will be quoted, and integer indexes will remain unquoted). We then join the parts up and prepend a dot.

Testing:

$ jq -r 'def kv_to_str($k;$v): "\($k): \($v)"; paths(scalars|true) as $p | kv_to_str("." + ($p|map([.]|tojson)|join("")); getpath($p)|tojson)' file
.["Reservations"][0]["Instances"][0]["ImageId"]: "ami-a"
.["Reservations"][0]["Instances"][0]["InstanceId"]: "i-a"
.["Reservations"][0]["Instances"][0]["InstanceType"]: "t2.micro"
.["Reservations"][0]["Instances"][0]["KeyName"]: "ubuntu"
$ jq -r '.["Reservations"][0]["Instances"][0]["InstanceType"]' file
t2.micro
1
  • I greatly appreciate the reversible application whereby I can pass the same identifier back into jq to modify the target value, it's a nice touch. Thank you.
    – James G
    Jan 29 at 9:21

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .