1

I'm new to jq and trying to extract key and value from selected fields.

{
    "StackEvents": [
        {
            "StackId": "arn:aws:cloudformation:us-east-1:1234567890123:stack/jenkinscf-s3/f64657b0-cb0a-11ed-848f-0a093f43c24f",
            "EventId": "06263740-cb0b-11ed-86ee-0e3b4b6ffec3",
            "StackName": "jenkinscf-s3",
            "LogicalResourceId": "jenkinscf-s3",
            "PhysicalResourceId": "arn:aws:cloudformation:us-east-1:1234567890123:stack/jenkinscf-s3/f64657b0-cb0a-11ed-848f-0a093f43c24f",
            "ResourceType": "AWS::CloudFormation::Stack",
            "Timestamp": "2023-03-25T12:46:12.647000+00:00",
            "ResourceStatus": "CREATE_COMPLETE"
        },
        {
            "StackId": "arn:aws:cloudformation:us-east-1:1234567890123:stack/jenkinscf-s3/f64657b0-cb0a-11ed-848f-0a093f43c24f",
            "EventId": "S3Bucket-CREATE_COMPLETE-2023-03-25T12:46:11.127Z",
            "StackName": "jenkinscf-s3",
            "LogicalResourceId": "S3Bucket",
            "PhysicalResourceId": "jenkinscf-s3-s3bucket-tmh6orxmeegs",
            "ResourceType": "AWS::S3::Bucket",
            "Timestamp": "2023-03-25T12:46:11.127000+00:00",
            "ResourceStatus": "CREATE_COMPLETE",
            "ResourceProperties": "{}"
        },
        {
            "StackId": "arn:aws:cloudformation:us-east-1:1234567890123:stack/jenkinscf-s3/f64657b0-cb0a-11ed-848f-0a093f43c24f",
            "EventId": "S3Bucket-CREATE_IN_PROGRESS-2023-03-25T12:45:50.261Z",
            "StackName": "jenkinscf-s3",
            "LogicalResourceId": "S3Bucket",
            "PhysicalResourceId": "jenkinscf-s3-s3bucket-tmh6orxmeegs",
            "ResourceType": "AWS::S3::Bucket",
            "Timestamp": "2023-03-25T12:45:50.261000+00:00",
            "ResourceStatus": "CREATE_IN_PROGRESS",
            "ResourceStatusReason": "Resource creation Initiated",
            "ResourceProperties": "{}"
        },
        {
            "StackId": "arn:aws:cloudformation:us-east-1:1234567890123:stack/jenkinscf-s3/f64657b0-cb0a-11ed-848f-0a093f43c24f",
            "EventId": "S3Bucket-CREATE_IN_PROGRESS-2023-03-25T12:45:49.665Z",
            "StackName": "jenkinscf-s3",
            "LogicalResourceId": "S3Bucket",
            "PhysicalResourceId": "",
            "ResourceType": "AWS::S3::Bucket",
            "Timestamp": "2023-03-25T12:45:49.665000+00:00",
            "ResourceStatus": "CREATE_IN_PROGRESS",
            "ResourceProperties": "{}"
        },
        {
            "StackId": "arn:aws:cloudformation:us-east-1:1234567890123:stack/jenkinscf-s3/f64657b0-cb0a-11ed-848f-0a093f43c24f",
            "EventId": "f64b87d0-cb0a-11ed-848f-0a093f43c24f",
            "StackName": "jenkinscf-s3",
            "LogicalResourceId": "jenkinscf-s3",
            "PhysicalResourceId": "arn:aws:cloudformation:us-east-1:1234567890123:stack/jenkinscf-s3/f64657b0-cb0a-11ed-848f-0a093f43c24f",
            "ResourceType": "AWS::CloudFormation::Stack",
            "Timestamp": "2023-03-25T12:45:46.136000+00:00",
            "ResourceStatus": "CREATE_IN_PROGRESS",
            "ResourceStatusReason": "User Initiated"
        }
    ]
}

Expected output

"LogicalResourceId": "jenkinscf-s3"
"ResourceStatus": "CREATE_COMPLETE"

The solutions I have tried only gets me the key or the value but not both.

.StackEvents | .[] | .LogicalResourceId, .ResourceStatus

"jenkinscf-s3"
"CREATE_COMPLETE"
"S3Bucket"
"CREATE_COMPLETE"
"S3Bucket"
"CREATE_IN_PROGRESS"
"S3Bucket"
"CREATE_IN_PROGRESS"
"jenkinscf-s3"
"CREATE_IN_PROGRESS"

.StackEvents[] | keys

[
  "EventId",
  "LogicalResourceId",
  "PhysicalResourceId",
  "ResourceStatus",
  "ResourceType",
  "StackId",
  "StackName",
  "Timestamp"
]

My end goal is to use this in a bash script. The script will detect drifts in the stack and output the differences. It currently runs and only outputs the values, so anyone unfamiliar with the script will get easily confused just by looking at the keys themselves.

7
  • It's a bit unclear what you want to do. You say you want to extract certain keys and values, but you don't really give any criteria for what keys you want. In your example, there's a StackEvents array with five elements. Each element is an object with a certain set of keys and their values. What do you want to get out from these five objects and what is the criteria for extracting that data?
    – Kusalananda ♦
    Mar 29 at 21:15
  • Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking.
    – Community Bot
    Mar 30 at 8:56
  • I'm looking to extract the "LogicalResourceId" and "ResourceStatus" keys and their values.
    – learner00
    Mar 30 at 12:46
  • @user9908 If you know the keys themselves, why do you want to extract them? Wouldn't it be enough to extract the values? Also, your input has an array of five elements, do you want to extract the values from each or just from one of them? In your question, it looks like you want the data from the first element, but it's unclear whether you always want only the first element's data or if there is a more intricate selection criteria (like the most recent Timestamp value).
    – Kusalananda ♦
    Apr 1 at 6:44
  • My end goal is to use this in a bash script. The script will detect drifts in the stack and output the differences. It currently runs and only outputs the values, so anyone unfamiliar with the script will get easily confused just by looking at the keys themselves.
    – learner00
    Apr 3 at 13:03

2 Answers 2

Reset to default
2

A quick sed solution:

$ sed -n 's/^ *"\(LogicalResourceId\|ResourceStatus\)": "\([^"]*\)",*/\1: \2/p' input.json
LogicalResourceId: jenkinscf-s3
ResourceStatus: CREATE_COMPLETE
LogicalResourceId: S3Bucket
ResourceStatus: CREATE_COMPLETE
LogicalResourceId: S3Bucket
ResourceStatus: CREATE_IN_PROGRESS
LogicalResourceId: S3Bucket
ResourceStatus: CREATE_IN_PROGRESS
LogicalResourceId: jenkinscf-s3
ResourceStatus: CREATE_IN_PROGRESS

Unlike your desired output, there's no quotes, but I'm not sure why would you need them if this output is intended to be processed by a bash script later. Still, if you do, just wrap \1 and \2 in quotes, in the replacement portion of the sed expression.

Doing it in jq is easy as well:

$ jq -r '.StackEvents[] | "LogicalResourceId: \(.LogicalResourceId)", "ResourceStatus: \(.ResourceStatus)"' input.json
LogicalResourceId: jenkinscf-s3
ResourceStatus: CREATE_COMPLETE
LogicalResourceId: S3Bucket
ResourceStatus: CREATE_COMPLETE
LogicalResourceId: S3Bucket
ResourceStatus: CREATE_IN_PROGRESS
LogicalResourceId: S3Bucket
ResourceStatus: CREATE_IN_PROGRESS
LogicalResourceId: jenkinscf-s3
ResourceStatus: CREATE_IN_PROGRESS

Adding quotes here is also easy if you need them, but you'll have to escape them with a backslash.

1

You can use jq with awk like this:

#!/bin/bash

input=$(cat input.json)

output=$(echo "$input" | jq -r '.StackEvents[] | {LogicalResourceId, ResourceStatus}')
echo "$output" | awk '{gsub(/[{}]/,"")}NF'

jq outputs:

{LogicalResourceId": "jenkinscf-s3",
 ResourceStatus": "CREATE_COMPLETE"}
{"LogicalResourceId": "S3Bucket",
 "ResourceStatus": "CREATE_COMPLETE"}

... etc

awk removes {} and empty lines.

The final ouput is:

  "LogicalResourceId": "jenkinscf-s3",
  "ResourceStatus": "CREATE_COMPLETE"
  "LogicalResourceId": "S3Bucket",
  "ResourceStatus": "CREATE_COMPLETE"
  "LogicalResourceId": "S3Bucket",
  "ResourceStatus": "CREATE_IN_PROGRESS"
  "LogicalResourceId": "S3Bucket",
  "ResourceStatus": "CREATE_IN_PROGRESS"
  "LogicalResourceId": "jenkinscf-s3",
  "ResourceStatus": "CREATE_IN_PROGRESS"

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .