5

The command aws s3api get-bucket-policy --bucket bucketname outputs:

{
    "Policy": "{\"Version\":\"2012-10-17\",\"Id\":\"S3SecureTransportPolicy\",\"Statement\":[{\"Sid\":\"ForceSSLOnlyAccess\",\"Effect\":\"Deny\",\"Principal\":{\"AWS\":\"*\"},\"Action\":\"s3:*\",\"Resource\":\"arn:aws:s3:::amn/*\",\"Condition\":{\"Bool\":{\"aws:SecureTransport\":\"false\"}}},{\"Sid\":\"AWSCloudTrailAclCheck20150319\",\"Effect\":\"Allow\",\"Principal\":{\"Service\":\"cloudtrail.amazonaws.com\"},\"Action\":\"s3:GetBucketAcl\",\"Resource\":\"arn:aws:s3:::amn\"},{\"Sid\":\"AWSCloudTrailWrite20150319\",\"Effect\":\"Allow\",\"Principal\":{\"Service\":\"cloudtrail.amazonaws.com\"},\"Action\":\"s3:PutObject\",\"Resource\":\"arn:aws:s3:::amn/AWSLogs/405042254276/*\",\"Condition\":{\"StringEquals\":{\"s3:x-amz-acl\":\"bucket-owner-full-control\"}}}]}"
}

I need to grep (i.e., extract) the value associated with the key aws:SecureTransport from this JSON (it would be false, in this case) and the value associated with the key Effect.

I've tried

aws s3api get-bucket-policy --bucket amn |
    grep -Po '"Bool": *\K"[^"]*"'

and

aws s3api get-bucket-policy --bucket amn |
    sed 's/.*\(aws:SecureTransport\)[^:]*:"\([0-9]*\)"'

How can I do this?

10

The JSON document that you get from your command seems to contain another encoded JSON document. It's from this encoded document you appear to want to get the data.

To get at the internal document, we may use jq:

aws ... |
jq -r '.Policy'

To get the value of the Effect key from the bit that contains that aws:SecureTransport key from this, we need to parse the document again:

aws ... |
jq -r '.Policy' |
jq -r '.Statement[] | select(.Condition.Bool."aws:SecureTransport").Effect'

The last jq call goes through all the elements of the Statement array, looking for one that has a key called .Condition.Bool."aws:SecureTransport". It then gets the value of the Effect key associated with that Statement element.

Running this on your data outputs the value Deny.

If you want the value of that .Condition.Bool."aws:SecureTransport" key (false in your document), use .Condition.Bool."aws:SecureTransport" in place of .Effect above.

Alternatively, use the fromjson instruction in jq instead of a second jq invocation:

aws ... |
jq -r '.Policy | fromjson | .Statement[] | select(.Condition.Bool."aws:SecureTransport").Effect'

Here, fromjson decodes the encoded JSON document and passes it to the later stages of processing.


Just for reference, the internal encoded JSON document looks like this (aws ... | jq -r '.Policy | fromjson'):

{
  "Version": "2012-10-17",
  "Id": "S3SecureTransportPolicy",
  "Statement": [
    {
      "Sid": "ForceSSLOnlyAccess",
      "Effect": "Deny",
      "Principal": {
        "AWS": "*"
      },
      "Action": "s3:*",
      "Resource": "arn:aws:s3:::amn/*",
      "Condition": {
        "Bool": {
          "aws:SecureTransport": "false"
        }
      }
    },
    {
      "Sid": "AWSCloudTrailAclCheck20150319",
      "Effect": "Allow",
      "Principal": {
        "Service": "cloudtrail.amazonaws.com"
      },
      "Action": "s3:GetBucketAcl",
      "Resource": "arn:aws:s3:::amn"
    },
    {
      "Sid": "AWSCloudTrailWrite20150319",
      "Effect": "Allow",
      "Principal": {
        "Service": "cloudtrail.amazonaws.com"
      },
      "Action": "s3:PutObject",
      "Resource": "arn:aws:s3:::amn/AWSLogs/405042254276/*",
      "Condition": {
        "StringEquals": {
          "s3:x-amz-acl": "bucket-owner-full-control"
        }
      }
    }
  ]
}
| improve this answer | |
3

First of all, do not do this! Use jq as described in Kusalananda's answer. That is a far more robust solution than fiddling around with regular expressions. That said, here's one way to do it with grep:

aws ... | grep -oP 'aws:SecureTransport.":."\K.+?(?=\\")'

That will look for aws:SecureTransport, then any character (.; this is just simpler than writing \\ to escape the slash), then a ". The \K will discard anything matched up to this point (so the part matching aws:SecureTransport.":."). Then, look for the shortest string (.+?) that is followed by \" ((?=\\")).

Running this on a file containing your example output gives:

$ grep -oP 'aws:SecureTransport.":."\K.+?(?=\\")' file
false
| improve this answer | |
  • 2
    that should be prefered over @kusalananda's answer, it looks way more cryptic, and is more error prone, so it is closer to unix philosophy. – Archemar Oct 7 at 14:20
  • 1
    @Archemar using regex instead of proper JSON parsing is going to be far more error prone than an actual JSON parser that was designed for this. – lights0123 Oct 7 at 19:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.