0

I have a JSON output from which I need to extract a few parameters in Linux.

This is the JSON output:

{
  items:[
    {
      provider_name:"ucp-ipg",
      subject_name:"rtm-instrumentation",
      dataset_name:"rtm-instrumentation-dataset-hour-sliced",
      dataset_key:[
        2018-03-06T06:00:00Z,
        "000394e3-a9eb-40b6-9463-fbd588d20ba4"
      ],
      record_count:21,
      state:"complete",
      version:0,
      etag:"a221df62",
      creation_timestamp:2018-03-06T06:10:46.294-00:00,
      created_by:"AAA",
      modification_timestamp:2018-03-06T06:10:46.294-00:00,
      modified_by:"AAA"
    },
    {
      provider_name:"ucp-ipg",
      subject_name:"rtm-instrumentation",
      dataset_name:"rtm-instrumentation-dataset-hour-sliced",
      dataset_key:[
        2018-03-06T06:00:00Z,
        "00097722-b02f-4938-bd4b-d935047c3837"
      ],
      record_count:17,
      state:"complete",
      version:0,
      etag:"aa4dbc25",
      creation_timestamp:2018-03-06T06:12:23.293-00:00,
      created_by:"AAA",
      modification_timestamp:2018-03-06T06:12:23.293-00:00,
      modified_by:"AAA"
    }

I want the output of

dataset_key:[
        2018-03-06T06:00:00Z,
        "00097722-b02f-4938-bd4b-d935047c3837"
      ]

I have already tried with below but not working :

file.txt | python -mjson.tool | grep 'dataset_key'
3
  • 3
    This does not seem to be a well formed JSON document. The keys are not quoted and some of the data is not quoted.
    – Kusalananda
    Sep 24, 2018 at 12:30
  • 1
    And not only are the keys not quoted, the dates (if they are supposed to be strings) are also not quoted. So the first step would be to get whatever produces this output to produce proper JSON, so you can use JSON tools like jq on it.
    – dirkt
    Sep 24, 2018 at 12:34
  • You are mixing json processing and grep processing: the python -mjson.tool just passes the input to the output, and grep works one one line at a time. You need to tell the json tool to do the grepping. Sep 24, 2018 at 16:03

3 Answers 3

5

Assuming that the JSON document is well formed and complete, as for example

{
  "items": [
    {
      "provider_name": "ucp-ipg",
      "subject_name": "rtm-instrumentation",
      "dataset_name": "rtm-instrumentation-dataset-hour-sliced",
      "dataset_key": [
        "2018-03-06T06:00:00Z",
        "000394e3-a9eb-40b6-9463-fbd588d20ba4"
      ],
      "record_count": 21,
      "state": "complete",
      "version": 0,
      "etag": "a221df62",
      "creation_timestamp": "2018-03-06T06:10:46.294-00:00",
      "created_by": "AAA",
      "modification_timestamp": "2018-03-06T06:10:46.294-00:00",
      "modified_by": "AAA"
    },
    {
      "provider_name": "ucp-ipg",
      "subject_name": "rtm-instrumentation",
      "dataset_name": "rtm-instrumentation-dataset-hour-sliced",
      "dataset_key": [
        "2018-03-06T06:00:00Z",
        "00097722-b02f-4938-bd4b-d935047c3837"
      ],
      "record_count": 17,
      "state": "complete",
      "version": 0,
      "etag": "aa4dbc25",
      "creation_timestamp": "2018-03-06T06:12:23.293-00:00",
      "created_by": "AAA",
      "modification_timestamp": "2018-03-06T06:12:23.293-00:00",
      "modified_by": "AAA"
    }
  ]
}

The second item element's dataset_key array may be had with jq:

$ jq '.items[1].dataset_key' file.json
[
  "2018-03-06T06:00:00Z",
  "00097722-b02f-4938-bd4b-d935047c3837"
]

Change [1] to [-1] to get the dataset_key from the last item element.

To get the raw data of the array elements:

$ jq -r '.items[1].dataset_key[]' file.json
2018-03-06T06:00:00Z
00097722-b02f-4938-bd4b-d935047c3837
1
  • 2
    @user3256289: If this works for you, then the example you gave in your question doesn't match your actual data, because AFAIK jq doesn't parse malformed JSON. So, please include the correct data the next time.
    – dirkt
    Sep 24, 2018 at 13:37
0

If you cannot make the json valid, e.g. for an output of an API you have no control of, this will return your desired output:

perl -0777 -ne '/(dataset_key:\[[^\]]*\])/ && print "$1\n"' file.txt

Note: This would break if the item contained a ].

0

an easy way to extract info from json is jtc (assuming your json is fixed):

bash $ jtc -w '<dataset_key>l+0' -r your.json 
[ "2018-03-06T06:00:00Z", "000394e3-a9eb-40b6-9463-fbd588d20ba4" ]
[ "2018-03-06T06:00:00Z", "00097722-b02f-4938-bd4b-d935047c3837" ]
bash $ 

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .