0

I want to transform json to new line delimited json. I was attempting to do it multiple times using jq in bash, but I have not been able to get even close to the final output.

INPUT:

{
    "windows124": {
        "updated": "2015-01-14",
        "attribution": [],
        "description": "",
        "notes": [],
        "alt_names": [],
        "sources": [],
        "urls": ["google.com", "google.co.uk"],
        "common_name": "test",
        "uuid": "7259334c-3218-4259-aaab-896d87507f4f"
    },
    "linux124": {
        "updated": "",
        "attribution": ["Naifdddkoscn"],
        "description": "",
        "notes": [],
        "alt_names": [],
        "sources": [],
        "urls": ["https://example.com/1.pdf", "https://example.com/1.pdf", "https://example.com/1.pdf"],
        "common_name": "121212",
        "uuid": "009db412-762d-4256-8df9-eb213be01ffd"
    },
    "wikipedia123": {
        "updated": "2018-07-31",
        "attribution": [],
        "description": "",
        "notes": [],
        "alt_names": [],
        "sources": [],
        "urls": ["https://example.com/1.pdf"],
        "common_name": "test343",
        "uuid": "4d8da0af-cfd7-4990-b211-af0e990vfda0"
    }
}

WANTED OUTPUT:

{"uuid": "7259334c-3218-4259-aaab-896d87507f4f","family": "windows124","updated": "2015-01-14","attribution": [],"description": "","notes": [],"alt_names": [],"sources": [],"urls": ["google.com", "google.co.uk"],"common_name": "test"}
{"uuid": "009db412-762d-4256-8df9-eb213be01ffd","family": "linux124", "updated": "","attribution": ["Naifdddkoscn"],"description": "","notes": [],"alt_names": [],"sources": [],"urls": ["https://example.com/1.pdf", "https://example.com/1.pdf", "https://example.com/1.pdf"],"common_name": "121212"}
{"uuid": "4d8da0af-cfd7-4990-b211-af0e990vfda0","family": "wikipedia123", "updated": "2018-07-31","attribution": [],"description": "","notes": [],"alt_names": [],"sources": [],"urls": ["https://example.com/1.pdf"],"common_name": "test343"}

What I have so far is: cat deserialize.json | jq -c '.|to_entries[]'

{"key":"windows124","value":{"updated":"2015-01-14","attribution":[],"description":"","notes":[],"alt_names":[],"sources":[],"urls":["google.com","google.co.uk"],"common_name":"test","uuid":"7259334c-3218-4259-aaab-896d87507f4f"}}
{"key":"linux124","value":{"updated":"","attribution":["Naifdddkoscn"],"description":"","notes":[],"alt_names":[],"sources":[],"urls":["https://example.com/1.pdf","https://example.com/1.pdf","https://example.com/1.pdf"],"common_name":"121212","uuid":"009db412-762d-4256-8df9-eb213be01ffd"}}
{"key":"wikipedia123","value":{"updated":"2018-07-31","attribution":[],"description":"","notes":[],"alt_names":[],"sources":[],"urls":["https://example.com/1.pdf"],"common_name":"test343","uuid":"4d8da0af-cfd7-4990-b211-af0e990vfda0"}}
2
  • Got closer with : jq '.|to_entries[]'
    – creed
    Nov 14 '18 at 16:06
  • You can add the -c (--compact-output) command line option if you don't want the output pretty-printed Nov 14 '18 at 16:26
2

You can use this jq filter:

<file jq 'to_entries|map(.value + {family:(.key)})[]'

As you found out, the to_entries function allows to get the key name in order add the property family.

So the filter only creates this family object and adds it to the content of the value given by to_entries function.

The map function does the add operation to all elements of the value array.

The final [] gets rid of the outer array.

Note the output is not order in the way you posted, but the content is same. If you want the keys to be sorted, use the option -S.

2
  • Thanks $oliv. Nicely done. Have you solved similar issue before, or how did you figure that out? If you can, please let me know how I could replicate your steps differently than doing brute force on all options.
    – creed
    Nov 14 '18 at 19:50
  • @creed The way to solve this type of problem is indeed to try out. Read the man pages and go to the jq website to get to know the functions and how to use it. In other words, learn the language.
    – oliv
    Nov 15 '18 at 7:26
0
jq -c 'with_entries(.value.family = .key)[]' file.json

or,

jq -c 'with_entries(.value += { family: .key })[]' file.json

This inserts a family key into each sub-element of the original top-level object, with the value of the key of that element (e.g. windows124 for the first one). It then returns the sub-element as a set, which is what you appear to want.

Output for the document given in the question:

{"updated":"2015-01-14","attribution":[],"description":"","notes":[],"alt_names":[],"sources":[],"urls":["google.com","google.co.uk"],"common_name":"test","uuid":"7259334c-3218-4259-aaab-896d87507f4f","family":"windows124"}
{"updated":"","attribution":["Naifdddkoscn"],"description":"","notes":[],"alt_names":[],"sources":[],"urls":["https://example.com/1.pdf","https://example.com/1.pdf","https://example.com/1.pdf"],"common_name":"121212","uuid":"009db412-762d-4256-8df9-eb213be01ffd","family":"linux124"}
{"updated":"2018-07-31","attribution":[],"description":"","notes":[],"alt_names":[],"sources":[],"urls":["https://example.com/1.pdf"],"common_name":"test343","uuid":"4d8da0af-cfd7-4990-b211-af0e990vfda0","family":"wikipedia123"}

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.