9

I can write

> echo '{"a": "arbiter", "b": "brisk"}{"a": "astound", "b": "bistro"}' | jq '.a, .b'
"arbiter"
"brisk"
"astound"
"bistro"

but if I do

> echo '{"a": "arbiter", "b": "brisk", "c": ["cloak", "conceal"]} {"a": "astound", "b": "bistro", "c": ["confer", "consider"]}' | jq '.a, .b, .c'

I get

"arbiter"
"brisk"
[
    "cloak",
    "conceal"
]
"astound"
"bistro"
[
    "confer",
    "consider"
]

How do I flatten the c arrays to get instead

"arbiter"
"brisk"
"cloak",
"conceal"
"astound"
"bistro"
"confer",
"consider"

Update

Since null safety is quite fashionable in several modern languages (and justifiably so), it is perhaps fitting to suppose that the question as asked above was incomplete. It's necessary to know how to handle the absence of a value.

If one of the values is null,

> echo '{"b": "brisk"}{"a": "astound", "b": "bistro"}' | jq '.a, .b'

we get a null in the output

null
"brisk"
"astound"
"bistro"

That may well be what we want. We could add a second step in the pipeline (watching out not to exclude "null"), but it's cleaner if jq itself excludes nulls. Just writing select(.a != null) does the trick, but introduces a {} level. What is the right way to discard nulls from within jq?

0

2 Answers 2

12

The jq expression

[.a, .b, .c]

extracts all the elements that we want from the input objects, and places them in an array. Some of these elements may be arrays, so we need to flatten all elements:

[.a, .b, .c] | flatten

For the input object

{
  "a": "arbiter",
  "b": "brisk",
  "c": [
    "cloak",
    "conceal"
  ]
}

this generates the array

[
  "arbiter",
  "brisk",
  "cloak",
  "conceal"
]

and you'll get a similar but separate array from your second object.

To merge these two arrays into one, we may simply pass the data through .[], but a shortcut way of writing flatten | .[] is flatten[]. Using this, we arrive at

[.a, .b, .c] | flatten[]

Summary:

echo '...as in the question...' |
jq '[.a, .b, .c] | flatten[]'

If you additionally want to weed out any null values, filter through select(. != null), or extract the values using [.a//empty,.b//empty,.c//emtpy], or filter through map(.//empty) before flatten[].


As a comment: Your input JSON is created using a simple echo in the question. However, to properly create JSON on the command line, consider using a tool like jo, which will additionally encode your data appropriately for inclusion in a JSON document:

Your second example JSON could be created using the two jo invocations

jo a=arbiter b=brisk  'c[]'=cloak  'c[]'=conceal
jo a=astound b=bistro 'c[]'=confer 'c[]'=consider
0
2

This answer addresses the question before the update.

Ok, that was rather trivial. No need for Python's delightful JSON library, nor even to lower the floor with the genericity of awk or sed for this one. It's enough to write:

> echo '{"a": "arbiter", "b": "brisk", "c": ["cloak", "conceal"]} {"a": "astound", "b": "bistro", "c": ["confer", "consider"]}' | jq '.a, .b, .c[]'

and we get:

"arbiter"
"brisk"
"cloak"
"conceal"
"astound"
"bistro"
"confer"
"consider"
5
  • 2
    or ... | jq 'flatten[]'
    – Freddy
    Jun 12, 2021 at 19:07
  • @Freddy Neat, but if in addition to a/b/c we also had, say, timestamp that we wanted to ignore, that would capture it. Other hints along the line you suggest?
    – Sam7919
    Jun 12, 2021 at 19:11
  • 1
    You could remove the timestamps first, e.g. ... | jq 'del(.timestamp) | flatten[]'.
    – Freddy
    Jun 12, 2021 at 20:17
  • 1
    @Freddy that's definitely worth an answer Jun 12, 2021 at 20:30
  • Well, I made my answer insufficient by making the question slightly harder. Take a look and tell me. I think the pair of questions should always arise together, not as two separate questions. And of course the question is open again.
    – Sam7919
    Jun 12, 2021 at 21:25

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .