I have a JSON file with thousands of records line by line in the following structure, with different values.


{"in": 5,"li": [{"st": 1508584161,"abc": 128416626,"ta": 33888}],"usr": {"is": "222108923573880","ie": "222108923573880"},"st2": 1508584161,"ei": {"ev": 0,"rt": 10},"rn": 947794,"st1": 1508584161}
{"in": 5,"li": [{"st": 1508584174,"abc": 128572802,"ta": 33504}],"usr": {"is": "222108923573880","ie": "222108923573880"},"st2": 1508584174,"ei": {"ev": 0,"rt": 19},"rn": 947795,"st1": 0}
{"in": 5,"li": [{"st": 1508584145,"abc": 279682,"ta": 50000}],"usr": {"is": "222108923573880","ie": "222108923573880"},"st2": 1508584145,"ei": {"ev": 0,"rt": 18},"rn": 947796,"st1": 1508584145}
{"in": 5,"li": [{"st": 1508584183,"abc": 1378680,"ta": 49840}],"usr": {"is": "222108923573880","ie": "222108923573880"},"st2": 1508584183,"ei": {"ev": 0,"rt": 10},"rn": 947797,"st1": 1508584186}
{"nt": 4}

I am trying to select objects (records) in the JSON file that match the following criteria and output to another file.

st1 is < or = st2

st1 is not 0

st2 is not 0

st1 is less than 2147483647

st2 is less than 2147483647

In the output, the footer of the original file ({"nt": 4}) should also be in the output file, so it can be edited with the new records count

Example of output file:

{"in": 5,"li": [{"st": 1508584161,"abc": 128416626,"ta": 33888}],"usr": {"is": "222108923573880","ie": "222108923573880"},"st2": 1508584161,"ei": {"ev": 0,"rt": 10},"rn": 947794,"st1": 1508584161} 
{"nt": 1}

I have the following:

jq -c 'select((.st1 > 0 and .st2 > 0 and .st1 < .st2) or (.st1 < 214748647 and .st2 < 214748647 and .st1 > 0 and .st2 > 0 and .st1 < .st2)) file.json

I have tried various permutations but it is not capturing the correct records.

  • 1
    the above json is invalid without commas , separating objects ... } , {... – RomanPerekhrest Nov 15 '17 at 11:39
  • Update your question with a VALID json structure – RomanPerekhrest Nov 15 '17 at 11:54
  • Another problem: 0002 is not valid in JSON, and neither is 00000000000" (missing quote?). – Satō Katsura Nov 15 '17 at 11:54
  • Updated the structure. – Booboo Nov 15 '17 at 12:04
  • @Booboo, still invalid, all objects should be items of an array. Otherwise, the above is just a text lines containing separate independent (therefore unrelated) objects – RomanPerekhrest Nov 15 '17 at 12:21

With the correct numbers, a straightforward translation of you conditions works:

$ jq -c 'select(.st1 <= .st2 and 
                .st1 > 0 and .st2 > 0 and 
                .st1 < 2147483647 and .st2 < 2147483647)' file.json 

Note the closing ', and no double parenthesis. I don't understand why you split the conditions into two and clauses connected by or, that's not what your conditions say.

Anyway, that captures the correct records; now we only have to add the footer. That's easiest with an additional step, shortening the select clause from above for brevity:

jq -c 'select ...' file.json > out.json
printf '{"nt":%i}\n' `wc -l < out.json` >> out.json 

I guess one can also do it with a complicated jq expression, but I didn't try that.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.