20

I'm familiar with "jq" for parsing json.

I work with one service that produces a json response where one of the properties is itself a json string. How do I convert that quoted value to a valid json string so I can then process it with jq?

For instance, if I just view the plain pretty-printed json from "jq .", here's a short excerpt of the output:

"someJsonString": "{\"date\":\"2018-01-08\", ...

I can use jq to get the value of that property, but I need to convert the quoted string to valid json by "unescaping" it.

I suppose I could pipe it into sed, removing the opening and ending double quotes, and removing all backslashes ("sed -e 's/^"//' -e 's/"$//' -e 's/\\//g'"). That seems to work, but that doesn't seem like the most robust solution.

Update:

Just to be a little clearer on what I'm doing, here are a couple of elided samples that show what I've tried:

% curl -s -q -L 'http://.../1524.json' | jq '.results[0].someJsonString' | jq .
"{\"date\":\"2018-01-08\",...
% echo $(curl -s -q -L 'http:/.../1524.json' | jq '.results[0].someJsonString') | jq .
"{\"date\":\"2018-01-08\",...

Update:

Here's a completely standalone example:

% cat stuff.json | jq .
{
  "stuff": "{\"date\":\"2018-01-08\"}"
}
% cat stuff.json | jq '.stuff'
"{\"date\":\"2018-01-08\"}"
% cat stuff.json | jq '.stuff' | jq .
"{\"date\":\"2018-01-08\"}"

Update:

If I tried to process that last output with a real jq expression, it does something like this:

% cat stuff.json | jq '.stuff' | jq '.date'
assertion "cb == jq_util_input_next_input_cb" failed: file "/usr/src/ports/jq/jq-1.5-3.x86_64/src/jq-1.5/util.c", line 371, function: jq_util_input_get_position
Aborted (core dumped)
  • If you use jq to get just the value of the string property, does it return it unescaped? If so, just pipe that into a fresh jq. – DopeGhoti Jan 8 '18 at 21:30
  • No, it does not return it unescaped. That's the point. – David M. Karr Jan 8 '18 at 21:30
  • How about echo $(jq statement here)? – DopeGhoti Jan 8 '18 at 21:31
  • Nope, no change. – David M. Karr Jan 8 '18 at 21:33
  • @DavidM.Karr, ok, If possible - extend your input with the actual crucial string and the final result – RomanPerekhrest Jan 8 '18 at 21:34
16

There is a raw flag for this

    -r      output raw strings, not JSON texts;

jq -rc .stuff stuff.json

Output

{"date":"2018-01-08"}
  • The difference is that with Roman's answer, you are guaranteed to get valid JSON output, or error messages if it's not valid JSON. – Kusalananda Apr 27 '18 at 18:48
  • Valid point, but if this is being used in automation, I think it would be unusual to suddenly not have valid json output. The most convenient form will be perfectly fine almost all the time. It's still good to know about more precise methods, however. – David M. Karr Apr 28 '18 at 22:30
  • @DavidM.Karr "unusual to suddenly not have valid json output" HA! Riiiight. Error handling in automation? Errors will never happen! Why bother! – Bruno Bronosky Feb 6 at 18:21
  • This requires piping to another jq for further JSON processing, whereas with Roman's approach you can continue the same jq expression. – Raman Mar 4 at 16:53
  • 1
    @cricket_007: tried it with jq 1.5, and confirmed it doesn't work: jq -rc '.stuff.date' produces jq: error (at <stdin>:0): Cannot index string with string "date". However: .stuff | fromjson | .date works fine. – Raman Mar 5 at 7:51
20

With jq's fromjson function:

Sample stuff.json contents:

{
  "stuff": "{\"date\":\"2018-01-08\"}"
}

jq -c '.stuff | fromjson' stuff.json

The output:

{"date":"2018-01-08"}
  • This seems unecssary. Provided alternative answer – cricket_007 Apr 27 '18 at 18:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.