44

I'm familiar with "jq" for parsing json.

I work with one service that produces a json response where one of the properties is itself a json string. How do I convert that quoted value to a valid json string so I can then process it with jq?

For instance, if I just view the plain pretty-printed json from "jq .", here's a short excerpt of the output:

"someJsonString": "{\"date\":\"2018-01-08\", ...

I can use jq to get the value of that property, but I need to convert the quoted string to valid json by "unescaping" it.

I suppose I could pipe it into sed, removing the opening and ending double quotes, and removing all backslashes ("sed -e 's/^"//' -e 's/"$//' -e 's/\\//g'"). That seems to work, but that doesn't seem like the most robust solution.

Update:

Just to be a little clearer on what I'm doing, here are a couple of elided samples that show what I've tried:

% curl -s -q -L 'http://.../1524.json' | jq '.results[0].someJsonString' | jq .
"{\"date\":\"2018-01-08\",...
% echo $(curl -s -q -L 'http:/.../1524.json' | jq '.results[0].someJsonString') | jq .
"{\"date\":\"2018-01-08\",...

Update:

Here's a completely standalone example:

% cat stuff.json | jq .
{
  "stuff": "{\"date\":\"2018-01-08\"}"
}
% cat stuff.json | jq '.stuff'
"{\"date\":\"2018-01-08\"}"
% cat stuff.json | jq '.stuff' | jq .
"{\"date\":\"2018-01-08\"}"

Update:

If I tried to process that last output with a real jq expression, it does something like this:

% cat stuff.json | jq '.stuff' | jq '.date'
assertion "cb == jq_util_input_next_input_cb" failed: file "/usr/src/ports/jq/jq-1.5-3.x86_64/src/jq-1.5/util.c", line 371, function: jq_util_input_get_position
Aborted (core dumped)
7
  • If you use jq to get just the value of the string property, does it return it unescaped? If so, just pipe that into a fresh jq.
    – DopeGhoti
    Jan 8 '18 at 21:30
  • No, it does not return it unescaped. That's the point. Jan 8 '18 at 21:30
  • How about echo $(jq statement here)?
    – DopeGhoti
    Jan 8 '18 at 21:31
  • Nope, no change. Jan 8 '18 at 21:33
  • @DavidM.Karr, ok, If possible - extend your input with the actual crucial string and the final result Jan 8 '18 at 21:34
37

There is a raw flag for this

    -r      output raw strings, not JSON texts;

jq -rc .stuff stuff.json

Output

{"date":"2018-01-08"}
7
  • The difference is that with Roman's answer, you are guaranteed to get valid JSON output, or error messages if it's not valid JSON.
    – Kusalananda
    Apr 27 '18 at 18:48
  • Valid point, but if this is being used in automation, I think it would be unusual to suddenly not have valid json output. The most convenient form will be perfectly fine almost all the time. It's still good to know about more precise methods, however. Apr 28 '18 at 22:30
  • @DavidM.Karr "unusual to suddenly not have valid json output" HA! Riiiight. Error handling in automation? Errors will never happen! Why bother! Feb 6 '19 at 18:21
  • This requires piping to another jq for further JSON processing, whereas with Roman's approach you can continue the same jq expression.
    – Raman
    Mar 4 '19 at 16:53
  • 3
    @cricket_007: tried it with jq 1.5, and confirmed it doesn't work: jq -rc '.stuff.date' produces jq: error (at <stdin>:0): Cannot index string with string "date". However: .stuff | fromjson | .date works fine.
    – Raman
    Mar 5 '19 at 7:51
50

With jq's fromjson function:

Sample stuff.json contents:

{
  "stuff": "{\"date\":\"2018-01-08\"}"
}

jq -c '.stuff | fromjson' stuff.json

The output:

{"date":"2018-01-08"}
2
  • This seems unecssary. Provided alternative answer Apr 27 '18 at 18:33
  • 3
    it is actually nice because you can do directly things like jq -c '.stuff | fromjson | .date' stuff.json for example; which you cannot do without the fromjson function
    – Tom
    Oct 5 '20 at 13:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.