67

I'm familiar with "jq" for parsing json.

I work with one service that produces a json response where one of the properties is itself a json string. How do I convert that quoted value to a valid json string so I can then process it with jq?

For instance, if I just view the plain pretty-printed json from "jq .", here's a short excerpt of the output:

"someJsonString": "{\"date\":\"2018-01-08\", ...

I can use jq to get the value of that property, but I need to convert the quoted string to valid json by "unescaping" it.

I suppose I could pipe it into sed, removing the opening and ending double quotes, and removing all backslashes ("sed -e 's/^"//' -e 's/"$//' -e 's/\\//g'"). That seems to work, but that doesn't seem like the most robust solution.

Update:

Just to be a little clearer on what I'm doing, here are a couple of elided samples that show what I've tried:

% curl -s -q -L 'http://.../1524.json' | jq '.results[0].someJsonString' | jq .
"{\"date\":\"2018-01-08\",...
% echo $(curl -s -q -L 'http:/.../1524.json' | jq '.results[0].someJsonString') | jq .
"{\"date\":\"2018-01-08\",...

Update:

Here's a completely standalone example:

% cat stuff.json | jq .
{
  "stuff": "{\"date\":\"2018-01-08\"}"
}
% cat stuff.json | jq '.stuff'
"{\"date\":\"2018-01-08\"}"
% cat stuff.json | jq '.stuff' | jq .
"{\"date\":\"2018-01-08\"}"

Update:

If I tried to process that last output with a real jq expression, it does something like this:

% cat stuff.json | jq '.stuff' | jq '.date'
assertion "cb == jq_util_input_next_input_cb" failed: file "/usr/src/ports/jq/jq-1.5-3.x86_64/src/jq-1.5/util.c", line 371, function: jq_util_input_get_position
Aborted (core dumped)
7
  • If you use jq to get just the value of the string property, does it return it unescaped? If so, just pipe that into a fresh jq.
    – DopeGhoti
    Jan 8, 2018 at 21:30
  • No, it does not return it unescaped. That's the point. Jan 8, 2018 at 21:30
  • How about echo $(jq statement here)?
    – DopeGhoti
    Jan 8, 2018 at 21:31
  • Nope, no change. Jan 8, 2018 at 21:33
  • @DavidM.Karr, ok, If possible - extend your input with the actual crucial string and the final result Jan 8, 2018 at 21:34

2 Answers 2

100

With jq's fromjson function:

Sample stuff.json contents:

{
  "stuff": "{\"date\":\"2018-01-08\"}"
}

jq -c '.stuff | fromjson' stuff.json

The output:

{"date":"2018-01-08"}
4
  • This seems unecssary. Provided alternative answer Apr 27, 2018 at 18:33
  • 6
    it is actually nice because you can do directly things like jq -c '.stuff | fromjson | .date' stuff.json for example; which you cannot do without the fromjson function
    – Tom
    Oct 5, 2020 at 13:11
  • very nice indeed, it works even for nasty nested cases: jq -c '.some.where | fromjson | .even.deeper | fromjson' stuff.json | jq . > interesting-part.json
    – ojdo
    Mar 10, 2022 at 16:34
  • This also validates the payload as JSON, which is nice.
    – Kusalananda
    Nov 10, 2023 at 5:30
44

There is a raw flag for this

    -r      output raw strings, not JSON texts;

jq -rc .stuff stuff.json

Output

{"date":"2018-01-08"}
7
  • 1
    The difference is that with Roman's answer, you are guaranteed to get valid JSON output, or error messages if it's not valid JSON.
    – Kusalananda
    Apr 27, 2018 at 18:48
  • Valid point, but if this is being used in automation, I think it would be unusual to suddenly not have valid json output. The most convenient form will be perfectly fine almost all the time. It's still good to know about more precise methods, however. Apr 28, 2018 at 22:30
  • 1
    @DavidM.Karr "unusual to suddenly not have valid json output" HA! Riiiight. Error handling in automation? Errors will never happen! Why bother! Feb 6, 2019 at 18:21
  • 2
    This requires piping to another jq for further JSON processing, whereas with Roman's approach you can continue the same jq expression.
    – Raman
    Mar 4, 2019 at 16:53
  • 3
    @cricket_007: tried it with jq 1.5, and confirmed it doesn't work: jq -rc '.stuff.date' produces jq: error (at <stdin>:0): Cannot index string with string "date". However: .stuff | fromjson | .date works fine.
    – Raman
    Mar 5, 2019 at 7:51

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .