1

I use jq as an efficient way to parse out large nested json files. I sometimes want to us jq with a json formatted file which has single quotes instead of double around keys and values like below:

{
  'aggs': {
    'two': {
      'composite': {
        'size': 700
       }
     }
  }
}

I know this is not considered an actual json format, but I was wondering if jq has some parameters I could set to read this since otherwise it is not able to read this file.

If not, what is the most efficient way in linux to replace all single quotes to double so it is readable by jq

1
  • 2
    I think single quotes are acceptable in yaml. Yaml tries to be a superset of json. You could try yq Jan 21 at 20:45

2 Answers 2

3

Using single quotes for the keys, or strings in general, is breaking the JSON format specification, but depending on the contents of the strings in the document, Mike Farah's yq may be able to parse it anyway (with caveats mentioned below), and to convert it into standard JSON:

$ yq -oj eval . file
{
  "aggs": {
    "two": {
      "composite": {
        "size": 700
      }
    }
  }
}

Note that it will convert '1"2' correctly into "1\"2", but that it does not handle '1\'2' (it gives an error), and that it converts '1\"2' into "1\\\"2".

It also does not seem to cope with single quoted strings containing other properly encoded values, such as literal newlines encoded as \n, or literal tabs encoded as \t, translating these into \\n and \\t respectively.

In short, it does not allow single quotes in single-quoted strings, and assumes that single quoted strings contain no JSON-encoded values.

Andrey Kislyuk's yq (which is a wrapper around jq) seems to have the same restrictions and caveats as Mike's yq:

$ yq . file
{
  "aggs": {
    "two": {
      "composite": {
        "size": 700
      }
    }
  }
}

Stephen Levine's yj, on the other hand, refuses to parse the document:

$ yj -jj <file
Error parsing JSON: invalid character '\'' looking for beginning of object key string
3

If in those files quotes are escaped just like json does (prepending \), then you could translate the quotes with tr before feeding the document to jq with something like this:

cat file_with_single_quotes | tr "'" '"' | jq ...

or the simpler:

tr "'" '"' < file_with_single_quotes | jq ...

For the text in your question:

$ tr "'" '"' < file_sq | jq .aggs.two
{
  "composite": {
    "size": 700
  }
}
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.