3

When parsing json, the command-line tool jshon converts numbers to scientific notation, and sometimes tries to round them.

To avoid these problems, I want jshon to consider these numbers as strings. For that, I have found that I need to place quotes around all numbers in the json file. After some unsuccessful googling, I have tried writing a sed command to quote the numbers, but I found it pretty unsafe, and have run into lots of issues already:

 sed -r 's/(" ?[:,] ?)"?([0-9]+(\.[0-9]+)?)"?([,}]|$)/\1"\2"\4/g' $file

I would like to know if there is some stable parser that can give me the desired result. I'm not including an example json file in the question, because I need this code for some little risky operations, and I will be parsing json from random websites.

  • I think that you are looking for a json parser here, you sure you want to use sed? It will be messy and error prone. – Braiam Jul 14 '14 at 0:39
  • @Braiam that was specified in the question. – Teresa e Junior Jul 14 '14 at 0:46
  • you want jq – mikeserv Jul 14 '14 at 1:01
  • @mikeserv I have tried their Try online! and, unfortunately, it converts the numbers to scientific notation and also rounds large numbers. – Teresa e Junior Jul 14 '14 at 1:19
  • Get the download, pipe the output as strings into bc or dc and what you want then back. Else, get node.js. – mikeserv Jul 14 '14 at 1:22
2

Here is a simple python3 script you can call as a filter in a shell. Save the script below in a file named something like filter.py, make executable, and call with ./filter.py <$file

Since I don't know the structure of your JSON file, I assume one json string per line. Let me know if you need it tweaked.

#!/usr/bin/python3
import json
import sys
for line in sys.stdin:
   try:
       print(json.dumps(json.loads(line, parse_float=str)))
   except (ValueError, TypeError):
       pass  # skip lines not parsable by json
       # print(line)  # uncomment to print non-json lines as-is
   # uncomment to ignore all other errors
   # except Exception:
   #    pass
exit(0)

Here is the output of my tests:

$ cat $file
{"a":200.12345678999999, "b":0.00001875}
{"a":200.1234567893029999, "b":0.03091875}
$ ./test.py <$file
{"a": "200.12345678999999", "b": "0.00001875"}
{"a": "200.1234567893029999", "b": "0.03091875"}
  • This is really great! I have also found that the janson lib used by jshon has problems parsing long integers, so I replaced parse_float=str with parse_int=str, parse_float=str. That's a lot of problems with janson, maybe I should have jq a better look! – Teresa e Junior Jul 14 '14 at 19:40
  • I don't think it is a problem with any json library in particular, but with floating point rounding errors native to the processor. I'm not sure about long integers, but I would guess it is a similar issue. – Oleg Jul 14 '14 at 23:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.