-2

I have a JSON string that returns some key value pairs but if the value is numeric it doesn't enclose it with double quotes so my JSON parsing isn't working for those cases.

My idea is to manipulate the string by searching and replacing any occurrences where it finds : with :" except when the : is followed by an open curly brace {

I would also need to close the double quotes at the end so I need to find a way to look for any 0-9 followed by a comma and replace that comma with ",

I am thinking I need SED to search/replace but I am very new to SED and RegEx so not sure how to start especially omitting places where it finds "{ and using RegEx to find [0-9],

Sample JSON string looks like:

{"data":{"project":{"issue":{"session":{"id":"625fdv6b95e232f08d6cy2686624f315","createdAt":1539849060000,"buildVersionId":"75492373","sdk":{"display":"1.11.1"},"os":{"platform":"unknown","modified":false},"memory":{"free":853000192,"used":1896611840},"storage":{"free":241791528960,"used":14197940224},"device":{"architecture":"arm64","manufacturer":"Samsung"}}}}}}

Desired finished JSON after the change would look like:

{"data":{"project":{"issue":{"session":{"id":"625fdv6b95e232f08d6cy2686624f315","createdAt":"1539849060000","buildVersionId":"75492373","sdk":{"display":"1.11.1"},"os":{"platform":"unknown","modified":false},"memory":{"free":"853000192","used":"1896611840"},"storage":{"free":"241791528960","used":"14197940224"},"device":{"architecture":"arm64","manufacturer":"Samsung"}}}}}}

pretty much the same thing but every number now has double quotes around it.

  • 1
    I see some evidence that JSON numbers should not be quoted. – Jeff Schaller Oct 24 '18 at 19:34
  • Jeff, yes, I understand numbers won't be numeric anymore and will be considered strings but that won't affect what my end result will be so it doesn't matter for me. I won't be sending this edited string to anything else for processing. It's only used to parse and extract data out of it. Thanks – ACCFAN Oct 24 '18 at 19:38
  • What JSON parser are you using to extract data from the document? – Kusalananda Oct 26 '18 at 11:01
  • @Kusalananda I'm not using any parser, that's the problem. I'd love to use jq but I'm working in an environment where jq is not installed and it's hard to get it installed. I'm basically doing what I put in my answer below. grepping for the correct key and trimming the value received. – ACCFAN Oct 30 '18 at 19:06
0

How about

sed 's/:\([0-9]*\)\([,}]\)/:"\1"\2/g' inp >out
  • s/:\([0-9]*\)\([,}]\) - search for : followed by digits, followed by either , or }. Place the digits into capture \1, the , or } into \2.
  • /:"\1"\2/g - replace with :" followed by capture \1 (the digits), followed by ", followed by capture \2 (the original suffix character)

Example

$ cat inp
{"data":{"project":{"issue":{"session":{"id":"625fdv6b95e232f08d6cy2686624f315","createdAt":1539849060000,"buildVersionId":"75492373","sdk":{"display":"1.11.1"},"os":{"platform":"unknown","modified":false},"memory":{"free":853000192,"used":1896611840},"storage":{"free":241791528960,"used":14197940224},"device":{"architecture":"arm64","manufacturer":"Samsung"}}}}}}
$ sed 's/:\([0-9]*\)\([,}]\)/:"\1"\2/g' inp >out
$ cat desired
{"data":{"project":{"issue":{"session":{"id":"625fdv6b95e232f08d6cy2686624f315","createdAt":"1539849060000","buildVersionId":"75492373","sdk":{"display":"1.11.1"},"os":{"platform":"unknown","modified":false},"memory":{"free":"853000192","used":"1896611840"},"storage":{"free":"241791528960","used":"14197940224"},"device":{"architecture":"arm64","manufacturer":"Samsung"}}}}}}
$ diff out desired
$
0

I took a different approach and instead of modifying the JSON data, I modified my grepping for numeric values not to look for the double quotes like so:

buildVersionId=`echo $jsonData | grep -m1 -oP '"buildVersionId"\s*:\s*\K[^"]+"'`

then I remove the last 2 characters ( ," ) from the end to get the actual number only:

buildVersionId=${buildVersionId::${#buildVersionId}-2}
0

This is a Perl script that will use the Perl JSON module to parse the JSON document, turn all non-string values in the resulting hash data structure into strings recursively, and then pretty-print the resulting hash converted back into JSON:

#!/usr/bin/env perl

use strict;
use warnings;

use JSON;

sub recursive_inplace_stringification {
    my $reftype = ref( $_[0] );
    if ( !length($reftype) ) {
        $_[0] = "$_[0]" if defined( $_[0] );
    }
    elsif ( $reftype eq 'ARRAY' ) {
        recursive_inplace_stringification($_) for @{ $_[0] };
    }
    elsif ( $reftype eq 'HASH' ) {
        recursive_inplace_stringification($_) for values %{ $_[0] };
    }
    elsif ( $reftype eq 'JSON::PP::Boolean' ) {
        if ( defined( $_[0] ) && $_[0] ) {
            $_[0] = "true";
        }
        else {
            $_[0] = "false";
        }
    }
    else {
        die("Unsupported reference to $reftype\n");
    }
}

# Get JSON parser
my $json = JSON->new();

# Read JSON from in-data and convert to Perl hash
my $hash = $json->decode(<>);

# Convert numbers and booleans into strings
recursive_inplace_stringification($hash);

# Convert to JSON and print
print $json->allow_nonref->utf8->pretty->encode($hash);

The code was adapted from an answer to a StackOverflow question about how to "stringify" JSON values. I extended it to include booleans since the example document contains an unquoted false value, and I turned it into a stand-alone script.

Testing it:

$ ./script.pl file.json
{
   "data" : {
      "project" : {
         "issue" : {
            "session" : {
               "buildVersionId" : "75492373",
               "id" : "625fdv6b95e232f08d6cy2686624f315",
               "sdk" : {
                  "display" : "1.11.1"
               },
               "os" : {
                  "modified" : "false",
                  "platform" : "unknown"
               },
               "device" : {
                  "architecture" : "arm64",
                  "manufacturer" : "Samsung"
               },
               "memory" : {
                  "used" : "1896611840",
                  "free" : "853000192"
               },
               "storage" : {
                  "free" : "241791528960",
                  "used" : "14197940224"
               },
               "createdAt" : "1539849060000"
            }
         }
      }
   }
}

Remove ->pretty from the last line of the script to get more compact output.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.