2

Let's say I have a file as below

{
    "fruit": "Apple",
}

I want to remove the comma at the end of the line, if and only if the next line contains "}". So, the output will be :

{
    "fruit": "Apple"
}

However, if the file is as below. I do not want to do any change. Since the ,s are not followed by a }

{
    "fruit": "Apple",
    "size": "Large",
    "color": "Red"
}

Anything with sed would be fantastic.

  • Sorry but I don't get what fruit has to do with it. Are you saying that if you have the line "colour": "Red", followed by }, then to leave it alone? – ctrl-alt-delor Nov 29 '18 at 19:52
  • Thanks for the question. All I am saying is, if the pattern fruit occurs and the immediate next line does not contain "}" (close parenthesis) I do not want to remove the comma at the end of the line where the pattern "fruit" exists. However, if "}" is present in the line immediately following the line where the pattern "fruit" exists, I want to remove the comma at the end of that line (where the pattern fruit is present). Let me know if you still have question. – Somy Nov 29 '18 at 19:55
  • 1
    This is an x-y problem. I'm fairly certain you are not concerned with the fruit value at all, you are concerned with making valid json objects. – Jesse_b Nov 29 '18 at 19:56
  • Same question. You just repeated your self. So I am thinking that you don't want valid JSON, if the line with color has a comma at the end, even though the next line is a }. – ctrl-alt-delor Nov 29 '18 at 19:57
  • I am just concerned with the"fruit" pattern. I already have the code in place to make the json valid except for the missing piece I just posted. I would say lets not think about json at all. Just think of the problem piece in general and how we can solve in unix. – Somy Nov 29 '18 at 20:01
2
sed -i.bak ':begin;$!N;s/,\n}/\n}/g;tbegin;P;D' FILE

sed -i.bak = creates a backup of the original file, then applies changes to the file

':begin;$!N;s/,\n}/\n}/g;tbegin;P;D' = anything ending with , followed by new line and }. Remove the , on the previous line

FILE = the file you want to make the change to

4

What makes this a non-trivial problem is that the JSON format does not care about whitespace that does not occur inside keys or data. Therefore,

{ "key": "data" }

is the same as

{ "key":
"data"
}

If you add the possibility of a "broken" JSON file, such as

{ "key":
"data", }

it becomes really difficult to properly parse the document with anything other than a JSON parser that knows how to relax the restrictions of the JSON format when parsing the data.

The Perl JSON module can do that, and also pretty-print the result:

$ cat file.json
{
    "fruit": "Apple",
}
$ perl -MJSON -e '@text=(<>);print to_json(from_json("@text", {relaxed=>1}), {pretty=>1})' file.json
{
   "fruit" : "Apple"
}

Here, we read in the whole text document into the array @text. We then decode this while relaxing the parsing (this enables the JSON document to have commas before } and ] and also to include # comments). We then immediately encode the resulting Perl data structure into JSON again and print it.

Another example:

$ cat file.json
{
    "fruit": "Apple",   # a comment
    "stuff": [1, 2, 3,],
}
$ perl -MJSON -e '@text=(<>);print to_json(from_json("@text", {relaxed=>1}), {pretty=>1})' file.json
{
   "fruit" : "Apple",
   "stuff" : [
      1,
      2,
      3
   ]
}

Without pretty printing:

$ perl -MJSON -e '@text=(<>);print to_json(from_json("@text", {relaxed=>1}))' file.json
{"fruit":"Apple","stuff":[1,2,3]}

(no newline at the end of the output)

For really large documents, you would want to use the module's incremental parsing capability and write a proper script for the conversion.

  • Thanks for sharing the solution. I went ahead with the sed solution so did'nt try the one you shared. But thanks anyway. – Somy Nov 29 '18 at 22:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.