0

There is a file with contents

{
  "first_name": "John",
  "last_name": "Smith",
  "is_alive": true,
  "age": 27,
  "address": {
    "street_address": "21 2nd Street",
    "city": "New York",
    "state": "NY",
    "postal_code": "10021-3100"
  },
  "phone_numbers": [
    {
      "type": "home",
      "number": "212 555-1234"
    },
    {
      "type": "office",
      "number": "646 555-4567"
    }
  ],
  "children": [
    "Catherine",
    "Thomas",
    "Trevor"
  ],
  "spouse": null
}

How to add content to a file before the last } character using Busybox utilities so that the file content becomes like below?

{
  "first_name": "John",
  "last_name": "Smith",
  "is_alive": true,
  "age": 27,
  "address": {
    "street_address": "21 2nd Street",
    "city": "New York",
    "state": "NY",
    "postal_code": "10021-3100"
  },
  "phone_numbers": [
    {
      "type": "home",
      "number": "212 555-1234"
    },
    {
      "type": "office",
      "number": "646 555-4567"
    }
  ],
  "children": [
    "Catherine",
    "Thomas",
    "Trevor"
  ],
  "spouse": null,
  "field1": "value1",
  "field2": "value2",
  "field3": "value3",
  "field4": "value4"
}

However, the } character is not necessarily the last character in the file and is not necessarily located on the last line.

So far I have found only this solution

tac file2 | sed '0,/}/s/}/}\n"field4": "value4"\n"field3": "value3",\n"field2": "value2",\n"field1": "value1",\n,/' | tac>tmp_file && mv tmp_file file2
5
  • @ilkkachu not, only after the last } in file
    – Maxim
    Mar 9 at 12:04
  • @ilkkachu yes I was wrong
    – Maxim
    Mar 9 at 12:19
  • Is the data guaranteed to already be a valid JSON string?
    – Kusalananda
    Mar 9 at 18:26
  • @Kusalananda yes the data is a valid json string
    – Maxim
    Mar 9 at 19:48
  • Good answer, spending time figure it out, no edit from others. Looks good enough ^^ Mar 9 at 20:09

5 Answers 5

2

Using any awk and any bourne-like shell:

$ cat tst.sh
#!/usr/bin/env bash

new='\
"field1": "value1",
"field2": "value2",
"field3": "value3",
"field4": "value4"\
'

awk '
    { lines[NR] = $0 }
    $0 == "}" { last = NR - 1 }
    END {
        for ( i=1; i<last; i++ ) {
            print lines[i]
        }
        print lines[i] ","

        indent = lines[i]
        sub(/[^ \t].*/,"",indent)
        gsub(/(^|\n)/,"&"indent,new)
        print new

        for ( ++i; i<=NR; i++ ) {
            print lines[i]
        }
    }
' new="$new" file

$ ./tst.sh file
{
  "first_name": "John",
  "last_name": "Smith",
  "is_alive": true,
  "age": 27,
  "address": {
    "street_address": "21 2nd Street",
    "city": "New York",
    "state": "NY",
    "postal_code": "10021-3100"
  },
  "phone_numbers": [
    {
      "type": "home",
      "number": "212 555-1234"
    },
    {
      "type": "office",
      "number": "646 555-4567"
    }
  ],
  "children": [
    "Catherine",
    "Thomas",
    "Trevor"
  ],
  "spouse": null,
  "field1": "value1",
  "field2": "value2",
  "field3": "value3",
  "field4": "value4"
}

Note that it sets the indent for the new block to be whatever the indent is for the line immediately before it so you don't have to hard-code that indent and it'll work even if you have lines of comments or blank lines after the last } line.

10
  • Works great, but how can I make sure that if a file matches the pattern \s*{\s*}\s*, there is no comma displayed before the added text?
    – Maxim
    Mar 9 at 19:22
  • @Maxim I'm not sure what you mean by that comment. Accept an answer to this question you asked and then ask a new question if you can't figure out how to do whatever else it is you want to do since Chameleon Questions are strongly discouraged.
    – Ed Morton
    Mar 9 at 19:27
  • Why not using the more portable -v variable ? And there's no here-documents in your post. Mar 10 at 0:24
  • Indent levels don't matter for JSON, and neither should key ordering. So it ought to be perfectly ok to add the data just after the initial curly brace too. Well, assuming you implement proper JSON processing in Awk.
    – Kusalananda
    Mar 10 at 6:09
  • @GillesQuénot -v isn't more portable than assigning a variable in the args list, it just makes that variables value available in the BEGIN section, which I'm not using, but it also interprets escape sequences so you can't use -v to pass literal strings into awk and idk what characters the real new string might contain. I removed the mention of here-docs, thanks.
    – Ed Morton
    Mar 10 at 12:20
2

Using the ed utility (tested in a BusyBox container):

ed file<<EOF
$ i
  ,
  "field1": "value1",
  "field2": "value2",
  "field3": "value3",
  "field4": "value4"
.
w
q
EOF
cat file

The output file pass the test.

2
  • Only works if } is on the last line
    – Maxim
    Mar 9 at 19:45
  • Yes, that's the case in your sample input ^^ Mar 9 at 20:06
2

If the last } is the last line then this can be done with head command. Include the last } in your added text:

$ cat addit
  "field1": "value1",
  "field2": "value2",
  "field3": "value3",
  "field4": "value4"
}

And what you want is:

$ (head -n -1 input_file ;cat addit)

You leave out the last } from your added text and instead pipe the above output to this sed:

$ sed '$ a \}'
0
1

If you are very confident that the file ends with }\n (or just } no newline at end of file), you could truncate the last 2 bytes (or 1 byte). Use tail -c to verify beforehand.

After truncate, the file would be missing the last } but be otherwise intact. At this point simply >> append new text (including the } you truncated).

Unfortunately, busybox truncate does not understand relative sizes (-2) so you need to do the math yourself and use stat -c %s to determine the size in bytes.

This method would also work with very large files, without needing to read all the contents first. awk, ed, head -n, etc. all read the entire file.

1
  • 1
    The } character is not necessarily the last character in the file; it may be followed by whitespace and newline characters.
    – Maxim
    Mar 9 at 19:26
1

Note: This answer does not restrict itself to using only utilities built into the Busybox multi-binary but uses the jq utility. This well-known JSON processing tool is available as a static binary for most common architectures, requiring no escalation of permission to install or use. You can also use it via a public Docker image.

Since Busybox multi-binaries may contain a wildly varying number of built-in utilities, depending on exactly what Busybox is being used (302 utilities on Alpine Linux, but an additional 100 extra in the public Busybox Docker image), and since this was not mentioned in the question, I'm assuming that the restriction is purely artificial and not terribly important.


Once you have installed jq you may use it in different ways to add your data, depending on in what form you provide the new data.

Assuming separate keys and unencoded strings, you would use jq with --arg key value to encode each string and create internal jq variables. For values that are already appropriately encoded (numbers, booleans, JSON strings, or JSON fragments) use --argjson instead of --arg. You may then use the key names prepended by $ ($key1, $var2, $lastname etc.) inside the jq expression to access the individual pieces of data, or you may access all such variables using the special $ARGS.named object:

jq  --arg field1 value1 \
    --arg field2 'value2 ("temporarily")' \
    --arg field3 value3 \
    --argjson field4 true \
    --argjson field5 '{ "some json fragment": [ "goes", "here" ] }' \
    '. += $ARGS.named' file.json

With your first example document in file.json, this would produce the following:

{
  "first_name": "John",
  "last_name": "Smith",
  "is_alive": true,
  "age": 27,
  "address": {
    "street_address": "21 2nd Street",
    "city": "New York",
    "state": "NY",
    "postal_code": "10021-3100"
  },
  "phone_numbers": [
    {
      "type": "home",
      "number": "212 555-1234"
    },
    {
      "type": "office",
      "number": "646 555-4567"
    }
  ],
  "children": [
    "Catherine",
    "Thomas",
    "Trevor"
  ],
  "spouse": null,
  "field1": "value1",
  "field2": "value2 (\"temporarily\")",
  "field3": "value3",
  "field4": true,
  "field5": {
    "some json fragment": [
      "goes",
      "here"
    ]
  }
}

You may obviously decide where the data is inserted in the document by specifying a path different from . in the expression above. For example, to add the data to any input object that represents a person who has at least one home phone number starting with "212":

select(
    .phone_numbers |
    map(.type == "home" and (.number | startswith("212"))) |
    any
) += $ARGS.named
1
  • Upvoted. jq is the real thing for this kind of files. Mar 10 at 14:09

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .