1

Using curl, I get some statistical information from my TP-Link router. I put that information to a text file. However, I need this info in JSON format so that I can use it elsewhere. The format of the text file is like this:

[0,0,0,0,0,0]0
enable=1
interval=10
action=0
[1,0,0,0,0,0]1
ipAddress=3232235848
macAddress=EC:0E:C4:4F:XX:XX
totalPkts=201
totalBytes=22914
currPkts=0
currBytes=0
currIcmp=0
currUdp=0
currSyn=0
currIcmpMax=0
currUdpMax=15
currSynMax=0
[2,0,0,0,0,0]1
ipAddress=3232235829
macAddress=00:16:E8:22:XX:XX
totalPkts=972
totalBytes=98730
currPkts=0
currBytes=0
currIcmp=0
currUdp=0
currSyn=0
currIcmpMax=0
currUdpMax=14
currSynMax=4
[3,0,0,0,0,0]1
ipAddress=3232235842
macAddress=70:14:A6:8B:XX:XX
totalPkts=154566
totalBytes=15116490
currPkts=0
currBytes=0
currIcmp=0
currUdp=0
currSyn=0
currIcmpMax=9
currUdpMax=58
currSynMax=60

I need a JSON something like:

{"ipAddress" : "3232235848",
"macAddress" : "EC:0E:C4:4F:XX:XX",
"totalPkts" : "201",
"totalBytes" : "22914",
"currPkts" : "0",
"currBytes" : "0",
"currIcmp" : "0",
"currUdp" : "0",
"currSyn" : "0",
"currIcmpMax" : "0",
"currUdpMax" : "15",
"currSynMax" : "0"}
{"ipAddress" : "3232235829", ...

Could you show me the most efficient way of doing this? I prefer a bash script if it is possible? Tried putting "| jq -R -s -c 'split("\n")'" to the end of curl command but then it looks like: {"ipAddress=3232235848","macAddress=EC:0E:C4:4F:XX:XX" ...

2
  • bash is just your shell, not a text editor
    – Jeff Schaller
    Jun 4, 2017 at 15:04
  • It looks like you created a second account and then tried to say thank you and ask a second question. Please accept an answer to this question before asking a separate one.
    – Jeff Schaller
    Jun 5, 2017 at 9:54

3 Answers 3

1

sed hack:

sed -E -e 's/\[([0-9],){5}[0-9]\][0-9]+/}{/g; s/([[:alnum:]]+)=([[:alnum:]]+)/"\1" : "\2",/; 
           s/,\n}/}/;  1s/.*/{/; $s/.*/&}/' yourfile | sed -Ez 's/,\n?}/ }\n/g'

The output:

{
"enable" : "1",
"interval" : "10",
"action" : "0" }
{
"ipAddress" : "3232235848",
"macAddress" : "EC",:0E:C4:4F:XX:XX
"totalPkts" : "201",
"totalBytes" : "22914",
"currPkts" : "0",
"currBytes" : "0",
"currIcmp" : "0",
"currUdp" : "0",
"currSyn" : "0",
"currIcmpMax" : "0",
"currUdpMax" : "15",
"currSynMax" : "0" }
{
"ipAddress" : "3232235829",
"macAddress" : "00",:16:E8:22:XX:XX
"totalPkts" : "972",
"totalBytes" : "98730",
"currPkts" : "0",
"currBytes" : "0",
"currIcmp" : "0",
"currUdp" : "0",
"currSyn" : "0",
"currIcmpMax" : "0",
"currUdpMax" : "14",
"currSynMax" : "4" }
{
"ipAddress" : "3232235842",
"macAddress" : "70",:14:A6:8B:XX:XX
"totalPkts" : "154566",
"totalBytes" : "15116490",
"currPkts" : "0",
"currBytes" : "0",
"currIcmp" : "0",
"currUdp" : "0",
"currSyn" : "0",
"currIcmpMax" : "9",
"currUdpMax" : "58",
"currSynMax" : "60" }
0
perl -lne 'BEGIN{*1 = sub {print q<{>, join(",\n", splice @A, 0, @A), q<}>}}
   next if $. == 1;
   /^\[(?:\d+,?)+\]\d+$/ and &1,next;
   push @A, join q/ : /, map qq/"$_"/, split /=/;
   eof && &1;
' yourfile
0

Using Miller (mlr) after some light preprocessing with sed:

sed 's/^\[.*//' file | mlr --irs lflf --ifs lf --ojson cat

This first uses sed to clear the lines that start with [. The empty lines that this creates later serve as record separators for Miller.

Miller then converts the input to JSON without further modifying the data. To make the data understandable to Miller, we specify that the input record separator is a double line-feed, and that the input field separator is a single line-feed.

This command, given the example data in the question, would produce the following output:

[
{
  "enable": 1,
  "interval": 10,
  "action": 0
},
{
  "ipAddress": 3232235848,
  "macAddress": "EC:0E:C4:4F:XX:XX",
  "totalPkts": 201,
  "totalBytes": 22914,
  "currPkts": 0,
  "currBytes": 0,
  "currIcmp": 0,
  "currUdp": 0,
  "currSyn": 0,
  "currIcmpMax": 0,
  "currUdpMax": 15,
  "currSynMax": 0
},
{
  "ipAddress": 3232235829,
  "macAddress": "00:16:E8:22:XX:XX",
  "totalPkts": 972,
  "totalBytes": 98730,
  "currPkts": 0,
  "currBytes": 0,
  "currIcmp": 0,
  "currUdp": 0,
  "currSyn": 0,
  "currIcmpMax": 0,
  "currUdpMax": 14,
  "currSynMax": 4
},
{
  "ipAddress": 3232235842,
  "macAddress": "70:14:A6:8B:XX:XX",
  "totalPkts": 154566,
  "totalBytes": 15116490,
  "currPkts": 0,
  "currBytes": 0,
  "currIcmp": 0,
  "currUdp": 0,
  "currSyn": 0,
  "currIcmpMax": 9,
  "currUdpMax": 58,
  "currSynMax": 60
}
]

To avoid that first object in the resulting array, change the cat operation to a filter operation that avoids outputting the first record:

sed 's/^\[.*//' file | mlr --irs lflf --ifs lf --ojson filter 'NR > 1'

Would you additionally prefer to not have the resulting JSON objects wrapped in an array, then use Miller with --no-jlistwrap:

sed 's/^\[.*//' file | mlr --irs lflf --ifs lf --ojson --no-jlistwrap filter 'NR > 1'

... or change to the one-object-per-line JSON output format by changing --ojson to --ojsonl:

sed 's/^\[.*//' file | mlr --irs lflf --ifs lf --ojsonl filter 'NR > 1'

... which would produce

{"ipAddress": 3232235848, "macAddress": "EC:0E:C4:4F:XX:XX", "totalPkts": 201, "totalBytes": 22914, "currPkts": 0, "currBytes": 0, "currIcmp": 0, "currUdp": 0, "currSyn": 0, "currIcmpMax": 0, "currUdpMax": 15, "currSynMax": 0}
{"ipAddress": 3232235829, "macAddress": "00:16:E8:22:XX:XX", "totalPkts": 972, "totalBytes": 98730, "currPkts": 0, "currBytes": 0, "currIcmp": 0, "currUdp": 0, "currSyn": 0, "currIcmpMax": 0, "currUdpMax": 14, "currSynMax": 4}
{"ipAddress": 3232235842, "macAddress": "70:14:A6:8B:XX:XX", "totalPkts": 154566, "totalBytes": 15116490, "currPkts": 0, "currBytes": 0, "currIcmp": 0, "currUdp": 0, "currSyn": 0, "currIcmpMax": 9, "currUdpMax": 58, "currSynMax": 60}

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .