Hot answers tagged

64

The availability of parsers in nearly every programming language is one of the advantages of JSON as a data-interchange format. Rather than trying to implement a JSON parser, you are likely better off using either a tool built for JSON parsing such as jq or a general purpose script language that has a JSON library. For example, using jq, you could pull out ...


48

jq has a filter, @csv, for converting an array to a CSV string. This filter takes into account most of the complexities associated with the CSV format, beginning with commas embedded in fields. (jq 1.5 has a similar filter, @tsv, for generating tab-separated-value files.) Of course, if the headers and values are all guaranteed to be free of commas and ...


43

jq . file.json is what I was looking for. I didn't realize that the . is a filter and not a placeholder for the piped in content: . The absolute simplest (and least interesting) filter is .. This is a filter that takes its input and produces it unchanged as output. And the man page makes it clear that the filter is a required argument.


33

Jq is the right tool for processing JSON data: jq '.items[].properties | to_entries[] | "\(.key) : \(.value)"' input.json The output: "content : \n#!/bin/bash\n\n# Set KAFKA specific environment variables here.\n\n# The java implementation to use.\nexport JAVA_HOME={{java64_home}}\nexport PATH=$PATH:$JAVA_HOME/bin\nexport PID_DIR={{kafka_pid_dir}}\nexport ...


31

I prefer to make each record an row in my CSV. jq '.data | map([.displayName, .rank, .value] | join(", ")) | join("\n")'


30

Given just this file, you can do something like: <testfile jq -r '.data | map(.displayName), map(.value) | join(", ")' The . operator selects a field from an object/hash. Thus, we start with .data, which returns the array with the data in it. We then map over the array twice, first selecting the displayName, then selecting the value, giving us two ...


30

Do it in jq jq -r '.host_components[].HostRoles.host_name | join(",")' No, that's wrong. This is what you need: jq -r '.host_components | map(.HostRoles.host_name) | join(",")' Demo: jq -r '.host_components | map(.HostRoles.host_name) | join(",")' <<DATA {"host_components":[ {"HostRoles":{"host_name":"one"}}, {"HostRoles":{"host_name":"two"}}, ...


28

$ jq 'map_values(tostring)' file.json { "id": "1", "customer": "user", "plate": "BMT-216-A", "country": "GB", "amount": "1000", "pndNumber": "20000", "zoneNumber": "4" } Redirect to a new file and then move that to the original filename. For a more thorough conversion of numbers in non-flat structures into strings, consider jq '(..|select(...


26

Please, please don’t get into the habit of parsing structured data with unstructured tools. If you’re parsing XML, JSON, YAML etc., use a specific parser, at least to convert the structured data into a more appropriate form for AWK, sed, grep etc. In this case, gron would help greatly: $ gron yourfile | grep -F .properties. json.items[0].properties.content ...


25

If you would use: $ cat members.json | \ python -c 'import json,sys;obj=json.load(sys.stdin);print obj;' you can inspect the structure of the nested dictonary obj and see that your original line should read: $ cat members.json | \ python -c 'import json,sys;obj=json.load(sys.stdin);print obj["hits"]["hits"][0]["_source"]["'$1'"]'; to the to ...


24

With jq's fromjson function: Sample stuff.json contents: { "stuff": "{\"date\":\"2018-01-08\"}" } jq -c '.stuff | fromjson' stuff.json The output: {"date":"2018-01-08"}


22

With GNU grep: N=10; grep -roP ".{0,$N}foo.{0,$N}" . Explanation: -o => Print only what you matched -P => Use Perl-style regular expressions The regex says match 0 to $N characters followed by foo followed by 0 to $N characters. If you don't have GNU grep: find . -type f -exec \ perl -nle ' BEGIN{$N=10} print if s/^.*?(.{0,$N}foo.{0,$...


20

Try to use this one: grep -r -E -o ".{0,10}wantedText.{0,10}" * -E tells, that you want to use extended regex -o tells, that you want to print only the match -r grep is looking for result recursively in the folder REGEX: {0,10} tells, how many arbitrary characters you want to print . represents an arbitrary character (a character itself wasn't ...


20

You can use jq to process json files in shell. For example, I saved your sample json file as raul.json and then ran: $ jq .message.temperature raul.json 409.5 25.1 409.5 $ jq .message.humidity raul.json null 40 null jq is available pre-packaged for most linux distros. There's probably a way to do it in jq itself, but the simplest way I found to get ...


19

If you look through the formal documentation of the python JSON library you see that the invocation of json.tool should be python -mjson.tool. This indicates that the program in the file tool.py under the json directory of your python installation, or that it is in the file __init__.py in the tool directory under json in your python installation. The file ...


19

VALUE=<<PERSON some data PERSON echo "$VALUE" No output. A here-document is a redirection, you can't redirect into a variable. When the command line is parsed, redirections are handled in a separate step from variable assignments. Your command is therefore equivalent to (note the space) VALUE= <<PERSON some data PERSON That is, it assigns ...


18

There is a raw flag for this -r output raw strings, not JSON texts; jq -rc .stuff stuff.json Output {"date":"2018-01-08"}


18

use the alternative operator: // so : $jq -r '.|[.login, .lastLoginTime // "-" , .lastLoginFrom // "-" ]|@tsv' test_json 050111 1529730115000 192.168.66.230 050112 - -


17

Simply pipe to keys function: Sample input.json: { "connections": { "host1": { "ip": "10.1.2.3" }, "host2": { "ip": "10.1.2.2" }, "host3": { "ip": "10.1.18.1" } } } jq -r '.connections | keys[] as $k | "\($k), \(.[$k] | .ip)"' input.json The output: host1, 10.1.2.3 host2, 10.1.2.2 host3, 10.1.18.1


15

You can use following python script to parse that data. Lets assume that you have JSON data from arrays in files like array1.json, array2.json and so on. import json import sys from pprint import pprint jdata = open(sys.argv[1]) data = json.load(jdata) print "InstanceId", " - ", "Name", " - ", "Owner" print data["Instances"][0]["InstanceId"], " - " ,data[...


15

Using jq : $ cat json [ { "item1": "value1", "item2": "value2", "sub items": [ { "subitem": "subvalue" } ] }, { "item1": "value1_2", "item2": "value2_2", "sub items_2": [ { "subitem_2": "subvalue_2" } ] } ] CODE: arr=( $(jq -r '.[].item2' json) ) printf '%s\n' "${arr[@]}" ...


14

(Posting @glennjackman comment as a community answer to prevent system from autodeleting the question) jq '.body.test2 = ["hi"]' will do it


13

Version 1.7 includes support for JSON: http://mama.indstate.edu/users/ice/tree/changes.html Per the man page (under XML/JSON/HTML OPTIONS): -J Turn on JSON output. Outputs the directory tree as an JSON formatted array. e.g. $ tree -J /home/me/trash/...


13

JSON=\''{"hostname": "localhost", "outdir": "'"$OUTDIR"'", "port": 20400, "size": 100000}'\' That is get out of the single quotes for the expansion of $OUTDIR. We did put that expansion inside double-quotes for good measure even though for a scalar variable assignment it's not strictly necessary. When you're passing the $JSON variable to echo, quotes are ...


12

Converting JSON to BSON There's this C implementation that includes a Makefile called json2bson.c. Converting BSON to JSON The MongoDB project provides a tool called bsondump. The bsondump converts BSON files into human-readable formats, including JSON. For example, bsondump is useful for reading the output files generated by mongodump. Usage $ ...


12

cat file.json | json_pp #perl utility cat file.json | jq . jq packs much more than just pretty-printing abilities.


12

jq solution: jq -r '.UserDetailList[] | .UserName as $u | .AttachedManagedPolicies[] | ([$u, .PolicyName] | @csv)' input.json The output: "citrix-xendesktop-ec2-provisioning","AmazonEC2FullAccess" "citrix-xendesktop-ec2-provisioning","AmazonS3FullAccess" "rundeck-read-only-iam-permissions","IAMReadOnlyAccess"


11

This unfortunately does not work with GNU coreutils. grep -o -c WORD file If it works on your platform, it's an elegant and fairly intuitive solution; but the GNU folks are still thinking.


11

There are two obvious ways to represent columnar data output in JSON: as an array of arrays and as an array of objects. In the former case you convert each line of the input to an array; in the latter, to an object. The commands listed bellow work at least with the output of procps-ng on Linux for the commands ps and ps -l. Option #1: array of arrays ...


11

Changed the code to this and it worked: content=$(curl -s -X GET -H "Header:Value" http://127.0.0.1:8200/etc) username=$( echo jq -r '.data.value' <<< "${content}" ) echo ${username}


Only top voted, non community-wiki answers of a minimum length are eligible