Hot answers tagged

119

jq . file.json is what I was looking for. I didn't realize that the . is a filter and not a placeholder for the piped in content: . The absolute simplest (and least interesting) filter is .. This is a filter that takes its input and produces it unchanged as output. And the man page makes it clear that the filter is a required argument.


85

Do it in jq jq -r '.host_components[].HostRoles.host_name | join(",")' No, that's wrong. This is what you need: jq -r '.host_components | map(.HostRoles.host_name) | join(",")' Demo: jq -r '.host_components | map(.HostRoles.host_name) | join(",")' <<DATA {"host_components":[ {"HostRoles":{"host_name":"one"}}, {"HostRoles":{"host_name":"two"}}, ...


77

The availability of parsers in nearly every programming language is one of the advantages of JSON as a data-interchange format. Rather than trying to implement a JSON parser, you are likely better off using either a tool built for JSON parsing such as jq or a general purpose script language that has a JSON library. For example, using jq, you could pull out ...


69

jq has a filter, @csv, for converting an array to a CSV string. This filter takes into account most of the complexities associated with the CSV format, beginning with commas embedded in fields. (jq 1.5 has a similar filter, @tsv, for generating tab-separated-value files.) Of course, if the headers and values are all guaranteed to be free of commas and ...


50

Solution: jq -r '.[]|select(.hostname | startswith("abcd"))' jjjj


48

With jq's fromjson function: Sample stuff.json contents: { "stuff": "{\"date\":\"2018-01-08\"}" } jq -c '.stuff | fromjson' stuff.json The output: {"date":"2018-01-08"}


44

I prefer to make each record a row in my CSV. jq '.data | map([.displayName, .rank, .value] | join(", ")) | join("\n")'


35

There is a raw flag for this -r output raw strings, not JSON texts; jq -rc .stuff stuff.json Output {"date":"2018-01-08"}


33

Given just this file, you can do something like: <testfile jq -r '.data | map(.displayName), map(.value) | join(", ")' The . operator selects a field from an object/hash. Thus, we start with .data, which returns the array with the data in it. We then map over the array twice, first selecting the displayName, then selecting the value, giving us two ...


33

Jq is the right tool for processing JSON data: jq '.items[].properties | to_entries[] | "\(.key) : \(.value)"' input.json The output: "content : \n#!/bin/bash\n\n# Set KAFKA specific environment variables here.\n\n# The java implementation to use.\nexport JAVA_HOME={{java64_home}}\nexport PATH=$PATH:$JAVA_HOME/bin\nexport PID_DIR={{kafka_pid_dir}}\nexport ...


32

With GNU grep: N=10; grep -roP ".{0,$N}foo.{0,$N}" . Explanation: -o => Print only what you matched -P => Use Perl-style regular expressions The regex says match 0 to $N characters followed by foo followed by 0 to $N characters. If you don't have GNU grep: find . -type f -exec \ perl -nle ' BEGIN{$N=10} print if s/^.*?(.{0,$N}foo.{0,$...


30

Try to use this one: grep -r -E -o ".{0,10}wantedText.{0,10}" * -E tells, that you want to use extended regex -o tells, that you want to print only the match -r grep is looking for result recursively in the folder REGEX: {0,10} tells, how many arbitrary characters you want to print . represents an arbitrary character (a character itself wasn't ...


30

You can use jq to process json files in shell. For example, I saved your sample json file as raul.json and then ran: $ jq .message.temperature raul.json 409.5 25.1 409.5 $ jq .message.humidity raul.json null 40 null jq is available pre-packaged for most linux distros. There's probably a way to do it in jq itself, but the simplest way I found to get ...


30

$ jq 'map_values(tostring)' file.json { "id": "1", "customer": "user", "plate": "BMT-216-A", "country": "GB", "amount": "1000", "pndNumber": "20000", "zoneNumber": "4" } Redirect to a new file and then move that to the original filename. For a more thorough conversion of numbers in non-flat structures into strings, consider jq '(..|select(...


29

use the alternative operator: // so : $jq -r '.|[.login, .lastLoginTime // "-" , .lastLoginFrom // "-" ]|@tsv' test_json 050111 1529730115000 192.168.66.230 050112 - -


28

If you would use: $ cat members.json | \ python -c 'import json,sys;obj=json.load(sys.stdin);print obj;' you can inspect the structure of the nested dictonary obj and see that your original line should read: $ cat members.json | \ python -c 'import json,sys;obj=json.load(sys.stdin);print obj["hits"]["hits"][0]["_source"]["'$1'"]'; to the to ...


28

Simply pipe to keys function: Sample input.json: { "connections": { "host1": { "ip": "10.1.2.3" }, "host2": { "ip": "10.1.2.2" }, "host3": { "ip": "10.1.18.1" } } } jq -r '.connections | keys[] as $k | "\($k), \(.[$k] | .ip)"' input.json The output: host1, 10.1.2.3 host2, 10.1.2.2 host3, 10.1.18.1


28

Please, please don’t get into the habit of parsing structured data with unstructured tools. If you’re parsing XML, JSON, YAML etc., use a specific parser, at least to convert the structured data into a more appropriate form for AWK, sed, grep etc. In this case, gron would help greatly: $ gron yourfile | grep -F .properties. json.items[0].properties.content ...


28

Using the del function in jq: jq 'del(.list[] | select(.name=="APP1"))' If you wanted to pass the app name as a shell variable to jq you can use the --arg option: jq --arg name "$name" 'del(.list[] | select(.name==$name))'


26

Using jq : $ cat json [ { "item1": "value1", "item2": "value2", "sub items": [ { "subitem": "subvalue" } ] }, { "item1": "value1_2", "item2": "value2_2", "sub items_2": [ { "subitem_2": "subvalue_2" } ] } ] CODE: arr=( $(jq -r '.[].item2' json) ) printf '%s\n' "${arr[@]}" ...


23

I was able to unpack the jsonlz4 by using lz4json: apt-get install liblz4-dev git clone https://github.com/andikleen/lz4json.git cd lz4json make ./lz4jsoncat ~/.mozilla/firefox/*/bookmarkbackups/*.jsonlz4


22

If you look through the formal documentation of the python JSON library you see that the invocation of json.tool should be python -mjson.tool. This indicates that the program in the file tool.py under the json directory of your python installation, or that it is in the file __init__.py in the tool directory under json in your python installation. The file ...


22

JSON=\''{"hostname": "localhost", "outdir": "'"$OUTDIR"'", "port": 20400, "size": 100000}'\' That is get out of the single quotes for the expansion of $OUTDIR. We did put that expansion inside double-quotes for good measure even though for a scalar variable assignment it's not strictly necessary. When you're passing the $JSON variable to echo, quotes are ...


22

Save this script in a file, e.g., mozlz4: #!/usr/bin/env python from sys import stdin, stdout, argv, stderr import os try: import lz4.block as lz4 except ImportError: import lz4 stdin = os.fdopen(stdin.fileno(), 'rb') stdout = os.fdopen(stdout.fileno(), 'wb') if argv[1:] == ['-c']: stdout.write(b'mozLz40\0' + lz4.compress(stdin.read())) elif ...


21

VALUE=<<PERSON some data PERSON echo "$VALUE" No output. A here-document is a redirection, you can't redirect into a variable. When the command line is parsed, redirections are handled in a separate step from variable assignments. Your command is therefore equivalent to (note the space) VALUE= <<PERSON some data PERSON That is, it assigns ...


21

Extracting the members jq -c '.children.values[]|[.path.components[0],.type,.size]' .children.values[] outputs every member of the array .values. | pipes the previous result through the next filter, rather like a shell pipe [...,...,...] makes all the terms inside appear in a single array The -c option produces "compact" format ie. one object per ...


20

Try this, jq '.items[].properties.content' t.json Add -r if you want to get rid of the double quotes


19

Piping stdout to cut with the -b flag; you can instruct grep's output to only bytes 1 through 400 per line. grep "foobar" * | cut -b 1-400


19

(Posting @glennjackman comment as a community answer to prevent system from autodeleting the question) jq '.body.test2 = ["hi"]' will do it


18

Version 1.7 includes support for JSON: http://mama.indstate.edu/users/ice/tree/changes.html Per the man page (under XML/JSON/HTML OPTIONS): -J Turn on JSON output. Outputs the directory tree as an JSON formatted array. e.g. $ tree -J /home/me/trash/...


Only top voted, non community-wiki answers of a minimum length are eligible