Questions tagged [json]

JSON (JavaScript Object Notation) is a lightweight data representation format that resembles JavaScript source and that is can be easily red by machines and edited by humans with some effort.

Filter by
Sorted by
Tagged with
1
vote
2answers
51 views

awk Group By & “Reduce”

Let's say I have data like the following: table_name id table_name col_1 table_name col_2 another_table_name id another_table_name col_1 How can I use awk to group by the first column and reduce the ...
-2
votes
1answer
26 views

Multiple JSON files to single CSV file for ML using Shell or Python

I am doing malware analysis using Cuckoo Sandbox(OpenSource), I have to collected multiple malware samples and have collected logs from Cuckoo Sandbox for analysis,The exported Results are in the ...
0
votes
1answer
20 views

MacOS/Windows - How to extract specific .json file from multiple zips and renaming the .json file as folder it was extracted from

I am dealing with cuckoo sandbox exported data having report.json file under each Zip file eg > 123456.zip each zip has the file in zipfile/reports/report.json i have multiple zip files in a folder ...
0
votes
1answer
55 views

AWK: field operations on “altered” FS and “chaining” operations together

I'm writing a script that uses awk to parse JSON data from MediaWiki API pages in order to retrieve information from Wikipedia tables. This is the sample I'm working with, that's being piped into awk. ...
1
vote
1answer
32 views

Add key/value to json object

I have a json object stored in a shell variable json: { "name": "foo", "array": [ { "name": "bar", "thing&...
0
votes
2answers
44 views

Bash replace multiple lines in a file between two patterns with the content of another file?

I have a long javascript file long.js which has this content as a part of it ... more lines ProcessCoverageController.createExampleQueries = function () { return [ { title: '--...
0
votes
0answers
51 views

jq - json to csv - print first string in array only once

I have this result from jq but I only need the string print only once. jq: cat file.json | jq -r '[.InstanceId] + (.Entries[] | [.IPV4, .IPV6, .MacAddress, .Name]) | @csv' >> final.csv id1,...
-2
votes
0answers
22 views

Error Message: jq: error (at <stdin>:17): string ("String…) has no keys

Please help to solve this issue. I have this command cat file.json | jq -r '(map(keys) | add | unique) as $cols | map(. as $row | $cols | map($row[.])) as $rows | $cols, $rows[] | ...
0
votes
0answers
19 views

JQ error - control characters from U+0000 through U+001F must be escaped

When using "jq" utility for validating Json file, getting error: parse error: Invalid string: control characters from U+0000 through U+001F must be escaped... While manually scanning the ...
5
votes
5answers
290 views

sed parameter substitution with multiline quoted string

Goals Replace the text "scripts: {" with the following string "scripts": { "watch": "tsc -w", in a json file. Attempts I created two variables for source ...
-2
votes
1answer
56 views

Is JSON and YAML considered Markup languages? [closed]

I have my Comp Tia Linux+ (XK0-004) exam tomorrow. I purchased the "Compt Tia Linux+ All in one Exam Guide) book off Amazon and this book is contradicting what Google searches are saying ...
1
vote
1answer
70 views

How to create JSON from associative array

I am writing a bash script on CentOS 7.5 that will execute some MongoDB commands. One of these commands will set replication servers. According to project, number of servers can be different. I have ...
0
votes
1answer
33 views

JQ flat objects to csv

I need to flatten this json array of flat objects [{ "key1a": "value1", "key1b": "value2", ... "key1z": "value50", }, { "...
0
votes
1answer
172 views

How to validate a json to make sure each json object is valid?

I have a json as shown below where I have multiple json objects inside rootLevel. I need to validate my json to make sure each json object should follow this below rules: Value in clientId, procId ...
0
votes
1answer
71 views

convert data in columns to json

I've data in text file in format: alex street1 253465873 alex123@google.com john street2 442893646 3jj33@google.com kevin street3 125234763 ke1vn@yahoo.com I need to convert this data to json format ...
0
votes
0answers
23 views

How to sync edits for a file across multiple linux machines?

I have 3 Linux machines which act as compilers to execute scripts in a parallel fashion. Now to make the execution better with less restarts of resources (where needed), I am developing a framework ...
0
votes
1answer
61 views

How to extract fields from encoded JSON object

Hi i'm trying to extract the tokens last_name, first_name and phone from the following JSON data: {"message":"{\"_\":\"user\",\"pFlags\":{\"contact\&...
1
vote
1answer
58 views

How can I extract two data fields (1 scalar & 1 array) per node out of a very large (> 100,000 lines) JSON file?

I've got a 139,000 line JSON file, whose structure basically looks like this (it's an extract from OpenStreetMap): { "type": "FeatureCollection", "generator": "...
1
vote
1answer
367 views

How to sort a stream of json objects by field value using jq

I'm starting with json that looks like this: { "object": "list", "data": [ { "id": "in_1HW85aFGUwFHXzvl8wJbW7V7", "object&...
5
votes
2answers
319 views

How to extract a value from a JSON file containing an encoded JSON object

The command aws s3api get-bucket-policy --bucket bucketname outputs: { "Policy": "{\"Version\":\"2012-10-17\",\"Id\":\"S3SecureTransportPolicy\&...
0
votes
1answer
43 views

Read all the JSON fields through shell script

I have below job lists. I need to prepare a CSV file for Job Name,Status,Last Start,Last End Joblists : 123.load.nw-data.025 123.load.nw-data.026 123.load.nw-data.027 123.load.nw-...
1
vote
2answers
82 views

Convert huge old json into new json format through some sort of linux commands?

I have original json like this. In general, my json is very large. To make things simpler and easier to understand I have reduced the entries. { "clientSettings":[ { "...
2
votes
1answer
259 views

Merge jq output into a comma separated string and new line

I'm trying to extract specific columns from this JSON file (which is probably incorrectly formatted). Here is a data sample: { "_id" : { "$oid" : "4ddb1e4a9a0881572c000008&...
3
votes
1answer
142 views

Get field and nested field at the same time using jq

I am trying to get two attributes I need from the JIRA API. Given the following input to JQ: { "expand": "names,schema", "startAt": 0, "maxResults": 50, ...
0
votes
3answers
40 views

Not able to get the values in JSON from Variable

I have written small Shell/bash script, where I have some data in sampleData.txt file which I want to convert in Base64 and to pass it in variable $scriptPayload: value=$(cat sampleData.txt) echo &...
0
votes
0answers
7 views

AWS metric filter for JSON event with time stamp

I have below formatted log events in cloudwatch log {"datetime": 2020, "NumberOfThreads": "Ok"} Above event can be filtered using {$.NumberOfThreads = "Ok"} ...
1
vote
0answers
74 views

Retrieve nested element with jq

I have a list of JSON objects(one object is given below) where the parent element will dynamically be changed ("Oauth2", "Oauth2c"), so that, how can I retrieve a nested level ...
0
votes
1answer
60 views

Find and replace based upon pattern in JSON

Based upon URL I need to find uri like Test123.elb.us-east-1.amazonaws.com and change the connectionId from hkl876 to xed763 For example: locate Test999.elb.us-east-1.amazonaws.com and update ...
2
votes
1answer
68 views

How to determine path to a deeply nested json field

How does one go about contructing the path to a deeply nested field from a large json document. The approach I've typically used is to flatten the JSON using a jq expression. map ( with_entries(...
0
votes
0answers
55 views

curl REST calls with Pagination

Curl and jq work well for simple REST calls but the workflow breaks if pagination is involved. For example Github uses links in the header: curl -i "https://api.github.com/search/code?q=addClass+...
1
vote
1answer
294 views

Jq: Select Key and Value of One Prop

I have a json translation file that is structured like this. { "STRING_ID_1": { "en": "Some englisch text.", "de": "Some german text." }, ...
4
votes
1answer
454 views

Limit output depth of jq

I want to explore arbitrary document using jq. To that end, I would like to limit the depth to which jq descends into documents and only show me the first n, e.g. 3, levels. Suppose I have the ...
0
votes
1answer
50 views

How to prettify json from ngrep capture?

Way I found to prettify JSON messages from ngrep command: sudo ngrep -t -W byline port 2012 -d lo | grep "^{" | jq . TL;DR We use to check the behavior of our app using ngrep, the messages ...
0
votes
4answers
106 views

how to capture parameter and value from line

we want to capture only the parameter - segment.bytes and his value example echo "$info" | grep "segment.bytes" {"version":1,"config":{"segment.bytes":...
1
vote
1answer
36 views

add values to json arrays

I have some JSON objects that I need to add a value to if it doesn't already exist. Each object will be the following except the contact_group inside each array (1-5) will vary: { "contact_groups&...
4
votes
3answers
485 views

help to grep/print value of a key in json that is stored in a variable?

I have a variable with nested json, a={"version":"3.0","user":"unknown_unknown","dateGenerated":"2020-07-08T11:53:23Z","status":&...
0
votes
1answer
42 views

recursively find out webpage video file url with wget or something like this

i'm trying to script a bash script which automatically can download video or audio files, maybe even "playlists" from youtube and other platforms like this. Especially the youtube video file ...
0
votes
5answers
89 views

How to parse a line which has a number after a certain string greater than a threshold?

I have a file (list_20.txt) that looks like this: [{"d_prime":"0.475425","variation1":"rs909776","r2":"0.057940","variation2":&...
1
vote
1answer
195 views

Parsing .json in Bash

Have a json file I am trying to parse from podfox to allow me to rename the downloaded files in a "friendly" way. Here's a snippet of json I am working with: { "episodes": [ ...
4
votes
3answers
324 views

within json data, how to transform integer to dollar amount using jq?

I have data like this { "dateCreated": "2020-06-12", "status": "pending", "depositDate": "2020-06-15", "amount": 41237 } { ...
1
vote
2answers
67 views

Change the value of a key in json while the key is mutable using jq

I have json-files which I wan't to modify. The problem is, that the target-key varies in the json-files. E.g. those two json: { "tasks": [{ "type": "type1", ...
3
votes
2answers
529 views

Convert Json table arrays to objects with jq

I am trying to convert a json structure that key1: array of values, key2: array of values ,.... to an array of objects. The size of arrays is same and each object is just aggregate of items at ...
-1
votes
1answer
658 views

Merging multiple JSON files removing duplicate objects by a field [closed]

I am trying to merging hundreds of thousand JSON files that have a single object. with the help of this answer I have successfully added all objects into a single object as an array. Now I want to ...
2
votes
2answers
1k views

Merging multiple json file into one object using JQ with argument

I have thousands of JSON file that I want to merge in one object. Those all are not in a similar format. Let me explain in detail. Here is the first sample of JSON [ { "value 1": 1, ...
0
votes
1answer
316 views

How should I replace a value in JSON file with its md5 value using jq command?

I have a big log file containing a line as below example : {"data_1":210,"target_number":1096748811,"extra_data":66} {"data_1":0,"target_number":7130881445,"extra_data":56} {"data_1":1712,"...
0
votes
1answer
95 views

Merging json files using jq

I am trying to merge two json files using command "jq -s . file1 file2". But I want these file's content to be comma separated, neither it should start with []. file 1: { "node1": { "...
1
vote
2answers
862 views

Parse curl response with sed

I'm trying to call a JSON API with the following curl command on macOS: curl https://api.ipify.org?format=json It returns something like this: {"ip":"xxx.xxx.xxx.xxx"} I would like to extract the ...
0
votes
1answer
80 views

How to print streaming JSON to file after processing with JQ and SED [duplicate]

I am using the following code to download streaming financial data curl -s -H "Content-Type: application/json" -H "Authorization: Bearer XXX..." "https://...=EUR_USD" | jq --raw-output '[.time, .bids[...
0
votes
0answers
54 views

specific filtering of Azure ARM JSON templates using jq

Hi I am trying to filter this JSON AZURE ARM template in order just to return a specific array. This jq is currently providing me with the arrays I need but I need to search through this and grab a ...
-1
votes
4answers
2k views

Bash: how to build JSON from multiline strings [closed]

My awk outputs a multiline string. awkresult=`awk '{...}'` echo "{ \"result\": \"$awkresult\" }" > result.json The multilines must be preserved in the result string. I think the best way is to ...

1
2 3 4 5
7