0

I am working with some location data and am querying an API with U.S. postal codes and returns a result like:

{ 
 "resultStatus":"SUCCESS",
  "zip5":"30301",
  "defaultCity":"ATLANTA",
  "defaultState":"GA",
  "defaultRecordType":"PO BOX",
  "citiesList":[],
  "nonAcceptList":[{"city":"ATL","state":"GA"}]
}

I need to parse this to create output that also includes the full state name as well like: ATLANTA, Georgia, GA, 30301

I created an associative array called States so that ${States[GA]} would return the value Georgia and tried to pass the array to jq as an argument like:

curl -sS <enpoint> |jq -r '"${States[\(.defaultState)]}, \(.defaultState), \(.defaultCity), \(.zip5)"'

Which resulted in the output ${States[GA]}, GA, ATLANTA, 30301

Is there anyway to pass and evaluate bash arrays in jq filters or similar?

Only option that I am seeing is to capture the output and pass through eval, but of course eval is evil... I am also going to be doing this thousands of times and combining with other external data from a file so I would prefer a better option than constructing convoluted strings with embedded bash arguments and eval'ing it.

EDIT

Forgot to mention I tried search jq 1.6 manual without luck and found this SO Post which led me to try passing States array as jq arg like so:

curl -sS <enpoint> |jq -r --arg states $States '"$states[\(.defaultState)], \(.defaultState), \(.defaultCity), \(.zip5)"'

but still no luck.

Workable script implementing steeldrivers answer:

#! /bin/bash

# Do once and save
statesJson=$(for state in "${!StatesArray[@]}"; do 
                printf '{"name":"%s",\n"value":"%s"}\n' $state "${StatesArray[$state]}";
            done \
            | jq -s 'reduce .[] as $i ({}; .[$i.name] = $i.value)');

# Read zip and 6 other values from SourceDataFile.csv
while IFS=',' read -r zip fileValue02 fileValue03 fileValue04 fileValue05 fileValue06 fileValue07; do 

    # Use zip from file to get expanded location data.
    expandedLocationData=$(curl -sS '<apiEndpoint>' -H <Headers> --data "zip=$zip" |jq -r --argjson states "${statesJson}" '"\United States, US, ($states[.defaultState]), \(.defaultState), \(.defaultCity), \(.zip5)"');

    # Do useful things with the completed data set.
    echo "${expandedLocationData}, ${fileValue02} ${fileValue03}, ${fileValue04}, ${fileValue05}, ${fileValue06}, ${fileValue07}" > ./DestinationDataFile.csv

done < ./SourceDataFile.csv
0

If you can turn the bash array into a valid JSON object, then (with minor tweaks to the string interpolation) you could pass it using --argjson. For example, given

$ declare -p States
declare -A States=([GA]="Georgia" [NY]="New York" [ME]="Maine" )

then referring to

then

$ for k in "${!States[@]}"; do printf '{"name":"%s",\n"value":"%s"}\n' $k "${States[$k]}"; done | jq -s 'reduce .[] as $i ({}; .[$i.name] = $i.value)'
{
  "GA": "Georgia",
  "NY": "New York",
  "ME": "Maine"
}

so that

$ jq -r --argjson states \
    "$(for k in "${!States[@]}"; do printf '{"name":"%s",\n"value":"%s"}\n' $k "${States[$k]}"; done | jq -s 'reduce .[] as $i ({}; .[$i.name] = $i.value)')" \
    '"\($states[.defaultState]), \(.defaultState), \(.defaultCity), \(.zip5)"' file.json
Georgia, GA, ATLANTA, 30301
| improve this answer | |
  • Took me a good several minutes to wrap my head around this. It worked once I figured I needed to Pipe my API results from curl into the first jq call and also remove the "file.json" at the end, which was causing errors. Thanks for the help... the --argjson option is something I can see being used often to inject external data. – DVS Mar 20 at 20:27
  • @DVS tbh it was so ugly I wasn't sure I should post it. It ends up that way because JSON is picky about trailing commas, and the only way I could find to sanitize it was the stream of objects --> array of objects --> object route – steeldriver Mar 20 at 20:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.