0

I have the following JSON data:

{
    "Name": "No.reply",
    "Email": "no.reply@xxxxxx.com",
    "ID": 5930,
    "details": 
    {
        "message": "Your name: john doe\nEmail: johndoe@xxxxxxx.com\nSubject: I need help with this\nDescription: I can find the download for the manual but I can only find the free updater for Windows or Mac. Can you help me please as I have a chrome notebook and Moto smart phone. Thank you. John doe"
    }
}

The name and email fields from the top level are irrelevant, as they are from the automated email. The information I need is in the message field and in the ID field, which is related to John Doe's info.

Anyway, this is what I need to be filtered and how it should be saved to a new file in this order:

  • Name: it should read the lines after this variable, regardless of the text.
  • Email: same as above
  • Subject: same as above
  • Description: same as above
  • ID: same as above

So, I need to remove the quotes, the newline character, assign those specific strings to a variable via bash, and read what it's after those strings.

I was able to come up with something, but it doesn't work for this JSON output: (only works if the text file is properly formatted)

while IFS=''
do  case "$line" in
    "Name:"*)             uservar="${line#*: }" ;;
    "Email:"*)            emailvar="${line#*: }" ;;
    "Subject:"*)          subject="${line#*: }" ;;
    "Message:"*)          message="${line#*: }" ;;
    "ID:"*)        ticketidvar="${line#*: }" ;;
    esac
done <<-EOF
$(pbpaste)
EOF
  • Welcome to U&L! That JSON seems malformed -- are you sure there's not a comma after "ID": 5930? – JigglyNaga Oct 23 '18 at 11:22
  • use this codebeautify.org/online-json-editor to validate your json data. – BlackCrystal Oct 23 '18 at 11:24
  • Hi to both. Thank you for your suggestions. I have now added a comma and validated the JSON data accordingly. – HV83 Oct 23 '18 at 11:31
  • You've said that the details field is irrelevant, but you appear to want to extract "your name", "subject" and "description" from message, which is inside details. Which is true? And which email do you want: the top-level Email member ("no.reply") or the one inside message ("johndoe")? – JigglyNaga Oct 23 '18 at 11:32
  • Sorry about that, I edited my question again to better explain what I want. The info I need is in the message, so I'll need the Email info inside message (john doe). The no.reply information is to be ignored. Thank you! – HV83 Oct 23 '18 at 11:41
5

This assumes that the Description: ... part of the message is a single line, and that the headers are in the canonical form (no " subJECT :hey", please).

It's using jq's @sh format spec to escape its output in a manner suitable for the shell (with single quotes). Thanks to @Stéphane Chazelas for corrections.

parse_json(){
  jq=$(jq -r '[.Email,.ID,(.details.message | split("\n")) | @sh] | join(" ")' -- "$1")
  eval "set -- $jq"
  email=$1; shift
  id=$1; shift
  for l; do
    case $l in
    "Your name: "*) name="${l#*: }";;
    "Subject: "*) subject="${l#*: }";;
    "Description: "*) description="${l#*: }";;
    # remove the following line if you want the .Email from json instead
    "Email: "*) email="${l#*: }";;
    esac
  done
  echo "id={$id}"
  echo "name={$name}"
  echo "email={$email}"
  echo "subject={$subject}"
  echo "description={$description}"
}

fz:/tmp% parse_json a.json
id={5930}
name={john doe}
email={johndoe@xxxxxxx.com}
subject={I need help with this}
description={I can find the download for the manual but I can only find the free updater for Windows or Mac. Can you help me please as I have a chrome notebook and Moto smart phone. Thank you. John doe

The case ... esac above could be replaced with something that will create variables with the same names as the headers with the non-alphanumeric characters replaced by underscores. This will only work with shells that support ${var//pat/repl} substitutions(bash, zsh, ksh93):

parse_json(){
  jq=$(jq -r '[.Email,.ID,(.details.message | split("\n")) | @sh] | join(" ")' -- "$1")
  eval "set -- $jq"
  Email=$1; shift
  ID=$1; shift
  for l; do
    v="${l#*: }"; k="${l%%: *}"; eval "${k//[!a-zA-Z0-9]/_}=\$v"
  done
}

show_vars(){
  for v in ID Your_name Email Subject Description; do
    eval "echo \"$v=[\$$v]\""
  done
}

fz:/tmp$ parse_json a.json
fz:/tmp$ show_vars
ID=[5930]
Your_name=[john doe]
Email=[johndoe@xxxxxxx.com]
Subject=[I need help with this]
Description=[I can find the download for the manual but I can only find the free updater for Windows or Mac. Can you help me please as I have a chrome notebook and Moto smart phone. Thank you. John doe]
  • Hi there, thank you so much for your help. One very noobish question: How do I run this code? Can I create a bash script with it? – HV83 Oct 23 '18 at 16:10
  • you should insert the parse_json() function in your script, with the echo ... lines deleted. When called as in the example, it will set set the name, email, subject, etc variables. Don't run it in a subcommand/subprocess. Notice that you'll have to modify it if you want the email to be that from the message instead of that from the top json object. – qubert Oct 23 '18 at 16:59
  • @HV83 I've updated the example to set the email to the one from inside the message, and added another one that generates variables based on the header names (Your name => $Your_name, etc). – qubert Oct 24 '18 at 12:25
  • Thank you very much for your help! This is exactly what I was looking for. – HV83 Oct 24 '18 at 13:18
2

You could use an associative array:

typeset -A field="($(jq -r '
   "[ID]=" + (.ID|@sh),
   (.details.message|capture("(?m)^(?<t>.*?): (?<v>.*)"; "g")|
     "["+(.t|@sh)+"]="+(.v|@sh))' file.json))"

Then you'd have the ID in ${field[ID]}, subject in ${field[Subject]}...

On your sample, typeset -p field outputs:

declare -A fields=(
  [Description]="I can find the download for the manual but I can only find the free updater for Windows or Mac. Can you help me please as I have a chrome notebook and Moto smart phone. Thank you. John doe"
  [ID]="5930"
  [Subject]="I need help with this"
  [Email]="johndoe@xxxxxxx.com"
  ["Your name"]="john doe"
)
0

I am not sure I understand you correctly, but if the intention is to create variables out of the contents of .details.message I would suggest the following.

First extract .details.message in raw mode with jq:

jq -r '.details.message' in.json

Now parse this with sed to make it bash-parsable:

parse.sed

/^Your name: /   s//uservar="/
/^Email: /       s//emailvar="/
/^Subject: /     s//subject="/
/^Description: / s//message="/
s/$/"/

Now you can source this in your script, e.g.:

. <(jq -r '.details.message' in.json | sed -f parse.sed)
echo $uservar, $emailvar

Output:

john doe, johndoe@xxxxxxx.com
0

Not so sophisticated (I suppose the shown text is in some file let us say json.txt or you can send the text to the first sed through a pipe):

sed s/'$'/:/ json.txt |\ ... add a : at the end of each line

sed s/'\\n'/:/g |\ ... change any \n to :

sed s/'"'//g |\ ... remove quots

awk 'BEGIN{RS="}";FS=":"}{if($7){print "Name: "$3"\nEmail: "$5"\nSubject: "$17"\nDescription: "$19"\nID: "$7}}'

In awk set the row separator to }, field separator to : and compose your output. There may left some ending commas, you can easily remove.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.