0

I have the following JSON data:

{
    "Name": "No.reply",
    "Email": "[email protected]",
    "ID": 5930,
    "details": 
    {
        "message": "Your name: john doe\nEmail: [email protected]\nSubject: I need help with this\nDescription: I can find the download for the manual but I can only find the free updater for Windows or Mac. Can you help me please as I have a chrome notebook and Moto smart phone. Thank you. John doe"
    }
}

The name and email fields from the top level are irrelevant, as they are from the automated email. The information I need is in the message field and in the ID field, which is related to John Doe's info.

Anyway, this is what I need to be filtered and how it should be saved to a new file in this order:

  • Name: it should read the lines after this variable, regardless of the text.
  • Email: same as above
  • Subject: same as above
  • Description: same as above
  • ID: same as above

So, I need to remove the quotes, the newline character, assign those specific strings to a variable via bash, and read what it's after those strings.

I was able to come up with something, but it doesn't work for this JSON output: (only works if the text file is properly formatted)

while IFS=''
do  case "$line" in
    "Name:"*)             uservar="${line#*: }" ;;
    "Email:"*)            emailvar="${line#*: }" ;;
    "Subject:"*)          subject="${line#*: }" ;;
    "Message:"*)          message="${line#*: }" ;;
    "ID:"*)        ticketidvar="${line#*: }" ;;
    esac
done <<-EOF
$(pbpaste)
EOF
5
  • Welcome to U&L! That JSON seems malformed -- are you sure there's not a comma after "ID": 5930?
    – JigglyNaga
    Oct 23, 2018 at 11:22
  • use this codebeautify.org/online-json-editor to validate your json data. Oct 23, 2018 at 11:24
  • Hi to both. Thank you for your suggestions. I have now added a comma and validated the JSON data accordingly.
    – HV83
    Oct 23, 2018 at 11:31
  • You've said that the details field is irrelevant, but you appear to want to extract "your name", "subject" and "description" from message, which is inside details. Which is true? And which email do you want: the top-level Email member ("no.reply") or the one inside message ("johndoe")?
    – JigglyNaga
    Oct 23, 2018 at 11:32
  • Sorry about that, I edited my question again to better explain what I want. The info I need is in the message, so I'll need the Email info inside message (john doe). The no.reply information is to be ignored. Thank you!
    – HV83
    Oct 23, 2018 at 11:41

5 Answers 5

5

This assumes that the Description: ... part of the message is a single line, and that the headers are in the canonical form (no " subJECT :hey", please).

It's using jq's @sh format spec to escape its output in a manner suitable for the shell (with single quotes). Thanks to @Stéphane Chazelas for corrections.

parse_json(){
  jq=$(jq -r '[.Email,.ID,(.details.message | split("\n")) | @sh] | join(" ")' -- "$1")
  eval "set -- $jq"
  email=$1; shift
  id=$1; shift
  for l; do
    case $l in
    "Your name: "*) name="${l#*: }";;
    "Subject: "*) subject="${l#*: }";;
    "Description: "*) description="${l#*: }";;
    # remove the following line if you want the .Email from json instead
    "Email: "*) email="${l#*: }";;
    esac
  done
  echo "id={$id}"
  echo "name={$name}"
  echo "email={$email}"
  echo "subject={$subject}"
  echo "description={$description}"
}

fz:/tmp% parse_json a.json
id={5930}
name={john doe}
email={[email protected]}
subject={I need help with this}
description={I can find the download for the manual but I can only find the free updater for Windows or Mac. Can you help me please as I have a chrome notebook and Moto smart phone. Thank you. John doe

The case ... esac above could be replaced with something that will create variables with the same names as the headers with the non-alphanumeric characters replaced by underscores. This will only work with shells that support ${var//pat/repl} substitutions(bash, zsh, ksh93):

parse_json(){
  jq=$(jq -r '[.Email,.ID,(.details.message | split("\n")) | @sh] | join(" ")' -- "$1")
  eval "set -- $jq"
  Email=$1; shift
  ID=$1; shift
  for l; do
    v="${l#*: }"; k="${l%%: *}"; eval "${k//[!a-zA-Z0-9]/_}=\$v"
  done
}

show_vars(){
  for v in ID Your_name Email Subject Description; do
    eval "echo \"$v=[\$$v]\""
  done
}

fz:/tmp$ parse_json a.json
fz:/tmp$ show_vars
ID=[5930]
Your_name=[john doe]
Email=[[email protected]]
Subject=[I need help with this]
Description=[I can find the download for the manual but I can only find the free updater for Windows or Mac. Can you help me please as I have a chrome notebook and Moto smart phone. Thank you. John doe]
4
  • Hi there, thank you so much for your help. One very noobish question: How do I run this code? Can I create a bash script with it?
    – HV83
    Oct 23, 2018 at 16:10
  • you should insert the parse_json() function in your script, with the echo ... lines deleted. When called as in the example, it will set set the name, email, subject, etc variables. Don't run it in a subcommand/subprocess. Notice that you'll have to modify it if you want the email to be that from the message instead of that from the top json object.
    – user313992
    Oct 23, 2018 at 16:59
  • @HV83 I've updated the example to set the email to the one from inside the message, and added another one that generates variables based on the header names (Your name => $Your_name, etc).
    – user313992
    Oct 24, 2018 at 12:25
  • Thank you very much for your help! This is exactly what I was looking for.
    – HV83
    Oct 24, 2018 at 13:18
2

You could use an associative array:

typeset -A field="($(jq -r '
   "[ID]=" + (.ID|@sh),
   (.details.message|capture("(?m)^(?<t>.*?): (?<v>.*)"; "g")|
     "["+(.t|@sh)+"]="+(.v|@sh))' file.json))"

Then you'd have the ID in ${field[ID]}, subject in ${field[Subject]}...

On your sample, typeset -p field outputs:

declare -A fields=(
  [Description]="I can find the download for the manual but I can only find the free updater for Windows or Mac. Can you help me please as I have a chrome notebook and Moto smart phone. Thank you. John doe"
  [ID]="5930"
  [Subject]="I need help with this"
  [Email]="[email protected]"
  ["Your name"]="john doe"
)
0

I am not sure I understand you correctly, but if the intention is to create variables out of the contents of .details.message I would suggest the following.

First extract .details.message in raw mode with jq:

jq -r '.details.message' in.json

Now parse this with sed to make it bash-parsable:

parse.sed

/^Your name: /   s//uservar="/
/^Email: /       s//emailvar="/
/^Subject: /     s//subject="/
/^Description: / s//message="/
s/$/"/

Now you can source this in your script, e.g.:

. <(jq -r '.details.message' in.json | sed -f parse.sed)
echo $uservar, $emailvar

Output:

john doe, [email protected]
0

Not so sophisticated (I suppose the shown text is in some file let us say json.txt or you can send the text to the first sed through a pipe):

sed s/'$'/:/ json.txt |\ ... add a : at the end of each line

sed s/'\\n'/:/g |\ ... change any \n to :

sed s/'"'//g |\ ... remove quots

awk 'BEGIN{RS="}";FS=":"}{if($7){print "Name: "$3"\nEmail: "$5"\nSubject: "$17"\nDescription: "$19"\nID: "$7}}'

In awk set the row separator to }, field separator to : and compose your output. There may left some ending commas, you can easily remove.

0

Assuming that all you need is the decoded .details.message string followed by a line with the ID:

jq -r '.details.message, "ID: \(.ID)"' file >outfile

This decodes the .details.message string and adds a separate line with the value of the top-level .ID key, prefixed by ID: .

For the given JSON document, this would generate the following output in the file outfile:

Your name: john doe
Email: [email protected]
Subject: I need help with this
Description: I can find the download for the manual but I can only find the free updater for Windows or Mac. Can you help me please as I have a chrome notebook and Moto smart phone. Thank you. John doe
ID: 5930

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .