-3

This question already has an answer here:

this is very short list of my json file:

  "slider-client" : {
    "properties_attributes" : { },
    "properties" : { }
  }
},
{
  "spark2-hive-site-override" : {
    "properties_attributes" : { },
    "properties" : {
      "hive.metastore.client.connect.retry.delay" : "5",
      "hive.server2.enable.doAs" : "false",
      "hive.server2.thrift.port" : "10016",
      "hive.server2.transport.mode" : "binary",
      "hive.metastore.client.socket.timeout" : "1800"
    }
  }
},
{
  "tez-env" : {
    "properties_attributes" : { },
    "properties" : {
      "heap_dump_location" : "/tmp",
      "content" : "\n# Tez specific configuration\nexport TEZ_CONF_DIR={{config_dir}}\n\n# Set HADOOP_HOME to point to a specific hadoop install directory\nexport HADOOP_HOME=${HADOOP_HOME:-{{hadoop_home}}}\n\n# The java implementation to use.\nexport JAVA_HOME={{java64_home}}",
      "enable_heap_dump" : "false",
      "tez_user" : "tez"
    }
  }
},

how we can capture only the lines that start with:

"properties" : {

and ended with

 }

example of expected output

    "properties" : { }
    "properties" : {
      "hive.metastore.client.connect.retry.delay" : "5",
      "hive.server2.enable.doAs" : "false",
      "hive.server2.thrift.port" : "10016",
      "hive.server2.transport.mode" : "binary",
      "hive.metastore.client.socket.timeout" : "1800"
    }

    "properties" : {
      "heap_dump_location" : "/tmp",
      "content" : "\n# Tez specific configuration\nexport TEZ_CONF_DIR={{config_dir}}\n\n# Set HADOOP_HOME to point to a specific hadoop install directory\nexport HADOOP_HOME=${HADOOP_HOME:-{{hadoop_home}}}\n\n# The java implementation to use.\nexport JAVA_HOME={{java64_home}}",
      "enable_heap_dump" : "false",
      "tez_user" : "tez"
    }

what I did until now is this syntax , is it the best approach ?

 awk '/"properties" : {/,/^[[:blank:]]*}$/'  file.json

marked as duplicate by Thor, Rui F Ribeiro, xhienne, Stephen Rauch, dhag Aug 17 '17 at 16:47

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

  • 3
    Your JSON excerpt is not valid JSON, important parts are missing. The simplest way to do what you want is use to use jq, but without knowing the exact JSON format, we can't make a ready-to-use answer. Have you tried reading the jq man page, and attempting it yourself? – dirkt Aug 16 '17 at 7:21
  • yes I read the jq but after few tests I cant to find the right way – jango Aug 16 '17 at 7:27
  • maybe awk that runs on the file – jango Aug 16 '17 at 7:32
  • There are plenty of questions here about parsing a JSON file and sensible answers will always be based on a JSON parser, not on sed/awk. Unless of course you have full control on the JSON output and thus can avoid all kind of side-effects, which you didn't state in your question. – xhienne Aug 17 '17 at 12:21
1

To be a valid JSON your input should be considered as an array of objects.

jq solution:

jq '.[] | to_entries[] | .value | if has("properties") then {"properties": .properties} else empty end' yourfile

The output (corrected):

{
  "properties": {}
}
{
  "properties": {
    "hive.metastore.client.connect.retry.delay": "5",
    "hive.server2.enable.doAs": "false",
    "hive.server2.thrift.port": "10016",
    "hive.server2.transport.mode": "binary",
    "hive.metastore.client.socket.timeout": "1800"
  }
}
{
  "properties": {
    "heap_dump_location": "/tmp",
    "content": "\n# Tez specific configuration\nexport TEZ_CONF_DIR={{config_dir}}\n\n# Set HADOOP_HOME to point to a specific hadoop install directory\nexport HADOOP_HOME=${HADOOP_HOME:-{{hadoop_home}}}\n\n# The java implementation to use.\nexport JAVA_HOME={{java64_home}}",
    "enable_heap_dump": "false",
    "tez_user": "tez"
  }
}
  • I run it but we get this "jq: error: Cannot check whether string has a string key" – jango Aug 16 '17 at 8:20
  • 1
    @jango, I repeat (as I wrote it in my answer): To be a valid JSON your input should be considered as an array of objects – RomanPerekhrest Aug 16 '17 at 8:21
  • 1
    @jango, what we can do? that means that your input is not a valid json or not a json at all. Do restructure your input to make it valid json, or don't call it json – RomanPerekhrest Aug 16 '17 at 8:28
  • 1
    @jango: Please, as has been mentioned multiple times edit your question and include a valid and representative example. Also read and follow these instructions – Thor Aug 16 '17 at 8:30
  • 1
    @jango: You do not listen, so no opinion – Thor Aug 16 '17 at 8:40
-1

to capture the properties until "}" we can do:

awk '/"properties" : {/,/^[[:blank:]]*}$/'  file.json
  • I will happy to get any comments ? – jango Aug 16 '17 at 8:42
  • awk/sed are not the right tools to work with json at all – RomanPerekhrest Aug 16 '17 at 8:55
  • yes but this awk do the job very well – jango Aug 16 '17 at 8:56
  • 2
    it can be called an arbitrary happy case, but not robust solution. It should not be advised to other people/community – RomanPerekhrest Aug 16 '17 at 8:57
  • Does it handle well properties with value { }? – xhienne Aug 17 '17 at 12:18

Not the answer you're looking for? Browse other questions tagged or ask your own question.