0

I'm using sed and I am having trouble finding out how to remove some of the first line of text from each block. Here is my original data:

  "s220823vaps2512":   {
"hostname": "s220823vaps2512",
"description": data",
"type": "Virtual",
"os": "data",
"idc": "data",
"environment": "data",
"deviceclass": "data",
"cores": "data",
"memory": "data",
"frontnet": "data",
"ipset":     {
  "backnet": "data",
  "storagenet": "data",
  "metroclusternet": "data"
},
"roles": "data",
"mounts": "data"
  },
   "s220823vaps2513":   {
"hostname": "s220823vaps2513",
"description": "data",
"type": "Virtual",
"os": "data",
"idc": "data",
"environment": "data",
"deviceclass": "data",
"cores": "data",
"memory": "data",
"frontnet": "data",
"ipset":     {
  "backnet": "data",
  "storagenet": "data",
  "metroclusternet": "data"
},

I want to use sed to successfully remove everything up until the { on the first line of each block leaving me with a data format similar to

{
 "hostname": "data",
  "etc": "etc",
},
{
 "hostname": "data",
  "etc": "etc",
},
  • 1
    Looks like JSON. Is it? If so, use a JSON parser. It's much much better that way. – Sobrique Oct 20 '15 at 9:12
2

This answer is predicated on the above looking rather a lot like JSON, so I've therefore assumed it probably is JSON. But it's incomplete in your sample, which I have also assumed is a typo. If this isn't the case... well, you already have sed answers.

Please - don't use a regex to parse JSON. It's nasty. regex is bad at recursive tagged data types like JSON/XML. It's at best a dirty hack which creates brittle code in future.

Similarly - JSON is important that it be complete - I've had to guess how your full JSON looks.

Assuming JSON like this (used http://jsonlint.com/ to tidy up the elements)

{
    "s220823vaps2512": {
        "hostname": "s220823vaps2512",
        "description": "data",
        "type": "Virtual",
        "os": "data",
        "idc": "data",
        "environment": "data",
        "deviceclass": "data",
        "cores": "data",
        "memory": "data",
        "frontnet": "data",
        "ipset": {
            "backnet": "data",
            "storagenet": "data",
            "metroclusternet": "data"
        },
        "roles": "data",
        "mounts": "data"
    },
    "s220823vaps2513": {
        "hostname": "s220823vaps2513",
        "description": "data",
        "type": "Virtual",
        "os": "data",
        "idc": "data",
        "environment": "data",
        "deviceclass": "data",
        "cores": "data",
        "memory": "data",
        "frontnet": "data",
        "ipset": {
            "backnet": "data",
            "storagenet": "data",
            "metroclusternet": "data"
        }
    }
}

Then the way to just get the bits you want:

#!/usr/bin/env perl
use strict;
use warnings;

use JSON;

my $input = ### as above, snipped for brevity. 

#decode/validated
my $json = decode_json ( $input );
#create new JSON array of elements. 
my $new_json = [map { $json -> {$_} } keys %$json];
#print it out. 
print to_json ( $new_json, { pretty => 1, canonical => 1 } );

And in this way, you create valid JSON output as well as handling cases where e.g. key ordering is different (which is entirely valid in JSON).

0

Simple with sed:

sed 's/^[^{]*{/{/' file
  • Just searches for lines starting with characters which are not { (^[^{]), zero or more times * followed by a {.
  • This all is replaced by a single {.

Edit: If you want to exclude some patterns from being replace (for example the "ipset": line) use this:

sed '/"ipset":/n;s/^[^{]*{/{/' file
  • If the line starts with "ipset":, continue with the next line n.
  • Will this not effect the ipset data? – CMS Oct 20 '15 at 7:10
  • Yes it does, if you don't want that, you need something to identify the other lines: is there a similarity in that names: s220823vaps2513? s-digit-vaps-digit or so? – chaos Oct 20 '15 at 7:15
  • Yes the names are similar in each line, probably around 10 variants though – CMS Oct 20 '15 at 7:17
  • Additionally can we use sed to ignore the ipset line as this is the only line that it would effect? – CMS Oct 20 '15 at 7:18
  • Yes, see my edited answer. – chaos Oct 20 '15 at 7:23
0

This seems to work, at least on the sample of data you provided:

sed -'s/^ .*{/{/' file 

It transforms the first line of each block "name" { (note the leading space) to just {

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.