10

Is there a convenient way to convert the output of the *nix command "tree" to JSON format?

Edit: I think I didn't describe my problem well enough. My goal is to convert something like:

.
|-- dir1
|   |-- dirA
|   |   |-- dirAA
|   |   `-- dirBB
|   `-- dirB
`-- dir2
    |-- dirA
    `-- dirB

into:

{"dir1" : [{"dirA":["dirAA", "dirAB"]}, "dirB"], "dir2": ["dirA", "dirB"]}
  • How would you expect to see that encapsulated in JSON? Could you give an example and expected results? – Drav Sloan Sep 10 '13 at 20:21
  • @DravSloan I edited the post to show an example – roundrobin Sep 10 '13 at 21:12
  • What would you expect to get if dir1/dirA had subdirectories? – cjm Sep 10 '13 at 21:22
  • {"dir1" : [{"dirA":["dirAA", "dirAB"]}, "dirB"], "dir2": ["dirA", "dirB"]} – roundrobin Sep 10 '13 at 21:42
  • @BausTheBig - I don't think you've thought this through all the way. The tree command isn't the right tool. I might be inclined to do ls -R or a find instead. – slm Sep 10 '13 at 22:12
6

Attempt 1

A solution using just perl, returning a simple hash of hashes structure. Before the OP clarified data format of JSON.

#! /usr/bin/perl

use File::Find;
use JSON;

use strict;
use warnings;

my $dirs={};
my $encoder = JSON->new->ascii->pretty;

find({wanted => \&process_dir, no_chdir => 1 }, ".");
print $encoder->encode($dirs);

sub process_dir {
    return if !-d $File::Find::name;
    my $ref=\%$dirs;
    for(split(/\//, $File::Find::name)) {
        $ref->{$_} = {} if(!exists $ref->{$_});
        $ref = $ref->{$_};
    }
}

File::Find module works in a similar way to the unix find command. The JSON module takes perl variables and converts them into JSON.

find({wanted => \&process_dir, no_chdir => 1 }, ".");

Will iterate down the file structure from the present working directory calling the subroutine process_dir for each file/directory under ".", and the no_chdir tell perl not to issue a chdir() for each directory it finds.

process_dir returns if the present examined file is not a directory:

return if !-d $File::Find::name;

We then grab a reference of the existing hash %$dirs into $ref, split the file path around / and loop with for adding a new hash key for each path.

Making a directory structure like slm did:

mkdir -p dir{1..5}/dir{A,B}/subdir{1..3}

The output is:

{
   "." : {
      "dir3" : {
         "dirA" : {
            "subdir2" : {},
            "subdir3" : {},
            "subdir1" : {}
         },
         "dirB" : {
            "subdir2" : {},
            "subdir3" : {},
            "subdir1" : {}
         }
      },
      "dir2" : {
         "dirA" : {
            "subdir2" : {},
            "subdir3" : {},
            "subdir1" : {}
         },
         "dirB" : {
            "subdir2" : {},
            "subdir3" : {},
            "subdir1" : {}
         }
      },
      "dir5" : {
         "dirA" : {
            "subdir2" : {},
            "subdir3" : {},
            "subdir1" : {}
         },
         "dirB" : {
            "subdir2" : {},
            "subdir3" : {},
            "subdir1" : {}
         }
      },
      "dir1" : {
         "dirA" : {
            "subdir2" : {},
            "subdir3" : {},
            "subdir1" : {}
         },
         "dirB" : {
            "subdir2" : {},
            "subdir3" : {},
            "subdir1" : {}
         }
      },
      "dir4" : {
         "dirA" : {
            "subdir2" : {},
            "subdir3" : {},
            "subdir1" : {}
         },
         "dirB" : {
            "subdir2" : {},
            "subdir3" : {},
            "subdir1" : {}
         }
      }
   }
}

Attempt 2

Okay now with different data structure...

#! /usr/bin/perl

use warnings;
use strict;
use JSON;

my $encoder = JSON->new->ascii->pretty;   # ascii character set, pretty format
my $dirs;                                 # used to build the data structure

my $path=$ARGV[0] || '.';                 # use the command line arg or working dir

# Open the directory, read in the file list, grep out directories and skip '.' and '..'
# and assign to @dirs
opendir(my $dh, $path) or die "can't opendir $path: $!";
my @dirs = grep { ! /^[.]{1,2}/ && -d "$path/$_" } readdir($dh);
closedir($dh);

# recurse the top level sub directories with the parse_dir subroutine, returning
# a hash reference.
%$dirs = map { $_ => parse_dir("$path/$_") } @dirs;

# print out the JSON encoding of this data structure
print $encoder->encode($dirs);

sub parse_dir {
    my $path = shift;    # the dir we're working on

    # get all sub directories (similar to above opendir/readdir calls)
    opendir(my $dh, $path) or die "can't opendir $path: $!";
    my @dirs = grep { ! /^[.]{1,2}/ && -d "$path/$_" } readdir($dh);
    closedir($dh);

    return undef if !scalar @dirs; # nothing to do here, directory empty

    my $vals = [];                            # set our result to an empty array
    foreach my $dir (@dirs) {                 # loop the sub directories         
        my $res = parse_dir("$path/$dir");    # recurse down each path and get results

        # does the returned value have a result, and is that result an array of at 
        # least one element, then add these results to our $vals anonymous array 
        # wrapped in a anonymous hash
        # ELSE
        # push just the name of that directory our $vals anonymous array
        push(@$vals, (defined $res and scalar @$res) ? { $dir => $res } : $dir);
    }

    return $vals;  # return the recursed result
}

And then running the script on the proposed directory structure...

./tree2json2.pl .
{
   "dir2" : [
      "dirB",
      "dirA"
   ],
   "dir1" : [
      "dirB",
      {
         "dirA" : [
            "dirBB",
            "dirAA"
         ]
      }
   ]
}

I found this pretty damn tricky to get right (especially given the "hash if sub directories, array if not, OH UNLESS top level, then just hashes anyway" logic). So I'd be surprised if this was something you could do with sed / awk ... but then Stephane hasn't looked at this yet I bet :)

  • Oh the format for sub dirs is somewhat different now, is the above output format going to be an issue? – Drav Sloan Sep 10 '13 at 22:18
  • Yeah I've been spinning on that format myself. I'm not sure it's standard in any kind of way, can't find much off the shelf that will provide it like so, but your approach is a definite improvement. – slm Sep 10 '13 at 22:19
  • Making any progress with this? 8-) – slm Sep 10 '13 at 23:06
  • I got side tracked with an slm-style-ascii-network-a-gram on another question (pit stop as this one was making my head spin). I'll make a cuppa to correct my caffiene/blood ratio and take another look. – Drav Sloan Sep 10 '13 at 23:08
  • asciio is the tool to make em – slm Sep 10 '13 at 23:11
13

Version 1.7 includes support for JSON:
http://mama.indstate.edu/users/ice/tree/changes.html

Per the man page (under XML/JSON/HTML OPTIONS):

-J     Turn on JSON output. Outputs the directory tree as an JSON formatted array.

e.g.

$ tree -J                                                                                                 

/home/me/trash/tree-1.7.0
[{"type":"directory","name": ".","contents":[
    {"type":"file","name":"CHANGES"},
    {"type":"file","name":"color.c"},
    {"type":"file","name":"color.o"},
    {"type":"directory","name":"doc","contents":[
      {"type":"file","name":"tree.1"},
      {"type":"file","name":"tree.1.fr"},
      {"type":"file","name":"xml.dtd"}
    ]},
    {"type":"file","name":"hash.c"},
    {"type":"file","name":"hash.o"},
    {"type":"file","name":"html.c"},
    {"type":"file","name":"html.o"},
    {"type":"file","name":"INSTALL"},
    {"type":"file","name":"json.c"},
    {"type":"file","name":"json.o"},
    {"type":"file","name":"LICENSE"},
    {"type":"file","name":"Makefile"},
    {"type":"file","name":"README"},
    {"type":"file","name":"strverscmp.c"},
    {"type":"file","name":"TODO"},
    {"type":"file","name":"tree"},
    {"type":"file","name":"tree.c"},
    {"type":"file","name":"tree.h"},
    {"type":"file","name":"tree.o"},
    {"type":"file","name":"unix.c"},
    {"type":"file","name":"unix.o"},
    {"type":"file","name":"xml.c"},
    {"type":"file","name":"xml.o"}
  ]},
  {"type":"report","directories":1,"files":26}
]
5

Here is one way using Perl and the JSON perl module.

$ tree | perl -e 'use JSON; @in=grep(s/\n$//, <>); \
     print encode_json(\@in)."\n";'

Example

Create some sample data.

$ mkdir -p dir{1..5}/dir{A,B}

Here's what it looks like:

$ tree 
.
|-- dir1
|   |-- dirA
|   `-- dirB
|-- dir2
|   |-- dirA
|   `-- dirB
|-- dir3
|   |-- dirA
|   `-- dirB
|-- dir4
|   |-- dirA
|   `-- dirB
`-- dir5
    |-- dirA
    `-- dirB

15 directories, 0 files

Here's a run using the Perl command:

$ tree | perl -e 'use JSON; @in=grep(s/\n$//, <>); print encode_json(\@in)."\n";'

Which returns this output:

[".","|-- dir1","|   |-- dirA","|   `-- dirB","|-- dir2","|   |-- dirA","|   `-- dirB","|-- dir3","|   |-- dirA","|   `-- dirB","|-- dir4","|   |-- dirA","|   `-- dirB","`-- dir5","    |-- dirA","    `-- dirB","","15 directories, 0 files"]

NOTE: This is just an encapsulation of the output from tree. Not a nested hierarchy. The OP changed the question after I suggested this!

  • sorry I think I didn't describe my problem well enough. My goal is to convert something like: |-- dir1 | |-- dirA | |-- dirB |-- dir2 | |-- dirA | |-- dirB into: {"dir1" : ["dirA","dirB"], "dir2": ["dirA","dirB"]} – roundrobin Sep 10 '13 at 20:32
  • @BausTheBig - no problem. Edit your answer and add an example of what you want. – slm Sep 10 '13 at 20:33
  • The data structure the OP seems to be after looks like a Python object. I have almost no knowledge of Python so I can't help but I would guess this kind of structure is easier to build there. – terdon Sep 11 '13 at 9:59
  • @terdon - I left it to Drav, it looked like a Hash of Hashes structure to us. – slm Sep 11 '13 at 11:39
2

I was also searching for a way to output a linux folder / file tree to some JSON or XML file. Why not use this simple terminal command:

tree --dirsfirst --noreport -n -X -i -s -D -f -o my.xml

So, just the Linux tree command, and config your own parameters. Here -X gives XML output. For me, that's OK, and I guess there's some script to convert XML to JSON.

1

You could try this command:

tree -a -J -o *filename*

Replace the filename with your desired output file name.

  • There is no such a flag J for the command tree!! – user88036 Sep 12 '18 at 17:45
  • Upvote: on tree v1.7.0 there is a flag J ... cheers – drl Oct 4 at 13:12
0

This does the job. https://gist.github.com/debodirno/18a21df0511775c19de8d7ccbc99cb72

import os
import sys
import json

def tree_path_json(path):
    dir_structure = {}
    base_name = os.path.basename(os.path.realpath(path))
    if os.path.isdir(path):
        dir_structure[base_name] = [ tree_path_json(os.path.join(path, file_name))\
         for file_name in os.listdir(path) ]
    else:
        return os.path.basename(path)
    return dir_structure

if len(sys.argv) > 1:
    path = sys.argv[1]
else:
    path = '.'

print json.dumps(tree_path_json(path), indent = 4, separators = (', ', ' : '))
  • I do not get it – Pierre.Vriens Oct 22 '18 at 4:54
  • So this converts the tree structure to json. Run this code on a directory and it will produce the json as indicated in the question. – Debodirno Chandra Oct 22 '18 at 6:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.