The Unix environment has some excellent tools for parsing text in various forms. However, of late, the data is not in the traditional (historical) formats (CSV, TSV, record-based or some other delimiter-based) it used to be before. Data these days is exchanged in structured formats like XML/JSON.

I know there are some good tools like sed, awk and Perl which can chew down nearly any form of data out there. However, to work with this sort of structured data, often one has to write a complete program, and, given the little time available to extract information, one has to sit down and figure out the whole logic of what one wants to query and put it down programmatically. Sometimes this is not OK – basically because the information extracted from those files acts as inputs for further work; also because of the time it takes to search for the appropriate solution and code it up. A command line tool is needed with sufficient switches to find, query and dump data.

I'm looking for tools that take a XML/JSON or other forms of structured data and dump it into other formats like csv, etc., so that from there one could use other commands to get any information out of it.

Are there any command line utilities you know of which do this kind of a job? Are there already awk/Perl scripts available to this?

2 Answers 2


for xml there is http://xmlstar.sourceforge.net/

XMLStarlet is a set of command line utilities (tools) which can be used to transform, query, validate, and edit XML documents and files using simple set of shell commands in similar way it is done for plain text files using UNIX grep, sed, awk, diff, patch, join, etc commands.

you can also use xsltproc and similar tools (saxon).

for json: i also think its better to just use python, ruby, perl and transform it.


I guess Perl, Python or Ruby modules can be used successfully for that. And any of these can be used for scripting.

  • My whole point was to avoid writing a script and try to do it using a command instead. Writing the script will defeat the whole purpose of this question.
    – kamaal
    Mar 4, 2011 at 8:47
  • Do you consider ruby -e 'program text' "writing a script"?
    – alex
    Mar 4, 2011 at 8:48
  • I think I didn't understand you properly. Where can I get the 'program text'?
    – kamaal
    Mar 4, 2011 at 9:25
  • @kamaal: dunno. Google? Docs for your scripting language of choice? CPAN comes to mind if you're using Perl.
    – alex
    Mar 4, 2011 at 17:21
  • @kamaal: As the data format gets more sophisticated, descriptions of the operations to perform get longer. For example, perl -0777 -MJSON -e '$t=from_json(<STDIN>); do_something_with($t)' lets you process JSON with a minimum of fuss. You still have to tell the computer what to do with the data somehow. Mar 4, 2011 at 21:33

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .