Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

Unix environment has some excellent tools for parsing text in various forms. However off late the data is not in the regular set of formats(CSV,TSV,Record based or some other delimiter based) it used to be before. Data these days is exchanged in structured formats like XML's/JSON's.

I know there are some good tools like sed, awk and Perl which can chew down nearly form of data out there. However to work with this sort of structured data often one has to write a complete program and given the little time available to extract information, one has to sit down and figure out the whole logic of what one wants to query and put it down programatically. Sometimes this is not OK. Basically because the information extracted from those files acts inputs for further work. Also because of the time it takes to search for the appropriate solution and coding it up. A command line tool is needed with sufficient switches to find, query and dump data.

Im looking for tools that take a XML/JSON or other forms of structured data and dump it into other formats like csv etc. So that from there one could use other commands to get any information out of it.

Are there any command line utilities you know of which do this kind of a job. Are there already awk/Perl scripts available to this?

share|improve this question

2 Answers 2

I guess Perl, Python or Ruby modules can be used successfully for that. And any of these can be used for scripting.

share|improve this answer
    
My whole point was to avoid writing a script and try to do it using a command instead. Writing the script will defeat the whole purpose of this question. –  kamaal Mar 4 '11 at 8:47
    
Do you consider ruby -e 'program text' "writing a script"? –  alex Mar 4 '11 at 8:48
    
I think I didn't understand you properly. Where can I get the 'program text'? –  kamaal Mar 4 '11 at 9:25
    
@kamaal: dunno. Google? Docs for your scripting language of choice? CPAN comes to mind if you're using Perl. –  alex Mar 4 '11 at 17:21
    
@kamaal: As the data format gets more sophisticated, descriptions of the operations to perform get longer. For example, perl -0777 -MJSON -e '$t=from_json(<STDIN>); do_something_with($t)' lets you process JSON with a minimum of fuss. You still have to tell the computer what to do with the data somehow. –  Gilles Mar 4 '11 at 21:33

for xml there is http://xmlstar.sourceforge.net/

XMLStarlet is a set of command line utilities (tools) which can be used to transform, query, validate, and edit XML documents and files using simple set of shell commands in similar way it is done for plain text files using UNIX grep, sed, awk, diff, patch, join, etc commands.

you can also use xsltproc and similar tools (saxon).

for json: i also think its better to just use python, ruby, perl and transform it.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.