Tag Info

New answers tagged

5

This should do the trick: awk -F '[<>]' ' NR!=1 && FNR==1{printf "\n"} FNR==1{sub(".*/", "", FILENAME); sub(".xml$", "", FILENAME); printf FILENAME} /double/{printf " %s", $3} END{printf "\n"} ' $path_to_xml/*.xml > final_table.csv Explanation: awk: use the program awk, I tested it with GNU awk 4.0.1 -F ...


4

Regarding glob expansion possibly exceeding a limit - yes and no. The shell's already running, and so it won't stop. But if you were to pass the entire globbed array as arguments to a single command, then yes, that is a definite possibility. The portable and robust way to handle this involves find... find . \! -name . -prune -name pattern -type f -exec cat ...


0

You can try this single liner for each file. The awk multiple delimiters does efficient splitting and tr concats all lines in memory , rather than on disk. for f in `ls *.xml` ; do echo $f,`grep double $f | awk -F '[<>]' '{print $3}' | tr '\n' ','`; done I can not profile this at my end - since I do not have the same data, but my hunch is ...


-1

You’re writing twice for every file. This is probably the most expensive part. You’ll instead want to try to keep the whole thing in memory, probably in an array. Then write once in the end. Look into ulimit if you start hitting memory limits. If you’re increasing this workload to 10-100x, you’re looking at maybe 10-100 GB of memory. You could batch this in ...



Top 50 recent answers are included