0

My scenario is I have a file that has the contents looking like this but longer. https://pastebin.com/q33wFdAX

I need to remove each occurrence of

        <category>Games</category>
</game>

If the category has data in there we don't want to remove it. So basically if I can just match this double line exact and invert like grep -v then I can output to a new file without those lines.

I will have more than just this category so I will need to be able to use several or statements. Example of the regex I'm using in notepad++ to accomplish this manually

(^\s+<category>Games</category>\s+</game>$|^\s+<category>Applications</category>\s+</game>$)

Bonus points if you can get me a simple way to do this in powershell too, I'd like to be able to do this on either OS. I'm not extremely good with advanced regex, awk etc.

5
  • It looks as if someone has botched an edit on the file. It would have been easy to work with the file using xmlstarlet, but since it's now a broken XML file, that won't work. Do you have the original file someplace (backup?) and what did you intend to do to that file? It's probably easier to do that than to fix up a broken XML document. – Kusalananda Apr 18 at 20:45
  • Yes I have the orginal. They're straight from Redump website. What I'm doing is removing all the mentions of Roms that aren't USA. The way I've been doing this is by say removing all likes that contain (Europe) but then we're left with what's in the past bin. I've been doing all this manually with notepad++ which doesn't take too long, but I would like to automate it – volitank Apr 18 at 20:53
  • Once it's in this position though in notepad++ it's real simple. Just bookmark all the lines matching ^\s+<category>Games</category>\s+</game>$ and then remove all bookmarked lines. – volitank Apr 18 at 20:55
  • Where is the original file. The edits should be done using an XML parser, not by hand, not by sed, not by awk. It is easy to do this with xmlstarlet. – Kusalananda Apr 18 at 20:58
  • redump.org/datfile/psx This is the original file. This is my regex string for the locals I wish to remove (\(Europe\)|\(Europe, Australia\)|\(Japan\)|\(Germany\)|\(Sweden\)|\(Korea\)|\(UK\)|\(Asia\)|\(Italy\)|\(Israel\)|\(Scandinavia\)|\(Spain\)|\(Netherlands\)|\(Finland\)|\(Denmark\)|\(Greece\)|\(Russia\)|\(UK, Australia\)|\(Japan, Asia\)|\(France\)|\(Australia\)|\(Portugal\)|\(China\)|\(India\)|\(Spain, Portugal\)|\(Switzerland\)|\(Belgium, Netherlands\)|\(Poland\)|\(Brazil\)|\(Latin America\)|\(Croatia\)|\(Japan, Korea\)|\(Austria\)|\(Norway\)|\(Ireland\)|\(Belgium\)) – volitank Apr 18 at 21:00
3

Given the original file, the following will delete all titles that does not mention (USA) in the title's rom node's name attribute:

xmlstarlet ed -d '//game[not(contains(rom/@name, "(USA)"))]' file-orig.xml >file-new.xml

The XPath expression //game[not(contains(rom/@name, "(USA)"))] selects all game nodes that has at least one rom sub-node that has a name attribute that does not contain the string (USA). These are selected for deletion.

The output is written to a new file using a redirection on the command line.

After looking at the XML data for a short while, I noticed that instead of looking at tho rom nodes, it appears to be enough to look at the main game node's name attribute:

xmlstarlet ed -d '//game[not(contains(@name, "(USA)"))]' file-orig.xml >file-new.xml

This deletes all games that does not contain (USA) in the name attribute of the game node.

The file that I downloaded will contain 1979 titles after deleting all non-(USA) titles.

To additionally filter out all non-Games categories:

xmlstarlet ed \
    -d '//game[not(contains(@name, "(USA)"))]' \
    -d '//game[category != "Games"]' file-orig.xml >file-new.xml

This leaves 1474 Games titles.

Just for fun, to sort the game titles by their total size:

xmlstarlet ed \
    -d '//game[not(contains(@name, "(USA)"))]' \
    -d '//game[category != "Games"]' file-orig.xml |
xmlstarlet sel -t -m '//game' \
    -v 'sum(rom/@size)' -o ' ' -v '@name' -nl | sort -n
5
  • I like where this is going. It seems like this would delete Games that are denoted region less (World) or multi-region (USA, Canada). Sometimes I come across these. I would rather do it reverse and delete the ones named (Region) rather than delete not (USA) if that makes sense. I know it would be a longer script, but would protect me from over deleting and missing a game. – volitank Apr 18 at 21:33
  • @volitank Just use xmlstarlet ed followed by a repeat of -d '//game[contains(@name, "(someregion)")]' -d '//game[contains(@name, "(someotherregion)")]' etc. for each region. – Kusalananda Apr 18 at 21:34
  • Interesting. Seems like this is the better way to work with it. Time to get some testing done. Thanks! – volitank Apr 18 at 21:39
  • Well this is a bit unfortunate but it seems like the windows version of xmlstarlet is old or something. The commands don't work the same way. – volitank Apr 18 at 23:09
  • Nothing wrong with the windows verson. Turns out is was a quotes issue. Changed it to ./xml.exe ed -d "//game[contains(@name, '(Japan)')]" test.dat for windows – volitank Apr 19 at 2:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.