Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I have a web server which stores configurations to PostgreSQL, and I plan to retrieve information from a certain table and then later parse it into csv format.

PAYLOAD

<?xml version="1.0" encoding="UTF-8" standalone="yes"?><rule roleDefinition="false" id="1168" type="COMMON" enabled="true" owner="admin" buildingBlock="false" scope="LOCAL"><name>Recon: Remote Windows Server Scanner</name><notes>Reports a remote host attempting reconnaissance or suspicious connections on common Windows server ports to more than 60 hosts in 10 minutes.</notes><testDefinitions><test uid="4" id="80" name="com.q1labs.semsources.cre.tests.ThresholdFunction_Test" group="Functions"><text>when any of these &lt;a href='javascript:editParameter("4", "1")' class='dynamic'&gt;BB:CategoryDefinition: Recon Events, BB:CategoryDefinition: Suspicious Events&lt;/a&gt; with the same &lt;a href='javascript:editParameter("4", "2")' class='dynamic'&gt;source IP&lt;/a&gt; more than &lt;a href='javascript:editParameter("4", "3")' class='dynamic'&gt;5&lt;/a&gt; times, across &lt;a href='javascript:editParameter("4", "4")' class='dynamic'&gt;more than&lt;/a&gt; &lt;a href='javascript:editParameter("4", "5")' class='dynamic'&gt;59&lt;/a&gt; &lt;a href='javascript:editParameter("4", "6")' class='dynamic'&gt;destination IP&lt;/a&gt; within &lt;a href='javascript:editParameter("4", "7")' class='dynamic'&gt;10&lt;/a&gt; &lt;a href='javascript:editParameter("4", "8")' class='dynamic'&gt;minutes&lt;/a&gt;</text>

Steps

perl -nle '
my @enabled = /(?<=enabled="").*?(?<=\"\")/g;
chop(@enabled);
chop(@enabled);
chomp(@enabled);
my @group = /(?<=group="").*?(?<=\"\")/g;
chop(@group);
chop(@group);
chomp(@group);
my @name = /(?<=<name>).*?(?=<\/name>)/g;chomp(@name);
my @text = /(?<=<text>).*?(?=<\/text>)/g;chomp(@text);
print "@enabled;@group;@name;@text"
' Rules.csv |sed 's/\&lt\;/</g'|sed 's/\&gt\;/>/g' | sed -e 's/<*>//g' -e '/ *$/d' > Rules2.csv

Rules2.csv shows no content, and empty file, however when I break the command I get some very positive results, please see pastebin (as the code) is bit too long. The pastebin analysis would show that somewhere in command section sed -e 's/<*>//g' -e '/ *$/d' it results into empty my Rules2.csv file. But I don't know why or what is causing this behavior.

Any help that helps me achieve the desired output as shown in pastebin would be wonderful.

share|improve this question
    
@Joseph thanks for edit:) –  user3066819 Jan 10 at 21:05
2  
Exact duplicate to superuser.com/questions/699955/… –  erik Jan 10 at 21:07
    
You're very welcome. Please try to insert a couple of newlines into your XML payload as well if that wouldn't interfere with your intended question. Also, welcome to U&L :) –  Joseph R. Jan 10 at 21:07
    
@JosephR. Let me do it. Thanks for mentioning. –  user3066819 Jan 10 at 21:10

1 Answer 1

up vote 1 down vote accepted

Although you should not parse XML with regexes, this works with your sample input:

perl -ple '
    ($enabled) = /(?<=enabled=").*?(?=")/g;
    ($group)   = /(?<=group=").*?(?=")/g;
    ($name)    = /(?<=<name>).*?(?=<\/name>)/g;
    ($text)    = /(?<=<text>).*?(?=<\/text>)/g;
    $_= "$enabled;$group;$name;$text";
    s/&lt;.*?&gt;//g;
' Rules.csv
true;Functions;Recon: Remote Windows Server Scanner;when any of these BB:CategoryDefinition: Recon Events, BB:CategoryDefinition: Suspicious Events with the same source IP more than 5 times, across more than 59 destination IP within 10 minutes

Notes:

  • your "enabled" and "group" lookarounds were broken
  • if you're working in perl, you don't need to pipe into sed
  • the reason s/<*>//g erases all output is that you're matching zero or more open brackets up to the last close bracket -- this matches your entire string.
    • you probably intended s/<[^>]*>//g
share|improve this answer
    
thanks for making me realize my error by explicit mentioning them.I will run this code now. –  user3066819 Jan 10 at 21:12

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.