0

I need to parse an XML without an xmlparser, is this possible? I started with xmlstarlet, which works nice, but I can not use any real parser on that machine.

I need it to be a one-liner. I just need this:

/dev/mdX /dev/sda5 /dev/sdma6 /dev/sdmb6 ... /dev/sdsb5

My Script looks like this currently:

cat xml.txt; j=!:1; \
for i in $( xmlstarlet sel -t -v '/spaces/space/device/lvm/raids/raid/@path' $j ); \
do echo -e \\n$i $( sed -n 's/[^\"]*\"\([^\"]*\)\"[^\"]*/\1 /gp' $j | cut -d " " -f2); \
done

which outputs to

/dev/md2 UTF-8 /volume1 X3BAVZ-PqyQ-BNvJ-mYaf-9lAR-Up24-icJSsM 11a49b00:599c89fe:09b6521c:2cea662f /dev/sdma6 /dev/sdmb6
/dev/sdmc6 /dev/sdna6 /dev/sdnb6 /dev/sdnc6 /dev/sdoa6 /dev/sdob6 /dev/sdoc6 /dev/sdpa6 /dev/sdpb6 /dev/sdqb6 /dev/sdqc6
/dev/sdra6 /dev/sdrb6 /dev/sdrc6 /dev/sdsa6 /dev/sdsb6 a11476dd:d11720f9:3ae54a77:a2fdb1eb /dev/sda5 /dev/sdb5
/dev/sdc5 /dev/sdd5 /dev/sde5 /dev/sdf5 /dev/sdg5 /dev/sdh5 /dev/sdi5 /dev/sdj5 /dev/sdma5 /dev/sdmb5 /dev/sdmc5 /dev/sdna5
/dev/sdnb5 /dev/sdnc5 /dev/sdoa5 /dev/sdob5 /dev/sdoc5 /dev/sdpa5 /dev/sdpb5 /dev/sdqb5 /dev/sdqc5 /dev/sdra5 /dev/sdrb5
/dev/sdrc5 /dev/sdsa5 /dev/sdsb5 /dev/vg1000/lv

or when I used the parser in the second part too, which is just what I need, but I need to eliminate xmlstarlet:

cat xml.txt; j=!:1;
for i in $( xmlstarlet sel -t -v '/spaces/space/device/lvm/raids/raid/@path' $j );
do echo -e \\n$i $( xmlstarlet sel -t -v '/spaces/space/device/lvm/raids/raid/disks/disk/@dev_path' $j );
done

output:

/dev/md3 /dev/sdma6 /dev/sdmb6 /dev/sdmc6 /dev/sdna6 /dev/sdnb6 /dev/sdnc6 /dev/sdoa6 /dev/sdob6 /dev/sdoc6 /dev/sdpa6 /dev/sdpb6 /dev/sdqb6 /dev/sdqc6 /dev/sdra6 /dev/sdrb6 /dev/sdrc6 /dev/sdsa6 /dev/sdsb6 /dev/sda5 /dev/sdb5 /dev/sdc5 /dev/sdd5 /dev/sde5 /dev/sdf5 /dev/sdg5 /dev/sdh5 /dev/sdi5 /dev/sdj5 /dev/sdma5 /dev/sdmb5 /dev/sdmc5 /dev/sdna5 /dev/sdnb5 /dev/sdnc5 /dev/sdoa5 /dev/sdob5 /dev/sdoc5 /dev/sdpa5 /dev/sdpb5 /dev/sdqb5 /dev/sdqc5 /dev/sdra5 /dev/sdrb5 /dev/sdrc5 /dev/sdsa5 /dev/sdsb5

/dev/md2 /dev/sdma6 /dev/sdmb6 /dev/sdmc6 /dev/sdna6 /dev/sdnb6 /dev/sdnc6 /dev/sdoa6 /dev/sdob6 /dev/sdoc6 /dev/sdpa6 /dev/sdpb6 /dev/sdqb6 /dev/sdqc6 /dev/sdra6 /dev/sdrb6 /dev/sdrc6 /dev/sdsa6 /dev/sdsb6 /dev/sda5 /dev/sdb5 /dev/sdc5 /dev/sdd5 /dev/sde5 /dev/sdf5 /dev/sdg5 /dev/sdh5 /dev/sdi5 /dev/sdj5 /dev/sdma5 /dev/sdmb5 /dev/sdmc5 /dev/sdna5 /dev/sdnb5 /dev/sdnc5 /dev/sdoa5 /dev/sdob5 /dev/sdoc5 /dev/sdpa5 /dev/sdpb5 /dev/sdqb5 /dev/sdqc5 /dev/sdra5 /dev/sdrb5 /dev/sdrc5 /dev/sdsa5 /dev/sdsb5

The file1.xml looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<spaces>
    <space path="/dev/vg1000/lv" reference="/volume1" uuid="w7hmDd-j6T3-yWCV-9fEv-HE2Y-Vfrm-FHXoLi" device_type="1" drive_type="0" container_type="1" limited_raidgroup_num="12" >
        <device>
            <lvm path="/dev/vg1000" uuid="X3BAVZ-PqyQ-BNvJ-mYaf-9lAR-Up24-icJSsM" designed_pv_counts="2" status="normal" total_size="135898876346368" free_size="17151620546560" pe_size="4194304" expansible="0" max_size="132713751296">
                <raids>
                    <raid path="/dev/md3" uuid="11a49b00:599c89fe:09b6521c:2cea662f" level="raid6" version="1.2">
                        <disks>
                            <disk status="normal" dev_path="/dev/sdma6" model="WD60EFRX-68L0BN1        " serial="WD-WX21DA5K1801" partition_version="8" partition_start="7814039104" partition_size="3906799104" slot="0">
                            </disk>
                            <disk status="normal" dev_path="/dev/sdmb6" model="WD60EFRX-68L0BN1        " serial="WD-WX31DB58YHUU" partition_version="8" partition_start="7814039104" partition_size="3906799104" slot="3">
                            </disk>
                            <disk status="normal" dev_path="/dev/sdmc6" model="WD60EFRX-68L0BN1        " serial="WD-WX31D95HZLA2" partition_version="8" partition_start="7814039104" partition_size="3906799104" slot="5">
                            </disk>
                            <disk status="normal" dev_path="/dev/sdna6" model="WD60EFRX-68L0BN1        " serial="WD-WX11DC580PX8" partition_version="8" partition_start="7814039104" partition_size="3906799104" slot="6">
                            </disk>
                            <disk status="normal" dev_path="/dev/sdnb6" model="WD60EFRX-68L0BN1        " serial="WD-WXB1HB4UA48R" partition_version="8" partition_start="7814039104" partition_size="3906799104" slot="1">
                            </disk>
                            <disk status="normal" dev_path="/dev/sdnc6" model="WD60EFRX-68L0BN1        " serial="WD-WX41DA57HNRZ" partition_version="8" partition_start="7814039104" partition_size="3906799104" slot="4">
                            </disk>
                            <disk status="normal" dev_path="/dev/sdoa6" model="WD60EFRX-68L0BN1        " serial="WD-WX11DB65Z87L" partition_version="8" partition_start="7814039104" partition_size="3906799104" slot="8">
                            </disk>
                            <disk status="normal" dev_path="/dev/sdob6" model="WD60EFRX-68L0BN1        " serial="WD-WX11DC6RDEYL" partition_version="8" partition_start="7814039104" partition_size="3906799104" slot="7">
                            </disk>
                            <disk status="normal" dev_path="/dev/sdoc6" model="WD60EFRX-68L0BN1        " serial="WD-WX41DA58PT53" partition_version="8" partition_start="7814039104" partition_size="3906799104" slot="2">
                            </disk>
                            <disk status="normal" dev_path="/dev/sdpa6" model="WD60EFRX-68L0BN1        " serial="WD-WX11DB5NET4F" partition_version="8" partition_start="7814039104" partition_size="3906799104" slot="10">
                            </disk>
                            <disk status="normal" dev_path="/dev/sdpb6" model="WD60EFRX-68L0BN1        " serial="WD-WX11D37D9SN2" partition_version="8" partition_start="7814039104" partition_size="3906799104" slot="9">
                            </disk>
                            <disk status="normal" dev_path="/dev/sdqb6" model="WD6002FFWX-68TZ4N0      " serial="K1JY4D5D" partition_version="8" partition_start="7814039104" partition_size="3906799104" slot="12">
                            </disk>
                            <disk status="normal" dev_path="/dev/sdqc6" model="WD6002FFWX-68TZ4N0      " serial="K1JXXWRD" partition_version="8" partition_start="7814039104" partition_size="3906799104" slot="11">
                            </disk>
                            <disk status="normal" dev_path="/dev/sdra6" model="WD6002FFWX-68TZ4N0      " serial="K1K9W1LD" partition_version="8" partition_start="7814039104" partition_size="3906799104" slot="17">
                            </disk>
                            <disk status="normal" dev_path="/dev/sdrb6" model="WD6002FFWX-68TZ4N0      " serial="K1K9PZJD" partition_version="8" partition_start="7814039104" partition_size="3906799104" slot="16">
                            </disk>
                            <disk status="normal" dev_path="/dev/sdrc6" model="WD6002FFWX-68TZ4N0      " serial="K1KAEVDD" partition_version="8" partition_start="7814039104" partition_size="3906799104" slot="15">
                            </disk>
                            <disk status="normal" dev_path="/dev/sdsa6" model="WD6002FFWX-68TZ4N0      " serial="K1K9PYWD" partition_version="8" partition_start="7814039104" partition_size="3906799104" slot="14">
                            </disk>
                            <disk status="normal" dev_path="/dev/sdsb6" model="WD6002FFWX-68TZ4N0      " serial="K1KAGJVD" partition_version="8" partition_start="7814039104" partition_size="3906799104" slot="13">
                            </disk>
                        </disks>
                    </raid>
                    <raid path="/dev/md2" uuid="a11476dd:d11720f9:3ae54a77:a2fdb1eb" level="raid6" version="1.2">
                        <disks>
                            <disk status="normal" dev_path="/dev/sda5" model="WD4000FYYZ-01UL1B0      " serial="WD-WCC130674014" partition_version="7" partition_start="9453280" partition_size="7804569728" slot="0">
                            </disk>
                            <disk status="normal" dev_path="/dev/sdb5" model="WD4000FYYZ-01UL1B0      " serial="WD-WCC130688271" partition_version="7" partition_start="9453280" partition_size="7804569728" slot="1">
                            </disk>
                            <disk status="normal" dev_path="/dev/sdc5" model="WD4000FYYZ-01UL1B0      " serial="WD-WCC130693259" partition_version="7" partition_start="9453280" partition_size="7804569728" slot="2">
                            </disk>
                            <disk status="normal" dev_path="/dev/sdd5" model="WD4000FYYZ-01UL1B0      " serial="WD-WCC130619691" partition_version="7" partition_start="9453280" partition_size="7804569728" slot="3">
                            </disk>
                            <disk status="normal" dev_path="/dev/sde5" model="WD4000FYYZ-01UL1B0      " serial="WD-WCC130651575" partition_version="7" partition_start="9453280" partition_size="7804569728" slot="4">
                            </disk>
                            <disk status="normal" dev_path="/dev/sdf5" model="WD4000FYYZ-01UL1B0      " serial="WD-WCC130681732" partition_version="7" partition_start="9453280" partition_size="7804569728" slot="5">
                            </disk>
                            <disk status="normal" dev_path="/dev/sdg5" model="WD4000FYYZ-01UL1B0      " serial="WD-WCC130622745" partition_version="7" partition_start="9453280" partition_size="7804569728" slot="6">
                            </disk>
                            <disk status="normal" dev_path="/dev/sdh5" model="WD4000FYYZ-01UL1B0      " serial="WD-WCC130726036" partition_version="7" partition_start="9453280" partition_size="7804569728" slot="7">
                            </disk>
                            <disk status="normal" dev_path="/dev/sdi5" model="WD4000FYYZ-01UL1B0      " serial="WD-WCC130730013" partition_version="7" partition_start="9453280" partition_size="7804569728" slot="8">
                            </disk>
                            <disk status="normal" dev_path="/dev/sdj5" model="WD4000FYYZ-01UL1B0      " serial="WD-WCC130746174" partition_version="7" partition_start="9453280" partition_size="7804569728" slot="9">
                            </disk>
                            <disk status="normal" dev_path="/dev/sdma5" model="WD60EFRX-68L0BN1        " serial="WD-WX21DA5K1801" partition_version="8" partition_start="9453280" partition_size="7804569728" slot="12">
                            </disk>
                            <disk status="normal" dev_path="/dev/sdmb5" model="WD60EFRX-68L0BN1        " serial="WD-WX31DB58YHUU" partition_version="8" partition_start="9453280" partition_size="7804569728" slot="13">
                            </disk>
                            <disk status="normal" dev_path="/dev/sdmc5" model="WD60EFRX-68L0BN1        " serial="WD-WX31D95HZLA2" partition_version="8" partition_start="9453280" partition_size="7804569728" slot="15">
                            </disk>
                            <disk status="normal" dev_path="/dev/sdna5" model="WD60EFRX-68L0BN1        " serial="WD-WX11DC580PX8" partition_version="8" partition_start="9453280" partition_size="7804569728" slot="16">
                            </disk>
                            <disk status="normal" dev_path="/dev/sdnb5" model="WD60EFRX-68L0BN1        " serial="WD-WXB1HB4UA48R" partition_version="8" partition_start="9453280" partition_size="7804569728" slot="11">
                            </disk>
                            <disk status="normal" dev_path="/dev/sdnc5" model="WD60EFRX-68L0BN1        " serial="WD-WX41DA57HNRZ" partition_version="8" partition_start="9453280" partition_size="7804569728" slot="14">
                            </disk>
                            <disk status="normal" dev_path="/dev/sdoa5" model="WD60EFRX-68L0BN1        " serial="WD-WX11DB65Z87L" partition_version="8" partition_start="9453280" partition_size="7804569728" slot="18">
                            </disk>
                            <disk status="normal" dev_path="/dev/sdob5" model="WD60EFRX-68L0BN1        " serial="WD-WX11DC6RDEYL" partition_version="8" partition_start="9453280" partition_size="7804569728" slot="17">
                            </disk>
                            <disk status="normal" dev_path="/dev/sdoc5" model="WD60EFRX-68L0BN1        " serial="WD-WX41DA58PT53" partition_version="8" partition_start="9453280" partition_size="7804569728" slot="10">
                            </disk>
                            <disk status="normal" dev_path="/dev/sdpa5" model="WD60EFRX-68L0BN1        " serial="WD-WX11DB5NET4F" partition_version="8" partition_start="9453280" partition_size="7804569728" slot="20">
                            </disk>
                            <disk status="normal" dev_path="/dev/sdpb5" model="WD60EFRX-68L0BN1        " serial="WD-WX11D37D9SN2" partition_version="8" partition_start="9453280" partition_size="7804569728" slot="19">
                            </disk>
                            <disk status="normal" dev_path="/dev/sdqb5" model="WD6002FFWX-68TZ4N0      " serial="K1JY4D5D" partition_version="8" partition_start="9453280" partition_size="7804569728" slot="22">
                            </disk>
                            <disk status="normal" dev_path="/dev/sdqc5" model="WD6002FFWX-68TZ4N0      " serial="K1JXXWRD" partition_version="8" partition_start="9453280" partition_size="7804569728" slot="21">
                            </disk>
                            <disk status="normal" dev_path="/dev/sdra5" model="WD6002FFWX-68TZ4N0      " serial="K1K9W1LD" partition_version="8" partition_start="9453280" partition_size="7804569728" slot="27">
                            </disk>
                            <disk status="normal" dev_path="/dev/sdrb5" model="WD6002FFWX-68TZ4N0      " serial="K1K9PZJD" partition_version="8" partition_start="9453280" partition_size="7804569728" slot="26">
                            </disk>
                            <disk status="normal" dev_path="/dev/sdrc5" model="WD6002FFWX-68TZ4N0      " serial="K1KAEVDD" partition_version="8" partition_start="9453280" partition_size="7804569728" slot="25">
                            </disk>
                            <disk status="normal" dev_path="/dev/sdsa5" model="WD6002FFWX-68TZ4N0      " serial="K1K9PYWD" partition_version="8" partition_start="9453280" partition_size="7804569728" slot="24">
                            </disk>
                            <disk status="normal" dev_path="/dev/sdsb5" model="WD6002FFWX-68TZ4N0      " serial="K1KAGJVD" partition_version="8" partition_start="9453280" partition_size="7804569728" slot="23">
                            </disk>
                        </disks>
                    </raid>
                </raids>
            </lvm>
        </device>
        <reference>
            <volumes>
                <volume path="/volume1" dev_path="/dev/vg1000/lv" uuid="w7hmDd-j6T3-yWCV-9fEv-HE2Y-Vfrm-FHXoLi" type="ext4">
                </volume>
            </volumes>
        </reference>
    </space>
</spaces>

file2.xml:

<?xml version="1.0" encoding="UTF-8"?>
<spaces>
<space path="/dev/md3" reference="/volume1" uuid="b9c08aae:78c4a659:658ead72:0b765dd7" device_type="2" drive_type="0" container_type="2" limited_raidgroup_num="12" >
    <device>
        <raid path="/dev/md3" uuid="b9c08aae:78c4a659:658ead72:0b765dd7" level="raid1" version="1.2">
            <disks>
                <disk status="normal" dev_path="/dev/sda3" model="WD1003FBYX-01Y7B0       " serial="WD-WCAW32967331" partition_version="8" partition_start="9437184" partition_size="1943883168" slot="0">
                </disk>
            </disks>
        </raid>
    </device>
    <reference>
        <volume path="/volume1" dev_path="/dev/md3" uuid="b9c08aae:78c4a659:658ead72:0b765dd7" type="ext4">
        </volume>
    </reference>
</space>
<space path="/dev/md2" reference="/volume2" uuid="f537fb36:6ab07e85:4ed56c52:02396e9e" device_type="2" drive_type="0" container_type="2" limited_raidgroup_num="12" >
    <device>
        <raid path="/dev/md2" uuid="f537fb36:6ab07e85:4ed56c52:02396e9e" level="raid1" version="1.2">
            <disks>
                <disk status="normal" dev_path="/dev/sdb3" model="ST2000DM001-9YN164      " serial="W1E15XDS" partition_version="8" partition_start="9437184" partition_size="3897387168" slot="0">
                </disk>
                <disk status="normal" dev_path="/dev/sdc3" model="ST2000DM001-1CH164      " serial="W1E4327N" partition_version="8" partition_start="9437184" partition_size="3897387168" slot="1">
                </disk>
            </disks>
        </raid>
    </device>
    <reference>
        <volume path="/volume2" dev_path="/dev/md2" uuid="f537fb36:6ab07e85:4ed56c52:02396e9e" type="ext4">
        </volume>
    </reference>
</space>
</spaces>

Is there a way to do this without xmlstarlet?

  • What is the reason you need to do it without xmlstarlet? If it's because "no additional software may be installed", then talk to your manager and ask them if they want a stable system or a hack. – Kusalananda Aug 24 '18 at 6:37
  • i remote into machines, but i can not install additional software on them, this does not really have anything to do with manager decisions. there are no package sources I can install anything from on this distribution i need to use. it was based on deabian once, but dkpg is not installed. Also those arent x86/x64 CPUs, so I would need to compile them first, which I don't have time for. – James Doe Aug 24 '18 at 8:16
  • Another option would be to transfer the file to a system that has XML tools installed, then transfer it back. – Jeff Schaller Aug 24 '18 at 10:05
  • Ah I just saw your comment. How would I do this in a automated way without running a ssh-server on my local system? – James Doe Aug 26 '18 at 11:18
  • I am thinking of a different approach : If I'm connected via SSH, can I somehow send the output of the xml file to my local terminal, send the file through xmlstarlet and show the output in my local terminal, then re-connect to ssh, so I can just copy/paste the output of xmstarlet to the remote machine again? Is this possible without running a ssh-server on my local machine? – James Doe Aug 28 '18 at 10:52
2

Try this,

awk -F '"' '/dev_path/ {print $4} /raid path/ {print $2} /raid>/ {print $5}' file.xml | grep -v "vg" | tr '\n' ' '  | sed 's#  #\n#g'

/dev/md3 /dev/sdma6 /dev/sdmb6 /dev/sdmc6 /dev/sdna6 /dev/sdnb6 /dev/sdnc6 /dev/sdoa6 /dev/sdob6 /dev/sdoc6 /dev/sdpa6 /dev/sdpb6 /dev/sdqb6 /dev/sdqc6 /dev/sdra6 /dev/sdrb6 /dev/sdrc6 /dev/sdsa6 /dev/sdsb6
/dev/md2 /dev/sda5 /dev/sdb5 /dev/sdc5 /dev/sdd5 /dev/sde5 /dev/sdf5 /dev/sdg5 /dev/sdh5 /dev/sdi5 /dev/sdj5 /dev/sdma5 /dev/sdmb5 /dev/sdmc5 /dev/sdna5 /dev/sdnb5 /dev/sdnc5 /dev/sdoa5 /dev/sdob5 /dev/sdoc5 /dev/sdpa5 /dev/sdpb5 /dev/sdqb5 /dev/sdqc5 /dev/sdra5 /dev/sdrb5 /dev/sdrc5 /dev/sdsa5 /dev/sdsb5
  • print the 4th field and 2nd field where the line has dev_path and raid path respectively with " as a delimiter (/raid>/ {print $5} is just to have two line result)
  • then exclude the volume group with grep -v
  • then transpose all newline to space

  • To have the two line output, replace double space with newline.

For you second script:

 awk -F '"' '/dev_path/ {print $4} /raid path/ {print $2} /reference>/ {print $5}' file.xml |grep -v "vg" | tr '\n' ' '  | sed 's#  #\n#g'

/dev/md3 /dev/sda3
/dev/md3
/dev/md2 /dev/sdb3 /dev/sdc3
/dev/md2

Option 2:

 grep -wo  '/dev/.*"'  file.xml | grep -v vg |awk -F '"' '{print $1}' |  uniq | tr '\n' ' ' | sed -e 's#\/dev\/md#\n\/dev\/md#g'
  • One more question though: sometimes I need to sort the HDDs by slot="0"> from 0 to x. How can I add this to your script? Is it also possible replace a massing HDD with "missing"? – James Doe Aug 23 '18 at 22:33
  • @JamesDoe Use sort command before tr – msp9011 Aug 24 '18 at 2:36
  • So I just used it on another space-file, but that didnt output the correct hdds. – James Doe Aug 24 '18 at 7:56
  • @JamesDoe check my second option and delete your answer.. since it's not the way. we should edit the question instead to give inputs. – msp9011 Aug 24 '18 at 8:29
  • Ok thank you!Is there some way I can make this uniserval the same same scipt? Like can I use if then inside awk? But I am not sure how that if then would have to look like. – James Doe Aug 24 '18 at 8:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.