Can anyone please guide me with writing Shell Script command for following scenario.

I have an html file with all tags in a single line like this,

<html><head></head><body><br><center><h2 style='display: block;font-size: 1.5em;weight: bold;'>Validation Report</h2></center></body><table style='border:1px solid;border-collapse: separate;border-spacing: 2px;border-color: gray;' align='center'width='85%' cellpadding ='4' cellspacing='1' bgcolor='#CFCFCF' ><tr style='background-color:#a6caf0;height:30px;font-size:12px;'><th style='width:467px;border: 1px solid;'>TestCaseID</th><th style='border: 1px solid;'>Severity</th><th style='width:125px;border: 1px solid;'>Result</th><th style='width:148px;border: 1px solid;'>Remark</th><tr style='background-color:#ebf3fc;' align='center'><td style='width:360px;border: 1px solid;'>TESTCASE-001</td><td style='width:360px;border: 1px solid;'>Critical</td><td style='width:125px;border: 1px solid;'>Error</td><td style='width:125px;border: 1px solid;'>None</td></tr><tr style='background-color:#ebf3fc;' align='center'><td style='width:360px;border: 1px solid;'>TESTCASE-002</td><td style='width:360px;border: 1px solid;'>Critical</td><td style='width:125px;border: 1px solid;'>Error</td><td style='width:125px;border: 1px solid;'>None</td></tr></table></body></html>

This file when viewed in a browser, will have a table look with entries of Testcase ID, their corresponding Severity, Result and Remark. This HTML file has totally 210 TestcaseIDs. Now I have to remove particular rows in a table by mentioning their TestcaseID. ( Say, I need to remove a row which has TESTCASE-002 )


You can remove the tag with XSL Transformations (XSLT).

Most distros should come with an xsltproc command (part of the libxslt package).

Here is the command to perform the transform, which can be used in a shellscript:

xsltproc --html main.xslt index.html

And here is the XSLT which will remove TESTCASE-002:

<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
  <xsl:output omit-xml-declaration="yes"/>
  <xsl:template match="node()|@*">
      <xsl:apply-templates select="node()|@*"/>
  <xsl:template match="tr[td = 'TESTCASE-002']"/>

The advantage of this approach is that it's declarative, reusable and separates the code that deals with XML/HTML from the rest of the shell script. It's also flexible and powerful. The disadvantage is that it's slightly verbose and cumbersome.

This method can also handle HTML that is not well formed.

  • 1
    Thank you so much. It works exactly. I never knew about XSL Transformations before. Now I'm learning and using it. Thanks once again.
    – Thiripu
    Oct 23 '15 at 6:42

Instead of a text processing tool like sed/grep, use a tool that understands HTML.

For example, in xsh, you can

open :F html file.html ;
delete //tr[td='TESTCASE-002'] ;
save :b ;

You have to fix the file first, though - remove the misplaced </body> and </head> endtags.

  • 2
    @Thiripu xsh is a shell scripting language, but unlike bash it is meant for working with XML (which also covers HTML)
    – Centimane
    Oct 22 '15 at 10:34
  • 1
    @Thiripu: You can run xsh from bash, i.e.: xsh 'open file.html ; delete //...'
    – choroba
    Oct 22 '15 at 10:48
  • 3
    You can't reliably parse html with regular expressions. Use an xml parser instead. There are many to choose from, including xsh mentioned here and xsltproc mentioned by Alexander. See stackoverflow.com/questions/1732348/…
    – cas
    Oct 22 '15 at 11:25
  • 2
    @Thiripu, you integrate it into your bash script in the same way you would an awk or sed or perl script. it's a scripting tool meant to be used from sh just as they are.
    – cas
    Oct 22 '15 at 11:27
  • 2
    @Thiripu if you're worried about efficiency, pretty much anything will be more efficient than the shell. So yes, calling an external program to do the processing from within your shell script is more efficient. See Why is using a shell loop to process text considered bad practice?
    – terdon
    Oct 22 '15 at 12:58

This string will pull out the first entry in your html block

awk -F"TESTCASE-" '{print $2}' test.html |sed -e 's/<[^>]*>/ /g'

And this will pull out the second

awk -F"TESTCASE-" '{print $3}' test.html |sed -e 's/<[^>]*>/ /g'

If you use the -v option with awk you can pass a bash variable and pull out whichever block you need.

  • This command removed all tags in that file. ( Including <head><body> everything )
    – Thiripu
    Oct 22 '15 at 12:04
  • @Thiripu: My apologies I misunderstood the question. I took it to mean you wished to have your script return the information for a row from within the block.
    – rcjohnson
    Oct 22 '15 at 15:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.