I want to retrieve whatever is between these two tags – <tr> </tr> – from an html doc. Now I don't have any specific html requirements that would warrant for an html parser. I just plain need something that matches <tr> and </tr> and gets everything in between and there could be multiple trs. I tried awk, which works, but for some reason it ends up giving me duplicates of each row extracted.

awk '
/<TR/{p=1; s=$0}
p && /<\/TR>/{print $0 FS s; s=""; p=0}
p' htmlfile> newfile

How to go about this?

  • IIUC your awk script should be: '/<tr/{p=1}; p; /<\/tr>/{p=0}'. Post some example input and expected output if it doesn't work.
    – Thor
    Commented Feb 13, 2013 at 14:23
  • since your awk is working but giving duplicates try to pass your awk's output from sort -u to get them distinct
    – igiannak
    Commented Jul 15, 2017 at 22:58

7 Answers 7


If you only want ... of all <tr>...</tr> do:

grep -o '<tr>.*</tr>' HTMLFILE | sed 's/\(<tr>\|<\/tr>\)//g' > NEWFILE

For multiline do:

tr "\n" "|" < HTMLFILE | grep -o '<tr>.*</tr>' | sed 's/\(<tr>\|<\/tr>\)//g;s/|/\n/g' > NEWFILE

Check the HTMLFILE first of the char "|" (not usual, but possible) and if it exists, change to one which doesn't exist.

  • 1
    That'll only work if the start and end tags are on the same line.
    – l0b0
    Commented Feb 13, 2013 at 12:01
  • echo "bla<tr>foo</tr>bla<tr>bar</tr>bla" | grep -o '<tr>.*</tr>' | sed 's/\(<tr>\|<\/tr>\)//g' gives fooblabar. The bla should not be there?
    – N.N.
    Commented Feb 13, 2013 at 12:03
  • @l0b0 correct. will go for a multiline-compatible one...
    – xx4h
    Commented Feb 13, 2013 at 12:07
  • grep -Po '<tr>.*?</tr>' would return one result per line in @N.N.'s case, but it's not portable.
    – l0b0
    Commented Feb 13, 2013 at 12:15
  • I'm not sure what you mean by 'specs' or 'spec-style' but note that your web browser uses an html parser and an html parser will parse html regardless of how it is written. It will not parse things that are not html, but then, neither will your browser, so no one would bother writing "html" that a parser cannot parse. In other words: A decent parser is absolutely certainly your best bet for doing this.
    – goldilocks
    Commented Feb 13, 2013 at 12:38

You do have a requirement that warrants an HTML parser: you need to parse HTML. Perl's HTML::TreeBuilder, Python's BeautifulSoup and others are easy to use, easier than writing complex and brittle regular expressions.

perl -MHTML::TreeBuilder -le '
    $html = HTML::TreeBuilder->new_from_file($ARGV[0]) or die $!;
    foreach ($html->look_down(_tag => "tr")) {
        print map {$_->as_HTML()} $_->content_list();
' input.html


python -c 'if True:
    import sys, BeautifulSoup
    html = BeautifulSoup.BeautifulSoup(open(sys.argv[1]).read())
    for tr in html.findAll("tr"):
        print "".join(tr.contents)
' input.html

sed and awk are not well suited for this task, you should rather use a proper html parser. For example hxselect from w3.org:

<htmlfile hxselect -s '\n' -c 'tr'
  • I dunno if hxselect is the best choice; I haven't used it but the man page does say it "reads a well-formed XML document" which many html documents are not. Probably worth a try tho. The html parser libs available for perl, python, et. al. will be much better, if that is an option.
    – goldilocks
    Commented Feb 13, 2013 at 12:44
  • 2
    @goldilocks: Best choice depends on the situation. In my experience hxselect does pretty good job with well-formed html/xml documents. Also, it's faster to use than perl, python and others. I think hxselect is a good middle-ground between sed/awk and parser libs.
    – Thor
    Commented Feb 13, 2013 at 13:00
  • 1
    If it works that's great! I was just adding a caveat for TechJack in case it didn't -- since I had also recommended using some kind of parser ;) The programming lib ones are of course more awkward but should deal with anything remotely passable as html.
    – goldilocks
    Commented Feb 13, 2013 at 13:56
  • Thor, hxselect looks good, will definitely explore it more. Thanks.
    – TechJack
    Commented Feb 13, 2013 at 17:16
  • @goldilocks: hxnormalize takes care of non well-formed html/xml files.
    – tokland
    Commented Nov 17, 2016 at 11:37

If ruby is available you can do the following

ruby -e 'puts readlines.join[/(?<=<tr>).+(?=<\/tr>)/m].gsub(/<\/?tr>/, "")' file

where file is your input html file. The command executes a Ruby one-liner. First, it reads all lines from file and joins them to a string, readlines.join. Then, from the string it selects anything between (but not including) <tr> and <\/tr> that is one character or longer irrespective of newlines, [/(?<=<tr>).+(?=<\/tr>)/m]. Then, it removes any <tr> or </tr> from the string, gsub(/<\/?tr>/, "") (this is necessary to handle nested tr tags). Finally, it prints the string, puts.

You said that a html parser is not warranted for you but it is very easy to use Nokogiri with ruby and it makes the command simpler.

ruby -rnokogiri -e 'puts Nokogiri::HTML(readlines.join).xpath("//tr").map { |e| e.content }' file

-rnokogiri loads Nokogiri. Nokogiri::HTML(readlines.join) reads all lines of file. xpath("//tr") picks out every tr element and map { |e| e.content } picks out the content for each element, i.e. what is between <tr> and </tr>.



Example using pup (which uses CSS selectors):

pup -f myfile.html tr

To print only text without tags, use: pup -f myfile.html tr text{}.

Here are few examples with curl:

curl -sL https://www.iana.org/ | pup tr text{}
pup -f <(curl -sL https://www.iana.org/) tr text{}


Example using xpup for HTML/XML parsing (which supports XPath):

xpup -f myfile.html "//tr"


To retrieve content within tr tag across multiple lines, pass it through xargs first, for example:

curl -sL https://www.iana.org/ | xargs | egrep -o "<tr>.*?</tr>"

To return only inner HTML, use:

curl -sL https://www.iana.org/ | xargs | grep -Po "<tr>\K(.*?)</tr>" | sed "s/..tr.//g"

Check the syntax for perlre extended patterns.

Note: For quicker performance, you may consider ripgrep which has similar syntax.

  • it printed out nicer looking without the xargs, came in handy to find inline javascript using egrep -o "<script .*?</script>"
    – Andrew
    Commented May 2, 2019 at 21:24

if it is just a quick listing of <tr>s this could help:

perl -ne 'print if /<tr>/../</tr>/' your.html > TRs.log


  • Have you tried it?
    – sergzach
    Commented Mar 29, 2022 at 9:45

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .