-3

If we have the following, how can we delete all the tags?

Study eases concerns about taking antidepressants during pregnancy and autism risk <a href="https://t.co/Cs0mdeYEBo" rel="nofollow noopener" dir="ltr" data-expanded-url="http://cbsn.ws/2oTosqU" class="twitter-timeline-link" target="_blank" title="http://cbsn.ws/2oTosqU" ><span class="tco-ellipsis"></span><span class="invisible">http://</span><span class="js-display-url">cbsn.ws/2oTosqU</span><span class="invisible"></span><span class="tco-ellipsis"><span class="invisible">&nbsp;</span></span></a><a href="https://t.co/rs5813GdLG" class="twitter-timeline-link u-hidden" data-pre-embedded="true" dir="ltr" >pic.twitter.com/rs5813GdLG</a>

The result after using the command should look like this:

Study eases concerns about taking antidepressants during pregnancy and autism risk

After using the following:

sed -e 's/<[^>]*>//g'

or

sed 's/<[^>]\+>//g'

I get:

Study eases concerns about taking antidepressants during pregnancy and autism risk http://cbsn.ws/2oTosqU&nbsp;pic.twitter.com/rs5813GdLG

Which is not exactly what I want. I need to do this using only either sed, awk,grep.

  • 1
    Possible duplicate of How to delete html tags from a file using sed ? – Tim Kennedy Apr 19 '17 at 16:47
  • @TimKennedy that's not a duplicate. The OP is using precisely what the accepted answer there is suggesting and it fails on this input file. That's because their file is wrong, of course, but still. – terdon Apr 19 '17 at 16:52
  • 4
    HTML is not a regular language, so attempting to parse it with regular expressions is the path to madness. – DopeGhoti Apr 19 '17 at 17:06
  • 1
    What you want is to remove the tags (which that sed does), and some of the text contained within some tags. Here you seem to want to remove <a>some url</a>, but you probably don't want to remove <p>some text</p>. You need to figure out which ones exactly you want to remove, and that would probably be much easier with an actual HTML parses library – ilkkachu Apr 19 '17 at 17:12
  • You might be looking for this – Jeff Schaller Apr 19 '17 at 17:14
1

The command is working perfectly, your file is malformed. You can see this using grep --color=yes <[^>]*>' file or, by adding a newline after every >:

$ sed -e 's/>/>\n/g' file 
Study eases concerns about taking antidepressants during pregnancy and autism risk <a href="https://t.co/Cs0mdeYEBo" rel="nofollow noopener" dir="ltr" data-expanded-url="http://cbsn.ws/2oTosqU" class="twitter-timeline-link" target="_blank" title="http://cbsn.ws/2oTosqU" >
<span class="tco-ellipsis">
</span>
<span class="invisible">
http://</span>
<span class="js-display-url">
cbsn.ws/2oTosqU</span>
<span class="invisible">
</span>
<span class="tco-ellipsis">
<span class="invisible">
&nbsp;</span>
</span>
</a>
<a href="https://t.co/rs5813GdLG" class="twitter-timeline-link u-hidden" data-pre-embedded="true" dir="ltr" >
pic.twitter.com/rs5813GdLG</a>

Note that the http://</span>, cbsn.ws/2oTosqU, &nbsp; and pic.twitter.com/rs5813GdLG are not inside html tags so they are left as is, quite correctly.

So, what you want isn't to remove the html tags, you want to remove the html tags and some other stuff but I have no idea how to know what you want and what you don't.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.