I've been slowly converting a blog of mine to markdown. The final thing to do is replacing all the html anchors with markdown.

I've come up this sed regex, which for all intents and purposes should do what I want, but it doesn't.

Source data:

$ cat /tmp/test
on <a href="https://www.reddit.com/" target="_blank" rel="noopener">reddit</a> or <a href="https://lifehacker.com/" target="_blank" rel="noopener">Lifehacker</a>

Sed command:

$ sed -r 's/<a.*?href="(.*?)".*?>(.*?)<\/a>/[\2](\1)/g' /tmp/test
on [Lifehacker](https://lifehacker.com/" target="_blank" rel="noopener)

What I want it to return:

on [Reddit](https://reddit.com/) or [Lifehacker](https://lifehacker.com/")
  • Is there a line break in /tmp/test or is that wrapping?
    – Quasímodo
    Apr 25 '20 at 11:01
  • Sorry, that pasted bad. It's on one line. Fixed it
    – devilkin
    Apr 25 '20 at 11:12
  • I dont think gnu sed supports non-greedy patterns. You can replace the 2nd .*? with [^"]*.
    – meuh
    Apr 25 '20 at 11:33
  • 1
    Using regular expressions to parse html is generally not a great idea (see the often quoted Stack Overflow answer on the topic). You may save yourself headaches by using a specialized html parser. Here, even pandoc may be a convenient solution - e.g. pandoc -f html -t markdown /tmp/test.
    – fra-san
    Apr 25 '20 at 11:45
  • @meuh while that fixes part of the issue, it still doesn't return both - only the last one is matched.
    – devilkin
    Apr 25 '20 at 12:04

sed uses basic and extended regular expressions (BRE/ERE). .*? is part of a Perl Compatible Regular Expression (PCRE).

To use PCRE, use perl:

$ perl -pe 's/<a.*?href="(.*?)".*?>(.*?)<\/a>/[\2](\1)/g' test
on [reddit](https://www.reddit.com/) or [Lifehacker](https://lifehacker.com/)
  • This is exactly the same expression as the original, but used with perl -p which reads & prints file line by line – like sed does

Here is a similar regex using ERE with sed:

$ sed -E 's/<a[^>]*href="([^"]*)[^>]*>([^<]*)[^>]*>/[\2](\1)/g' test
on [reddit](https://www.reddit.com/) or [Lifehacker](https://lifehacker.com/)
  • PCRE uses a ? following a quantifier to match the shortest repetition, standard regular expressions do not
  • Negated character classes are used to work around this
  • Thanks :) I was thinking of having a look at perl, but I was breaking my head why sed didn't work. Now I know... TIL :)
    – devilkin
    Apr 25 '20 at 12:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.