0

Toy problem:

$ echo "foo <a href="/topic/null-hypothesis/" data-sc="text link:topic link">Null hypothesis</a> bar" | sed -E 's@.*<a href=/topic/[^>]*>([^<]*)</a>.*@\1@'
Null hypothesis

Real world (sed does not filter anything):

$ cat *html | grep '<a href="/topic' | sed -E 's@.*<a href=/topic/[^>]*>([^<]*)</a>.*@\1@'
                <a href="/topic/null-hypothesis/" data-sc="text link:topic link">Null hypothesis</a>, 
                <a href="/topic/approximation/" data-sc="text link:topic link">Approximation</a>, 
                <a href="/topic/estimation-methods/" data-sc="text link:topic link">Estimation methods</a>, 
                <a href="/topic/statistical-variance/" data-sc="text link:topic link">Statistical variance</a>, 
                <a href="/topic/identifiability/" data-sc="text link:topic link">Identifiability</a>, 
                <a href="/topic/preliminary-estimates/" data-sc="text link:topic link">Preliminary estimates</a>, 
                <a href="/topic/matrix-inversion/" data-sc="text link:topic link">Matrix inversion</a>

What change needs to made to get 'Null hypothesis'?

PS:

$ cat *html | grep -n10 '<a href="/topic' | sed -E 's@.*<a href=/topic/[^>]*>([^<]*)</a>.*@\1@'
538-
539-                
540-
541-
542-
543-    
544-        <div class="topics-list mtl">
545-            <p class="hide">You can always find the topics here!</p>
546-            <strong>Topics:</strong>
547-            
548:                <a href="/topic/null-hypothesis/" data-sc="text link:topic link">Null hypothesis</a>, 
549-            
550:                <a href="/topic/approximation/" data-sc="text link:topic link">Approximation</a>, 
551-            
552:                <a href="/topic/estimation-methods/" data-sc="text link:topic link">Estimation methods</a>, 
553-            
554:                <a href="/topic/statistical-variance/" data-sc="text link:topic link">Statistical variance</a>, 
555-            
556:                <a href="/topic/identifiability/" data-sc="text link:topic link">Identifiability</a>, 
557-            
558:                <a href="/topic/preliminary-estimates/" data-sc="text link:topic link">Preliminary estimates</a>, 
559-            
560:                <a href="/topic/matrix-inversion/" data-sc="text link:topic link">Matrix inversion</a>
561-            
562-        </div>
563-
564-        <div class="mvl left">
565-            
566-
567-
568-
569-<div id="flag-description" aria-live="assertive">
570-    <a class="hover" data-qa="give-feedback" data-toggle="flag-reason" href="#" title="Give feedback on the topics for this item.">

ṔS2: the complete *html file: https://pastebin.com/RLnWXKWe

  • 1
    How could href=/topic/ match href="/topic/. Double quotes can't be just inserted anywhere. – choroba Jun 30 at 19:52
  • Because echo "foo <a href="/topic/null-hypothesis/" data-sc=" has a double-quoted string foo <a href=, an unquoted string /topic/null-hypothesis/, a second double-quoted string ` data-sc=, and so on, meaning the quotes are not present when sed` sees the text. I believe it would work otherwise in both cases with the appropriate changes. – Michael Homer Jun 30 at 22:39
  • This is what the toy thing should have been: $ echo "foo <a href=\"/topic/null-hypothesis/\" data-sc=\"text link:topic link\">Null hypothesis</a> bar" | sed -E 's@.*<a href=\"/topic/[^>]*>([^<]*)</a>.*@\1@' Null hypothesis Actually, not necessary to escape the dq in the replacement. – Erwann Jul 1 at 1:59
0

cat *html | grep -oE '\"\/.*\/\"' | awk -F'/' '{print $(NF-1)}'
This should work fine.

0

You should try running your first command just up to (but not including) the | — i.e., just the echo command.

$ echo "foo <a href="/topic/null-hypothesis/" data-sc="text link:topic link">Null hypothesis</a> bar"
foo <a href=/topic/null-hypothesis/ data-sc=text link:topic link>Null hypothesis</a> bar

See anything wrong?  Were you expecting href= and data-sc= to be followed by quoted strings?

Your echo command is wrong.  The " at the beginning doesn’t match the " at the end; it matches the first one it finds:

$ echo "foo <a href="/topic/null-hypothesis/" data-sc="text link:topic link">Null hypothesis</a> bar"
       ↑▲▲▲▲▲▲▲▲▲▲▲▲↑.......................↑▲▲▲▲▲▲▲▲▲↑....................↑▲▲▲▲▲▲▲▲▲▲▲▲▲▲▲▲▲▲▲▲▲▲▲▲↑

The characters with symbols under them are between quotes — so you have successfully quoted the < and > characters, which would have caused a mess it they weren’t quoted.  The characters with . under them aren't quoted.  And the quotes themselves aren’t quoted!

The simplest way to quote a quote is to use the other kind of quote — i.e., change the first and last " to '.

Then fix your sed command to work with the correct toy data, like this:

sed -E 's@.*<a href="/topic/[^>]*>([^<]*)</a>.*@\1@'
Note the added " after the href=.

and you should be fine with the real data.

  • "You should try running your first command just up to (but not including) the | — i.e., just the echo command." this answer comes after this comment, so I'm not sure it's still relevant. – Erwann Jul 1 at 3:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.