Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I am extracting URLs from a website using cURL as below.

curl www.somesite.com | grep "<a href=.*title=" > new.txt

My new.txt file is as below.

<a href="http://website1.com" title="something">
<a href="http://website1.com" information="something" title="something">
<a href="http://website2.com" title="some_other_thing">
<a href="http://website2.com" information="something" title="something">
<a href="http://websitenotneeded.com" title="something NOTNEEDED">

However, I need to extract only the below information.

<a href="http://website1.com" title="something">
<a href="http://website2.com" information="something" title="something">

I am trying to ignore the <a href which have information in them and whose title end with NOTNEEDED.

How can I modify my grep statement?

share|improve this question
    
Is the output you're showing here correct? The text describing it doesn't make sense along with this example. –  slm Feb 10 at 19:55
1  
Aren't you looking for curl www.somesite.com | grep "<a href=.*title=" | grep -v NOTNEEDED > new.txt? –  terdon Feb 10 at 21:33
    
@terdon, exactly that was what I was looking for. I can accept it as an answer if you post it. –  Ramesh Feb 10 at 21:36
    
Ramesh, it's basically @slm's answer. I have just edited it so you can accept it. –  terdon Feb 10 at 21:36
    
oh yeah, I did not realize pipe was this powerful. I have accepted it as an answer. Thanks! –  Ramesh Feb 10 at 21:38

1 Answer 1

up vote 3 down vote accepted

I'm not fully following your example + the description but it sounds like what you want is this:

$ grep -v "<a href=.*title=.*NOTNEEDED" sample.txt 
<a href="http://website1.com" title="something">
<a href="http://website1.com" information="something" title="something">
<a href="http://website2.com" title="some_other_thing">
<a href="http://website2.com" information="something" title="something">

So for your example:

$ curl www.example.com | grep -v "<a href=.*title=" | grep -v NOTNEEDED > new.txt
share|improve this answer
    
I have a class in the <a href section. Basically, I do not want that in my output. –  Ramesh Feb 10 at 21:31

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.