Sign up ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. It's 100% free, no registration required.

I got a bunch of URLs (more than 1,000) and I am wondering if there is any CLI script to validate URL for http schema?

share|improve this question

closed as not a real question by Gilles, Renan, Stéphane Gimenez, jasonwryan, jw013 Aug 29 '12 at 18:55

It's difficult to tell what is being asked here. This question is ambiguous, vague, incomplete, overly broad, or rhetorical and cannot be reasonably answered in its current form. For help clarifying this question so that it can be reopened, visit the help center.If this question can be reworded to fit the rules in the help center, please edit the question.

If you google for something like "perl validate url" you will get there... – Johan Jul 15 '12 at 7:23
Thanks. I got this… and it works as expected. – Drake Jul 15 '12 at 7:46
What do you mean by URL? Are you trying to see if it's well-formed according to RFC1738? Do you want heuristics to detect unencoded space and extra final punctuation? Do you want to know whether the URL points to a reachable web page? – Gilles Jul 15 '12 at 23:20

2 Answers 2

up vote 4 down vote accepted

You can use Perl's Regexp::Common::URI::http. From the CPAN documentation:

use Regexp::Common qw /URI/;

while (<>) {
    /$RE{URI}{HTTP}/       and  print "Contains an HTTP URI.\n";
share|improve this answer
Even wget can do that to a certain extent (wish I could write an answer here, but the ever-present SE "cerberuses" have chosen to close this question). * Sigh * So here goes ... You may (mis)use wget -nv --spider <someURL> and evaluate its return code (echo $?). This is only for checking whether the link is valid (respectively the server still up) or not, though. – syntaxerror Jun 24 at 21:20

Something like egrep "^https?:\/\/" yourlistofurls or do you want to check that that url points to valid http content, too?

share|improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.