I got a bunch of URLs (more than 1,000) and I am wondering if there is any CLI script to validate URL for http schema?

  • 1
    If you google for something like "perl validate url" you will get there... – Johan Jul 15 '12 at 7:23
  • Thanks. I got this stackoverflow.com/questions/2599047/… and it works as expected. – Drake Guan Jul 15 '12 at 7:46
  • What do you mean by URL? Are you trying to see if it's well-formed according to RFC1738? Do you want heuristics to detect unencoded space and extra final punctuation? Do you want to know whether the URL points to a reachable web page? – Gilles 'SO- stop being evil' Jul 15 '12 at 23:20

You can use Perl's Regexp::Common::URI::http. From the CPAN documentation:

use Regexp::Common qw /URI/;

while (<>) {
    /$RE{URI}{HTTP}/       and  print "Contains an HTTP URI.\n";
| improve this answer | |
  • Even wget can do that to a certain extent (wish I could write an answer here, but the ever-present SE "cerberuses" have chosen to close this question). * Sigh * So here goes ... You may (mis)use wget -nv --spider <someURL> and evaluate its return code (echo $?). This is only for checking whether the link is valid (respectively the server still up) or not, though. – syntaxerror Jun 24 '15 at 21:20

Something like egrep "^https?:\/\/" yourlistofurls or do you want to check that that url points to valid http content, too?

| improve this answer | |

Not the answer you're looking for? Browse other questions tagged or ask your own question.