I would like to verify if a URL exists without downloading. I am using below with curl:

if [[ $(curl ftp://ftp.somewhere.com/bigfile.gz) ]] 2>/dev/null;
 then
  echo "This page exists."
 else
  echo "This page does not exist."
fi

or using wget:

if [[ $(wget ftp://ftp.somewhere.com/bigfile.gz) -O-]] 2>/dev/null;
 then
  echo "This page exists."
 else
  echo "This page does not exist."
fi

This works great if the URL doesn't exist. If it exists, it downloads the file. In my case, the files are really big and I do not want it to download. I just want to know if that URL exists.

  • If you are trying to save disk space, try redirecting output of the command to /dev/null. – Lewis M Oct 11 at 11:48
up vote 0 down vote accepted

You're close. The correct way to approach this is using the HEAD method.

With cURL:

if curl --head --silent --fail ftp://ftp.somewhere.com/bigfile.gz 2> /dev/null;
 then
  echo "This page exists."
 else
  echo "This page does not exist."
fi

or using wget:

if wget -q --method=HEAD ftp://ftp.somewhere.com/bigfile.gz;
 then
  echo "This page exists."
 else
  echo "This page does not exist."
fi
if curl --output /dev/null --silent --head --fail "ftp://ftp.somewhere.com/bigfile.gz"
then
  echo "This page exists."
 else
  echo "This page does not exist."
fi

Try curl --head (HTTP FTP FILE) Fetch the headers only!

status=$(curl --head --silent ftp://ftp.somewhere.com/bigfile.gz | head -n 1)
if echo "$status" | grep -q 404
  echo "file does not exist"
else
  echo "file exists"
fi

Your Answer

 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.