I wanted to backup all my .vcf files from my carddav server (ownCloud). The script is very simple and is the following:

$ wget -Avcf -r -np -l0 --no-check-certificate -e robots=off --user=user \
    --password='password' https://cloud.domain.com/foo/carddav

The total number of .vcf files is about 400, after downloading about 70 of them, wget returns this error:

   ss of error

original URL: http://oi40.tinypic.com/2ch9itt.jpg

Which kills the process because the system is "out of memory". The system is a Debian Wheezy virtual machine, hosted on Windows 7. I tried to raise the RAM to 1024MB instead of the actual 128MB, but the problem still exists.

Any suggestions on how to work around this or alternative ways to accomplish this?

  • HTTrack might be an alternative to wget in your case. It is available in Debian.
    – jofel
    Jan 7, 2014 at 14:13
  • 1
    wget was consuming about 530MB of memory at the time it was killed, so that's very abnormal. Could you try with curl?
    – mveroone
    Jan 7, 2014 at 14:38
  • What is this -f switch? wget doesn't seem to have such an option.
    – psusi
    Jan 7, 2014 at 15:24
  • @jofel: thank you for the suggestion, I will use it as a last option if I can't do it with wget.
    – giovi321
    Jan 7, 2014 at 16:22
  • @psusi: there is no -f switch in my command, probably you saw the -A switch followed by the .vcf extension, without the dot. It is "-Avcf", where vcf is the argument of the switch -A.
    – giovi321
    Jan 7, 2014 at 16:23

1 Answer 1


It seems wget loops causes memory overflow.
The natural first suggestion is to increase again memory of your cloud instance from 1Gb to 2Gb. This solved a similar issue recently.

If this is not possible or doesn't solve the problem the second solution is to run wget within 2 steps:

  1. Retrieve files list. As I see in your screenshot the files are in the directory cloud.some_domain.com/remote.php/carddav/addressbooks/your_name/.
    So, run wget to get the directory index:
    wget https://cloud.some_domain.com/remote.php/carddav/addressbooks/your_name/
    This will give you an index.html file.

  2. Now you can parse it to retrieve the filenames to download:

    grep ".vcf" index.html | awk -F"href=" '{print $2}' | awk -F\" '{print $2}' > ALL_VCF_FILES.lst
    for elt in `cat ALL_VCF_FILES.lst`
    wget https://cloud.some_domain.com/remote.php/carddav/addressbooks/your_name/$elt
    rm ALL_VCF_FILES.lst index.html
  • thank you very much that's the best solution ti my problem! I'll let you know asap if it works.
    – giovi321
    Jan 7, 2014 at 16:24
  • You welcome ! I edited the code parts to match best your case (HTTPS)
    – Slyx
    Jan 7, 2014 at 16:30
  • I have to thank @Slyx! Your code was almost perfect. The only problem was that the ALL_VCF_FILES.lst list was a list like /remote.php/carddav/addressbooks/user/contacts/xxxxxxxxx.vcf this means that the link is missing the "cloud.domain.com". Using sed I made a simple find and replace like this: sed 's/\/remote.php\/carddav\/addressbooks\/user\/contacts\//https:\/\/www.cloud.domain.com\/remote.php\/carddav\/addressbooks\/user\/contacts\//g' ALL_VCF_FILES.lst > all_vcf_files.lst Maybe you can incorporate this in your answer to make the code complete. It works like a charm!
    – giovi321
    Jan 7, 2014 at 16:48
  • wget --no-check-certificate -e robots=off --user=user --password='pswd' blabla grep ".vcf" index.html | awk -F"href=" '{print $2}' | awk -F\" '{print $2}' > ALL_VCF_FILES.lst sed 's/\/remote.php\/carddav\/addressbooks\/user\/contacts\//https:\/\/cloud.domain.com\/remote.php\/carddav\/addressbooks\/user\/contacts\//g' ALL_VCF_FILES.lst > all_vcf_files.lst for elt in cat all_vcf_files.lst do wget -nc --no-check-certificate -e robots=off --user=user --password='pswd' $elt done rm ALL_VCF_FILES.lst index.html all_vcf_files.lst mv *.vcf /home/scripts/owncloud_backup/vcf/
    – giovi321
    Jan 7, 2014 at 16:51
  • @user3161330 instead of adding a sed line we can just add the missing part of the URL before the $elt variable. See my edit. 2nd : you have the pathchain in your index.html because you requested this specific file in your wget commad.
    – Slyx
    Jan 7, 2014 at 16:57

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .