0

I have seen this but it's not working. The remote/ftp directory structure is this:

domain.com/
  toplevel/
    foo1/
      ARCHIVE/
        README.stuff
        DATA/
          README.txt
          ...other nested folders
      wantedstuff.zip
      wantedstuff2/
        morewantedstuff.zip
        ...otherstuffwanted

I want everything within /toplevel EXCEPT everything inside the /ARCHIVE folders for each nested folder in /toplevel/*.

I tried this:

wget --continue -r --exclude-directories=/ARCHIVE/ ftp://domain.com/toplevel/

and these:

wget --continue -r --exclude-directories=ARCHIVE ftp://domain.com/toplevel/
wget --continue -r --exclude-directories=ARCHIVE/ ftp://domain.com/toplevel/
wget --continue -r X /ARCHIVE/ ftp://domain.com/toplevel/
wget --continue -r -X '*/ARCHIVE/*' ftp://domain.com/toplevel/
wget --continue -r -X '*/ARCHIVE' ftp://domain.com/toplevel/
wget --continue -r --reject-regex '.*/ARCHIVE/.*' ftp://domain.com/toplevel/

But none seem to work, it still downloads that ARCHIVE folder. Wondering how to prevent it from downloading.

  • From your post, it sure looks like ARCHIVE is inside foo1, not toplevel. In which case, I would say your command is incorrect. Are you saying the tree is domain.com/toplevel/foo1/ARCHIVE or domain.com/toplevel/ARCHIVE? – number9 Nov 7 '18 at 0:00
  • it is domain.com/toplevel/foo1/ARCHIVE , but I am doing wget domain.com/toplevel/, because I want all the subfolders (and all their recursive content). – Lance Pollard Nov 7 '18 at 0:08
  • Then you still have to use the toplevel... I will answer – number9 Nov 7 '18 at 0:11
2

You have to include the toplevel/foo. See a previous example The solution should be:

wget --continue -r --exclude-directories=/toplevel/foo/ARCHIVE/ ftp://domain.com/toplevel/

For some reason I prefer:

wget --continue  -X /toplevel/foo/ARCHIVE/ -r ftp://domain.com/toplevel/

To exclude all ARCHIVE directories in the first two top levels, do:

wget --continue  -X */*/ARCHIVE/ -r ftp://domain.com/toplevel/

but that is personal preference.

  • Thank you, will try around with that. In my system it would be more like /toplevel/*/ARCHIVE, wondering how to do that then (making it dynamic). Otherwise I might have to just copy/paste a thousand subfolders have a thousand wget commands in a script. – Lance Pollard Nov 7 '18 at 0:19
  • I think you can try -X */*/ARCHIVE – number9 Nov 7 '18 at 0:23
  • Yay, */*/ARCHIVE works. – Lance Pollard Nov 7 '18 at 0:42
  • Cool, I did not have my webserver at the moment to test. I will add that to my answer. – number9 Nov 7 '18 at 0:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.