0

I have seen this but it's not working. The remote/ftp directory structure is this:

domain.com/
  toplevel/
    foo1/
      ARCHIVE/
        README.stuff
        DATA/
          README.txt
          ...other nested folders
      wantedstuff.zip
      wantedstuff2/
        morewantedstuff.zip
        ...otherstuffwanted

I want everything within /toplevel EXCEPT everything inside the /ARCHIVE folders for each nested folder in /toplevel/*.

I tried this:

wget --continue -r --exclude-directories=/ARCHIVE/ ftp://domain.com/toplevel/

and these:

wget --continue -r --exclude-directories=ARCHIVE ftp://domain.com/toplevel/
wget --continue -r --exclude-directories=ARCHIVE/ ftp://domain.com/toplevel/
wget --continue -r X /ARCHIVE/ ftp://domain.com/toplevel/
wget --continue -r -X '*/ARCHIVE/*' ftp://domain.com/toplevel/
wget --continue -r -X '*/ARCHIVE' ftp://domain.com/toplevel/
wget --continue -r --reject-regex '.*/ARCHIVE/.*' ftp://domain.com/toplevel/

But none seem to work, it still downloads that ARCHIVE folder. Wondering how to prevent it from downloading.

3
  • From your post, it sure looks like ARCHIVE is inside foo1, not toplevel. In which case, I would say your command is incorrect. Are you saying the tree is domain.com/toplevel/foo1/ARCHIVE or domain.com/toplevel/ARCHIVE?
    – number9
    Commented Nov 7, 2018 at 0:00
  • it is domain.com/toplevel/foo1/ARCHIVE , but I am doing wget domain.com/toplevel/, because I want all the subfolders (and all their recursive content).
    – Lance
    Commented Nov 7, 2018 at 0:08
  • Then you still have to use the toplevel... I will answer
    – number9
    Commented Nov 7, 2018 at 0:11

1 Answer 1

2

You have to include the toplevel/foo. See a previous example The solution should be:

wget --continue -r --exclude-directories=/toplevel/foo/ARCHIVE/ ftp://domain.com/toplevel/

For some reason I prefer:

wget --continue  -X /toplevel/foo/ARCHIVE/ -r ftp://domain.com/toplevel/

To exclude all ARCHIVE directories in the first two top levels, do:

wget --continue  -X */*/ARCHIVE/ -r ftp://domain.com/toplevel/

but that is personal preference.

4
  • Thank you, will try around with that. In my system it would be more like /toplevel/*/ARCHIVE, wondering how to do that then (making it dynamic). Otherwise I might have to just copy/paste a thousand subfolders have a thousand wget commands in a script.
    – Lance
    Commented Nov 7, 2018 at 0:19
  • I think you can try -X */*/ARCHIVE
    – number9
    Commented Nov 7, 2018 at 0:23
  • Yay, */*/ARCHIVE works.
    – Lance
    Commented Nov 7, 2018 at 0:42
  • Cool, I did not have my webserver at the moment to test. I will add that to my answer.
    – number9
    Commented Nov 7, 2018 at 0:44

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .