How am I able to download files from a FTP server that have only been added yesterday? For all subfolders + keep the same structure.


I can't think of a straight forward way to do this without scripting. Wget does not have a built in command to filter by date. Curl has -z:

curl -z "Oct 01 2015" "file"

But curl doesn't support wildcards for ftp / lacks wget's -r switch to recursively download an entire dir. With wget you can use --timestamping to only download new/changed files but that's for when you have a local mirror to compare against.

You're probably best off mounting the server with something like curlftpfs and then just using find and cp. I'm sure there's a neat way to script it with just wget but I'll leave that to someone more knowledgeable.

  • How can I do it with a script? I want to set a cronjob to download every days previous data – Teddy291 Oct 2 '15 at 13:59
  • If you're doing it every day you can just do wget --mirror once and then cron wget --timestamped every night and it'll update your local archive with the new data. – tolgraven Oct 2 '15 at 16:31
  • See section 5 of the wget manual for details. Probably the most straightforward way to go, just mirror and then do any kind of other processing (like if you want to sort the files in different folders per date or whatever) locally in the same cronjob. – tolgraven Oct 2 '15 at 16:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.