I'm using mget in an ftp script and would like mget to skip getting files that already exist in my local directory. Is there such a way? If so, is there also a way for mget to proceed if the local and remote files are different?

EDIT: Both are excellent answers, I went with the 2nd because it provided an example which I modified to:

lftp $HOST -u $USER,$PASSWD -e 'mirror --verbose=3 --only-newer --include "my-extended-regex-expression-of-include-file-patterns" / transfer/'

There are many different ftp implementations. I don't know of any ftp program that has an mget command that checks local files before downloading.

There are many programs that can download files over FTP and that do have what you want.

  • lftp

    lftp ftp.example.com <<EOF
    mirror --only-missing --file=/path/to/*.ext .

    Replace --only-missing by --only-newer to download only newer files.

  • wget

    wget -nc 'ftp://ftp.example.com/path/to/*.ext'

    Add the option -N to download only newer files.

Alternatively, and this is the best solution to do complex things, mount the remote filesystem and then use normal file manipulation tools. You can mount an FTP directory with CurlFtpFS.

mkdir mnt
curlftpfs ftp://ftp.example.com/ mnt
rsync -a --ignore-existing mnt/*.ext .   # replace --ignore-existing by -u to download only newer files
fusermount -u mnt
| improve this answer | |

ftp by itself can't do this (nor do most variations of sftp). That's usually done using a more capable program such as rsync or lftp.

If you're unable to use rsync, there are applications which use ftp that can be configured for the first part (existing files), at least. For example:

But (having used both that and the related Perl script), rsync and lftp warrant some study:


| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.