I am running an Amazon EC2 instance that is limited in disk space. I have a very large folder that I need to FTP to my local machine, preferably without creating an intermediate tar file, since there might not be enough disk space.

Can I use a pipe for this?

Note While FTP isn't the best way of transferring the files, is my only option in this case.


Use just about any FTP client other than the basic ftp program.

  • Store your password in ~/.netrc and use wget:

    wget -r ftp://username@example.com/path/to/directory

    Keep in mind that the -r option has a default maximum depth of 5. You can use -l to increase it.

  • Use lftp:

    open username@example.com
    mirror /path/to/directory
  • Use ncftp:

    open -u username example.com
    get -R /path/to/directory

    Or its ncftpget command:

    ncftpget ftp://username@example.com//path/to/directory

Alternatively, mount the FTP server as a directory and access it like you would any other directory. There are several FUSE filesystems for FTP.

  • AVFS:

    cat file_containing_your_password >~/.avfs/#ftp_ctl:username@example.com/password
    cp -Rp ~/.avfs/#ftp:username@example.com/path/to/directory .
  • CURLFTPFS: store your password in ~/.netrc, then

    mkdir ~/example.com
    curlftpfs -o user=username example.com ~/example.com
    cp -Rp example.com/path/to/directory .
  • Using mget with the basic ftp clients should work for one directory (but not sub directories). – AProgrammer Nov 30 '11 at 20:15

For this kind of transfer FTP may not be your best choice. Using rsync or scp may work better for this. Try something like:

scp -r user@host:folder  .

If you need to fetch updates from the server, then rsync should work better.

EDIT: If you are stuck with FTP, then the use a recursive wget command, or get an FTP client which supports recursive gets.

  • For all kinds of transfers FTP is not the best choice! – symcbean Nov 1 '11 at 14:16
  • I know it's not best, but that's the system I have. The FTP server is running on a NAS and I can't change that. – gallamine Nov 1 '11 at 14:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.