Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I am trying to use FTP to download several hundred files within a directory. Is there a way to only download specific files with FTP?

Ideally, I would like to match specific files using a regular expression. Unfortunately, the directory is massive and it's too large to download the entire directory and then use find or grep locally to manipulate certain files. Is there a simply way to do what I am trying to do on Unix?

If not, I plan to just write a script in perl or python. Thanks in advance for the help.

share|improve this question
add comment

4 Answers

You can use shell globbing on the remote system by making sure glob is on, then doing mget.

See http://www.proftpd.org/docs/howto/Globbing.html for more information.

I believe that's as far as you can get with FTP. If you can get ssh acess instead, there is much more that you can do, including grepping and modifying the files on the server.

share|improve this answer
    
thanks for the link, this worked! –  dr.bunsen Oct 20 '11 at 0:10
add comment

Very often large file repositories will have an index.txt file somewhere about. If you download that it contains a list of all the files inside that folder. Quite useful for quick greps of package lists at least.

share|improve this answer
add comment

Mount the FTP server as a local directory with one of the several FUSE-based filesystems for FTP, then use your favorite tool (such as cp plus shell wildcards, or rsync) to match and copy the files you want to retrieve. For example:

mkdir example-somedir
curlftpfs ftp://ftp.example.com/somedir example-somedir
cd example-somedir
rsync -a --exclude 'boring.ext' --include='*.ext' --exclude='*' . /path/to/local/copy
fusermount -u example-somedir
share|improve this answer
add comment

If you want a GUI solution and you have the Konqueror browser, you can type ftp:// and the ftp address and browse the directory structure.

It sounds like you may want a command line solution, however.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.