Copy specific folders via lftp

Question

I want to copy data using lftp login from server. Here is my command to copy all the data.

lftp -u uid,pwd -e 'mirror -c /home/dcr96/TCGA' sftp://rob2056@serapeum2.qib.pbtech

This copies the data from sftp to the local dir. However, I want to copy only specific folders, which are in a list. How should I do that?

What form is this list in? A single shell variable? An array? Delimited in a file somehow? — Jeff Schaller, May 3, 2017 at 19:22
These are folders with data.I can put the folder names in a list any format.Rather prefer putting them in new lines. — Ron, May 3, 2017 at 19:23
Also I was looking at options in mirror such as -i .But how can i include a list of folder names in this ? Or Just write the folder names(dont know whether to tab separate or comma separate on command line,as both did not work) in -i ? — Ron, May 3, 2017 at 19:26

rudimeier · Accepted Answer · 2017-05-03 22:43:45Z

You can simply use more than one mirror commands like this:

lftp -u uid,pwd \
  -e 'mirror -c /home/dcr96/TCGA/dir1; mirror -c /home/dcr96/TCGA/dir2; exit;' \
  sftp://rob2056@serapeum2.qib.pbtech

So all you need is some shell code which produces such command line from your list.

You could also generate such lftp script, which might be more easy to review and debug before executing.

#!/usr/bin/lftp -f
open -u uid,pwd serapeum2.qib.pbtech
cd /home/dcr96/TCGA

# auto-generated from the list
mirror -c dir1
mirror -c dir2
# ...

exit

Note the mirror command accepts a local target directory. You would need this to keep the complete paths in case of longer paths:

mirror -c path/to/dir1 path/to/dir1

1 Answer 1