How do I list files in the same order as scp -rp copies them?
I need to know this because sometimes I need to ctrl-C an scp and later want to copy the remaining files.
3 Answers
While I would use rsync here, this an attempt as a direct answer to the question.
With OpenSSH 7.4p1 on Debian GNU/Linux amd64 at least, and when copying remotely (that is, where either the source or destination is on a remote host), it looks like scp -r processes the files in the order it reads them from the directories.
That's the same order you'd get with:
printf '%s\n' **/*(D.oN)
in zsh for instance. The oN glob qualifier is to disable sorting. See also tree -U:
$ tree -U 1
1
├── 2
│ ├── 4
│ │ └── 2-4
│ ├── 2
│ │ └── 2-2
│ ├── 3
│ │ └── 2-3
│ └── 1
│ └── 2-1
├── 3
│ ├── 4
│ │ └── 3-4
│ ├── 2
│ │ └── 3-2
│ ├── 3
│ │ └── 3-3
│ └── 1
│ └── 3-1
└── 1
├── 4
│ └── 1-4
├── 2
│ └── 1-2
├── 3
│ └── 1-3
└── 1
└── 1-1
$ print -rl 1/**/*(D.oN)
1/2/4/2-4
1/2/2/2-2
1/2/3/2-3
1/2/1/2-1
1/3/4/3-4
1/3/2/3-2
1/3/3/3-3
1/3/1/3-1
1/1/4/1-4
1/1/2/1-2
1/1/3/1-3
1/1/1/1-1
$ scp -r localhost:1 2
2-4 100% 5 11.6KB/s 00:00
2-2 100% 5 38.9KB/s 00:00
2-3 100% 5 19.6KB/s 00:00
2-1 100% 5 19.6KB/s 00:00
3-4 100% 5 32.1KB/s 00:00
3-2 100% 5 38.8KB/s 00:00
3-3 100% 5 37.5KB/s 00:00
3-1 100% 5 28.4KB/s 00:00
1-4 100% 5 23.8KB/s 00:00
1-2 100% 5 39.1KB/s 00:00
1-3 100% 5 42.8KB/s 00:00
1-1 100% 5 40.9KB/s 00:00
answered Mar 6, 2017 at 16:25
Well, the obvious thing to do is just use -v so you can see their names as they're being copied:
scp -vrp from/here to/there
However, fiddling about with the file names is not really the best approach. If you find yourself needing to do this often, you should be using rsync instead. Since it has a -u option (only copy if the local file is newer than the remote), it will skip any files you've already copied, and if it finds the same file on both source and destination, it will only copy the differences to make the remote file the same as the local one.
-
scp -rpis already showing the names of the files which are being copied (Ubuntu 16.04). The problem is with a directory of 20-100 files, where I have interruptedscp -rp ...with ctlr-C. I want to be able to simplyls -...the files in the order that they were being copied.– fadedbeeCommented Mar 3, 2017 at 10:04 -
@chrisdew why? Isn't it to be able to resume transferring?– terdon ♦Commented Mar 3, 2017 at 10:06
-
1@chrisdew well, I would expect that
echo /path/to/**would probably list them in the same order, but I have no idea howscplists its files internally when given a directory. I also really don't see why that would ever be relevant unless it is to resume a transfer a task for whichscpis the wrong tool. All of these problems go away (and your transfers are significantly sped up) if you usersyncinstead.– terdon ♦Commented Mar 3, 2017 at 10:11 -
1@chrisdew Beware that relying on the directory order is fragile. For example, if you remove or add a file, then the order of other files can change. If
fsckruns on the system, the order can change. If the files are backed up and restored, the order can change. Also, if you interruptscpwhile it's copying a file, then the partially-copied file will remain in the destination, and scp wouldn't know that it needs to finish that file. The sane solution is to use rsync. Commented Mar 3, 2017 at 22:49
GNU ls has the --sort=none (-U in abbreviated form) option ("do not sort; list entries in directory order").
rsyncinstead.ls -1f(orprint -rl -- *(oN)) lists them (that is the order the entries are stored in the directories).