2

How do I list files in the same order as scp -rp copies them?

I need to know this because sometimes I need to ctrl-C an scp and later want to copy the remaining files.

  • 2
    You should probably use rsync instead. – Stéphane Chazelas Mar 3 '17 at 9:53
  • On my system, it seems to be transfering the files in the same order as ls -1f (or print -rl -- *(oN)) lists them (that is the order the entries are stored in the directories). – Stéphane Chazelas Mar 3 '17 at 10:16
1

While I would use rsync here, this an attempt as a direct answer to the question.

With OpenSSH 7.4p1 on Debian GNU/Linux amd64 at least, and when copying remotely (that is, where either the source or destination is on a remote host), it looks like scp -r processes the files in the order it reads them from the directories.

That's the same order you'd get with:

printf '%s\n' **/*(D.oN)

in zsh for instance. The oN glob qualifier is to disable sorting. See also tree -U:

$ tree -U 1
1
├── 2
│   ├── 4
│   │   └── 2-4
│   ├── 2
│   │   └── 2-2
│   ├── 3
│   │   └── 2-3
│   └── 1
│       └── 2-1
├── 3
│   ├── 4
│   │   └── 3-4
│   ├── 2
│   │   └── 3-2
│   ├── 3
│   │   └── 3-3
│   └── 1
│       └── 3-1
└── 1
    ├── 4
    │   └── 1-4
    ├── 2
    │   └── 1-2
    ├── 3
    │   └── 1-3
    └── 1
        └── 1-1
$ print -rl 1/**/*(D.oN)
1/2/4/2-4
1/2/2/2-2
1/2/3/2-3
1/2/1/2-1
1/3/4/3-4
1/3/2/3-2
1/3/3/3-3
1/3/1/3-1
1/1/4/1-4
1/1/2/1-2
1/1/3/1-3
1/1/1/1-1
$ scp -r localhost:1 2
2-4      100%    5    11.6KB/s   00:00
2-2      100%    5    38.9KB/s   00:00
2-3      100%    5    19.6KB/s   00:00
2-1      100%    5    19.6KB/s   00:00
3-4      100%    5    32.1KB/s   00:00
3-2      100%    5    38.8KB/s   00:00
3-3      100%    5    37.5KB/s   00:00
3-1      100%    5    28.4KB/s   00:00
1-4      100%    5    23.8KB/s   00:00
1-2      100%    5    39.1KB/s   00:00
1-3      100%    5    42.8KB/s   00:00
1-1      100%    5    40.9KB/s   00:00
2

Well, the obvious thing to do is just use -v so you can see their names as they're being copied:

scp -vrp from/here to/there

However, fiddling about with the file names is not really the best approach. If you find yourself needing to do this often, you should be using rsync instead. Since it has a -u option (only copy if the local file is newer than the remote), it will skip any files you've already copied, and if it finds the same file on both source and destination, it will only copy the differences to make the remote file the same as the local one.

  • scp -rp is already showing the names of the files which are being copied (Ubuntu 16.04). The problem is with a directory of 20-100 files, where I have interrupted scp -rp ... with ctlr-C. I want to be able to simply ls -... the files in the order that they were being copied. – fadedbee Mar 3 '17 at 10:04
  • @chrisdew why? Isn't it to be able to resume transferring? – terdon Mar 3 '17 at 10:06
  • 1
    @chrisdew well, I would expect that echo /path/to/** would probably list them in the same order, but I have no idea how scp lists its files internally when given a directory. I also really don't see why that would ever be relevant unless it is to resume a transfer a task for which scp is the wrong tool. All of these problems go away (and your transfers are significantly sped up) if you use rsync instead. – terdon Mar 3 '17 at 10:11
  • 1
    @chrisdew Beware that relying on the directory order is fragile. For example, if you remove or add a file, then the order of other files can change. If fsck runs on the system, the order can change. If the files are backed up and restored, the order can change. Also, if you interrupt scp while it's copying a file, then the partially-copied file will remain in the destination, and scp wouldn't know that it needs to finish that file. The sane solution is to use rsync. – Gilles Mar 3 '17 at 22:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.