I tried to transfer a file of 200 MB last week to the remote machine. Then I got a 30 MB file. Today I tried to scp a whole directory and a significant number of files were lost.

Is it normal with scp? How to prevent this?


I wrote a expect script to scp file, could it be relevant?

this script is quite simple:

set file [lindex $argv 0];
set ip [lindex $argv 1];
spawn scp -r $file $ip
expect {
    "*password*" {send "password\r"}
    "*(yes/no)?*" {send "yes\r";exp_continue}
expect eof

There was no error message, but simply a lost of files were not copied.

  • 4
    No. You have other problems. – Ignacio Vazquez-Abrams Oct 25 '17 at 11:39
  • @IgnacioVazquez-Abrams Hi, thnx. I scp files with a expect script to automate it. Could it relevant? – J.R. Oct 25 '17 at 11:41
  • 4
    We don't know what you did, or what happened other than that you used expect to run scp and lost some files. You did not say what commands were executed or where the files were lost (on server or client). You also do not mention any error messages or warnings. Please edit your question to provide more details. – Kusalananda Oct 25 '17 at 11:45
  • @Kusalananda thnx. I edited. – J.R. Oct 25 '17 at 11:56
  • What happens when you simply issue the scp command outside the expect script? Does it run normally with all files completely transferred? ... in my experience using public key auth with a passphraseless key used on an unprivileged account, limited to a particular ip or sub, and a particular command works very well for scripted backup purposes... – RubberStamp Oct 25 '17 at 13:02

Scp cannot guarantee that a copy will happen correctly. No software can guarantee that a copy will take place correctly. There is always the possibility that the power or the network will go down, or a source file turns out to be unreadable, or the destination volume fills up, etc.

The most a file copy software can do is to guarantee that when it exits, either the copy was correct, or the software reports an error. The reliable way to check for errors is to check the return status of the process: 0 for success, any other value for error. Scp, like any halfway decent program, does return 0 on success and a nonzero value on failure.

Your expect script does not check the return status of scp, therefore you have no way to know whether scp told you that the copy was successful.

Expect is not a useful tool here. You should really use public key authentication. If you really, really can't set up keys, then use sshpass rather than rolling your own.

Also, you should use rsync rather than scp. Rsync has a major advantage: if a copy is interrupted, you can run it again and it'll take where it left off. Be sure to tell it to preserve timestamps, and also other metadata unless you have a good reason not to:

| improve this answer | |
  • Might be worth mentioning that rsync is better at reporting failures... – jasonwryan Oct 25 '17 at 20:11
  • @jasonwryan In what way? (But thanks anyway, your comment reminds me I meant to mention rsync to resume interrupted copies.) – Gilles 'SO- stop being evil' Oct 25 '17 at 20:16
  • In that it prints the failed files and, as Stéphane answers, can be rerun with -n for a comprehensive report. – jasonwryan Oct 25 '17 at 20:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.