scp aborts when encountering a file witch it cannot get access to.

on host 1: /whatever/ contains ~10.000 files each 2-100kB in size daemons running on the host are creating new files in /whatever/ Creating a new file takes ~10 sec, and it happens every ~50 sec.

on host 2: an hourly cronjob runs scp -q me@host1:/whatever/* /local_copy_of_whatever/

scp fails when processing the files the the writing daemons is still accessing. I suspect that it is because it cannot gain access to the files being created. It seems as scp simply aborts in this situation. At the next run the problematic file will be copy'ed without any complains.

I would like a way to make scp ignore the problematic files - and continue quietly. (knowing that my cron job will get the file later)

closed as unclear what you're asking by Gilles, Jeff Schaller, roaima, Michael Homer, countermode Jan 25 '17 at 8:04

Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • 1
    Use find with ! -newer to create a tarball of files that haven't been modified in the last X minutes. Then scp the tarball. – DepressedDaniel Jan 23 '17 at 23:53
  • 2
    Your suspicion is unfounded. If scp fails, it will print an error message explaining why. The message might be hard to understand, in which case we can help, but there will be a message. There is no such thing as “blocked” files, so your question is not answerable. You need to tell us more about what is going on, and what the problem is. In what way does something fail? What behavior are you observing, and what did you expect to happen instead? – Gilles Jan 24 '17 at 23:42
  • Create the files with a .tmp suffix, and when the file creation is complete rename the file to remove the .tmp suffix. Ensure your scp command doesn't copy *.tmp files. – roaima Jan 25 '17 at 0:13
  • @DepressedDaniel: Thanks for the suggestion, that might be the way to go. – Henrik Jan 26 '17 at 0:13
  • @roaima: Thanks for the suggestions, a good workaround – Henrik Jan 26 '17 at 0:27

Browse other questions tagged or ask your own question.