Output progess of the scp/sftp command to both standard out and a file on linux server

Question

I am writing a release script, which uses scp/sftp to get the the file from a remote server, we want to log the output of this script to a release.log file as well. I am using "tee" to achieve this, which works perfectly fine for everything, i mean standard out and error, but when it comes to progress of the file download(which takes time to copy), it doesn't show up on the console or in the file as well.

Output of scp without tee

frrinstd@loneqflocfd0034.uk.db.com:/apps/flowrisk/valservice/release/London/uat2/vs-server-2.1.16-21190-linux-amd64-frod-uat2-London-dist.zip /apps/flowrisk/uatldn/valservice
vs-server-2.1.16-21190-linux-amd64-frod-uat2-London-dist.zip                                                               100%   78MB  39.1MB/s   00:02

Output of scp WITH tee

frrinstd@loneqflocfd0034.uk.db.com:/apps/flowrisk/valservice/release/London/uat2/vs-server-2.1.16-21190-linux-amd64-frod-uat2-London-dist.zip /apps/flowrisk/uatldn/valservice

Can someone please tell me how to make it to output the progress of scp/sftp command on the console with tee as well, so that user know what is happening?

Answer 1

Having the progress meter in the log file doesn't sound very useful. How about running scp with its standard output connected to the terminal? This does have the downside that if the script is interrupted mid-transfer, there will be nothing in the log to indicate progress (but that's a very minor downside, as the size of the partial transfer ).

scp remotehost:/path/to/file /local/file >/dev/tty

If you want to save the progress output from scp, you can use script.

script -q /dev/stdout -c 'scp remotehost:/path/to/file /local/file' | tee scp.log

Alternatively, you can insert pv in the pipeline to get a progress indication. This isn't as nice as scp because pv only knows the number of bytes transferred, not the total.

ssh remotehost 'cat /path/to/file' | pv -bpt >/local/file

Answer 2

It seems that scp disables the progress meter if stdout is not a tty. The article I linked to contains some pseudo terminal hackery that might work for you; or you could go into the scp source and disable the isatty() check.