Uploading directories with sftp?

Question

I'm having some trouble uploading directories(which contain other directories a few levels deep) by sftp. I realize I could work around this by gzipping, but I don't see why that's necessary.

Anyway, I try

sftp> put bin/
Uploading bin/ to /home/earlz/blah/bin
bin/ is not a regular file
sftp> put -r bin/
Uploading bin/ to /home/earlz/blah/bin
Couldn't canonicalise: No such file or directory
Unable to canonicalise path "/home/earlz/blah/bin"

I think the last error message is completely stupid. So the directory doesn't exist? Why not create the directory?

Is there anyway around this issue with sftp, or should I just use scp?

Answer 1

I don't know why sftp does this but you can only recursive copy if the destination directory already exists. So do this...

sftp> mkdir bin
sftp> put -r bin

Answer 2

CORRECTED: I initially claimed wrongly that OpenSSH did not support put -r. It does, but it does it in a very strange way. It seems to expect the destination directory to already exist, with the same name as the source directory.

sftp> put -r source
 Uploading source/ to /home/myself/source
 Couldn't canonicalize: No such file or directory
 etc.
sftp> mkdir source
sftp> put -r source
 Uploading source/ to /home/myself/source
 Entering source/
 source/file1
 source/file2

What's especially strange is that this even applies if you give a different name for the destination:

sftp> put -r source dest
 Uploading source/ to /home/myself/dest
 Couldn't canonicalize: ...
sftp> mkdir dest
sftp> put -r source dest
 Uploading source/ to /home/myself/dest/source
 Couldn't canonicalize: ...
sftp> mkdir dest/source
sftp> put -r source dest
 Uploading source/ to /home/myself/dest/source
 Entering source/
 source/file1
 source/file2

For a better-implemented recursive put, you could use the PuTTY psftp command line tool instead. It's in the putty-tools package under Debian (and most likely Ubuntu).

Alternately, Filezilla will do what you want, if you want to use a GUI.

Answer 3

You might be interested in using rsync instead. The command for that would be

 rsync --delete --rsh=ssh -av bin/ remote-ip-or-fqdn:/home/earlz/blah/bin/

This will copy everything in bin/ and place it in on the remote server in /home/earlz/blah/bin/. As an added benefit, it will first check to see if the file on the remote side hasn't changed, and if it hasn't, it won't re-send it. Additionally, you can add a -z option and it will compress it for you.

Answer 4

lcd: your local folder (with subfolders)

cd: your remote folder

put -r .

Answer 5

May I suggest a somewhat complicated answer, without zipping, but including tar?

Here we go:

tar -cf - ./bin | ssh target.org " ( cd /home/earlz/blah ; tar -xf - ) "

This will pack the directory ./bin with tar (-cf:=create file), filename - (none, stdout) and pipe it through the ssh-command to target.org (which might as well be an IP) where the command in quotes is performed, which is: cd to blah, and tar -xf (extract file) - none, no name, just stdin.

It's as if you pack a package at home, bring it to the post, then drive to work, where you expect the package and open it.

Maybe there is a much more elegant solution which just uses sftp.

Answer 6

You can use yafc (Yet anoter FTP/SFTP client). The -r option works there very well.

Answer 7

You can use rsync, which is a very powerful alternative for scp and sftp, especially when updating the copies from machine A to machine B, as it doesn't copy the files that haven't been altered; it's also able to remove files from machine B that have been deleted from machine A (only when it's told to of course).

for example :

rsync -zrp /home/a/ user@remote.host.com:/home/b/  

The -r option is for recursively copying files, -z enables compression during the transfer, and -p preserves the file permissions (file creation, edit, etc.) when copying, which is something that scp doesn't do AFAIK. Many more options are possible; as usual, read the man pages.
Original answer by Karolos

Answer 8

Login to the remote server with ssh, use sftp to connect back to your box, then use the get -r command to transfer directories to the remote server. The get command allows you to transfer directories recursively without having the directory already created.

ssh remote ip
sftp local ip 
get -r whichever-dir

Answer 9

SFTP case:

I needed to copy that structure on my ftp:

mainfolder --- folder --- subfolder
                  |           |
              file1.txt   file2.txt

That solved my problem:

cd ./mainfolder
mkdir folder
put -r /from/source/folder/* /mainfolder/folder/
cd ./folder
mkdir subfolder
put -r /from/source/folder/subfolder/* /mainfolder/folder/subfolder/

Answer 10

I just learned from the Arch Linux Wiki that it is possible to mount the sftp-share using sshfs. I'm running an sftp-server with chroot and jail and sshfs works very well.

1. Mount: sshfs <sftpuser>@<server>:<read/writable/directory> <your/local/mount/directory>
2. Unmount: fusermount -u <your/local/mount/directory>

Answer 11

Given the following local structure (inspired by by @nikita-malovichko):

~ --- mainfolder --- folder --- subfolder
                        |           |
                    file1.txt   file2.txt

I use this one-liner to upload a whole local directory mainfolder:

# Powershell:
Write-Output 'put mainfolder' | sftp -r -i "~/.ssh/ssh.pem" ubuntu@my-url.com:~/my-remote-target

Bash
echo 'put mainfolder' | sftp -r -i "~/.ssh/ssh.pem" ubuntu@my-url.com:~/my-remote-target

If you want to upload everything in the directory mainfolder but unpack it (so to speak) use put mainfolder/* or as the whole command:

# Powershell:
Write-Output 'put mainfolder/*' | sftp -r -i "~/.ssh/ssh.pem" ubuntu@my-url.com:~/my-remote-target

Bash
echo 'put mainfolder/*' | sftp -r -i "~/.ssh/ssh.pem" ubuntu@my-url.com:~/my-remote-target