Hot answers tagged

324

To copy a file from B to A while logged into B: scp /path/to/file username@a:/path/to/destination To copy a file from B to A while logged into A: scp username@b:/path/to/file /path/to/destination


89

Master connection It's easiest if you plan in advance. Open a master connection the first time. For subsequent connections, route slave connections through the existing master connection. In your ~/.ssh/config, set up connection sharing to happen automatically: ControlMaster auto ControlPath ~/.ssh/control:%h:%p:%r If you start an ssh session to the ...


53

cat < file1 > file2 is not a UUOC. Classically, < and > do redirections which correspond to file descriptor duplications at the system level. File descriptor duplications by themselves don’t do a thing (well, > redirections open with O_TRUNC, so to be accurate, output redirections do truncate the output file). Don’t let the < > symbols ...


49

%CPU should be low during a copy. The CPU tells the disk controller "grab data from sectors X–Y into memory buffer at Z". Then it goes and does something else (or sleep, if there is nothing else). The hardware triggers an interrupt when the data is in memory. Then the CPU has to copy it a few times, and tells the network card "transmit packets at memory ...


42

Install sshfs: sudo apt-get install sshfs create a empty dir mkdir /home/user/testdir "link" or "mount" the two directories sshfs user@server.com:/remote/dir /home/user/test "unlink" the dirs fusermount -u /home/youruser/remotecomp For more see here, linuxjournal.com


42

TL;DR Don't use -t. -t involves a pseudo-terminal on the remote host and should only be used to run visual applications from a terminal. Explanation The linefeed character (also known as newline or \n) is the one that when sent to a terminal tells the terminal to move its cursor down. Yet, when you run seq 3 in a terminal, that is where seq writes 1\n2\...


41

There's but one way to determine the optimal block size, and that's a benchmark. I've just made a quick benchmark. The test machine is a PC running Debian GNU/Linux, with kernel 2.6.32 and coreutils 8.5. Both filesystems involved are ext3 on LVM volumes on a hard disk partition. The source file is 2GB (2040000kB to be precise). Caching and buffering are ...


39

SSH does support a few commands, via the escape character (~ by default): $ ~? Supported escape sequences: ~. - terminate connection (and any multiplexed sessions) ~B - send a BREAK to the remote system ~C - open a command line ~R - Request rekey (SSH protocol 2 only) ~^Z - suspend ssh ~# - list forwarded connections ~& - background ...


36

Are you using a 64-bit version of Linux with a lot of memory? In that case the problem could be that Linux can lock for minutes on big writes on slow devices like for example SD cards or USB sticks. It's a known bug that should be fixed in newer kernels. See http://lwn.net/Articles/572911/ Workaround: as root issue: echo $((16*1024*1024)) > /proc/...


34

This is a job for rsync. There's no benefit to doing this manually with a shell loop unless you want to move the file rather than copy them. rsync -a /images/ /images2/ If images with the same name exist in both directories, the command above will overwrite /images2/SOMEPATH/SOMEFILE with /images/SOMEPATH/SOMEFILE. If you want to replace only older files, ...


27

Use rsync with the --partial option rsync -av --partial sourcedir user@desthost:/destinationdir The --partial will keep partially transferred files. When you resume the rsync transfer after a ssh broken connection, partially transferred files will start resuming from the point where the ssh connection was lost, and also successfully transferred files will ...


25

Those are all very complicated methods. You can mount the remote file system on your local machine with sshfs: mkdir -p /mnt/sshfs root@IS1300:~# sshfs 192.168.1.2:/ /mnt/sshfs root@IS1300:~# umount /mnt/sshfs Then you can copy paste the file with nautilus, gnome, konqueror, dolphin, bash or whatever.


21

Patrick has it more or less correct, but here's why. The way you copy a file under UNIX works like this: Try to read some (more) bytes from fileA. If we failed to get bytes because we're at (or past) the end of the file, we're done; quit. Otherwise, write the bytes to fileB and loop back to step 1. Knowing that, and knowing it's as simple as that, lets ...


21

It's easy using the install program from the coreutils that is typically used for this very purpose by build systems like automake: install -D /path/to/source /path/to/destination Note that install creates all parts of the path just like mkdir -p does, see man install. I'm curious why you didn't include why you want to do that. Calling mkdir and cp is ...


21

How about something like this in bash: for file in ABC.*; do cp "$file" "${file/ABC/DEF}";done you can test it by putting echo in front of the cp command: for file in ABC.*; do echo cp "$file" "${file/ABC/DEF}";done


21

As Celada mentioned, there would be no point to using multiple threads of execution since a copy operation doesn't really use the cpu. As ryekayo mentioned, you can run multiple instances of cp so that you end up with multiple concurrent IO streams, but even this is typically counter-productive. If you are copying files from one location to another on the ...


20

Many GNU tools such as cp, mv and tar support creating backup files when the target exists. That is, when copying foo to bar, if there is already a file called bar, the existing bar will be renamed, and after the copy bar will contain the contents of foo. By default, bar is renamed to bar~, but the behavior can be modified: # ...


20

If you have rsync (remove --dry-run to do it for real): rsync --dry-run --remove-source-files -avHAX /unencrypted/ /encrypted Otherwise, using bash4+ and GNU stat: #!/bin/bash set -e shopt -s nullglob globstar for from in /unencrypted/**/*; do to="${from/\/un//}" if [[ -d "$from" ]]; then echo mkdir -p "$to" echo chmod "$(stat -...


19

dd dates from back when it was needed to translate old IBM mainframe tapes, and the block size had to match the one used to write the tape or data blocks would be skipped or truncated. (9-track tapes were finicky. Be glad they're long dead.) These days, the block size should be a multiple of the device sector size (usually 4KB, but on very recent disks ...


18

rsync --verbose --archive --dry-run /oldisk/a/ /newdisk/a/ The --dry-run (or -n) will do a dry run, showing you what it would do without actually doing anything. If it looks ok, run the rsync without the -n option. This will be a copy, not a move, which isn't quite what you're doing, but is safer. The --archive (or -a) ensures all the ownership and ...


18

It looks OK. Although why are you using the -z option to compress the transfer? This option is normally used when you are copying to a remote rsync server over a slow network. In this instance it will compress and instantly decompress the files which will only increase your CPU uage with no benefit. The -a (archive) option implies the --progress and -r (...


17

I think you should do something like the GUI applications do. My idea for doing this is to write two functions for Copy and Paste, where Copy writes path of files to be copied to a temporary file and Paste reads those paths and simply calls cp command. My implementation (to be put in .bashrc file) is like below: function Copy { touch ~/.clipfiles ...


17

Try to use such next function for such situation: copy_wdir() { mkdir -p -- "$(dirname -- "$2")" && cp -- "$1" "$2" ; } and use it as copy_wdir aaa/deep/sea/blob.psd bbb/deep/sea/blob.psd By the way, GNU cp has a --parents option. It's really close to what you want, but not exactly. It will also create aaa directory that seems you don't need. ...


16

This could, possibly, be a faster alternative, and you won't clog the network for two days: Take one or two large USB (USB 3 if you have it) or FireWire disks, connect it to the server and copy the files to the disk. Carry the disk to your local machine. Copy the files to the machine.


16

Use the PSCP tool from the putty download page: http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html PSCP is the putty version of scp which is a cp (copy) over ssh command. PSCP needs to be installed on your windows computer (just downloaded, really, there's no install process). Nothing needs to be installed on the school's servers. PSCP and ...


16

Sometimes you need to get fancy with tar: tar -C / -cf - \ opt/widget etc/widget etc/cron.d/widget etc/init.d/widget \ --exclude=opt/widget/local.conf | ssh otherhost tar -C / -xvf -


15

The quick way The quickest way to transfer files over a LAN is likely not rsync, unless there are few changes. rsync spends a fair bit of time doing checksums, calculating differences, etc. If you know that you're going to be transferring most of the data anyway, just do something like this: user@dest:/target$ nc -q 1 -l -p 1234 | tar xv user@source:/...


15

It is not, because as others have pointed out, the behavior in question is shell-dependent. As you (the OP) have pointed out, this is a bit of a pedantic, maybe even humorous?, sort of topic. However, on GNU systems, your initial premise has another solution available: cp --no-preserve=all file1 file2. Try this out, I think it will satisfy your described ...



Only top voted, non community-wiki answers of a minimum length are eligible