We’re rewarding the question askers & reputations are being recalculated! Read more.
1005

Syntax: scp <source> <destination> To copy a file from B to A while logged into B: scp /path/to/file username@a:/path/to/destination To copy a file from B to A while logged into A: scp username@b:/path/to/file /path/to/destination


174

This is a job for rsync. There's no benefit to doing this manually with a shell loop unless you want to move the file rather than copy them. rsync -a /path/to/source/ /path/to/destination In your case: rsync -a /images2/ /images/ (Note trailing slash on images2, otherwise it would copy to /images/images2.) If images with the same name exist in both ...


110

In case if you need an alternate approach. Install sshfs. if you use ubuntu/debian: sudo apt-get install sshfs or, if you use centos/rhel: sudo yum install fuse-sshfs Create an empty dir mkdir /home/user/testdir "link" or "mount" the two directories sshfs user@server.com:/remote/dir /home/user/testdir "unlink" the dirs fusermount -u /home/user/...


90

This puts folder A into folder B: rsync -avu --delete "/home/user/A" "/home/user/B" If you want the contents of folders A and B to be the same, put /home/user/A/ (with the slash) as the source. This takes not the folder A but all of it's content and puts it into folder B. Like this: rsync -avu --delete "/home/user/A/" "/home/user/B" -a Do the sync ...


85

Are you using a 64-bit version of Linux with a lot of memory? In that case the problem could be that Linux can lock for minutes on big writes on slow devices like for example SD cards or USB sticks. It's a known bug that should be fixed in newer kernels. See http://lwn.net/Articles/572911/ Workaround: as root issue: echo $((16*1024*1024)) > /proc/...


79

X11 If using X11 (the most common GUI on traditional Unix or Linux based systems), to copy the content of a file to the X11 CLIPBOARD selection without displaying it, you can use the xclip or xsel utility. xclip -sel c < file Or: xsel -b < file To store the content of file as the CLIPBOARD X11 selection. Note that it should be stored using an UTF-...


70

Another (IMO) easy way would be: # to remote host cat localfile.conf | ssh user@hostname 'cat -> /tmp/remotefile.conf' # from remote host ssh user@hostname 'cat /tmp/remotefile.conf' > /tmp/localfile.conf Or if you prefer something GUI-like, try Midnight Commander. They call the feature Shell-Link. Most distros have em in their package systems as mc....


60

TL;DR Don't use -t. -t involves a pseudo-terminal on the remote host and should only be used to run visual applications from a terminal. Explanation The linefeed character (also known as newline or \n) is the one that when sent to a terminal tells the terminal to move its cursor down. Yet, when you run seq 3 in a terminal, that is where seq writes 1\n2\...


58

cat < file1 > file2 is not a UUOC. Classically, < and > do redirections which correspond to file descriptor duplications at the system level. File descriptor duplications by themselves don’t do a thing (well, > redirections open with O_TRUNC, so to be accurate, output redirections do truncate the output file). Don’t let the < > symbols ...


53

SSH does support a few commands, via the escape character (~ by default): $ ~? Supported escape sequences: ~. - terminate connection (and any multiplexed sessions) ~B - send a BREAK to the remote system ~C - open a command line ~R - Request rekey (SSH protocol 2 only) ~^Z - suspend ssh ~# - list forwarded connections ~& - background ...


53

%CPU should be low during a copy. The CPU tells the disk controller "grab data from sectors X–Y into memory buffer at Z". Then it goes and does something else (or sleep, if there is nothing else). The hardware triggers an interrupt when the data is in memory. Then the CPU has to copy it a few times, and tells the network card "transmit packets at memory ...


51

It looks OK. Although why are you using the -z option to compress the transfer? This option is normally used when you are copying to a remote rsync server over a slow network. In this instance it will compress and instantly decompress the files which will only increase your CPU uage with no benefit. The -a (archive) option implies the -r (recursive) option ...


46

Sometimes you need to get fancy with tar: tar -C / -cf - \ opt/widget etc/widget etc/cron.d/widget etc/init.d/widget \ --exclude=opt/widget/local.conf | ssh otherhost tar -C / -xvf -


43

The quick way The quickest way to transfer files over a LAN is likely not rsync, unless there are few changes. rsync spends a fair bit of time doing checksums, calculating differences, etc. If you know that you're going to be transferring most of the data anyway, just do something like this (note: there are multiple implementations of netcat; check the ...


43

The best choice, as already posted, is of course rsync. Nevertheless also unison would be a great piece of software to do this job. Both can be used in several operating systems. Rsync rsync synchronizes in one direction from source to destination. Therefore the following statement rsync -avh --progress Source Destination syncs everything from Source to ...


41

Use the PSCP tool from the putty download page: http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html PSCP is the putty version of scp which is a cp (copy) over ssh command. PSCP needs to be installed on your windows computer (just downloaded, really, there is no install process. In the Packaged Files section, pscp.exe is already included). ...


41

How about something like this in bash: for file in ABC.*; do cp "$file" "${file/ABC/DEF}";done you can test it by putting echo in front of the cp command: for file in ABC.*; do echo cp "$file" "${file/ABC/DEF}";done


40

For clarity's sake - the real mechanics is more complicated to give even better security - you can imagine the write-to-disk operation like this: application writes bytes (1) the kernel (and/or the file system IOSS) buffers them once the buffer is full, it gets flushed to the file system: the block is allocated (2) the block is written (3) the file and ...


36

From man rsync: Usages with just one SRC arg and no DEST arg will list the source files instead of copying. this as explanation below the invocation options, for you the invocation matches: Pull: rsync [OPTION...] [USER@]HOST::SRC... [DEST]


32

Use rsync with the --partial option rsync -av --partial sourcedir user@desthost:/destinationdir The --partial will keep partially transferred files. When you resume the rsync transfer after a ssh broken connection, partially transferred files will start resuming from the point where the ssh connection was lost, and also successfully transferred files will ...


29

You can use tar as a buffer process cd .rubies tar cf - ruby-2.1.3 | ( cd /opt && sudo tar xvfp - ) The first tar runs as you and so can read your home directory; the second tar runs under sudo and so can write to /opt.


28

rsync --verbose --archive --dry-run /oldisk/a/ /newdisk/a/ The --dry-run (or -n) will do a dry run, showing you what it would do without actually doing anything. If it looks ok, run the rsync without the -n option. This will be a copy, not a move, which isn't quite what you're doing, but is safer. The --archive (or -a) ensures all the ownership and ...


28

I believe you can use rsync to do this. The key observation would be in needing to use the --existing and --update switches. --existing skip creating new files on receiver -u, --update skip files that are newer on the receiver A command like this would do it: $ rsync -avz --update --existing src/ dst Example Say ...


27

Here's the solution on non-embedded Linux and Cygwin: cp -as SOURCE/ COPY Note that SOURCE must be an absolute path and have a trailing slash. If you want to give a relative path, you can use cp -as "$(pwd)/SOURCE/" COPY


27

rsync copies to temporary filenames (e.g. see Rsync temporary file extension and rsync - does it create a temp file during transfer?) unless you use the --inplace option. It renames them only after the file has been transferred successfully. rsync also deletes any destination files that were only partially transferred (e.g. due to disk full or other error)....


26

As Celada mentioned, there would be no point to using multiple threads of execution since a copy operation doesn't really use the cpu. As ryekayo mentioned, you can run multiple instances of cp so that you end up with multiple concurrent IO streams, but even this is typically counter-productive. If you are copying files from one location to another on the ...


25

It's easy using the install program from the coreutils that is typically used for this very purpose by build systems like automake: install -D /path/to/source /path/to/destination Note that install creates all parts of the path just like mkdir -p does, see man install. I'm curious why you didn't include why you want to do that. Calling mkdir and cp is very ...


25

Your shell code has two issues: The echo should not be there. The variable $i is mistyped as $1 in the destination file name. To make a copy of a file in the same directory as the file itself, use cp thefile thecopy If you insert anything else in there, e.g. cp thefile theotherthing thecopy then it is assumed that you'd like to copy thefile and ...


23

You can throttle a pipe with pv -qL (or cstream -t provides similar functionality) tar -cf - . | pv -q -L 8192 | tar -C /your/usb -xvf - -q removes stderr progress reporting. The -L limit is in bytes. More about the --rate-limit/-L flag from the man pv: -L RATE, --rate-limit RATE Limit the transfer to a maximum of RATE bytes per second. A ...


Only top voted, non community-wiki answers of a minimum length are eligible