Tag Info

Hot answers tagged

47

Master connection It's easiest if you plan in advance. Open a master connection the first time. For subsequent connections, route slave connections through the existing master connection. In your ~/.ssh/config, set up connection sharing to happen automatically: ControlMaster auto ControlPath ~/.ssh/control:%h:%p:%r If you start an ssh session to the ...


40

%CPU should be low during a copy. The CPU tells the disk controller "grab data from sectors X–Y into memory buffer at Z". Then it goes and does something else (or sleep, if there is nothing else). The hardware triggers an interrupt when the data is in memory. Then the CPU has to copy it a few times, and tells the network card "transmit packets at memory ...


24

SSH does support a few commands, via the escape character (~ by default): $ ~? Supported escape sequences: ~. - terminate connection (and any multiplexed sessions) ~B - send a BREAK to the remote system ~C - open a command line ~R - Request rekey (SSH protocol 2 only) ~^Z - suspend ssh ~# - list forwarded connections ~& - background ...


21

There's but one way to determine the optimal block size, and that's a benchmark. I've just made a quick benchmark. The test machine is a PC running Debian GNU/Linux, with kernel 2.6.32 and coreutils 8.5. Both filesystems involved are ext3 on LVM volumes on a hard disk partition. The source file is 2GB (2040000kB to be precise). Caching and buffering are ...


21

Patrick has it more or less correct, but here's why. The way you copy a file under UNIX works like this: Try to read some (more) bytes from fileA. If we failed to get bytes because we're at (or past) the end of the file, we're done; quit. Otherwise, write the bytes to fileB and loop back to step 1. Knowing that, and knowing it's as simple as that, lets ...


21

It depends on what you're doing. The install command is normally used in installation scripts that come with packages and source code for installing a binary to your system. It can also be used to install any other file or directory. In addition to the -d and -c options you have -m for specifying the new permissions of the file to be installed, so you don't ...


20

If you have rsync (remove --dry-run to do it for real): rsync --dry-run --remove-source-files -avHAX /unencrypted/ /encrypted Otherwise, using bash4+ and GNU stat: #!/bin/bash set -e shopt -s nullglob globstar for from in /unencrypted/**/*; do to="${from/\/un//}" if [[ -d "$from" ]]; then echo mkdir -p "$to" echo chmod "$(stat ...


19

To copy a file from B to A while logged into B: scp /path/to/file username@a:/path/to/destination To copy a file from B to A while logged into A: scp username@b:/path/to/file /path/to/destination


17

rsync -va -n /oldisk/a/ /newdisk/a/ The -n will do a dry run, showing you what it would do without actually doing anything. If it looks ok, run the rsync without the -n option. This will be a copy, not a move, which isn't quite what you're doing, but is safer.


17

It's easy using the install program from the coreutils that is typically used for this very purpose by build systems like automake: install -D /path/to/source /path/to/destination Note that install creates all parts of the path just like mkdir -p does, see man install. I'm curious why you didn't include why you want to do that. Calling mkdir and cp is ...


16

How about something like this in bash: for file in ABC.*; do cp "$file" "${file/ABC/DEF}";done you can test it by putting echo in front of the cp command: for file in ABC.*; do echo cp "$file" "${file/ABC/DEF}";done


14

Try to use such next function for such situation: copy_wdir() { mkdir -p -- "$(dirname -- "$2")" && cp -- "$1" "$2" ; } and use it as copy_wdir aaa/deep/sea/blob.psd bbb/deep/sea/blob.psd By the way, GNU cp has a --parents option. It's really close to what you want, but not exactly. It will also create aaa directory that seems you don't need. ...


13

Those are all very complicated methods. You can mount the remote file system on your local machine with sshfs: mkdir -p /mnt/sshfs root@IS1300:~# sshfs 192.168.1.2:/ /mnt/sshfs root@IS1300:~# umount /mnt/sshfs Then you can copy paste the file with nautilus, gnome, konqueror, dolphin, bash or whatever.


13

I think you should do something like the GUI applications do. My idea for doing this is to write two functions for Copy and Paste, where Copy writes path of files to be copied to a temporary file and Paste reads those paths and simply calls cp command. My implementation (to be put in .bashrc file) is like below: function Copy { touch ~/.clipfiles ...


13

Are you using a 64-bit version of Linux with a lot of memory? In that case the problem could be that Linux can locks for minutes on big writes on slow devices like for example SD cards or USB sticks. It's a known bug that should be fixed in newer kernels. See http://lwn.net/Articles/572911/ Workaround: as root issue: echo $((16*1024*1024)) > ...


11

This could, possibly, be a faster alternative, and you won't clog the network for two days: Take one or two large USB (USB 3 if you have it) or FireWire disks, connect it to the server and copy the files to the disk. Carry the disk to your local machine. Copy the files to the machine.


10

mkdir ~/dst find source -name "*.xxx" -exec mv -i {} -t ~/dst \;


9

rsync is able to do this. rsync --ignore-existing <src> <dest> You can also various kinds of updates. Just have a look at the man page.


9

Your definition of efficient is backwards. A more efficient implementation wastes less cpu time. On the local copy you are averaging about 74 MB/s of throughput ( read + write ), which is about as good as a single hard disk is going to get.


8

The magic of rsync filter rules: $ rsync -av --filter="+ */" --filter="-! *blah*" /source /dest Consult the rsync man page for the details on filter rules, but here's the condensed version for this particular need. --filter="+ */" means "include everything that is a directory" --filter="-! *blah* means "exclude everything that does NOT include blah in ...


8

If fileA.big is grown during the copy, the copy will include the data that was appended. If the file is truncated shorter than where the copy is currently at, the copy will abort right where its at and the destination file will contain what was copied up to the time it aborted.


8

dd dates from back when it was needed to translate old IBM mainframe tapes, and the block size had to match the one used to write the tape or data blocks would be skipped or truncated. (9-track tapes were finicky. Be glad they're long dead.) These days, the block size should be a multiple of the device sector size (usually 4KB, but on very recent disks ...


8

As root, set up a named pipe: # mkfifo /tmp/fifo # chmod o+w /tmp/fifo Then, transfer your data as me: $ tar cfzp - foldertocopy | ssh me@machine "cat > /tmp/fifo" But read it as root: # tar -xfzp /tmp/fifo


8

It would be a hell to tell find what to do in this case. Better use the shell: for i in **/*.{xrt,ini,moo}; do FILE=$(basename "$i") DIR=~/dst/${FILE%.*} echo mkdir -p -- "$DIR" echo mv -i -t "$DIR" -- "$i" done Use shopt -s globstar to make the ** glob work (or use zsh!). And remove the echos later if the command prints what you want.


8

This does the whole job in one go - in all child directories, all in a single stream without any filename problems. It'll copy from smallest to largest every file you have. You will need to mkdir ${DESTINATION} if it doesn't already exist. find . ! -type d -print0 | du -b0 --files0-from=/dev/stdin | sort -zk1,1n | sed -zn 's/^[^0-9]*[0-9]*[^.]*//p' | tar ...


7

If you want to move all the files inside a directory hierarchy to a single destination directory, in bash ≥4 (put shopt -s globstar in your ~/.bashrc) or zsh: mkdir ~/new-directory mv -i **/*.xxx ~/new-directory In other shells: mkdir ~/new-directory find . -name '*.xxx' -exec mv -i {} ~/new-directory \; How to read this find command: .: traverse the ...


7

i suggest xclip. i use it all the time for this. but you don't need a script. just use functions. i see no need to use these scripts, in particular, because if you are doing what i think you are doing, you will find the scripts already mentioned (on the helpful linux tidbits blog) annoying because they prompt you for input wherever possible, just to use ...


7

You can do: sed -n wfile.merge file1 file2 Or: awk '{print > "file.merge"}' file1 file2 Or: sh -c 'cat file1 file2 > file.merge' (note that depending on the implementation, the first two may not work properly with binary files).



Only top voted, non community-wiki answers of a minimum length are eligible