Tag Info

Hot answers tagged

97

In appearance, dd is a tool from an IBM operating system that's retained its foreign appearance (its parameter passing), which performs some very rarely-used functions (such as EBCDIC to ASCII conversions or endianness reversal… not a common need nowadays). I used to think that dd was faster for copying large blocks of data on the same disk (due to more ...


86

You can send dd a certain signal using the kill command to make it output its current status. The signal is INFO on BSD systems (including OSX) and USR1 on Linux. In your case: kill -INFO $PID You can find the process id ($PID above) with the ps command; or see pgrep and pkill alternatives on mac os x for more convenient methods. More simply, as AntoineG ...


34

Under OS X (didn't try on Linux), you can simply type Ctrl+T in the terminal running dd. It will print the same output as kill -INFO $PID, plus the CPU usage: load: 1.40 cmd: dd 34536 uninterruptible 3.49u 64.58s 5020305+0 records in 5020304+0 records out 2570395648 bytes transferred in 4284.349974 secs (599950 bytes/sec) I found out about it reading ...


32

There's but one way to determine the optimal block size, and that's a benchmark. I've just made a quick benchmark. The test machine is a PC running Debian GNU/Linux, with kernel 2.6.32 and coreutils 8.5. Both filesystems involved are ext3 on LVM volumes on a hard disk partition. The source file is 2GB (2040000kB to be precise). Caching and buffering are ...


30

You can switch bs and skip options: dd bs=1131 skip=1 if=filtered.dump of=trimmed.dump This way the operation can benefit from a greater block. Otherwise, you could try with tail (although it's not safe to use it with binary files): tail -c +1132 filtered.dump >trimmed.dump Finally, you may use 3 dd instances to write something like this: dd ...


25

As far as the end result is concerned, they will do the same. The difference is in how dd would process data. And actually, both your examples are quite extreme in that regard: the bs parameter tells dd how much data it should buffer into the memory before outputting it. So, essentially, the first command would try to read 2GB in two chunks of 1GB, and the ...


23

It depends on whether the disk image is a full disk image, or just a partition. Washing the partition(s) If the disk is in good working condition, you will get better compression if you wash the empty space on the disk with zeros. If the disk is failing, skip this step. If you're imaging an entire disk then you will want to wash each of the partitions on ...


21

When you use dd on /dev/sdb instead of /dev/sdb1 or /dev/sdb2, you copy all the partitions from the said drive into one file. You must mount each partition separately. To mount a partition from a file, you must first find out where in the file that partition resides. Using your output from file -s sdb.img we find the startsectors for each partition: ...


19

You're observing a combination of the peculiar behavior of dd with the peculiar behavior of Linux's /dev/random. Both, by the way, are rarely the right tool for the job. Linux's /dev/random returns data sparingly. It is based on the assumption that the entropy in the pseudorandom number generator is extinguished at a very fast rate. Since gathering new ...


18

For dd, you can send a signal. For other commands that are reading or writing to a file, you can watch their position in the file with lsof. lsof -o -p1234 # where 1234 is the process ID of the command lsof -o /path/to/file If you plan in advance, pipe the data through pv.


18

If you use the conv=notrunc argument, you can replace just the first however many bytes. e.g. dd conv=notrunc if=small.img of=large.img root@debian:~/ddtest# dd if=/dev/zero of=file1.img bs=1M count=10 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 1.14556 s, 9.2 MB/s root@debian:~/ddtest# dd if=/dev/urandom of=file2.img bs=1M count=1 1+0 ...


16

Though the "best answer" was given, this site states otherwise: Actually, it stands for Copy and Convert' and was renamed toddonly becausecc' was reserved for the C compiler! This is the authentic information I got from the man pages of our Unix-V7 on our university PDP 11.


16

The result will be the same but in the first case dd will write two 1GB blocks while in the second one 2GB block. The difference is that dd keeps the copied block in memory. You will need 1GB of RAM in the first case and 2GB in the second. In my opinion there is no need to use such large blocks. You can do a couple of tests but in my case I achieve a ...


16

If you check Reading from /dev/urandom gives EOF after 33554431 bytes and follow the discussion, it points to another bug report where Ted Tso states... ...that commit 79a8468747c5 causes reads larger than 32MB results in a only 32MB to be returned by the read(2) system call. That is, it results in a short read. POSIX always allows for a short ...


15

The dd command includes LOTS of options that cat is not able to accommodate. Perhaps in your usage cases cat is a workable substitute, but it is not a dd replacement. One example would be using dd to copy part of something but not the whole thing. Perhaps you want to rip out some of the bits from the middle of an iso image or the partition table from a hard ...


14

You can create a sparse file like this with dd: dd of=file bs=1 seek=2G count=0 $ du file 0 disk $ du --apparent-size file 2097152 disk


14

If your intent is to backup a remote computer's HDD A via SSH to a single file that's on your local computer's HDD, you could do one of the following. Examples run from remote computer $ dd if=/dev/sda | gzip -1 - | ssh user@local dd of=image.gz run from local computer $ ssh user@remote "dd if=/dev/sda | gzip -1 -" | dd of=image.gz Live example $ ...


13

Wikipedia (dd) asserts it was named after IBM JCL command DD which stands for Data Description. I always thought it would mean data duplicate, though.


13

No one has yet mentioned that you can use dd to create sparse files, though truncate can also be used for the same purpose. dd if=/dev/zero of=sparse-file bs=1 count=1 seek=10GB This is almost instant and creates an arbitrary large file that can be used as a loopback file for instance: loop=`losetup --show -f sparse-file` mkfs.ext4 $loop mkdir myloop ...


13

You want dd_rescue. dd_rescue -a -b 8M /dev/sda1 /mount/external/backup/sda1.raw


13

Well, assuming you have stat and bash, you can get the file size with: stat -c %s your_file If you want to extract the last $amount bytes for that file with dd, you could: dd if=your_file of=extracted_part \ bs=1 count=$amount \ skip=$(( $(stat -c %s your_file) - $amount )) But the saner approach would be to use tail: tail -c $(( 1024*1024 )) ...


12

dd dates from back when it was needed to translate old IBM mainframe tapes, and the block size had to match the one used to write the tape or data blocks would be skipped or truncated. (9-track tapes were finicky. Be glad they're long dead.) These days, the block size should be a multiple of the device sector size (usually 4KB, but on very recent disks ...


12

/dev/null is a special file, of type character device. The driver for that character device ignores whatever you try to write to the device and writes are always successful. If a write to /dev/null fails, it means that you've somehow managed to remove the proper /dev/null and replace it by a regular file. You might have accidentally removed /dev/null; then ...


11

"dump data". JCL is irrelevant.


11

From the dd(1) man page: status=noxfer suppress transfer statistics thus: dd if=boot1h of="/dev/r$temp1" status=noxfer This still outputs the 0+1 records in 0+1 records out garbage when dd exits, so redirecting to a data sink really is your only option.


11

The difference between cat <input >output and dd if=input of=output is in the block size they will use. The default for dd if no bs parameter is given is 512 bytes (the size of a disk sector), whereas cat will use whatever blocksize it is coded to work with, which based on a quick inspection of the source code is either 32 KB or the st_blksize value ...


11

From man 4 random on a RHEL 5 box: When read, the /dev/random device will only return random bytes within the estimated number of bits of noise in the entropy pool. I get files of size 213 bytes on that machine. Back to man 4 random: When read, /dev/urandom device will return as many bytes as are requested. I get 2048 bytes from ...


11

This is slow because of the small block size. Using a recent GNU dd, the simplest way is to use the skip_bytes and count_bytes options: in_file=1tb start=12345678901 end=19876543212 block_size=4096 copy_size=$(( $end - $start )) dd if="$in_file" iflag=skip_bytes,count_bytes,fullblock bs="$block_size" \ skip="$start" count="$copy_size" Update ...


10

A more general way is to use iotop that displays the current amount of disk reading / writing per program. EDIT: iotop -o show only programs which perform currently I/O operations (thanks Jason C for this comment).


10

No, dd can be not sufficient. As examples: If you have a multi-track cd-rom with data and audio track mixed, using dd you will copy only the first data session. Many old video-games cd (on Play-Station 1) use as copy-protection some fake session on the cd. You have to replicate them to obtain a working cd. I successfully used cdparanoia to backup quite ...



Only top voted, non community-wiki answers of a minimum length are eligible