Tag Info

Hot answers tagged

87

In appearance, dd is a tool from an IBM operating system that's retained its foreign appearance (its parameter passing), which performs some very rarely-used functions (such as EBCDIC to ASCII conversions or endianness reversal… not a common need nowadays). I used to think that dd was faster for copying large blocks of data on the same disk (due to more ...


71

You can send dd a certain signal using the kill command to make it output its current status. The signal is INFO on BSD systems (including OSX) and USR1 on Linux. In your case: kill -INFO $PID You can find the process id ($PID above) with the ps command; or see pgrep and pkill alternatives on mac os x for more convenient methods. More simply, as AntoineG ...


25

You can switch bs and skip options: dd bs=1131 skip=1 if=filtered.dump of=trimmed.dump This way the operation can benefit from a greater block. Otherwise, you could try with tail (although it's not safe to use it with binary files): tail -c +1132 filtered.dump >trimmed.dump Finally, you may use 3 dd instances to write something like this: dd ...


25

As far as the end result is concerned, they will do the same. The difference is in how dd would process data. And actually, both your examples are quite extreme in that regard: the bs parameter tells dd how much data it should buffer into the memory before outputting it. So, essentially, the first command would try to read 2GB in two chunks of 1GB, and the ...


23

There's but one way to determine the optimal block size, and that's a benchmark. I've just made a quick benchmark. The test machine is a PC running Debian GNU/Linux, with kernel 2.6.32 and coreutils 8.5. Both filesystems involved are ext3 on LVM volumes on a hard disk partition. The source file is 2GB (2040000kB to be precise). Caching and buffering are ...


21

When you use dd on /dev/sdb instead of /dev/sdb1 or /dev/sdb2, you copy all the partitions from the said drive into one file. You must mount each partition separately. To mount a partition from a file, you must first find out where in the file that partition resides. Using your output from file -s sdb.img we find the startsectors for each partition: ...


19

You're observing a combination of the peculiar behavior of dd with the peculiar behavior of Linux's /dev/random. Both, by the way, are rarely the right tool for the job. Linux's /dev/random returns data sparingly. It is based on the assumption that the entropy in the pseudorandom number generator is extinguished at a very fast rate. Since gathering new ...


16

It depends on whether the disk image is a full disk image, or just a partition. Washing the partition(s) If the disk is in good working condition, you will get better compression if you wash the empty space on the disk with zeros. If the disk is failing, skip this step. If you're imaging an entire disk then you will want to wash each of the partitions on ...


15

For dd, you can send a signal. For other commands that are reading or writing to a file, you can watch their position in the file with lsof. lsof -o -p1234 # where 1234 is the process ID of the command lsof -o /path/to/file If you plan in advance, pipe the data through pv.


15

The result will be the same but in the first case dd will write two 1GB blocks while in the second one 2GB block. The difference is that dd keeps the copied block in memory. You will need 1GB of RAM in the first case and 2GB in the second. In my opinion there is no need to use such large blocks. You can do a couple of tests but in my case I achieve a ...


14

Under OS X (didn't try on Linux), you can simply type Ctrl+T in the terminal running dd. It will print the same output as kill -INFO $PID, plus the CPU usage: load: 1.40 cmd: dd 34536 uninterruptible 3.49u 64.58s 5020305+0 records in 5020304+0 records out 2570395648 bytes transferred in 4284.349974 secs (599950 bytes/sec) I found out about it reading ...


14

The dd command includes LOTS of options that cat is not able to accommodate. Perhaps in your usage cases cat is a workable substitute, but it is not a dd replacement. One example would be using dd to copy part of something but not the whole thing. Perhaps you want to rip out some of the bits from the middle of an iso image or the partition table from a hard ...


14

You can create a sparse file like this with dd: dd of=file bs=1 seek=2G count=0 $ du file 0 disk $ du --apparent-size file 2097152 disk


14

If you use the conv=notrunc argument, you can replace just the first however many bytes. e.g. dd conv=notrunc if=small.img of=large.img root@debian:~/ddtest# dd if=/dev/zero of=file1.img bs=1M count=10 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 1.14556 s, 9.2 MB/s root@debian:~/ddtest# dd if=/dev/urandom of=file2.img bs=1M count=1 1+0 ...


13

Wikipedia (dd) asserts it was named after IBM JCL command DD which stands for Data Description. I always thought it would mean data duplicate, though.


12

No one has yet mentioned that you can use dd to create sparse files, though truncate can also be used for the same purpose. dd if=/dev/zero of=sparse-file bs=1 count=1 seek=10GB This is almost instant and creates an arbitrary large file that can be used as a loopback file for instance: loop=`losetup --show -f sparse-file` mkfs.ext4 $loop mkdir myloop ...


12

You want dd_rescue. dd_rescue -a -b 8M /dev/sda1 /mount/external/backup/sda1.raw


12

Well, assuming you have stat and bash, you can get the file size with: stat -c %s your_file If you want to extract the last $amount bytes for that file with dd, you could: dd if=your_file of=extracted_part \ bs=1 count=$amount \ skip=$(( $(stat -c %s your_file) - $amount )) But the saner approach would be to use tail: tail -c $(( 1024*1024 )) ...


12

/dev/null is a special file, of type character device. The driver for that character device ignores whatever you try to write to the device and writes are always successful. If a write to /dev/null fails, it means that you've somehow managed to remove the proper /dev/null and replace it by a regular file. You might have accidentally removed /dev/null; then ...


11

Though the best answer was given, this site states otherwise: Actually, it stands for Copy and Convert' and was renamed toddonly becausecc' was reserved for the C compiler! This is the authentic information I got from the man pages of our Unix-V7 on our university PDP 11.


11

This is slow because of the small block size. Using a recent GNU dd, the simplest way is to use the skip_bytes and count_bytes options: in_file=1tb start=12345678901 end=19876543212 block_size=4096 copy_size=$(( $end - $start )) dd if="$in_file" iflag=skip_bytes,count_bytes,fullblock bs="$block_size" \ skip="$start" count="$copy_size" Update ...


10

"dump data". JCL is irrelevant.


10

From the dd(1) man page: status=noxfer suppress transfer statistics thus: dd if=boot1h of="/dev/r$temp1" status=noxfer This still outputs the 0+1 records in 0+1 records out garbage when dd exits, so redirecting to a data sink really is your only option.


10

From man 4 random on a RHEL 5 box: When read, the /dev/random device will only return random bytes within the estimated number of bits of noise in the entropy pool. I get files of size 213 bytes on that machine. Back to man 4 random: When read, /dev/urandom device will return as many bytes as are requested. I get 2048 bytes from ...


9

dd dates from back when it was needed to translate old IBM mainframe tapes, and the block size had to match the one used to write the tape or data blocks would be skipped or truncated. (9-track tapes were finicky. Be glad they're long dead.) These days, the block size should be a multiple of the device sector size (usually 4KB, but on very recent disks ...


9

No, dd can be not sufficient. As examples: If you have a multi-track cd-rom with data and audio track mixed, using dd you will copy only the first data session. Many old video-games cd (on Play-Station 1) use as copy-protection some fake session on the cd. You have to replicate them to obtain a working cd. I successfully used cdparanoia to backup quite ...


9

The first one will take up 1G of memory, fill it with data from the input file and output it, two times. The second one will take up 2G of memory, fill it with data from the input file and output it.


9

No question, rsync will be faster. dd will have to read and write the whole 1.5TB and it will hit every bad block, triggering multiple read retries which will further slow an already long process. rsync will only have to read blocks that matter, and since it is unlikely that every bad block occurs in existing files or directories, rsync will encounter ...


9

That's not exactly how to go about it. What you'll want to do is mount the disk image as a loopback device: mount -o ro,loop -t ntfs disk.image /mnt/test The contents of the image will be available in /mnt/test (but you can choose to mount it anywhere you like). You can copy individual files (or entire directory trees) from it. Use umount /mnt/test1 to ...


9

I think you can use UDEV to do what you want. Creating a rules file such as/etc/udev/rules.d/99-thumbdrives.rules you'd simply add a rule that will allow either a Unix group or user access to arbitrary USB thumb drives. KERNEL=="sd*", SUBSYSTEM=="block", ENV{DEVTYPE}=="disk", OWNER="<user>", GROUP="<group>", MODE="0660" Would create the device ...



Only top voted, non community-wiki answers of a minimum length are eligible