Tag Info

Hot answers tagged

76

In appearance, dd is a tool from an IBM operating system that's retained its foreign appearance (its parameter passing), which performs some very rarely-used functions (such as EBCDIC to ASCII conversions or endianness reversal… not a common need nowadays). I used to think that dd was faster for copying large blocks of data on the same disk (due to more ...


58

You can send dd a certain signal using the kill command to make it output its current status. The signal is INFO on BSD systems (including OSX) and USR1 on Linux. In your case: kill -INFO $PID You can find the process id ($PID above) with the ps command; or see pgrep and pkill alternatives on mac os x for more convenient methods. More simply, as AntoineG ...


25

As far as the end result is concerned, they will do the same. The difference is in how dd would process data. And actually, both your examples are quite extreme in that regard: the bs parameter tells dd how much data it should buffer into the memory before outputting it. So, essentially, the first command would try to read 2GB in two chunks of 1GB, and the ...


19

You can switch bs and skip options: dd bs=1131 skip=1 if=filtered.dump of=trimmed.dump This way the operation can benefit from a greater block. Otherwise, you could try with tail (although it's not safe to use it with binary files): tail -c +1132 filtered.dump >trimmed.dump Finally, you may use 3 dd instances to write something like this: dd ...


19

There's but one way to determine the optimal block size, and that's a benchmark. I've just made a quick benchmark. The test machine is a PC running Debian GNU/Linux, with kernel 2.6.32 and coreutils 8.5. Both filesystems involved are ext3 on LVM volumes on a hard disk partition. The source file is 2GB (2040000kB to be precise). Caching and buffering are ...


18

When you use dd on /dev/sdb instead of /dev/sdb1 or /dev/sdb2, you copy all the partitions from the said drive into one file. You must mount each partition separately. To mount a partition from a file, you must first find out where in the file that partition resides. Using your output from file -s sdb.img we find the startsectors for each partition: ...


18

You're observing a combination of the peculiar behavior of dd with the peculiar behavior of Linux's /dev/random. Both, by the way, are rarely the right tool for the job. Linux's /dev/random returns data sparingly. It is based on the assumption that the entropy in the pseudorandom number generator is extinguished at a very fast rate. Since gathering new ...


15

The result will be the same but in the first case dd will write two 1GB blocks while in the second one 2GB block. The difference is that dd keeps the copied block in memory. You will need 1GB of RAM in the first case and 2GB in the second. In my opinion there is no need to use such large blocks. You can do a couple of tests but in my case I achieve a ...


13

The dd command includes LOTS of options that cat is not able to accommodate. Perhaps in your usage cases cat is a workable substitute, but it is not a dd replacement. One example would be using dd to copy part of something but not the whole thing. Perhaps you want to rip out some of the bits from the middle of an iso image or the partition table from a hard ...


12

For dd, you can send a signal. For other commands that are reading or writing to a file, you can watch their position in the file with lsof. lsof -o -p1234 # where 1234 is the process ID of the command lsof -o /path/to/file If you plan in advance, pipe the data through pv.


12

/dev/null is a special file, of type character device. The driver for that character device ignores whatever you try to write to the device and writes are always successful. If a write to /dev/null fails, it means that you've somehow managed to remove the proper /dev/null and replace it by a regular file. You might have accidentally removed /dev/null; then ...


11

No one has yet mentioned that you can use dd to create sparse files, though truncate can also be used for the same purpose. dd if=/dev/zero of=sparse-file bs=1 count=1 seek=10GB This is almost instant and creates an arbitrary large file that can be used as a loopback file for instance: loop=`losetup --show -f sparse-file` mkfs.ext4 $loop mkdir myloop ...


11

Under OS X (didn't try on Linux), you can simply type Ctrl+T in the terminal running dd. It will print the same output as kill -INFO $PID, plus the CPU usage: load: 1.40 cmd: dd 34536 uninterruptible 3.49u 64.58s 5020305+0 records in 5020304+0 records out 2570395648 bytes transferred in 4284.349974 secs (599950 bytes/sec) I found out about it reading ...


10

Though the best answer was given, this site states otherwise: Actually, it stands for Copy and Convert' and was renamed toddonly becausecc' was reserved for the C compiler! This is the authentic information I got from the man pages of our Unix-V7 on our university PDP 11.


10

From man 4 random on a RHEL 5 box: When read, the /dev/random device will only return random bytes within the estimated number of bits of noise in the entropy pool. I get files of size 213 bytes on that machine. Back to man 4 random: When read, /dev/urandom device will return as many bytes as are requested. I get 2048 bytes from ...


9

From the dd(1) man page: status=noxfer suppress transfer statistics thus: dd if=boot1h of="/dev/r$temp1" status=noxfer This still outputs the 0+1 records in 0+1 records out garbage when dd exits, so redirecting to a data sink really is your only option.


9

No, dd can be not sufficient. As examples: If you have a multi-track cd-rom with data and audio track mixed, using dd you will copy only the first data session. Many old video-games cd (on Play-Station 1) use as copy-protection some fake session on the cd. You have to replicate them to obtain a working cd. I successfully used cdparanoia to backup quite ...


9

Well, assuming you have stat and bash, you can get the file size with: stat -c %s your_file If you want to extract the last $amount bytes for that file with dd, you could: dd if=your_file of=extracted_part \ bs=1 count=$amount \ skip=$(( $(stat -c %s your_file) - $amount )) But the saner approach would be to use tail: tail -c $(( 1024*1024 )) ...


9

No question, rsync will be faster. dd will have to read and write the whole 1.5TB and it will hit every bad block, triggering multiple read retries which will further slow an already long process. rsync will only have to read blocks that matter, and since it is unlikely that every bad block occurs in existing files or directories, rsync will encounter ...


9

That's not exactly how to go about it. What you'll want to do is mount the disk image as a loopback device: mount -o ro,loop -t ntfs disk.image /mnt/test The contents of the image will be available in /mnt/test (but you can choose to mount it anywhere you like). You can copy individual files (or entire directory trees) from it. Use umount /mnt/test1 to ...


9

This is slow because of the small block size. Using a recent GNU dd, the simplest way is to use the skip_bytes and count_bytes options: in_file=1tb start=12345678901 end=19876543212 block_size=4096 copy_size=$(( $end - $start )) dd if="$in_file" iflag=skip_bytes,count_bytes,fullblock bs="$block_size" \ skip="$start" count="$copy_size" Update ...


9

I think you can use UDEV to do what you want. Creating a rules file such as/etc/udev/rules.d/99-thumbdrives.rules you'd simply add a rule that will allow either a Unix group or user access to arbitrary USB thumb drives. KERNEL=="sd*", SUBSYSTEM=="block", ENV{DEVTYPE}=="disk", OWNER="<user>", GROUP="<group>", MODE="0660" Would create the device ...


8

Override specific segments of a hard-drive with something is a common example. For example you might want to delete your MBR using this command: dd if=/dev/zero of=/dev/sda bs=446 count=1 Also you can create empty files with it (say for loop disk images): dd if=/dev/zero of=10mb.file bs=1024k count=10


8

It depends on whether the disk image is a full disk image, or just a partition. Washing the partition(s) If the disk is in good working condition, you will get better compression if you wash the empty space on the disk with zeros. If the disk is failing, skip this step. If you're imaging an entire disk then you will want to wash each of the partitions on ...


8

Based on what you have described you should do something like this: dd if=/dev/urandom of=testfile bs=1M count=699 mkisofs -o test_cdrom.iso testfile Once done you can read and write to and from the optical media to your hearts content. One thing that I would suggest is that instead of pretesting the optical media and then attempting to write the actual ...


8

Generally speaking, just use dd, but as you mention the use of KVM virtualization, you might consider using qemu-img: qemu-img create -f raw disk 2G It does the same as the dd command in the answer of Chris Down, effectively. Regardless of what command you use, for use in virtualization, I would strongly suggest to use fallocate to pre-allocate blocks in ...


8

To overwrite the start of the destination file without truncating it, give the notrunc conversion directive: $ dd if=out/one.img of=out/go.img conv=notrunc If you wanted the source file's data appended to the destination, you can do that with the seek directive: $ dd if=out/one.img of=out/go.img bs=1k seek=9 This tells dd that the block size is 1 kiB, ...


8

/dev/sgxx is a SCSI-generic device, which allows sending and receiving of raw SCSI commands. When you write to the device, you are expected to start the write with a SCSI header, which defines the operation you wish to do. Writing random data to an sg device is really a bad idea. You'll be sending random SCSI commands, which might not even exist (hence ...


7

dd is very useful for backing up the boot sector of a hard drive or other storage device (dd if=/dev/sda of=boot_sector.bin bs=512 count=1 ) and then later rewriting it (dd if=boot_sector.bin of=/dev/sda). It is similarly useful for backing up the headers of encrypted volumes. cat might be able to be twisted into doing that but I wouldn't trust it on the ...


7

You can use ddrescue or dd_rescue or myrescue to clone the failing disk, without aborting on any unreadable sector. (Myrescue is less configurable but has a better default strategy as it tries to skip over unreadable regions.) This will copy everything including blank space and won't let you set priorities. However, such a low-level approach has an advantage ...



Only top voted, non community-wiki answers of a minimum length are eligible