Hot answers tagged dd
59
In appearance, dd is a tool from an IBM operating system that's retained its foreign appearance (its parameter passing), which performs some very rarely-used functions (such as EBCDIC to ASCII conversions or endianness reversal… not a common need nowadays).
I used to think that dd was faster for copying large blocks of data on the same disk (due to more ...
32
You can send dd a certain signal using the kill command to make it output its current status. The signal is INFO on BSD systems (including OSX) and USR1 on Linux. In your case:
kill -INFO $PID
You can find the process id ($PID above) with the ps command; or see pgrep and pkill alternatives on mac os x for more convenient methods.
As an example on Linux, ...
24
As far as the end result is concerned, they will do the same. The difference is in how dd would process data. And actually, both your examples are quite extreme in that regard: the bs parameter tells dd how much data it should buffer into the memory before outputting it.
So, essentially, the first command would try to read 2GB in two chunks of 1GB, and the ...
17
When you use dd on /dev/sdb instead of /dev/sdb1 or /dev/sdb2, you copy all the partitions from the said drive into one file.
You must mount each partition separately.
To mount a partition from a file, you must first find out where in the file that partition resides.
Using your output from file -s sdb.img we find the startsectors for each partition:
...
16
You can switch bs and skip options:
dd bs=1131 skip=1 if=filtered.dump of=trimmed.dump
This way the operation can benefit from a greater block.
Otherwise, you could try with tail (although it's not safe to use it with binary files):
tail -c +1132 filtered.dump >trimmed.dump
Finally, you may use 3 dd instances to write something like this:
dd ...
15
There's but one way to determine the optimal block size, and that's a benchmark. I've just made a quick benchmark. The test machine is a PC running Debian GNU/Linux, with kernel 2.6.32 and coreutils 8.5. Both filesystems involved are ext3 on LVM volumes on a hard disk partition. The source file is 2GB (2040000kB to be precise). Caching and buffering are ...
14
The result will be the same but in the first case dd will write two 1GB blocks while in the second one 2GB block. The difference is that dd keeps the copied block in memory. You will need 1GB of RAM in the first case and 2GB in the second.
In my opinion there is no need to use such large blocks. You can do a couple of tests but in my case I achieve a ...
14
You're observing a combination of the peculiar behavior of dd with the peculiar behavior of Linux's /dev/random. Both, by the way, are rarely the right tool for the job.
Linux's /dev/random returns data sparingly. It is based on the assumption that the entropy in the pseudorandom number generator is extinguished at a very fast rate. Since gathering new ...
11
/dev/null is a special file, of type character device. The driver for that character device ignores whatever you try to write to the device and writes are always successful. If a write to /dev/null fails, it means that you've somehow managed to remove the proper /dev/null and replace it by a regular file. You might have accidentally removed /dev/null; then ...
10
The dd command includes LOTS of options that cat is not able to accommodate. Perhaps in your usage cases cat is a workable substitute, but it is not a dd replacement.
One example would be using dd to copy part of something but not the whole thing. Perhaps you want to rip out some of the bits from the middle of an iso image or the partition table from a hard ...
9
Though the best answer was given, this site states otherwise:
Actually, it stands for Copy and
Convert' and was renamed toddonly
becausecc' was reserved for the C
compiler! This is the authentic
information I got from the man pages
of our Unix-V7 on our university PDP
11.
9
From the dd(1) man page:
status=noxfer
suppress transfer statistics
thus:
dd if=boot1h of="/dev/r$temp1" status=noxfer
This still outputs the
0+1 records in
0+1 records out
garbage when dd exits, so redirecting to a data sink really is your only option.
9
No, dd can be not sufficient. As examples:
If you have a multi-track cd-rom with data and audio track mixed, using dd you will copy only the first data session.
Many old video-games cd (on Play-Station 1) use as copy-protection some fake session on the cd. You have to replicate them to obtain a working cd.
I successfully used cdparanoia to backup quite ...
9
That's not exactly how to go about it.
What you'll want to do is mount the disk image as a loopback device:
mount -o ro,loop -t ntfs disk.image /mnt/test
The contents of the image will be available in /mnt/test (but you can choose to mount it anywhere you like). You can copy individual files (or entire directory trees) from it. Use umount /mnt/test1 to ...
8
For dd, you can send a signal. For other commands that are reading or writing to a file, you can watch their position in the file with lsof.
lsof -o -p1234 # where 1234 is the process ID of the command
lsof -o /path/to/file
If you plan in advance, pipe the data through pv.
8
Well, assuming you have stat and bash, you can get the file size with:
stat -c %s your_file
If you want to extract the last $amount bytes for that file with dd, you could:
dd if=your_file of=extracted_part \
bs=1 count=$amount \
skip=$(( $(stat -c %s your_file) - $amount ))
But the saner approach would be to use tail:
tail -c $(( 1024*1024 )) ...
8
From man 4 random on a RHEL 5 box:
When read, the /dev/random device will only return random bytes
within the estimated number of bits of noise in the entropy pool.
I get files of size 213 bytes on that machine. Back to man 4 random:
When read, /dev/urandom device will return as many bytes as
are requested.
I get 2048 bytes from ...
8
No question, rsync will be faster. dd will have to read and write the whole 1.5TB and it will hit every bad block, triggering multiple read retries which will further slow an already long process. rsync will only have to read blocks that matter, and since it is unlikely that every bad block occurs in existing files or directories, rsync will encounter ...
8
Based on what you have described you should do something like this:
dd if=/dev/urandom of=testfile bs=1M count=699
mkisofs -o test_cdrom.iso testfile
Once done you can read and write to and from the optical media to your hearts content.
One thing that I would suggest is that instead of pretesting the optical media and then attempting to write the actual ...
8
Generally speaking, just use dd, but as you mention the use of KVM virtualization, you might consider using qemu-img:
qemu-img create -f raw disk 2G
It does the same as the dd command in the answer of Chris Down, effectively.
Regardless of what command you use, for use in virtualization, I would strongly suggest to use fallocate to pre-allocate blocks in ...
8
To overwrite the start of the destination file without truncating it, give the notrunc conversion directive:
$ dd if=out/one.img of=out/go.img conv=notrunc
If you wanted the source file's data appended to the destination, you can do that with the seek directive:
$ dd if=out/one.img of=out/go.img bs=1k seek=9
This tells dd that the block size is 1 kiB, ...
7
No one has yet mentioned that you can use dd to create sparse files, though truncate can also be used for the same purpose.
dd if=/dev/zero of=sparse-file bs=1 count=1 seek=10GB
This is almost instant and creates an arbitrary large file that can be used as a loopback file for instance:
loop=`losetup --show -f sparse-file`
mkfs.ext4 $loop
mkdir myloop
...
6
You can use a sub-shell and two dd calls like this:
$ ( dd bs=1131 count=1 of=dev_null && dd bs=4K of=out.mp3 ) < 100827_MR029_LobbyControl.mp3
1+0 records in
1+0 records out
1131 bytes (1.1 kB) copied, 7.9691e-05 s, 14.2 MB/s
22433+1 records in
22433+1 records out
91886130 bytes (92 MB) copied, 0.329823 s, 279 MB/s
$ ls -l *
-rw------- 1 max ...
6
dd dates from back when it was needed to translate old IBM mainframe tapes, and the block size had to match the one used to write the tape or data blocks would be skipped or truncated. (9-track tapes were finicky. Be glad they're long dead.) These days, the block size should be a multiple of the device sector size (usually 4KB, but on very recent disks ...
6
Override specific segments of a hard-drive with something is a common example. For example you might want to delete your MBR using this command:
dd if=/dev/zero of=/dev/sda bs=446 count=1
Also you can create empty files with it (say for loop disk images):
dd if=/dev/zero of=10mb.file bs=1024k count=10
6
I think that the simplest answer is that dd, dd_rescue and ddrescue are not designed to defeat copy protection schemes. They make no assumptions about the format of the data and try to maintain the integrity of the whole of the original on disk data.
In the case of dd I suspect that it is terminating due to an intentional read error on the disk that is part ...
6
The problem is your designated output from dd goes to STDERR and not STDOUT so you have to redirect STDERR as well and not only STDOUT.
For bash and zsh you can use |& instead of | which will also redirect STDERR to STDIN of the second command, e.g:
dd if=/dev/urandom of=/dev/null bs=1K count=10000 |& awk '/copied/ {print $8 " " $9}'
The more ...
6
You write to the disk (/dev/sdb), not to the file system you created (/dev/sdb1).
Since an ISO image already contains a file system (which you don't want) you can simply mount the ISO (with option -o loop) and copy the data to /dev/sdb1. That way the file system is preserved.
6
The right thing to do is something like mdadm --add /dev/md0 /dev/sdb1. Use the correct array in place of md0 and the correct partition in place of sdb1.
The key thing is the array is running. Its completely unambiguous which data to copy: the data that is currently running. If you have bitmaps enabled, the resync will be fairly fast as it'll only copy what ...
Only top voted, non community-wiki answers of a minimum length are eligible
