The Stack Overflow podcast is back! Listen to an interview with our new CEO.

Questions tagged [dd]

dd is a traditional utility for binary data copies

Filter by
Sorted by
Tagged with
6
votes
1answer
86 views

Has 'sync' been proved to be necessary when writing to a block device with 'dd'?

Every time I've written a file to an empty raw block device, e.g. # dd if=image.iso of=/dev/sdb status=progress I've never used any type of sync (i.e. sync; conv=fsync; conv=fdatasync; oflag=sync; ...
5
votes
2answers
894 views

Resizing a virtual hard drive

I found a very nice tutorial on how to create virtual hard disks, and I am considering using these for my work in order to store reliably and portably large datasets with associated processing results....
5
votes
2answers
1k views

How to prevent dd's progress from being meaningless on Linux?

When running dd command to copy an ISO on Linux, I get a single progress print that stays open for a long time (many minutes). Then another at the end. The problem seems to be that a very large cache ...
5
votes
2answers
649 views

Errors on cloning disk with cat

I wish to clone a large disk (a 500GB SSD, for what it's worth), and I am leaning toward using cat, as suggested by Gilles here. But what gave me pause is that I do not really know what cat does upon ...
5
votes
2answers
2k views

If I dd a CDROM, then I always get the exact copy of the CDROM?

If I: dd if=/dev/cdrom of=cdrom.iso then I will always get the exact same, bit-by-bitly same image that is the same as the original CDROM? Or are there any methods that prevents copying all the ...
5
votes
2answers
1k views

How large is the img that dd creates?

This is the first time I use the dd command. I execute: dd if=/dev/sdb2 of=/mnt/sdc1/Hdd1.img bs=512 conv=noerror,sync Where the sdb is the destroyed hdd (size: 500 GB). I copying the partition sdb2 ...
5
votes
1answer
9k views

Why is dd so slow with a bs of 100M [closed]

I just tried to overwrite a fast ssd using dd. Using the ubuntu boot image I typed in: dd if=/dev/zero of=/dev/sda bs=100M error writing '/dev/sda': No space left on device blah blah 256 GB copied, ...
5
votes
2answers
537 views

dd writes to CF with delay

I have an old 64MB CF card which I use in my laptop(kernel 2.6.38) with CardBus adapter. If I write a 64MB image to this CF card, then the write speed is more than 200MB/s: T42 ~ # fdisk -lu Disk /...
5
votes
1answer
2k views

Partially overwrote /dev/sda, can I recover anything?

I was messing around with my new hard drive, which only contains some personal files, and to check its write speed, I typed: sudo dd if=/dev/zero of=/dev/sda, /dev/sda/ being the new 2 TB connected to ...
5
votes
1answer
187 views

System lags when doing large R/W operations on external disks

I am having some issues with system-wide latency/lagging when doing large disk imaging operations on an Ubuntu 18.04 system. Here's the system specs: Processor: Intel Core i7 (never near capacity on ...
5
votes
6answers
9k views

Check command success in bash

I'm currently writing a small script to backup bulks of floppy disks and format them afterward for later use. I use dd to copy the disk's image and cp to copy all the files on the disk. Here are the ...
5
votes
3answers
1k views

how do I do this: dd if=coolstuff.iso of=(ssh kev@mypi )

the card reader on my laptop doesn't want to work. can I use dd (or some other tool) to write an image to a networked disk. I am trying to replace one raspberry pi distro with another. The SD card ...
5
votes
1answer
3k views

How do I create a USB image with a partition table?

I want to create an image of some directory tree that is writable directly to a USB drive, just like the images of many Linux distributions. For example, with openSUSE you can download an ISO image ...
5
votes
1answer
308 views

How to fill a device with zeros, without overwriting the bytes that are already zeros?

I have a USB flash drive usb 3, the reading speed is much more than the writing speed. Let's say that 99% of the flash memory is already full with zeros, and I would like to fill it with zeros until ...
5
votes
1answer
812 views

Can DD be used to add a buffer to a pipe?

I'm reading and writing audio from a named pipe. I'm running into an issue where my read process lags, causing the pipe's buffer to fill up and drop frames. I'd like to insert a buffer of arbitrarily ...
5
votes
2answers
910 views

Why is dd reading from and writing to the same disk?

I've been using dd in Linux to overwrite an external USB hard drive. When I use the default block size of 512 bytes, with this command: # dd if=/dev/zero of=/dev/sdb throughout the whole operation, ...
5
votes
2answers
775 views

Creating a large file with random content: short-cut by copying?

Assume I want to create a large encrypted drive stored in a file using cryptsetup, the first step is to create a random file, assume this should be of size 3T: dd if=/dev/urandom of=$FILE bs=1G count=...
5
votes
3answers
1k views

Can I overwrite LUKS partition with it's decrypted content?

I have HDD with my data (partial backup of most important data available on other HDD), currently it's formatted as Ext4 over LVM over LUKS. I want to remove LUKS layer, but reformatting and data ...
5
votes
3answers
2k views

from a sparse file to a block device over the network

If I have a sparse file representing a block device, then how can I efficiently restore the file over a network? Ideally, I'd use rsync -S host:file /dev/sdb, but rsync won't write to block devices. ...
5
votes
2answers
3k views

What is actually the speed reported by dd?

I always thought that when doing: dd if=/dev/sdx of=/dev/sdy the report which is displayed once the command finishes, such as: 79304319+0 records in 79304319+0 records out 40603811328 bytes (41 GB) ...
5
votes
3answers
1k views

Help with unzipping a .dd.gz, possibly corrupted

I recieved this file that I was told was a copy of a hard drive (or disk). The file is "sheeva-mem.dd.gz". I tried to unzip the file in a linux Fedora machine. This yielded "unexpected end of file". ...
5
votes
1answer
2k views

Why can't I simply dd an installation iso to a bootable partition on an USB flash drive?

I have a copy of the Slackware 64 bit dvd installation iso. I've burned this image to a dvd and used it to properly install the distro. I wanted to try also having this image available on a usb flash ...
5
votes
2answers
5k views

why do people use “notrunc” when making an ISO from a DVD?

There are many articles on the web that describe a command-line to make an ISO image from a DVD-Video disc like this: dd if=/dev/sr0 of=mydvd.iso conv=noerror,sync,notrunc I'd like to uderstand why ...
5
votes
2answers
2k views

Red Hat cannot boot after clone with dd

I used dd to clone a x4100 server with Linux Red Hat. The hardware is identical for old and new servers. Disks are sda, sdb and sdc(sdc1 and sdc2). But newly restored server gives following errors and ...
5
votes
0answers
216 views

Why does ddrescue not utilize full bandwidth?

I am running ddrescue from disk A to disk B. I had expected the slower of the disks would be running at full speed all the time and the other be running enough to keep up. Imagine my surprise when ...
5
votes
2answers
318 views

confused about dd write speed and VFS page cache

After reading some articles on the Linux VFS page cache and the tunable parameters like dirty_ratio i was under the impression that page cache would operate as both read and write caching layer. But ...
5
votes
1answer
562 views

Partitions same size but different block [closed]

I tried to backup my partition using dd on another disk and I created a partition of the same size. When I finished the copy I had problems mounting the partition since I got bad geometry: block ...
4
votes
4answers
2k views

How to fill a file with a stream from /dev/urandom with a specified number of lines ?

I am trying to fill a file with a sequence of random 0 and 1s with a user-defined number of lines and number of characters per line. the first step is to get a random stream of 0 and 1s: cat /dev/...
4
votes
3answers
1k views

Why does a gunzip to dd pipeline slow down at the end?

my command: gunzip -c serial2udp.image.gz | sudo dd of=/dev/mmcblk0 conv=fsync,notrunc status=progress bs=4M my output: 15930949632 bytes (16 GB, 15 GiB) copied, 1049 s, 15.2 MB/s 0+331128 ...
4
votes
2answers
11k views

dd command indicates not enough disk space - trying to format sd card for raspberry pi

I have been trying to format an sd card with the lastest debian jessie-lite image for use with raspberry pi. When using the dd command, it states that there is no space left on device after copying ...
4
votes
2answers
10k views

Can I mount a partition from a full drive image? [duplicate]

Note: While I agree that this question basically is a duplicate of the above question, I feel @alienth's answer (below) is more concise, so I suggest you take a look at it before going to the other ...
4
votes
3answers
971 views

How do I DD over the network to an external HD?

I'm trying to clone a hard drive on my raspberry pi which is on the same network to an external harddrive of the computer I'm currently on. However I'm having issues piping the command correctly. I've ...
4
votes
4answers
2k views

dd remove range of bytes

Given this file $ cat hello.txt hello doge world I would like to remove a range of bytes to end up with this $ cat hello.txt heorld I would like to do this with dd if possible. The reason is ...
4
votes
1answer
11k views

What is the difference between `dd`, `cp` and `rsync`? [closed]

Besides the explanations from the man pages and --help information, in which way do the commands dd, cp and rsync differ when used to copy files? In which context is each of these superior to the ...
4
votes
4answers
1k views

How can I convert two-valued text data to binary (bit-representation)

I have a text file with two (2) only possible characters (and maybe new lines \n). Example: ABBBAAAABBBBBABBABBBABBB (Size 24 bytes) How can I convert this to a binary file, meaning a bit ...
4
votes
3answers
924 views

Why is dd running forever on a 8 GB drive

I have a filesystem handled by ZFS and it's about 8 GB. I tried to zero out the filesystem by running: dd if=/dev/zero of=/EMPTY bs=1M | true rm -f /EMPTY And normally on an ext4 filesystem, it ...
4
votes
1answer
1k views

Use backup file from dd to restore hard disk and also extract files

I made regular complete backups of my hard disk (with all partitions) using dd tool as backup.img file. I have several such files. Now that my old hard disk died, how can I use backup.img file to ...
4
votes
3answers
21k views

Terminal: create bootable USB from iso

How can I create a bootable USB stick from an iso image? I thought dd should do the work, but so far I were unsuccesful. This is what I've tried: umount /dev/sdx deleted every partition on sdx with ...
4
votes
3answers
7k views

HDD image file checksum does not match with device checksum

I have a 660297728 byte HDD image with MD5 hash f5a9d398e974617108d26c1654fe7bcb: root@T42# ls -l image -rw-rw-r-- 1 noc noc 660297728 Sep 29 19:00 image root@T42# md5sum image ...
4
votes
1answer
211 views

Error(?) in output of the dd command

When I run the following command: dd if=/dev/sda of=output bs=512 count=1 In the middle of the output, I see this text: GRUB ^@Geom^@Hard Disk^@Read^@ Error^M What does it mean?
4
votes
3answers
4k views

Is creating a sparse image using dd appropriate for backup/restore from a RAID?

I have two non-networked servers, each with 1.2TB of storage in a RAID5 config on an LSI MegaRaid 9240-8i controller. I have been trying without success to get Symantec Ghost to image one machine to ...
4
votes
3answers
1k views

How to overwrite beginning of a device with random bits or zeros?

Just looking to overwrite a small portion of a device with zeros or random bits. More precisely I would like to overwrite the first 1% of all the sectors or few MiB. Is there an easy way to do that ?
4
votes
1answer
4k views

What is the fastest way to write all sectors on a drive?

The goal in my case is not to securely erase data, but simply to force sector reallocation by the drives firmware. I have a drive that has had a few file corruptions. It seems to be due to getting ...
4
votes
1answer
1k views

How do I interpret the output of dd when the option status=progress is chosen?

The dd manual page is very limited on this: ` status=LEVEL The LEVEL of information to print to stderr; 'none' suppresses everything but error messages, 'noxfer' ...
4
votes
3answers
4k views

How can I copy three partitions of my disk with a specific beginning and ending using dd?

Using the fdisk -l command I got the following answer: Device Boot Start End Blocks Id System /dev/sda1 * 2048 28266495 14132224 27 Hidden NTFS WinRE /dev/...
4
votes
6answers
2k views

How one can re-read image with dd so it will match one you just wrote?

I have a ~1GB image that I'm writing to a 8GB SD card via the dd tool. I'd like to verify that it was written without corruption by reading it back and comparing its hash with original one. Obviously,...
4
votes
1answer
1k views

How to inspect the raw contents of a partition?

I'm trying to see the contents in raw bytes of a dummy partition /dev/sda3, which I've made to be 100MiB through gparted. I want to test writing and reading on that partition. Is there a tool that ...
4
votes
2answers
14k views

How to write a file at various offsets to a filesystem partition with dd command

Requirement I want to write a file at various offsets into the partition Partition /dev/part2 is mounted at /mypart I tried the command below: dd if=/dev/urandom of=/mypart/aaa bs=1024 seek=0 ...
4
votes
2answers
5k views

dd to clone to smaller sdcard

One of my Raspberry Pi's is using a 4GB SDcard for the root/boot filesystem. As seen when mounted on another system: Disk /dev/sdc: 3904 MB, 3904897024 bytes 121 heads, 62 sectors/track, 1016 ...
4
votes
1answer
1k views

What does this dd command do exactly?

I have an embedded device with this SD card: [root@(none) ~]# busybox fdisk -l Disk /dev/mmcblk0: 3965 MB, 3965190144 bytes 4 heads, 16 sectors/track, 121008 cylinders Units = cylinders of 64 * 512 = ...