Questions tagged [dd]

dd is a traditional utility for binary data copies

Filter by
Sorted by
Tagged with
147
votes
13answers
150k views

How do I know if dd is still working?

I've not used dd all that much, but so far it's not failed me yet. Right now, I've had a dd going for over 12 hours - I'm writing an image back to the disk it came from - and I'm getting a little ...
120
votes
8answers
31k views

dd vs cat — is dd still relevant these days?

I recently realized we can use cat as much as dd, and it's actually faster than dd I know that dd was useful in dealing with tapes where block size actually mattered in correctness, not just ...
110
votes
20answers
25k views

What is the fastest way to send massive amounts of data between two computers? [closed]

This is a situation I am frequently in: I have a source server with a 320GB hard-drive inside of it, and 16GB of ram (exact specs available here, but as this is an issue I run into frequently on ...
83
votes
3answers
131k views

How to dd a remote disk using SSH on local machine and save to a local disk

How can I create a backup of a remote disk using SSH on my local machine and save it to a local disk? I've tried the following: ssh hostname@my.ip.address "sudo dd if=/dev/sdX " | \ dd of=/home/...
69
votes
5answers
44k views

Is there a way to determine the optimal value for the bs parameter to dd?

On occasion I've seen comments online along the lines of "make sure you set 'bs=' because the default value will take too long," and my own extremely-unscientific experiences of, "well that seemed to ...
66
votes
6answers
62k views

Is it possible to mount a gzip compressed dd image on the fly?

I like create an image backup for the first time I'm backing up a system. After this first time I use rsync to do incremental backups. My usual image backup is as follows: Mount and zero out the ...
61
votes
7answers
65k views

Best way to remove bytes from the start of a file?

Today I had to remove the first 1131 bytes from an 800MB mixed text / binary file, a filtered subversion dump I'm hacking for a new repository. What's the best way to do this? To start with I tried ...
59
votes
2answers
13k views

When is dd suitable for copying data? (or, when are read() and write() partial)

Short version: In what circumstances is dd safe to use for copying data, safe meaning that there is no risk of corruption due to a partial read or write? Long version — preamble: dd is often used to ...
42
votes
4answers
18k views

What does dd stand for?

I know very well what the command does, but man dd, info dd tell me: 'Convert and copy a file', as does GNU Coreutils. Google says its an abbreviation of everything between medicine and bad webchat ...
35
votes
8answers
23k views

Clone whole partition or hard drive to a sparse file

I like to clone a whole partition or a whole hard drive onto a larger external disk but like to create a sparse file. I often use dd for cloning, but it doesn't support sparse files. As a workaround I ...
34
votes
5answers
121k views

Full DD copy from hdd to hdd

ORIGINAL QUESTION: If I have 2 identical hard drives with the following characteristics: SATA 6.0 Gb/s 5400 rpm 3TB How long should a full dd copy take to complete? So far it's been running for 5 ...
33
votes
7answers
26k views

How do I suppress dd output?

I have a bash script that creates some file using dd. The problem is dd throws a great amount of output that is going to mess with the output of my script. Searching around I've found a solution: dd ...
32
votes
2answers
9k views

Patching a binary with dd

I have read this quote (below) several times, most recently here, and am continually puzzled at how dd can be used to patch anything let alone a compiler: The Unix system I used at school, 30 years ...
30
votes
4answers
20k views

Is dd able to overwrite parts of a file? [duplicate]

I have a 1TB big file (disk-image from a damaged drive) and a 1.3MB small file (beginning of a disk-file). Using the contents of the small file, I want to overwrite portions of the big file. That is, ...
28
votes
2answers
39k views

Running dd. Why resource is busy?

I just formatted microSD card, and would like to run a dd command. Unfortunately dd command fails: $ sudo dd bs=1m if=2016-02-26-raspbian-jessie-lite.img of=/dev/rdisk2 dd: /dev/rdisk2: Resource busy ...
24
votes
4answers
15k views

Why does dd from /dev/random give different file sizes?

I am running the following command on an ubuntu system: dd if=/dev/random of=rand bs=1K count=2 However, every time I run it, I end up with a file of a different size. Why is this? How can I ...
23
votes
5answers
5k views

On-the-fly stream compression that doesn't spill over into hardware resources?

I have 200 GB free disk space, 16 GB of RAM (of which ~1 GB is occupied by the desktop and kernel) and 6 GB of swap. I have a 240 GB external SSD, with 70 GB used1 and the rest free, which I need to ...
23
votes
4answers
3k views

What's the difference between these two dd commands?

What's the difference between two dd commands that have different bs and count values, as long as they multiply to the same? For example: dd if=/dev/random of=aa bs=1G count=2 dd if=/dev/random of=aa ...
23
votes
4answers
30k views

Benchmark ssd on linux: How to measure the same things as crystaldiskmark does in windows

I want to benchmark a ssd (possibly with encrypted filesystems) and compare it to benchmarks done by crystaldiskmark on windows. So how can I measure approximately the same things as crystaldiskmark ...
23
votes
2answers
34k views

Create file image container

I would like to create a 100GB file to use as a file-system. The file will be stored on an encrypted volume, so the file/file-system itself does not need to be (further) encrypted. Searching has ...
22
votes
4answers
34k views

Force dd not to cache or not to read from cache

I'm working on an embedded system with the busybox version of dd. I'm trying to test an erase to the drive from some outside utility, however dd does not read from the disc again after the erase, but ...
20
votes
2answers
11k views

Is it better to use cat, dd, pv or another procedure to copy a CD/DVD?

Background I'm copying some data CDs/DVDs to ISO files to use them later without the need of them in the drive. I'm looking on the Net for procedures and I found a lot: Use of cat to copy a medium: ...
19
votes
2answers
51k views

What is the difference between 'bs', 'count' and 'seek' in dd command?

I've read many guides and forum posts describing how to use dd, but one thing I've noticed is that people always use different values for the bs=, count= and seek= switches. Please can someone ...
19
votes
3answers
20k views

How do I mount an `img` created with /bin/dd of a hard drive?

I used dd to backup a 80GB drive dd if=/dev/sdb of=~/sdb.img Now I need to access some files on that drive, but I don't want to copy the ".img" back over the drive. mount ~/sdb.img /mnt/sdb doesn'...
19
votes
4answers
41k views

'seek' argument in command dd

Can some explain me what is happening in the following lines? dd if=/dev/urandom bs=4096 seek=7 count=2 of=file_with_holes especially seek part is not clear Man pages says : seek=BLOCKS ...
19
votes
4answers
11k views

Read the middle of a large file

I have a 1 TB file. I would like to read from byte 12345678901 to byte 19876543212 and put that on standard output on a machine with 100 MB RAM. I can easily write a perl script that does this. ...
19
votes
5answers
12k views

How to ignore write errors while zeroing a disk?

Say you want to zero-out a failing hard disk. You want to overwrite as much as possible with zeros. What you don't want is: the process aborts on the first write-error. How to do that? AFAICS, plain ...
18
votes
6answers
77k views

What does `dd if=/dev/zero of=/dev/sda` do

Edited: do not run this to test it unless you want to destroy data. Could someone help me understand what I got? dd if=/dev/zero of=/dev/sda bs=4096 count=4096 Q: Why specifically 4096 for count? ...
18
votes
1answer
3k views

Why does piping `tar` into `dd` not stop until the disk is full?

I have a tar archive of a single disk image. The image inside this tar file is about 4GB in size. I pipe the output of tar xf into dd to write the disk image to an SD card. The diskdump never stops ...
18
votes
2answers
26k views

Use `dd` to cut file end part

There is probably simple trick to do this, but I can't figure from man page. How do I cut last 1MB from file with undetermined size for example, by using dd?
18
votes
3answers
3k views

How can you trim files using the command line?

What is the console equivalent of the following Python code: target = file("disk", "w") # create a file target.seek(2*1024*1024*1024) # skip to 2 GB target.write("\0") target.close() Maybe some ...
17
votes
3answers
15k views

Create random data with dd and get “partial read warning”. Is the data after the warning now really random?

I create a 1TB file with random data with dd if=/dev/urandom of=file bs=1M count=1000000. Now I check with kill -SIGUSR1 <PID> the progress and get the following: 691581+0 Datensätze ein 691580+...
17
votes
2answers
1k views

Why do these duplicated SD cards have different sha1sums for their content?

I have a bunch of Class 10 UHS-1 SDHC SD cards from different manufacturers. They are all partitioned as follows $ sudo fdisk -l /dev/sdj Disk /dev/sdj: 14.9 GiB, 15931539456 bytes, 31116288 sectors ...
16
votes
3answers
8k views

dd: write to multiple disks?

I have 2 exactly same formatted, same size and same brand SD-cards. I would like to dd image to /dev/disk2 and to /dev/disk3 at the same time. Pseudocode sudo dd bs=1m if=/Users/masi/2016-05-10-...
16
votes
3answers
6k views

What does the two numbers mean respectively in dd's “a+b records” stats?

The first 2 lines in dd stats have the following format: a+b records in c+d records out Why 2 numeric values? What does this plus sign mean? It's usually a+0, but sometimes when I use bigger block ...
15
votes
5answers
5k views

Does dd do any kind of verification?

I'm using dd to copy data from an old hard drive to a new one. I want to be sure that the data's integrity is secure. On this answer, Gilles says If [dd] terminated successfully, then the backup ...
14
votes
6answers
14k views

Fast Way to Randomize HD?

I've read about how to make hard drives secure for encryption, and one of the steps is to write random bits to the drive, in order to make the encrypted data indistinguishable from the rest of the ...
14
votes
4answers
18k views

How to pad a file to a desired size?

I have a file that I want to pad until it reaches 16 MiB (16777216 bytes). Currently it is 16515072 bytes. The difference is 262144 bytes. How do I pad it? This doesn't seem to be working: cp ...
14
votes
2answers
5k views

Why does dd from /dev/urandom stops early? [duplicate]

On my current Linux system (Debian Jessie amd64), I'm getting different behavior for dd using /dev/urandom (/dev/random behavior is properly documented). If I naively want 1G of random data: $ dd if=/...
13
votes
4answers
13k views

Why dd takes too long?

I need to copy one disk to another. I tried with the command below and it takes nearly a day to copy 1 TB of disk. dd if=/dev/sda of=/dev/sdb I have tried the same on a Unix system with the ...
13
votes
4answers
7k views

dd: multiple input files

I need to concatenate chunks from two files: if I needed concatenate whole files, I could simply do cat file1 file2 > output But I need to skip first 1MB from the first file, and I only want 10 ...
13
votes
5answers
1k views

Is it safe to take the drive image of the current working drive?

I have to backup my hard disk. I want to use dd and put the image on an external hdd. Can I do this using dd from the OS that resides on the hdd itself or do I have to boot from another device, for ...
13
votes
5answers
34k views

can 'dd' be used to clone to a smaller HDD, knowing that partitions will need editing?

I've used dd to clone disks like this: dd if=/dev/sdb of=/dev/sda bs=4096 conv=notrunc,noerror,sync And it's always worked fine. Any and all docs on 'dd' take pains to remind you that the target ...
13
votes
3answers
12k views

Are there alternatives for `dd`?

What other tools should I use to read and write files with truncation, seeking and skipping. dd's command line options seem inconvenient and foreign and I don't like choosing between slow, but precise ...
12
votes
2answers
4k views

Will dd if=/dev/zero of=/dev/sda wipe out a pre-existing partition table?

Will # dd if=/dev/zero of=/dev/sda wipe out a pre-existing partition table? Or is it the other way around, i.e, does # fdisk /dev/sda g (for GPT) wipe out the zeros written by /dev/zero?
12
votes
1answer
11k views

Can I uncompress a zip file containing a disk image and then save that to an SD card all in one step?

I am under the following restrictions: I have a 1.0 GB .zip file on my computer which contains one file, a disk image of raspbian. When uncompressed, this file is 3.2 GB large and named 2015-11-21-...
12
votes
3answers
4k views

saving data from a failing drive

An external 3½" HDD seems to be in danger of failing — it's making ticking sounds when idle. I've acquired a replacement drive, and want to know the best strategy to get the data off of the dubious ...
11
votes
4answers
2k views

Why does writing random data using dd result in disk partitions?

Prior to running the dd command, the command lsblk returned the output below: NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 931.5G 0 disk The ...
11
votes
2answers
3k views

bash can't store hexvalue 0x00 in variable

I'm trying to do some tricks with dd. I thought it would be possible to store some hexvalues in a variable called "header" to pipe it into dd. My first step without a variable was this: $ echo -ne ...
11
votes
5answers
11k views

Why specify block size when copying devices of a finite size?

In online tutorials it is often suggested to use the following command to copy a CDROM to an iso image: $ dd if=/dev/dvd of=foobar.iso bs=2048 Why must the byte size be specified? I notice that in ...