Questions tagged [dd]

dd is a traditional utility for binary data copies

117 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
5
votes
0answers
205 views

Why does ddrescue not utilize full bandwidth?

I am running ddrescue from disk A to disk B. I had expected the slower of the disks would be running at full speed all the time and the other be running enough to keep up. Imagine my surprise when ...
5
votes
2answers
295 views

confused about dd write speed and VFS page cache

After reading some articles on the Linux VFS page cache and the tunable parameters like dirty_ratio i was under the impression that page cache would operate as both read and write caching layer. But ...
4
votes
0answers
3k views

Using dd on Mac OS: conv=fsync equivalent?

I'm running an experiment where I benchmark the speed of reading and writing with and without caching. On Linux, using 'conv=fsync' as an operand makes sure that the output file is entirely physically ...
4
votes
0answers
325 views

How do I use the color palette (FBIOPUTCMAP)?

I am trying to track down the following issue in Debian Jessie on PowerPC. Per bogl-bterm package maintainer the issue is simply that bterm makes uses of FBIOPUTCMAP, see ref here. So before even ...
4
votes
1answer
38 views

Looking for a way to backup/mirror a drive by swapping

I want to duplicate a drive. I am unable to hook up both the old and new drive at the same time (the old powerbrick died; so I have one powerbrick to swap between two drives). In years and years ...
4
votes
0answers
355 views

How to write to an .img file with two different partitions?

I have been working on building a Linux based OS for Raspberry-pi using buildroot. I am generating the required empty .img file using "dd" and "fdisk" utilities. I am creating two empty partitions as ...
3
votes
1answer
899 views

dd, ddrescue, dd_rescue performance according to OS (Ubuntu, macOS)

I put this question: recover files from HSF+ USB External HD, test on CentOS 7 to diagnostic And this https://askubuntu.com/questions/955325/using-gpart-testdisk-to-recover-gpt-for-files-from-...
3
votes
1answer
394 views

Full clone of encrypted disk not bootable on other computers

I have made a full disk clone of my encrypted S2 disk to an external USB disk with dd if=/dev/nvme0n1 of=/dev/sdb status=progress The S2 disk was not mounted when cloning, I used a live distribution ...
3
votes
0answers
170 views

Have I destroyed data on drive?

I stupidly ran the following command on an external ext4 USB disk, typing /dev/sdb instead of /dev/sda which was the new disk I was setting up: dd if=/dev/zero of=/dev/sda bs=512 count=100 These ...
3
votes
0answers
169 views

Hard Drive reporting all 1's after parted

So I have a 3TB hard drive /dev/sdc that I am trying to create a partition on. Before this point, I had the issues described below, I transfered the drive to a Windows 7 computer and created a GPT on ...
3
votes
0answers
110 views

Is this a correct way to back up one big drive to two smaller ones?

I have a pre-fail drive, and while I'm waiting for replacement I am trying to do full back up to two smaller drives. Given that drive is 3TG and ones I have are 1.5 and 2TB, is this a correct way to ...
3
votes
1answer
390 views

Equivalent of memmove on files: copy a range of bytes to an earlier position

What is an equivalent of memmove on files with the same input and output file, which works with large files? dd works when I move the file contents towards the right, but not if I move the contents ...
2
votes
0answers
29 views

dd measurement question

I am wrestling with some perceived performance issues on a 3-node HPE Apollo cluster running Linux. (The cluster supports a backup application.) These are very $$ servers. In our setup each node ...
2
votes
0answers
16 views

Avoid pipeline stalls from write caching to usb drive

I have the following process I use for backups: btrfs send snapshots/home-2016-06-04 | zstd --verbose -T4 - | gpg --batch --passphrase-file /tmp/secret --compress-algo none --symmetric - > /mnt/...
2
votes
0answers
48 views

dd copied UUID - How to make GRUB2 find the correct partition?

Using Grub2 on Ubuntu LTS Xenial On sda1, I dd the partition to sdb1(which already has GRUB2 on the MBR) with sudo dd if=/dev/sda1 of=/dev/sdb1 bs=1M. I found that it also copied the UUID. So, both /...
2
votes
0answers
54 views

md5sum of partition and img do not match

I booted Ubuntu on a live USB and ran: dd if=/dev/sda1 of=/mnt/usb/backup.img bs=30M oflag=sync I then performed a comparison (using 'md5sum') of the above input partition and output img, but they ...
2
votes
0answers
41 views

Strange memory usage pattern using LUKS

I'm setting up and external HDD encrypted with LUKS. After adding the LUKS partition on it, to 1) scramble it a bit (which could be pointless) and 2) check that I can write on all the disk, I did: dd ...
2
votes
0answers
215 views

Dumping a single partition with dd (not the entire disk)

I have dumped the raw partition dumped via ssh from an external system using the command: ssh root@10.10.10.10 'dd if=/dev/xvda2 bs=1M | gzip' | gunzip | dd of=xvda2.raw As you can see, only the ...
2
votes
0answers
568 views

Why does md5sum give inconsistent results on device files?

Backing up some USB flash-drives on Lubuntu 16.04.3 LTS. Plug USB drive into laptop. Automounter does its stuff. I remount all partitions as read-only sudo mount -o remount,ro /dev/sdxn Copy USB ...
2
votes
0answers
5k views

How to get RAM map?

I had read a topic by Gabriele Tolomei about map of Linux memory. I want to make a memory map for my Linux Ubuntu 16.04, using basic utilities, like dd and xxd, and I want to compare my result with ...
2
votes
0answers
164 views

How to extract and stitch together a multipart tar arcvhive without the initial file

I have a large folder with multi part tar files each 2gb in size, file.partaa through file.partgy, however I am missing the initial tar file. Is there any way that I can extract each part ...
2
votes
0answers
385 views

dd command 4M block size - buff cache

dd if=/dev/urandom of=file1 bs=4M count=100 procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------ ---timestamp--- r b swpd free buff cache si so bi bo ...
2
votes
0answers
875 views

how to copy a piece of a text file byte-by-byte to another text file? dd, head, or?

I need to grab the first lines of a long text file for some bugfixing on a smaller file (a Python script does not digest the large text file as intended). However, for the bugfixing to make any sense, ...
2
votes
0answers
263 views

IOSTAT report doesn't match with DD benchmark

I am running a DD benchmark but the obtained results are not matching with the IOSTAT values. This is the DD sample: $ sudo dd bs=4096 count=690 if=/dev/zero of=/mnt/tmp/test oflag=dsync 690+0 ...
2
votes
0answers
1k views

Making an ISO of MBR + X amounts of partitions

I am trying to make an ISO of a Windows Backup disc that I "burned" to my microSD card, which I did in desperation due to lack of HDD storage at the time. Now I want to get it out of there and onto ...
2
votes
1answer
596 views

dd'ing SDD, decreasing write speed

My hard drive is a SanDisk SSD. The drive is new. I've booted into ArchLinux Live System, with the intention of preparing the hard drive for full disk encryption. I've noticed a significant drop of ...
2
votes
0answers
2k views

Use dd to backup disk image within an OpenVZ container

I want to use dd to backup the disk of an OpenVZ VPS in addition to traditional cold tar backups. Because I may need to migrate to a different VPS provider. From end user point of view, OpenVZ ...
1
vote
0answers
34 views

Piping Backups through Compression and Encryption

I am trying to backup a disk image of a local system to a remote system, in a way I can automate by bash script at a later point. Backing up and restoring was going well until I introduced encryption, ...
1
vote
0answers
43 views

Issue in following dd command

I'm new with this so bear with me... I have this command : tr '\\0' '\\377' < /dev/zero | dd bs=4096 count=1484095 of=/path/one Output: 1+1484094 records in 1+1484094 records out 8412124 bytes ...
1
vote
0answers
31 views

Fast way to create virtual disks?

I want to create virtual 500GB ext4 disks on external usb disks and samba/nfs shares. For test purposes, I first tried with a 1GB file using the following commands. sudo dd if=/dev/zero of=/mnt/...
1
vote
1answer
58 views

Can recover data from disk but not from image of same disk?

So I'm creating this training scenario whereby participants have to take an image of a broken disk, loop mount and recover the data. It's supposed to be fairly basic, but I've come accross this weird ...
1
vote
0answers
68 views

loop device not matching file size

I have a sparse backing file (NTFS) (made via dd seek=) that's a partial image of a damaged disk. The size as given by ls matches the original disk. However, when looping it, /proc/partitions shows it ...
1
vote
0answers
52 views

Cannot access mounted partition after using dd

I have used following command to create an image of mounted partition sda1: dd if=/dev/sda1 of=/dev/sdb1 'conv=noerror,sync' where sda1 is the one I want to "backup" and sdb1 is obviously target, ...
1
vote
0answers
53 views

When using TAB completion with dd command, options (like 'if' or 'of') disappear

When I type dd if= then hit TAB to complete, the file name is added, but the if= portion vanishes (and of= too, for that matter). So for example, instead of # dd if=input_file.iso of=/dev/sdb I get ...
1
vote
0answers
48 views

Reset all files after reboot

I have a little question. It is possible to reset all files of the system to the saved one? I thought about saving state with dd to other disk, then create "man in the middle" so after boot it will ...
1
vote
0answers
177 views

Why can't dd write to /dev/ram0 on Core Linux?

I created an ext2 file-system on /dev/ram0 ramdisk, and mounted it at /mnt/ram. mkdir /mnt/ram mkext2fs /dev/ram0 sudo mount /dev/ram0 /mnt/ram -o rw Reading from ramdisk with dd works. dd if=/dev/...
1
vote
0answers
70 views

Good fdisk output for backup image but can't mount to loop device even when using offset

I've recently created a backup file for my Ubuntu 16.04.4 server using dd in order to prepare for migrating to a larger SSD. I want to mount the backup image through a separate Ubuntu VM running on my ...
1
vote
2answers
232 views

bash output create file with dd /dev/zero with dialog --gauge

I am trying to create a new file (blank file) with: dd if=/dev/zero of=file.txt count=10240 bs=10240 I also want to use dialog output --gauge, something like this: dd if=/dev/zero | pv | dd of=...
1
vote
0answers
544 views

Why is this `dd` command producing `Resource busy` errors?

I'm attempting to recover anything possible from a known-bad hard disk (failing, or failed). I began with dd and realized I didn't unmount. After unmounting volumes, I was able to correctly begin the ...
1
vote
2answers
92 views

How to identify relevant parts of an ext4 partition

Background: I need to clone ext4 partitions from an eMMC using uboot (and if neccessary custom bare metal code). I copied the whole thing using mmc read and found that most of the partition is empty, ...
1
vote
0answers
106 views

Can gzip change original file size?

/dev/sdf is a 32G SD. As far as I can tell, I had performed the following, but upon attempting to perform the final step and write to a SD, my file image appears to be one byte to large for the SD. ...
1
vote
1answer
377 views

dd is working with bs argument but gives IO error without

I have a HDD 4TB WD 40EFRX (red series) and try to check the health of that disc. Actually, I erased all partitions and the table via gparted and build a new table + ex4 partition. Subsequently, I ...
1
vote
1answer
378 views

dd creates a file bigger than the source

I use dd for creating an image of my filesystem . I use Debian 8(Jessie) on ARM 32 bit system. Command I run is : dd if=/dev/mmcblk0 bs=1M | gzip -c > /mnt/usb/<image_name.gz> For some ...
1
vote
0answers
169 views

What does dd do when not enough free space

I just made backup of SD card with dd command: dd if=/dev/mmcblk0 of=backup.img bs=64k When I started copying, Gnome Files said that there is just 8.5GB of free space from total 400GB. I forgot that ...
1
vote
1answer
154 views

Partitioning SSD for multiple OS's

I installed a SanDisk 240GB SSD. My old HDD will now be my storage. I've partitioned the SSD into 4 roughly 55GB partitions. I've also done a pvcreate, vgcreate, and an lvcreate for each partition ...
1
vote
0answers
152 views

Optimize cloning image without cloning unused space

I have a pi with a 32g sd. michael@pi3:/ $ df Filesystem 1K-blocks Used Available Use% Mounted on /dev/root 29804860 1773224 26791008 7% / devtmpfs 468152 0 468152 ...
1
vote
0answers
57 views

Problems mounting and dumping to external storage (Mac) from dedicated server (Ubuntu)

I have set up a Kimsufi KS-4A - 16G i7-920 1x2TB Server, and been following this guide on how to create my own Windows template for Kimsufi. I have come to the last steps of the tutorial. What I want ...
1
vote
0answers
437 views

Zero out deprecated ZFS Label from Disk with dd

I recently created a new zpool over 8 disks with 4 mirrors. Today I attached the disks to a linux server and an old deprecated zpool is now showing up with zpool import which uses the same disks as ...
1
vote
0answers
58 views

Strange df output after ddrescue clone

My old harddisk was having I/O error last week so I tried to copy the data out to another new disk intending to rescue the data. I use ddrescue: ddrescue -f /dev/sda1 /dev/sdb1 my_mapfile After I ...
1
vote
1answer
1k views

dd script with timings

Trying to figure out the best approach to a script that will do the following: Create a loop script that creates a 10GB file using dd where, upon completion, removes the file, then repeats 25 times ...