Questions tagged [dd]
dd is a traditional utility for binary data copies
849
questions
9
votes
3
answers
2k
views
How can I copy arbitrary part of binary file with reasonable speed?
How can I copy arbitrary part of a binary file with reasonable speed?
Apparently, dd with bs=1 is very slow, while setting bs to another value makes it impossible to copy arbitrary parts.
Is it wrong? ...
- 2,967
-1
votes
1
answer
77
views
what is "dd if=/dev/zero of=/storage2/test4.img bs=1M count=256" usage?
I would be thankful if someone can explain me what does following command exactly doing:
dd if=/dev/zero of=/storage2/test4.img bs=1M count=256
0
votes
1
answer
42
views
Can't boot from SSD after it was moved into an external case
I used a SSD in an ASMT 2105 harddrive docking station connected to my Pi4 for installation (Raspbian Pi OS).
After moving the SSD into a smaller external SSD case, the Pi won't boot anymore (no ssh ...
- 226
2
votes
1
answer
135
views
How to read constant size buffers from a named pipe, and run a command for each buffer
There is a function in terminal: redis-cli -x PUBLISH myChannel. It reads data from stdin and does its job.
FIFO itself doesn't give chunked data. So I need to create chunks from the stream, and run ...
- 31
0
votes
1
answer
31
views
How to Tranfser huge amount of Files to slow USB Drive (dd, cp, rsync, 7z, tar)
I have like 5GB of data and a very slow USB-Drive.
Should I use dd, cp or rsync?
Should I compress them first into for example 7z / tar or not?
In short: What is the best way and the best practices ...
- 27
0
votes
1
answer
61
views
What makes sfill so slow?
sfill -Illv makes a file with data from /dev/urandom until it fills the partition.
Making a large file like this takes a lot of time. iotop tells me it is writing about 1 MB/s.
Though dd if=/dev/...
- 3
0
votes
1
answer
44
views
I can't specify DD to overwrite a usb 2.0 at its theoretical speed limit
I'm trying to learn about filesystem and i/o operations in practice, i mean, acting in a conscious way and my task is to overwrite a USB 2.0 flash drive with the dd command in a Linux. But as dd is ...
- 1
0
votes
0
answers
18
views
How to move a partition to the beginning of a drive
Say you have some drive /dev/sda that is 1TB in size. There is a partition /dev/sda2 of size 750GB at the end of the drive, and there is 250GB of unallocated space at the beginning of the drive. How ...
- 121
1
vote
1
answer
101
views
dd: invalid argument 'progress' to 'status'
In my Buildroot Linux image I try do
dd if=/mnt/disk.img of=/dev/sda bs=4M conv=fsync status=progress
But I get error
dd: invalid argument 'progress' to 'status'
Why?
0
votes
1
answer
38
views
Extract first n bytes from .tar.gz and output as a .tar.gz in a single command
I have a .tar.gz as input and want to extract the first 128 MiB of it and output as a .tar.gz in a single command. I tried:
sudo tar xzOf input.tar.gz | sudo dd of=output bs=1M count=128 iflag=...
- 3,466
1
vote
0
answers
73
views
How can I quickly perform a whole-drive dd backup over ssh?
I am trying to periodically back up a machine I am hosting remotely with dd. This is the method that I have found works in the past for machines with small drives. But my new machine has a 75 GiB luks-...
- 232
0
votes
1
answer
52
views
What does it mean if the dd clock is ticking very slowly (every 12s only)?
I have an Intenso external 1TB 2,5" HDD here. It is somehow faulty (disconnects randomly on Windows) so I decided to run this command to check it:
dd if=/dev/sdb of=/dev/null bs=4M status=...
- 779
1
vote
0
answers
89
views
Is it possible to copy a disk image to a smaller disk?
Let me explain, for a greater detail (although this casuistry will serve for many more cases): I have a cloned image of a 512GB mSATA disk, which in practice, really dlos comanoes not reach 10GB used ...
- 1,086
4
votes
1
answer
98
views
Disk ran out of space when cloning with dd, but dd continues to copy
I am using the dd tool to make back-ups of Raspberry Pi SD card images to my local machine. When cloning one of these cards, the system drive ran out of free space. My machine had 112 GB of free space ...
- 41
1
vote
1
answer
41
views
Unable to access or safely remove encrypted device
Details: I have a bootable encrypted HDD (running Linux mint). I have cryptsetup (2:2.4.3-1ubuntu1.1) installed on my main machine.
I can access the unencrypted partition of the 1TB HDD. The system ...
- 23
0
votes
1
answer
41
views
Mount compressed disk image or clone with dd respecting free space
I intend to mount locally an image obtained as a clone through dd. The image is corresponds to a mSATA disk of 512GB of storage, with a minimum version of Ubuntu barely configured. So usefully, really ...
- 1,086
1
vote
1
answer
52
views
Extra space issue while creating a hard disk image using dd command
I have a Ubuntu OS installed on 1 Tb hard disk. Currently, around 200 GB hard disk is used while around 800 GB is free. Now, I have to take a complete (iso) image of hard disk for backup so that I can ...
1
vote
2
answers
610
views
Accidentally used dd command
I have been following a tutorial online to teach myself some basic command line stuff and accidentally I used the dd command (which I haven't gotten to and don't understand), but it seems to have done ...
- 23
0
votes
1
answer
63
views
Make a bootable, preinstalled and persistent USB bootable linux system
I have a debian 10 VM / Physical server, that I want to clone completely into a USB (DOK) Drive, and make it bootable.
Additionally - I need this DOK to be Read/Write and not only ROM like CDFS.
I ...
0
votes
1
answer
55
views
How to calculate the checksum while writing random data to a disk?
I have a disk which is perhaps broken. I want to write random data to the disk and later verify the md5 checksum.
I write to the disk like this:
dd if=/dev/urandom of=/dev/sda bs=4M status=progress
...
- 779
1
vote
1
answer
63
views
md5sum of image and sd-card partition differ
I am creating an image of an sd-card partition (dd) and eventually the checksum (md5sum) of the image and the partition are not the same.
What am i doing wrong?
My sd card is inserted into an external ...
- 13
4
votes
1
answer
193
views
Mounting a drive from an embedded device with bytes swapped
I have a device from about twenty years ago that uses an IDE hard drive as storage. I would like to keep using it, but it has a USB 1 interface that is extremely slow.
I can take out the hard drive ...
- 81
0
votes
1
answer
33
views
Is there a way to be sure a file is not truncated compared to the file size stored in the inode table? (Does a dd command do?)
I work on an ext4 filesystem. I have doubts about the accuracy of a directory entry regarding the description of a file that I have copied from an NTFS filesystem and that might have spanned some bad ...
- 260
0
votes
2
answers
22
views
dc3dd, why tab completion doesn't work?
On Slackware 15.
Using dd I do
dd if=/home/myuser/my..TAB and complete dir path after pressing tab
using dc3dd I did
dc3dd if=/home/myuser/my..TAB and nothing happen.
Why?
I have tried with rlwrap but ...
- 11.7k
0
votes
1
answer
38
views
Performance test went wrong and the dd command created 13TB of data on /dev/mapper/device. Why system didn't crash? HDD-250GB
So I wanted to do some performance test with encrypted and normal data storage on my embedded device.
That is not what I was expected to see at all!
Can you please explain it to me what just happend. ...
1
vote
0
answers
103
views
Is ddrescue/dd cloning compatible with GPT/UEFI boot?
I used ddrescue many times for failing HDDs with MBR partition tables. It always worked perfectly and if the failing HDD was bootable, the clone booted, too.
Now I have a possibly failing SSD, but ...
- 11
2
votes
1
answer
110
views
mkfs fails if called after a 'dd' operation
I want to delete all files on one my eMMC blocks (e.g. /dev/mmcblk0p1) and re-create the filesystem afterward (with ext4).
So I unmount and erase:
umount /dev/mmcblk0p1
dd if=/dev/zero of=/dev/...
- 764
0
votes
1
answer
183
views
New on Linux, I m afraid I detroyed an SSD, did I? What can I do, please?
I'm new on the Linux field. I have a real big problem.
I have installed several times Linux Mint 21 on an old - but very nice computer - without any issues.
I'm not at all a beginner in the field of ...
0
votes
1
answer
48
views
Using dd to do a bare-metal restoration w/ stored individual partitions
I have a disk that I make regular backups of. The disk is made of a few different partitions. Previously I've been using the dd command to back up /dev/sdX all at once and it's all gone very well. I ...
- 101
1
vote
1
answer
55
views
dd stops copying mid-way through a CD
I got a couple of old disks I'm trying to copy over to a HDD. Some of the disks worked fine and I was able to copy them using just mount and rsync, however as is it with old disks, some of the disks ...
- 35
1
vote
1
answer
91
views
dd: `iflag=nocache` ineffective for the first usage
The flag iflag=nocache of dd seems to take effect only at the second time onwards.
I don't understand why.
I was thinking maybe it is related to readahead cache, but it doesn't explain the results in ...
- 2,375
1
vote
1
answer
25
views
what does dd's metadata mean in the context of a non-file write?
So apparently fdatasync is like fsync but doesn't sync the metadata, I can understand what the metadata refers to in case of like copying a file, but what if you are just zeroing a drive or writing ...
- 143
1
vote
0
answers
126
views
ZFS On Linux - recover lacked MBR after physical problem
We had a physical problem in our backup server (we suppose an electrical shock due to the thunderstorm). The system is Linux Debian on a disk and the storage in on a two 4 TB disks RAID-1 ZFS (ZFS On ...
- 135
1
vote
2
answers
202
views
how to duplicate rootfs to another partition (dd cause problems)
I've been using dd in order to duplicate my rootfs into another partition and boot the system using the new rootfs copy.
sudo dd if=/dev/disk/by-partuuid/342853ea-cab9-4a4f-b935-1a704c1cc17f conv=sync,...
- 159
1
vote
1
answer
72
views
Does not putting conv=fsync mean that if you shut down your computer the copy might not be complete?
So from what I know conv=fsync means that the cache is fully written before dd quits. But does that mean otherwise the writing is not complete when dd quits? Like if I shut down the computer (after dd ...
- 143
4
votes
3
answers
1k
views
Auto detect RAM and create double the swap memory
I'm able to auto detect RAM in GB as below and round off to the nearest integer:
printf "%.f\n" $(grep MemTotal /proc/meminfo | awk '$3=="kB"{$2=$2/1024^2;$3="GB";} 1' | ...
- 417
1
vote
1
answer
58
views
SSD performance different results in dd and gnome-disk
I wanted to measure my SSD read/write performance and found some recommendations for using dd and gnome-disks. (from my understanding, hdpart is not relevant since it's not using the disk itself but ...
- 159
1
vote
0
answers
504
views
cp and dd stuck when writing ISO image to USB flash drive
I have a hybrid ISO image (Debian netinst, size is around 473MB) which I tried to write to a USB flash drive on my Debian system. When I tried to do this with any of the following commands (sdX is the ...
- 11
1
vote
1
answer
255
views
Why not always use 'oflag=direct' in 'dd'?
Using oflag=direct in dd seems always to make operations much faster than either not using it, or using pv etc. to do the operation instead.
Under what circumstances might I not want to use oflag=...
- 3,853
16
votes
3
answers
5k
views
How can I verify that my hard disk has been zeroed / wiped?
I used shred to wipe my external hard disk:
sudo shred -vz /dev/sdb
I should also add that the disk had 5 bad sectors.
I want to verify the disk has been zeroed, per https://superuser.com/questions/...
- 447
24
votes
1
answer
2k
views
Why does `dd` use `=` in its parameters?
Why does dd use parameters with a = (if=/dev/zero) and not the more common approach of something like -i /dev/zero?
I'm just curious if someone knows the historical reason or what the pros and cons ...
- 341
1
vote
1
answer
194
views
Copying image with dd results in ext4 filesystem as unknown
I've been trying to make a backup of this raspberry pi SD card and I keep failing. I just needed it a little bit smaller so it would fit on another 32GB SD card that I have. After a lot of failures ...
- 113
0
votes
1
answer
80
views
Accidently dd to /dev/sda
I used Kali Linux, and I accidently used dd to /dev/sda while I'm trying to backup my files.
It completes after a few seconds with a total size about 10 - 20 mb. After my computer was rebooted, it ...
- 9
0
votes
1
answer
94
views
Why does dd create a 34MB file with bs=1G? [duplicate]
When typing the command:
dd if=/dev/urandom of=outputfile bs=1G count=1
I should receive a file at the size of 1GB if I understand correctly.
Yet, I receive a file of the following values:
33554431 ...
- 3
1
vote
0
answers
21
views
Differences in performance of HDD secure wipe commands
I am trying to secure wipe a USB connected HDD(not SSD) on macOS by overwriting it with random data.
For some reason, only diskutil is very fast among these commands.
Why is this?
diskutil
diskutil ...
- 11
0
votes
1
answer
184
views
Restore from Full Disk Image to Single Partition
For major version upgrades, Tails recommends this convoluted upgrade path where you write an image on a fresh USB drive, then clone the OS partition onto your original USB drive. I'm trying to figure ...
- 103
1
vote
2
answers
320
views
DD copy of DVD stops with read error at a fixed position
First of all: I know, that you need to unlock the access to a DVD and the drive by first performing a handshake, for example with VLC. Otherwise the ISO will be roughly 3.7 MB in this case and fail ...
0
votes
0
answers
361
views
iostat returns disk utilization greater than 100% while profiling a Beaglebone Black board
I need to profile the performance of software running on a BeagleBone Black (BBB). The BBB has an ARM Cortex-A8 up to 1GHz frequency, 512MB RAM, and 4GB eMMC onboard flash storage. You can find more ...
- 1
0
votes
0
answers
76
views
backup of ubuntu
I have Ubuntu installed on a single 1TB drive. What I want to do is a full backup of the system. I know I can use tar/gzip or rsync for most of the data, but I'm not sure what exactly do I need to do ...
- 111
0
votes
1
answer
65
views
Why does cat randomFile.txt mess up with my terminal? [duplicate]
I am simply generating a pseudo-random content file with.
dd if=/dev/urandom bs=1M count=1 of=./filetone.txt
To then cat it
cat ./fileone.txt
The issue is that upon completing with this command it ...
