Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I'm trying to dd an .img file over to my micro sd (8 GB). The rom (.img) is 2.66 GB, but in my output of DD (live) it says its transfered 9 GB. I don't get how it's turning a 2 GB image into 9 GB. I'm running OSX Lion and using a Belkin card read. The card reader is not the problem, I have used disk utility with it with no problem. I think the problem is dd or my card (I hope it's not my card :( ).

EDIT:Here is the command I use to copy the file

dd if=honey.img of=/dev/disk1 bs=1m

This is the code I use to check the status:

killall -INFO dd
share|improve this question
    
Please add the command line you used to your question. –  Kyle Jones Apr 29 '12 at 2:03
    
Done, any ideas? –  Sirens Apr 29 '12 at 2:05
1  
Nope. Run ls -ls honey.img and show the output of that. –  Kyle Jones Apr 29 '12 at 2:50
1  
stat honey.img could be more interesting than ls. –  Mat Apr 29 '12 at 7:12
1  
The output of ls is 234881026 8588302 -rw-r--r-- 1 ***** staff 0 3670016000 "Apr 28 21:49:50 2012" "Feb 7 19:11:05 2011" "Apr 28 19:11:02 2012" "Feb 7 19:11:05 2011" 4096 7168000 0 honey.img –  Sirens Apr 29 '12 at 16:08
show 3 more comments

2 Answers

up vote 2 down vote accepted

The behavior you describe and the nature of the file make me suspect that this is a sparse file. Sparse files are a primitive compression technique, in which large sequences of null bytes in a file are not stored on the disk. Here is an example where I create a sparse file:

$ echo a | dd seek=999999999 >sparse
0+1 records in
0+1 records out
2 bytes (2 B) copied, 6.614e-05 s, 30.2 kB/s
$ ls -l a
-rw-r--r-- 1 gilles gilles 511999999490 Apr 30 00:03 sparse
$ du sparse
16      sparse

The file sparse contains 511999999490 bytes (999999999 blocks of 512 bytes, all zero, plus the two bytes a followed by a newline). Yet the total disk space used by the file is 16kB (4kB for the final block, and 3 other blocks containing only metadata related to the location of the other blocks — all of them absent).

If honey.img is a disk image which was created carefully enough, it may be sparse where the disk had unused space.

When you read from a file, there is nothing to mark it as sparse. So if honey.img is a large disk image, dd may be reading gigabyte upon gigabyte containing only null bytes.

Running ls -l and du on the file (or, on OSX, ls -ls) would show the number of bytes and the number of blocks used for storage. If the bytes wouldn't fit in the number of blocks, the file is sparse. As I write, you haven't posted legible data that could confirm or infirm this.

The one tool I know on OSX that can copy sparse files efficiently is rsync. However, what you're doing here is not copying a file from one filesystem to another, but copying a byte stream (that happens to come from a file) onto a disk. You can only do this if the data actually fits on the target disk.

share|improve this answer
    
ls -l reports the apparent size of the file ( counting the sparse space ), not how much disk space it is actually using, as your example clearly shows. –  psusi Apr 30 '12 at 13:45
    
or, on OSX ls -ls? Why not on every unix since the beginning of time ls -ls? minnie.tuhs.org/cgi-bin/utree.pl?file=V1/man/man1/ls.1 –  Wumpus Q. Wumbley Jun 22 '13 at 22:01
add comment

I ran into a very similar problem trying to transfer a Debian live binary.img to a 2GB usb flash drive. The image size was listed as 1.6GB. Matt's comment above regarding using stat was very helpful and after calculating the block size at 4096 times the number of blocks (3,109,040), it puts the size at 12,734,627,840 which is nearly 8 times the 1.6GB size listed. I noticed that on the dd command I was using the bs=1M also, so I retried with the bs=4096 as listed in the stat command, and it copied the file as originally expected at 1.6GB. So my impression is that if you copy bigger chunks than the block size, it just copies block by block. But if you copy at the block size (and possibly smaller, although untested) dd evaluates what is inside the block and trims superfluous garbage. Thanks for the tip Matt.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.