I just backed up the microSD card from my Raspberry Pi on my PC running a Linux distro using this command:

dd if=/dev/sdx of=file.bin bs=16M

The microSD card is only 3/4 full so I suppose there's a few gigs of null bytes at the end of the tremendous file. I am very sure I don't need that. How can I strip those null bytes from the end efficiently so that I can later restore it with this command?

cat file.bin /dev/zero | dd of=/dev/sdx bs=16M
  • 2
    Stripping off a few bytes affects the physical size of the file, which may cause problems when trying to mount it or write it to a device, and I don't recommend it. I strongly recommend compressing it instead. – Mukesh Sai Kumar Jan 13 '18 at 8:41
  • 4
    I would't be so sure about them all being "null bytes", the card contains at least one filesystem, probably two, and certain structures may be spread over the disk. Why not compress file.bin and use zcat when a restore is needed? – Gerard H. Pille Jan 13 '18 at 8:44
  • I don't think file layout is an issue given that I have provided the way I would restore the image. – iBug Jan 13 '18 at 9:07
  • 1
    The point is that you suppose there are null bytes at the end of the file and you are not sure whether they are actually null bytes or how large those null bytes occupies. It is not impossible that the last byte is not a null byte, depending on various conditions. – Weijun Zhou Jan 13 '18 at 9:20
  • @WeijunZhou Yep, you're right. I'm not sure how much is there nor am I sure if it exists. – iBug Jan 13 '18 at 9:22

To create a backup copy of a disk while saving space, use gzip:

gzip </dev/sda >/path/to/sda.gz

When you want to restore the disk from backup, use:

gunzip -c /path/to/sda.gz >/dev/sda

This will likely save much more space than merely stripping trailing NUL bytes.

Removing trailing NUL bytes

If you really want to remove trailing NUL bytes and you have GNU sed, you might try:

sed '$ s/\x00*$//' /dev/sda >/path/to/sda.stripped

This might run into a problem if a large disk's data exceeds some internal limit of sed.

  • I finally changed my backup command to this: pv -s $SIZE /dev/sdx | gzip > file.bin.gz where $SIZE is obtained by other means. – iBug Jan 13 '18 at 13:30
  • Anyway, the GNU sed solution is a good one if I later want to work with some small files. – iBug Jan 13 '18 at 15:56
  • Note that if you forget the '<' parameter in gzip it will might try to do some dangerous operation! – Luciano Andress Martini Jan 15 '18 at 11:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.