Hot answers tagged

78

Such an utility is zerofree. From its description: Zerofree finds the unallocated, non-zeroed blocks in an ext2 or ext3 file-system and fills them with zeroes. This is useful if the device on which this file-system resides is a disk image. In this case, depending on the type of disk image, a secondary utility may be able to reduce the size of the disk ...


47

If one is interested only in block storage devices, one can use lsblk from widely-available util-linux package: $ lsblk -o KNAME,TYPE,SIZE,MODEL KNAME TYPE SIZE MODEL sda disk 149.1G TOSHIBA MK1637GS sda1 part 23.3G sda2 part 28G sda3 part 93.6G sda4 part 4.3G sr0 rom 1024M CD/DVDW TS-L632M It lends itself well to scripting with many ...


34

The best command to use is lsblk -f. It will list all the devices and partitions, how they are mounted (if at all) and the tree structure of the devices in the case of using LVM, crypto_LUKS, or multiple volume groups on the same drive.


33

Summary of the methods (as mentioned in this question and elsewhere) to clear unused space on ext2/ext3/ext4: Zeroing unused space File system is not mounted If the "disk" your filesystem is on is thin provisioned (e.g. a modern SSD supporting TRIM, a VM file whose format supports sparseness etc.) and your kernel says the block device understands it, you ...


21

Before we discuss the specifics regarding pdflush, kjournald, andkswapd`, let's first get a little background on the context of what exactly we're talking about in terms of the Linux Kernel. The GNU/Linux architecture The architecture of GNU/Linux can be thought of as 2 spaces: User Kernel Between the User Space and Kernel Space sits the GNU C Library (...


19

RHEL6 LVM Admin Guide According to the RHEL 6 Logical Volume Administration Guide it's recommended that if you're going to use an entire drive as a physical volume in a LVM volume group, that you should still partition it: excerpt from the guide "RHEL6 Logical Volume Manager Administration LVM Administrator Guide" 2.1.2. Multiple Partitions on a Disk ...


19

Using MD5 sums is a good way, but the canonical way to use it is: cd to the directory of the source files and issue: md5sum * >/path/to/the/checksumfile.md5 If you have directories with many levels, you can use shopt -s globstar and replace * by **/*. Notice that the file specs in the MD5 file are exactly as provided in the command line (relative ...


18

The dd does not bypass the kernel disk caches when it writes to a device, so some part of data may be not written yet to the USB stick upon dd completion. If you unplug your USB stick at that moment, the content on the USB stick would be inconsistent. Thus, your system could even fail to boot from this USB stick. Sync flushes any still-in-cache data to the ...


15

sfill from secure-delete can do this and several other related jobs. e.g. sfill -l -l -z /mnt/X UPDATE #1 There is a source tree that appears to be used by the ArchLinux project on github that contains the source for sfill which is a tool included in the package Secure-Delete. https://github.com/BlackArch/secure-delete Also a copy of sfill's man page ...


13

You don't have to export the VG, that's used to migrate a VG from one system to another. Simply vgchange -an vgname to deactivate all logical volumes on the volume group you wish to unplug. Later, after plugging the device back in, vgchange -ay vgname will reactivate all logical volumes in your vgname VG and then you can mount LVs and use. Device ...


13

You could trawl through the output of lshw and extract details about devices in the disk or tape class (and maybe others - storage class gives you details on storage controllers, scsi, sata, sas, etc). e.g. lshw -class disk -class tape The -short option gives a nice compact summary. e.g. on my home zfsonlinux server/workstation/experiment-box (no tape ...


12

Yes you can use dd to skip the blocks. A="file1" B="file2" BLOCKSIZE=512 # default bs for dd size_b=$(stat -c "%s" "$B") skip_blocks=$((size_b / BLOCKSIZE)) dd if="$A" of="$B" skip=$skip_blocks seek=$skip_blocks bs=$BLOCKSIZE The important parameters here are skip as well as seek: skip: skip BLOCKS ibs-sized blocks at start of input seek: skip BLOCKS ...


12

Unmount, eject, and remount the device. Then use diff -r source destination In case you used rsync to do the copy, rsync -n -c might be very convenient, and it is nearly as good as diff. It doesn't do a bit-for-bit comparison though; it uses an MD5 checksum. There are some similar answers with other details at: Verifying a large directory after copy ...


11

The walk is over the different software components (drivers) that handle the device; this corresponds by and large to the hardware devices and buses that are involved in connecting to the device. This is mostly unrelated to the physical arrangement of the devices: most of them are inside the same chip anyway. Taking this example from the top: First we have ...


10

This works for me: ls -la /dev/disk/by-uuid If you want to check what type the partition is, use: df -Th and it will show you if you have ext3 or ext2. Today it helped me because there was a formatted ext2 partition and I thought it was ext3, which was causing the mount to fail.


10

You want rsync: rsync -a --append "$A" "$B"


10

It's preferable to have some commonly recognized descriptors (meta-data) and MBR does quite stand as such a descriptor. Even GPT uses old MBR-based partition table to indicate its presence. Indeed you lose some diskspace but it's rather negligible meanwhile advantage of understanding what's on the disk (and where) is self-evident.


9

(Old question, but useful info for those searching) If you want to fully test an SD card (destructively) you can check the entire data space with the F3 tools which have been ported to Linux They let you write a variety of patterns to the disk and then check to see if there are any failures. You could also use badblocks, but badblocks uses repeating ...


9

QEMU comes with the qemu-img program to convert between image formats. qemu-img convert -f qcow2 -O raw my-qcow2.img /dev/sdb


9

I've used LVM2 defragmenter before. It is not the simplest or best written, but it does the trick.


9

“Input/output error” indicates something that shouldn't happen somewhere in the input/output stack. The intended meaning is a hardware failure: your hard disk is failing. Often this error turns up in cases where the problem is a software bug or a misuse of a special-purpose filesystem. But here, you're reading from a system directory, so a hardware failure ...


8

To only get the UUID of a specific disk device (for example to be used in a script) you can use: sudo blkid -s UUID -o value /dev/sdXY where /dev/sdXY is the name of the device.


8

If you have e2fsprogs 1.42.9, then you can use e2image to create the partition image without the free space in the first place, so you can skip the zeroing step.


8

Generally you can't really refresh the whole disk without reading/writing all of it. fsck is unlikely to provide what you need - it works with the file system not the underlying device hence it mostly just scans file system meta data (inodes and other file system structures). badblocks -n might be an option to dd if=X of=X. In any case you probably want to ...


8

UPDATE So, I really didn't think I would be researching NTFS this morning, but, thanks mostly to @AndrewMedico's comments below, I learned something. The truth is file streams are weird, and they confuse me, but apparently it gets deeper. Behaving in a way very like NTFS file streams, Transactional NTFS commits file changes to some alternate cache until ...


7

If you're looking for advanced filesystems for general-purpose computers in the Linux world, there are two candidates: ZFS and BTRFS. ZFS is older and more mature, but it's originally from Solaris and the port to Linux isn't seamless. BTRFS is still under heavy development, and not all features are ready for prime time yet. Both filesystems offer per-file ...


7

No, having Snap images which consume 100% of their filesystem is perfectly acceptable. In fact, it's supposed to work that way. A snap is a squashfs file carrying content and a bit of metadata that tells the system how to manipulate it. - https://docs.snapcraft.io/snaps/metadata Because Snap uses SquashFS, which is a compressed read-only filesystem, the ...


7

If you're just looking to copy an SD card exactly from one to another then you can do so with dd on the command line. You should NOT do this from your raspberry pi from it's own OS. This is because the OS may write to the SD card while copying and corrupt the copy. To copy an sd card, plug both into your two readers (it doesn't matter whether or not they ...


6

lsblk -o +uuid,name You can see all the outputs that can be added to the -o (--output) with lsblk --help Also this will do the job # blkid


6

There is a special option in rsync: --remove-source-files This tells rsync to remove from the sending side the files (meaning non-directories) that are a part of the transfer and have been successfully duplicated on the receiving side. Note that you should only use this option on source files that ...


Only top voted, non community-wiki answers of a minimum length are eligible