Questions tagged [storage]

The tag has no usage guidance, but it has a tag wiki.

Filter by
Sorted by
Tagged with
144
votes
10answers
326k views

linux: How can I view all UUIDs for all available disks on my system?

My /etc/fstab contains this: # / was on /dev/sda1 during installation UUID=77d8da74-a690-481a-86d5-9beab5a8e842 / ext4 errors=remount-ro 0 1 There are several other disks on ...
73
votes
6answers
86k views

Clear unused space with zeros (ext3,ext4)

How to clear unused space with zeros ? (ext3,ext4) I'm looking for something smarter than cat /dev/zero > /mnt/X/big_zero ; sync; rm /mnt/X/big_zero Like FSArchiver is looking for "used space" ...
31
votes
9answers
176k views

Finding all storage devices attached to a Linux machine

I have a need to find all of the writable storage devices attached to a given machine, whether or not they are mounted. The dopey way to do this would be to try every entry in /dev that corresponds ...
29
votes
7answers
19k views

How can I use DD to migrate data from an old drive to a new drive?

I am upgrading the internal SATA hard drive on my laptop from a 40G drive to a 160G drive. I have a Linux/Ubuntu desktop which has a SATA card. I would actually like to do the same thing for a couple ...
21
votes
4answers
34k views

What is the best practice for adding disks in LVM

According to the Linux manpages you can add raw disks as well as partitions to a volume group. In other documentation (RedHat, CentOS or openSUSE), all examples refer to adding partitions to the VG ...
20
votes
3answers
20k views

How can I check the condition of an SD card?

How can I check the health condition of an SD card? When I want to check a hard drive, I can use SMART, how should I check an SD card? Is there a universal approach?
18
votes
1answer
9k views

How do pdflush, kjournald, swapd, etc interoperate?

Recently saw a question that sparked this thought. Couldn't really find an answer here or via the Google machine. Basically, I'm interested in knowing how the kernel I/O architecture is layered. For ...
18
votes
2answers
10k views

Defragmentation of LVM2 logical volumes

Question: Is there a tool supporting de-fragmentation of LVM2 logical volumes? (Making their extents into consistent sequences) Preferably by defining desired order of extents (something like "...
14
votes
2answers
6k views

How to properly unplug/plug removable LVM2 device?

How to properly unplug (remove) and later plug (insert) removable (hot-plug, usb etc) storage device with LVM2 on it ? I've found this note Remove (unplug, eject): lvchange -an /dev/...
11
votes
5answers
2k views

Check data integrity after copying thousands of files [duplicate]

I copied thousands of files into an exFAT MicroSD card. The number of files and bytes is identical, but how do I know whether the data is corrupt or not? It would be good if the JackPal Android ...
11
votes
2answers
14k views

Why is sync so important when making a bootable linux usb stick?

From the Arch Linux Wiki: https://wiki.archlinux.org/index.php/USB_flash_installation_media # dd bs=4M if=/path/to/archlinux.iso of=/dev/sdx status=progress && sync [...] Do not miss ...
11
votes
5answers
4k views

how do I wipe a server that I don't have physical access to?

I have a Godaddy dedicated server that I would like to cancel. Before I do that I'd like to do a clean format on the server to make sure that the next person who gets the server isn't able to undelete ...
11
votes
1answer
19k views

Understand output of `udevadm info -a -n /dev/sdb`

I plug an external HDD into my laptop. I try to find out the information by $ udevadm info -a -n /dev/sdb Udevadm info starts with the device specified by the devpath and then walks up the chain of ...
11
votes
3answers
3k views

Ubuntu - No space left on device (NTFS)

I have a server which stores a lot of data, I keep many nested directories with millions of files in them. While trying to move one big directory (~700GB) from one disk to another I get an error: mv ...
10
votes
1answer
9k views

Move qcow2 image to physical hard drive

I have a qcow2 image that I use to boot an installation of linux using QEMU; I would like to place the contents of that image directly onto a physical hard drive so that I can boot to that linux ...
9
votes
2answers
7k views

Clear unused space with zeros (btrfs)

How to clear unused space with zeros ? (I am asking for tool for btrfs filesystem) I'm looking for something smarter than cat /dev/zero > /mnt/X/big_zero ; sync; rm /mnt/X/big_zero Like ...
9
votes
1answer
5k views

Is it possible to change the sector size of a partition for zfs raidz pool in linux?

I've been migrating a zfs raidz pool on linux to new discs via virtual devices which were sparse files. I've used partitions on the discs as the discs are different sizes of 1.9T each. The last disc ...
8
votes
3answers
6k views

Resume interrupted copying process

How to resume securely and reliably process of copying file $A into backup location $B done with pv "$A" > "$B" or cat "$A" > "$B" ? (let's assume file $A is very big, e.g. LVM2 snapshot file) ...
8
votes
2answers
3k views

How do I refresh the magnetic state on a disks with backups?

I have a couple of large disks with backup/archive material on them. They're ext4. Regarding the ones of those that will be stored for a couple of years without reading the whole disc again I've been ...
8
votes
1answer
1k views

Can I make a USB port on my Linux computer look like a mass storage device?

I'm running Kubuntu 12.04 on my PC. I want to make one USB port on my computer appear as if it is a USB mass storage device to other devices. In other words, something connected to this USB port will ...
8
votes
1answer
265 views

How to re-size partitions in a complete hard drive image?

I have taken a complete image of a hard drive using: dd if=/dev/sda of=/home/user/harddriveimg bs=4M It would seem to me, that I should be able to re-size the partitions within it after suitably ...
7
votes
1answer
646 views

Volume Management: How to move space from one partition to another?

I am setting up a redhat ec2 instance and by default the software I am using (called qradar) created the following volumes on the two 500g ebs storage devices attached to the instance: $ lvs LV ...
6
votes
3answers
7k views

How to resize a VirtualBox HDD image

I initially created a dynamically expanding VirtualBox image, and have tried to increase its hard drive space: VBoxManage modifyhd "VirtualBox VMs/JHBuild/JHBuild.vdi" --resize 16384 After that, I ...
6
votes
2answers
8k views

Why can I not see some files in my USB stick on Windows that I can see on Linux?

I'm having a little problem with my USB stick. There are files and folders on the USB stick I can see and manipulate on my Linux machine (Fedora 20) but some of those files and folders appear nowhere ...
6
votes
1answer
13k views

“Found duplicate PV”

SERVER:~ # pvs Found duplicate PV Wb0b2UTCKtpUtSki0k2NnIB24qNj4CEP: using /dev/mapper/36005076304ffc2500000000000004903 not /dev/mapper/36005076304ffc2990000000000004903 PV ...
6
votes
3answers
3k views

FreeBSD 10 with Root-on-ZFS using only a part of the disk

I'm having a bit of trouble here. I'll try and be as concise as possible in describing my objective: I'm setting up a storage server for my employer, using two Intel DC S3500 series 120GB solid state ...
6
votes
1answer
285 views

How to restore an LVM container to a bigger physical partition?

The setup is an encrypted LVM container/PV that holds a few ext4 partitions. I am moving these to a new machine, but there is more room on the disk. So, I have created a bigger encrypted LVM ...
5
votes
3answers
5k views

ReFS but for Linux

Microsoft is going to bring a "Resilient FileSystem" with Windows 8, but only for servers. I'd like that on a Linux desktop, but my search reveals no contender. There are so many filesystems for ...
5
votes
3answers
284 views

RAID 1 in linux

I'd like to setup RAID 1 with two drives in linux. I don't have a hardware RAID controller and don't want to buy one, so what is the best software RAID option for linux? Note: If it makes a ...
5
votes
5answers
158 views

How do you layout extra storage? [closed]

When I was doing dual boot, all my extra storage was simple in /win/d, /win/e, /win/f, ... and formatted NTFS. Now the desktop only runs Windows in a VM, and I access all my partitions from samba (...
5
votes
3answers
34k views

Mount same disk to two linux machines - SAN/NAS

I do not know anything from storage and filesystems that's why i have created this question. I want to mount the same disk to two linux machines and both machine must be able to read/write on the ...
5
votes
2answers
493 views

Does using full-disk encryption affect the probability of losing data in case of storage errors?

So.. if there is a bad block on the HDD, what can survive better? not using FDE (full disc encryption) using FDE - since the whole disk has "1 partition" - the encrypted one, isn't it more likely to ...
5
votes
2answers
104 views

Report how much space is used and available in storage in ZFS on FreeBSD

When using ZFS on FreeBSD, how can we see how much space is used and how much is available in storage drives (hard disk, SSD, etc.)? For example, spinning up a virtual machine instance of FreeBSD on ...
5
votes
2answers
635 views

Can you identify transport via sysfs?

I'm writing a script, and I'm interested in being able to identify the transport class (fc - "fibre channel", scsi, iscsi, etc.) for a given block device. I can retrieve this information via ls -l /...
5
votes
4answers
5k views

Compress and remove original directory with limited storage

On my 3 TB external hard drive, I have a 2.7 TB directory containing relatively small files. I would like to compress this 2.7 TB directory and remove it to keep only the compressed version. The issue ...
5
votes
1answer
952 views

What are the implications of using an inode_ratio of 16384 in terms of storage use on ext4?

I'm assuming that this means that if the average file stored (including directories etc) is less than 16384 bytes, it may be possible to run out of inodes before using the full storage capacity of the ...
5
votes
0answers
4k views

`crontab -e` sometimes fails with “Creation of temporary crontab file failed”

From time to time crontab -e fails on me on a Ubuntu box. It is the same for all users including the root and crontab -eu <user> under the root. Goes like this on a regular basis, a success here ...
4
votes
1answer
309 views

rsync move files, leave directories?

I'm using rsync to copy data between my internal and external hard drives. It's done what I need, but I'm running out of room on my internal hard drive. So, is there any way to get rsync to function ...
4
votes
1answer
354 views

Disc light comes on when pressing keys in Iceweasel Preferences? Is this malevolent?

On Linux,I'm using the Iceweasel web browser (it's effectively Firefox), and in Edit/Preferences/General/Startup, when I type a URL into the "Home Page" box, the hard disc light comes on for EVERY ...
4
votes
1answer
1k views

What are potential consequences of going nuts with sparse file based vdevs for a LVM to ZFS migration?

I've been playing around with ZFS having finally accepted that it's mature enough I shouldn't get burned. Now to migrate the home NAS box which is currently an LVM JBOD set up that's quite full, but ...
4
votes
1answer
18k views

extending a partition by resize2fs

I have a 4 GB SD card. Before the image load root@ubuntu# fdisk -l Disk /dev/sdb: 3965 MB, 3965190144 bytes 49 heads, 48 sectors/track, 3292 cylinders, total 7744512 sectors Units = sectors of 1 * ...
4
votes
2answers
4k views

tmpfs usage and resizing

I am trying to understand the stuff. I have a machine with 80G storage. It looks like that: Filesystem Size Used Avail Use% Mounted on /dev/mapper/centos-root 50G 7.1G 43G 15% /...
4
votes
1answer
3k views

Convert a single-drive LVM volume to a striped volume across 3 drives

On my server using LVM, I have a simple linear LV on a single drive (PV). Now, I added 2 more (same size) drives (PVs) to the server. I want to convert my existing linear LV to a striped LV (RAID0 ...
4
votes
1answer
2k views

Protecting data against bit rot

I have realized that I need to protect all of my photographs against bit rot (file corruption occurring at random due to errors in hard drives or network transfer). I recently discovered par2 which ...
4
votes
1answer
1k views

How does ZFS grow/scale?

I am wondering how would one design a ZFS system to build something that is massively scalable, along the lines of your favorite cloud storage, but with all the neat benefits the come with ZFS. A ...
4
votes
3answers
1k views

slow media - disk cache tuning?

Is there any way to make the system more consistent when using LUKS? (or slow storage in general) as it is everything is snappy until the write buffer is full, then everything grinds to a halt as the ...
4
votes
1answer
291 views

ZFS on Linux: cannot destroy snapshot, dataset is busy

I'm trying to delete a bunch of old ZFS snapshots but I get errors saying that the datasets are busy: [root@pool-01 ~]# zfs list -t snapshot -o name -S creation | grep ^pool/nfs/public/mydir | xargs -...
4
votes
3answers
803 views

Store a custom serial number on a USB flash drive, immune to overwriting

I'm working with massive amounts of USB flash memory. I'd like to be able to track events regarding the USB drives. Specifically: I perform f3 tests, format the drives and write data to them. I'm ...
4
votes
2answers
3k views

Parallel vs Distributed vs Traditional File system

I am trying to understand the differences between these three file system at a very basic level. Distributed FS: HDFS Parallel FS : Lustre Traditional FS : ext4/ext3/ NTFS/FAT etc. I want to ...
4
votes
1answer
1k views

Something like VMware “Changed Block Tracking” (CBT) in linux

I'm wondering if is there any possibility to track changed blocks in linux with an ext3/ext4 filesystem (or a specific one supporting it). I mean a feture like the Changed Block Tracking (CBT) ...