Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I have a situation where I need dynamically expandable storage space on a Linux machine running in Amazon EC2, and ZFS came to mind. If I remember properly, ZFS supports a way to dynamically expand a non-redundant JBOD-like array.

I have a few requirements for a project I'm working on:

  1. I should be able to move the array to another EC2 instance if need be.
  2. I should be able to dynamically expand the ZFS array, adding new "drives" to the ZFS volume without any downtime.

Is this possible? If ZFS provides the ability to dynamically expand the array, I should be able to run a script which would dynamically create new EBS volumes, attach them to the EC2 instance, and have ZFS add them to its pool, all dynamically without downtime.

share|improve this question

2 Answers 2

up vote 5 down vote accepted

ZFS on Linux does not have a critical mass following, whatever that may be. ZFS will effectively lock you in. The underlying format is incompatible with rescue disks, and distributions for which you will find ZFS is rare.

I would tend to overlook these limits due to my bias towards ZFS. You may want to google for a service provider that provides SmartOS/ZFS based Cloud housting.

Linux file systems ext3/ext4 and probably various others do allow for dynamic growing. The volume manager built into Linux, LVM, also allows you to dynamically expand volumes onto new disks. For anything other than experimentation this would have to be the current best Linux recommendation, though that would change once ZFS got a critical mass following on Linux.

The process for Linux + LVM + ext3 is:

Example: LVM volume group myvg, mounted volume name uservol1, and disk device in Linux is /dev/sdf

  1. Allocate the disk to the VM. (in Amazon Management Console, Create the EBS volume, write down its ID, and allocate it to the instance)
  2. The EC2 instance should have some udev rules for creating the device node. So you should see a new disk in /dev/sd* ... Log in on the instance and check that the EBS volume is visible, eg fdisk -l /dev/sdf, cat /proc/partitions, run blkid.
  3. Create partition table if needed: fdisk / sfdisk
  4. Initialize for LVM use: pvcreate /dev/sdf
  5. Add the disk (physical volume) to the LVM volume group vgextend myvg /dev/sdf
  6. Grow the Volume size: lvextend -L +1024G /dev/myvg/uservol1
  7. Grow the ext3/ext4 file system: resize2fs /dev/myvg/uservol1
  8. Check (df -h) and you should see that the mounted file system now have more space.

QED.

share|improve this answer
    
Is point one possible? IE: With LVM, can I migrate the array to another EC2 instance? –  Naftuli Tzvi Kay Apr 4 '13 at 17:25
    
Yes. Use the Amazon Control Panel to allocate all the disks in the volume group to the new instance. –  Johan Apr 5 '13 at 10:15
    
The process is well documented, you should export it, and you must import it on the new host/instance. I just found this procedure: tldp.org/HOWTO/LVM-HOWTO/recipemovevgtonewsys.html –  Johan Apr 5 '13 at 10:17
1  
First, you can just as easily install ZFS in a rescue environment as you can on a production one. Second, Debian, Ubuntu, Fedora, Arch, Gentoo, Funtoo, Sabayon, CentOS, and RHEL all support installing ZFS on Linux and also support installing packages in the rescue environment. Further, I would be vary careful for advocating LVM. Not only does it provide a stacked block device setup, it's also non-trivial to administer. Resizing filesystems is certainly easier than flat partitions, however. serverfault.com/questions/279571/lvm-dangers-and-caveats is a good read. –  Aaron Toponce May 7 '13 at 15:36
    
Non-trivial to administer is not much of an argument. Once you learned how to do something it is easy enough. But I get all your points regarding the ability to use and set up ZFS. I am a self-confessed Solaris bigot, and a ZFS-on-Linux user, and even installed it on a large production system recently. I love ZFS. But I don't think it is a good fit for everyone! –  Johan May 8 '13 at 10:10

Yes, you can most definitely auto expand the array with ZFS. You just need to set the following property:

# zpool set autoexpand=on pool
# zpool set expandsize=on pool

These properties will give you two benefits.

  1. As you add any VDEV to the pool, it will be automatically expanded
  2. As you replace 1TB drives with 2TB drives, when all drives are replaced, the pool will automatically resize to accommodate the new drive sizes.

There are a couple of caveats:

  1. You cannot remove a drive nor can you remove a VDEV from the pool. So, while you can grow your pool, you cannot shrink it. However, you can detach a drive from a mirror only.
  2. All drives should be the same specifications: same size, same speed, etc. Same make and model is not necessary.
  3. All VDEVs should be the same VDEV. If you have one RAIDZ in your pool, then all VDEVs should be RAIDZ.
  4. All VDEVs should be the same size. If one VDEV is 2TB. All VDEVs should be 2TB.
share|improve this answer
    
Don't particularly points 2 in these lists contradict each other? –  Michael Kjörling Jun 17 '13 at 12:07

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.