9

I have a zpool called storage that contains a five device raidz1 array.

Today I went and bought another 3TB device and put it in my enclosure. However, instead of creating a new pool and adding that device to it, I made a mistake and added it to my existing storage pool.

Now I have a top-level device that I want to remove called sdg (that's the new drive). Every time I try to remove it I get:

cannot remove sdg: only inactive hot spares, cache, top-level, or log devices can be removed.

So how do I remove this device now? If this device fails, my entire pool will be unavailable. I'm thinking I should go buy another drive and at least it will be mirrored, but I just can't believe there isn't a proper way to do this.

This is my status dump:

pool: storage
state: ONLINE
scan: scrub canceled on Wed Jul 23 17:26:08 2014

config:

 NAME                                 STATE     READ WRITE CKSUM
 storage                              ONLINE       0     0     0
   raidz1-0                           ONLINE       0     0     0
     ata-ST3000DM001-1CH166_Z1F1PYM6  ONLINE       0     0     0
     ata-ST3000DM001-1CH166_W1F24CSC  ONLINE       0     0     0
     ata-ST3000DM001-1CH166_W1F2372R  ONLINE       0     0     0
     ata-ST3000DM001-1CH166_W1F24BTK  ONLINE       0     0     0
     ata-ST3000DM001-1CH166_Z1F2KKLW  ONLINE       0     0     0
   sdg                                ONLINE       0     0     0

errors: No known data errors
Improve this question
10
  • 1
    My understanding is that sdg is now part of the striped storage pool. Because the pool is striped rather than mirrored, it is impossible to remove sdg from the pool. The only way to get it back is to copy your data from the storage pool to another device, destroy the pool and recreate it without sdg. Then create a new pool for sdg to do whatever you wanted to use it for.
    – Warwick
    Jul 24, 2014 at 1:13
  • Everything I read in the zfs documentation said that a stripe could not grow like that. In fact, people have come up with some rather interesting things to do so that you can increase the same array.
    – chronic
    Jul 24, 2014 at 1:29
  • If I can't remove it, can I add a couple more drives and make another stripe out of sdg and the two new drives?
    – chronic
    Jul 24, 2014 at 1:29
  • Once you added the drive to the pool, you can't remove it.
    – Ask and Learn
    Jul 24, 2014 at 1:33
  • 1
    I don't know for sure, but I don't think that you are able to add a couple more drives and make another stripe out of sdg and the two new drives. I believe that the sdg would have had to be specified as a striped volume with the two extra disks when added to the pool. There is no zfs convert functionality.
    – Warwick
    Jul 24, 2014 at 1:48

2 Answers 2

Reset to default
3

Unfortunately, removing a vdev (which this drive is) from a pool is not supported by ZFS currently (but there is work ongoing to support it). The work around would be to dump your data out of the pool and recreate it, then put the data back.

Improve this answer
0

Follow-up: for those who are reading this since then, this is possible at least from FreeBSD 12.2-RELEASE (however, man page for zpool doesn't mention this in the remove section in 12.2, however zpool remove allows the removal on non-redundant top vdevs too, if the pool has enough freespace).

Improve this answer
2
  • 2
    This was interesting, so to test I made a 4-device RAIDZ1 vdev, then concatenated a non-redundant 5th vdev. Trying to remove the non-redundant vdev failed: cannot remove md4: invalid config; all top-level vdevs must have the same sector size and not be raidz. I destroyed that, and created a new tank: a 2-drive mirror, concatenated with another 2-drive mirror, concatenated with a non-redundant fifth device. That allowed me to remove the fifth device, then work backward to detach/remove the second mirror, etc. It appears any top-level RAIDZ is a deal-breaker. FreeBSD 13.0.
    – Jim L.
    Nov 13, 2021 at 0:04
  • This has been my experience. I was rolling mirrors this whole time and was surprised my new RAID-Z2 vdevs couldn't be removed.
    – Sawtaytoes
    Oct 17 at 3:06

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .