How to replace a disk in a non-redundant ZFS pool?

Question

I've been doing a bit of reading, and it looks like ZFS doesn't like disks being removed from non-redundant arrays:

You can use the zpool detach command to detach a device from a mirrored storage pool. For example:

# zpool detach zeepool c2t1d0

However, this operation is refused if there are no other valid replicas of the data. For example:

# zpool detach newpool c1t2d0
cannot detach c1t2d0: only applicable to mirror and replacing vdevs

The basic problem is understandable: removing the only copy of a piece of data (whether metadata or payload data) from an array would render that data unavailable.

The examples for replacing devices in a ZFS storage pool give a basic step-by-step description for how to replace a device in a storage pool: offline the disk, remove the disk, insert the replacement disk, run zpool replace to inform ZFS of the change and online the disk. This obviously requires that the array does not depend on the disk being replaced, hence the array must have redundancy; if it does depend on the drive in question, this approach presents the same problem as above.

What is the recommended way of replacing a disk in a non-redundant ZFS array?

Assume that the existing disk is working properly, and assume that the replacement disk is at least the same size as the disk being replaced. (If the existing disk has failed, clearly all one could do is add a new disk and restore all files affected by the disk failure from backup.)

Answer 1

zpool attach pool old-disk new-disk

That will convert old-disk into a mirror with both old-disk and new-disk and start resilvering after which you can detach the old disk.

zpool detach pool old-disk

Tested with loop devices:

# truncate -s 1G a b
# truncate -s 1200M c
# losetup  /dev/loop0 a
# losetup  /dev/loop1 b
# losetup  /dev/loop2 c
# zpool create test loop{0,1}
# zpool attach test loop1 loop2
# zpool status
      pool: test
 state: ONLINE
  scan: resilvered 62K in 0h0m with 0 errors on Wed Jul 31 13:43:19 2013
config:

        NAME        STATE     READ WRITE CKSUM
        test        ONLINE       0     0     0
          loop0     ONLINE       0     0     0
          mirror-1  ONLINE       0     0     0
            loop1   ONLINE       0     0     0
            loop2   ONLINE       0     0     0
# zpool detach test loop1
# zpool status
  pool: test
 state: ONLINE
  scan: resilvered 62K in 0h0m with 0 errors on Wed Jul 31 13:43:19 2013
config:

        NAME        STATE     READ WRITE CKSUM
        test        ONLINE       0     0     0
          loop0     ONLINE       0     0     0
          loop2     ONLINE       0     0     0

Answer 2

Don't know if things were that different in `13 but 'zfs replace' works on non-redundant pools. You just run the 1 command instead of detaching first.

d1 is 1G, d2 is 2G, both are empty files in /tmp:

/t/test #> zpool create test /tmp/test/d1
/t/test #> zpool set autoexpand=on test
/t/test #> zpool status
  pool: test
 state: ONLINE
  scan: none requested
config:

    NAME            STATE     READ WRITE CKSUM
    test            ONLINE       0     0     0
      /tmp/test/d1  ONLINE       0     0     0

errors: No known data errors
/t/test #> zpool list
NAME    SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
test   1008M   258K  1008M         -     0%     0%  1.00x  ONLINE  -


/t/test #> zpool replace test /tmp/test/d1 /tmp/test/d2


/t/test #> zpool status
  pool: test
 state: ONLINE
  scan: resilvered 61K in 0h0m with 0 errors on Sun Sep 18 18:55:32 2016
config:

    NAME            STATE     READ WRITE CKSUM
    test            ONLINE       0     0     0
      /tmp/test/d2  ONLINE       0     0     0

errors: No known data errors
/t/test #> zpool list
NAME    SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
test   1.98G   408K  1.98G         -     0%     0%  1.00x  ONLINE  -