For about a year I had a debian server running with data on dm-crypt/ luks-encrypted hdds underneath zfs (pool name zdata), 8 hdds in a mirrored mode, so 4 mirrors with 2 hdds each. A zpool status from the healthy times:
# zpool status pool: zdata state: ONLINE scrub: scrub completed after (blahblah) config: NAME STATE READ WRITE CKSUM zdata ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 dm-name-n8_2 ONLINE 0 0 0 dm-name-n8_3 ONLINE 0 0 0 mirror-1 ONLINE 0 0 0 dm-name-n8_0 ONLINE 0 0 0 dm-name-n8_1 ONLINE 0 0 0 mirror-2 ONLINE 0 0 0 dm-name-n8_4 ONLINE 0 0 0 dm-name-n8_5 ONLINE 0 0 0 mirror-3 ONLINE 0 0 0 dm-name-n8_7 ONLINE 0 0 0 dm-name-n8_6 ONLINE 0 0 0 errors: No known data errors
(Of course, no backup of this 34TB raw data. I can give up on almost all data, but there are some files I consider precious to me.)
Two weeks ago after upgrading to stable I noticed several read, write and checksum errors beginning with dm-name-n8_2. A scrub didn't do it, because zfs began a resilvering, but never ended it, as it began over and over again. That was the moment, where also dm-name-n8_3, the other hdd in this mirror, showed same kind of errors and I began to realize, that both hhds respectively their controllers were dying.
I think, than I did silly things, not everything I can comprehend at the moment (zpool history shows nothing). Among others I initiated a replace with the "better" disks. In the meantime I worked on a defect home appliance which caused a power outage, and after booting up I wasn't able to import the pool again. The bad disks are now dd'ed and substituted. Eventually I cannot import the pool again:
# zpool import -f pool: zdata id: 1343310357846896221 state: UNAVAIL status: One or more devices were being resilvered. action: The pool cannot be imported due to damaged devices or data. config: zdata UNAVAIL missing device mirror-0 DEGRADED dm-name-n8_2 UNAVAIL dm-name-n8_3 ONLINE mirror-1 ONLINE n8_0 ONLINE n8_1 ONLINE mirror-2 DEGRADED dm-name-n8_4 ONLINE dm-name-n8_5 UNAVAIL mirror-3 ONLINE n8_7 ONLINE dm-name-n8_6 ONLINE # zpool import zdata -f cannot import 'zdata': one or more devices is currently unavailable
Samesame with zpool import zdata -fnF
Note the changed device names.
The pool has at least one device online in each mirror, why does it complain about a missing device?
How do I get to know, which device is missing? Are there ways to get the pool online again and how do I do it?
(I read the oracle docs and the chapter on zfs at the freebsd handbook, learned some things, but didn't find an answer to my questions.)
The output from zdb is a bit lengthy, it can be seen here.