2

I'm running a personal server at home with a CentOS 7 OS and a 12TB zpool. It's been running for a couple of years and yesterday I noticed some problems so I went in to have a look.

At first it seemed like one of my drives had failed, with zpool import giving the following results:

pool: media
id: 1363376331138686016
state: DEGRADED
status: One or more devices contains corrupted data.
action: The pool can be imported despite missing or damaged devices.  
The
fault tolerance of the pool may be compromised if imported.
   see: http://zfsonlinux.org/msg/ZFS-8000-4J
 config:

media                                DEGRADED
  raidz1-0                           DEGRADED
    ata-ST3000DM001-1ER166_W500G55Q  ONLINE
    ata-ST3000DM001-1CH166_Z1F278KB  UNAVAIL
    sdc                              ONLINE
    sde                              ONLINE
    sdf                              ONLINE

This looks ok, however I can't seem to import the pool directly.

Running zpool import media gives me:

cannot import 'media': I/O error
Destroy and re-create the pool from
a backup source.

I've looked around to figure out the problem but everything I've found has given me nothing.

Some other things I've tried:

  • zpool import -fFX: zpool could not be imported
  • zpool import -fFV: zpool imported with FAULTED status
  • zpool status: no pools available

Is anyone able to point me in the right direction? I'm not sure what my next course of diagnosis should be.

2 Answers 2

0

Your best bet is to destroy the pool, recreate it with a replacement for the failed drive ata-ST3000DM001-1CH166_Z1F278KB, and then restore from backup.

If that's not an option (it should be - neither ZFS nor RAID are a substitute for backups! nor were they ever intended to be), then you could try taking the zpool offline until you have a replacement for ata-ST3000DM001-1CH166_Z1F278KB ready to be installed.

When you have acquired a replacement drive, try to import the pool in DEGRADED state and immediately replace the failed drive with the good new drive.

BTW, https://serverfault.com/questions/548568/zfs-recover-from-faulted-pool-state may have some useful information for you.

4
  • Thanks, unfortunately backup is not an option :( I'm actually in the process of picking an offsite backup option but that hasn't happened yet. I've actually already got a spare drive to replace it with, the problem being I can't seem to import it in DEGRADED state, any suggestions on how else I can achieve that?
    – Ben Wong
    Aug 13, 2017 at 12:39
  • if none of the options in that serverfault link helped, you might try either rebooting (to reset the drives), or perhaps using sg_reset on each of the drives in the pool, then try importing them.
    – cas
    Aug 13, 2017 at 13:01
  • 2
    also, you probably should import with -d /dev/disk/by-id/ - "/dev/sdX" device names are NOT guaranteed to remain the same across reboots.
    – cas
    Aug 13, 2017 at 13:03
  • Thanks, I tried the rewind option and even tried -T and none of those solutions work. I keep getting 'some devices unavailable' no matter what tag ID I use. Fortunately I don't have any critical data on this server so I'm just going to destroy it and recreate it with the replacement drive.
    – Ben Wong
    Aug 13, 2017 at 21:42
0

What kind of problems did you notice that made you go and look at the volume? Your zpool command listed shows the drive as unavailable. Can you see it otherwise, or is it dead?

It appears there are sufficient drives in the raidz pool where you should be able to import it.

Normally first with a zpool import -nF to determine if the pool could be imported by discarding the last few transactions.

Sometimes a zpool clear along with a zpool scrub can help. You might also be able to try a zpool replace on the suspect/bad/failing drive with a new drive.

And when using large SATA drives in a home fileserver, it's best not to get too greedy. There are tons of various chains on this whether you're using ZFS, another kind of software raid, as well as even using "enterprise" hardware raid, so I'm not going to go into great detail.

Drives fail. It's going to happen. It is after all just spinning rust. Even if you have a setup where multiple drives could fail, what's to say another drive won't fail during the rebuild to a replacement?--hello multi-terabyte drives. For this reason a lot of people are starting to use 3-way mirrors. Luckily drives aren't as expensive as they used to be.

With ZFS, you could also try increasing the amount of copies (zfs set copies=<# copies> <pool>) to increase the amount of copies for a file to limit possible data loss.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.