Recovering From a Failed ZFS Root Mirror Disk Recovering From a Failed ZFS Root Mirror Disk

Posted by smokris on 2010.11.14 @ 18:13

Filed under:

Hypothetical. Say, for example, you’ve got a ZFS mirror on disk0 and disk1, from which your Nexenta system boots. A year or two later, disk0 crashes, and starts spewing messages such as the following:

 ahci: [ID 860969 kern.warning] WARNING: ahci0: ahci_port_reset port 0 the device hardware has been initialized and the power-up diagnostics failed

Shortly thereafter there’s a storm and the power goes out for several hours and drains your entire humongous UPS, and when the power finally comes back on

, the system won’t boot — it stops at the grub> prompt; issuing a standard boot command fails.

You take out the bad disk, and try to boot with the mirror, but the mirror doesn’t boot in normal mode. You scratch your head, and boot successfully in safe mode.

From there, you update the boot archive —

 # bootadm update-archive
 # touch /reconfigure

allowing it to again boot in normal mode.

Then you insert a replacement disk0, boot again, and install GRUB on the new disk (and make sure the existing disk is up-to-date) —

 # cd /boot/grub
 # installgrub -f -m stage[12] [your-boot-device, e.g., /dev/rdsk/c1t0d0s0]
 # installgrub -f -m stage[12] [your-other-boot-device, e.g., /dev/rdsk/c1t1d0s0]

And tell ZFS the drive is again available for use —

 # zpool replace syspool c1t0d0s0

And then you have a happy hypothetical ZFS root mirror again.

Post new comment

The content of this field is kept private and will not be shown publicly.
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd> <img> <blockquote> <pre> <div> <span> <b> <i> <br> <table> <tr> <td> <th> <small> <sub> <tex> <equation> <equations> <s>
  • You can enable syntax highlighting of source code with the following tags: [code], [blockcode], [applescript], [bash], [c], [cpp], [css], [diff], [drupal5], [drupal6], [html4], [java], [javascript], [latex], [lisp], [lua], [m68k], [make], [mysql], [objc], [pascal], [perl], [php], [python], [ruby], [sql], [xml].
  • You may insert videos with [video:URL]
  • You can use Markdown syntax to format and style the text. Also see Markdown Extra for tables, footnotes, and more.
  • Adds typographic refinements.
  • LaTex commands embedded in text will be interpreted and rendered. Additional information can be found at DruTex Documentation Pages
    • Allowed commands include: \atop \binom \cdot \cfrac \choose \frac \int \ln \over \sum \to
      Allowed environments include: align array equation equations gather matrix split
    • Provides different environments to create rendered images (especially maths).
    • Assists automatic numbering of tex, equation, and equations environments.

    More information about formatting options