I rebooted my server for the first time since i was monitoring a drive rebuild and setting a new drive as a hot spare through dellmgr, and now i get the following message:
Unresolved configuration mismatch between disk(s) and NVRAM on the adapter for PERC 3/QC
I'm assuming using the dellmgr utility through RedHat didn't update something, but when I do the Ctrl+M to enter the BIOS, i see all the changes I've made through dellmgr, so not sure what it thinks is different.
How can I resolve the config mismatch without totally destroying all the data on my arrays?
What you are seeing is most likely not a RedHat issue. The NVRAM mismatch means that the controller's record of the RAID configuration differs from the disk metadata. This can occasionally happen if there is a power surge and is usually recoverable. While there is a possibility this is happening because the array is corrupt and the data is lost, it is most likely you just need to reset the config in the controller and have it read back from the disks and you'll be good to go again. Here are the steps to do that:
1. Power down and remove all disks; make sure to keep them organized so you know which disk came from which slot
2. Power up with no disks, get into the controller BIOS with Ctrl+M, go to Objects > Adapter > Fast Init -- turn this OFF as a precaution in case we need to re-tag later
3. Go back to the first menu, choose Configure, and choose Clear -- this is what will remove the configuration from NVRAM that is "mismatched"
4. Escape out and Save changes if prompted
5. Power down, install the drives in same slots they were removed from.
6. Power back up, Ctrl+M to get into the Controller BIOS again
7. Under Physical Disks you should see all your disks and everything should look OK -- no changes needed
8. Go to the Configure menu, View/Add Configuration, and you should see the correct RAID configuration you had earlier and now can add it.
9. Once you confirm that looks right then you can add the hot spare.
Hopefully now everything is good and you can boot back into RedHat! If not you may have one other option, which is retagging the array. However if that does not work you will most likely not have any other options other than to start from scratch with new arrays. For a retag you just delete the configuration detected on the disks and then create a new one right on top of it without initializing -- that is why we turned Fast Init off. You must specify the exact same configuration the disks had prior to the retag otherwise you will not be able to access the disks except by formatting. If the retag works, then the disk metadata will match the actual RAID data and you should be able to boot ok.
Let us know how it goes and if you need more information or clarification at any step!