Dell EMC Unity: Reinitializing the Unity Operating Environment (OE) with the same version as on the Array (Dell EMC Correctable)

Introduction

Reinitializing the Dell EMC Unity OE should be considered a last resort, and by nature is fully destructive, completely removing all existing data and configurations. This procedure resets the system to a 'back-to-factory' state based on the OE version that was last in use. You cannot use this procedure to change out the system drives (drives 0-3) from one type or size to another, because the reinit process itself access the Unity image that is stored on the system drives.

When this procedure might apply:

1. If the Converged Network Adapter (CNA) ports (4 and 5) on the Service Processors (SPs) have been set to a protocol that is not wanted (the ports can be initially set as Fibre Channel (FC) or Ethernet), then the only way to change the protocol personality of the ports is to reinitialize the system ('back to factory state').
2. If the system has been installed without the CNA ports populated (Management IP assigned, Unisphere Initialization wizard run, but is not yet in production), and you have ETHERNET Small Form-Factor Pluggables (SFPs), you can physically add the SFPs to the CNA ports. Perform the Commit IO Ports by navigating to Unisphere > System > System View > Enclosures > DPE. Committing I/O ports requires multiple SP reboots. 
   If the CNA ports are empty and you must add FC SFPs, this step will NOT work. The system requires reinstallation with the FC SFPs installed in the ports.
3. If the wrong I/O Modules have been initialized on the system, you can use the reinit procedure to change the modules to a different type.
4. If some other condition exists whereby the system cannot be restored to Normal Mode but has no apparent hardware issues (and is not in production).

Performing a System reinitialization from Unisphere

1. If the system is accessible from Unisphere, you can perform the system reinit, or reinitialization, by navigating to the System > Service > Service Tasks section. Here, you must first verify if the SPs are in Normal Mode and then place each SP into Service Mode by running the Enter Service Mode function. If both SPs are in Normal Mode, place the non-Primary SP into Service Mode first, wait for it to completely reboot, and then place the Primary SP into Service Mode (The UI shows as Storage Processor X (Primary).
2. If you are changing the SLIC type, do the following, or else go to step 3. Once both SPs are in Service Mode, you may remove SLICs from the SP slots, or SFPs from the CNA ports, and replace them with different SLICs or SFP type. For example, if you must change the CNA port protocol from FC to Ethernet, or vice-versa.      
   It is possible that pulling out and changing I/O modules will panic the SPs. Wait for the SPs to completely reboot back to Service Mode, then continue with the procedure.

3. With both SPs in Service Mode (aka, Rescue Mode), you must log in to Unisphere as the service user with service password and then run the System > Service > Service Task called Reinitialize, which reinstalls the Unity OE from an image stored on the System Drives. 
4. Once a system has been reinitialized, which can take approximately an hour, it requires IP initialization (Connection Utility) and Unisphere Initialization wizard steps. 

Performing a System reinitialization using CLI with SSH connection

The following procedure re-initializes the system at the version that is already on the system.
As a recommended precaution, you should always inject root to the system, when possible, before performing a reinit if recovery steps are required.
 

1. Use PuTTY to SSH to the system management IP and log in with service user credentials.

2. Determine if SPs are running in Normal Mode versus Rescue Mode (Service Mode):

3. Before reinitializing, both SPs must be in Rescue/Service Mode. Use the following example to put an SP into Service Mode:

4. Once SPs are in Service Mode, you can remove or rearrange SLICs, swap SFPs in the CNA ports to change from FC to Ethernet protocol, or vice-versa (SPs may reboot after removing SLICs, if so, they will come back in Service Mode)
5. Perform the system reinitialization from either SP, using the following:

6. Answer the system prompt, then wait approximately an hour for the system to reset to a factory fresh state.
7. Verify SP Fault LEDs show the correct behavior before running CU and the Configuration wizard.

Performing a System reinitialization using the IPMITool (No IP network)
Use this method only if you cannot log in to the system with Unisphere or SSH over the network.

1. Download and install the IPMI Tool 1.0 on your laptop (available from support.emc.com). The IPMI Tool is a Windows command prompt utility that is used to establish a Serial-over-LAN connection to the SPs.
2. Use the following steps to connect and log in to the SPs:

1. Connect your laptop to a portable switch and run Ethernet cables from the switch to each SP's Service LAN port. If you do not have a switch, you can connect to a single Service LAN port directly from your laptop, but are limited to one SP, though you can still ssh peer to access the other SP from the command line.
2. Configure the laptop with an IP address for the Service LAN network: 128.221.1.249, netmask 255.255.255.0, no gateway required
3. Open two command prompts on the laptop, and use the following syntax to connect to each SP:
   c:\> ipmitool.exe -I lanplus -C 3 -U console -P <unity_serial_number> -H 128.221.1.252 sol activate     -->SPA Internal IP
c:\> ipmitool.exe -I lanplus -C 3 -U console -P <unity_serial_number> -H 128.221.1.253 sol activate     -->SPB Internal IP
   For the IPMI command, the User is console, and the password is the Serial Number of the system for the SOL activation. If you have issues connecting, disable Cisco AnyConnect services and your wireless network adapter.
4. Once the IPMI connection is established, press the Enter key once. You should see a login prompt. At this point, log in using the service user account and password. If the service user password has never been changed, it is service by default. IPMI sessions can be fickle and may drop back to the command prompt at times. Arrow up to the IPMI command to reconnect. It should not require logging in again.

3. Once logged in, verify whether each SP is in Normal or Rescue Mode using the svc_diag command.
4. Place each SP into Rescue Mode using the following methodology:
   As with other reinit procedures, it is recommended to place the non-Primary SP into Service Mode first, reboot it, wait for it to completely reboot, then place the Primary SP into Service Mode. You can identify the Primary SP when the following command returns a running process for the ECOM service.
   spa:~$ pidof ECOM
   28771           --> The non-Primary SP does not run the ECOM service
   spb:~$ svc_rescue_state -s
   spb:~$ svc_shutdown -r
   Wait for the SP to come back up, and repeat the process on the other SP, as needed

5. Perform the system reinitialization from either SP and wait at least an hour for it to complete.

6. Verify SP Fault LEDs show correct behavior before performing the IP initialization (Connection Utility) and running the Unisphere Configuration Wizard.
   Use svc_initial_config to assign a name and Management IP to the system in lieu of the CU, then run the Configuration Wizard from a browser connection. The default login to Unisphere is user  admin  and  Password123#.
    

1. When the Unity OE installation is complete, the SP Fault LEDs should display the following, indicating that the system is ready for IP initialization: One second Amber, followed by a 3 seconds Blue, on each SP

2. Or, if DHCP is set up on the customer network, the SP fault LED may show a solid Blue LED condition.
   In either case, the system is ready for IP initialization. Refer to the Unity Installation Guide for a complete chart of system LED indicator conditions.

3. If the SP Fault LEDs are alternating one second Amber, then Blue, the SP is in Service Mode, which indicates that the system is NOT READY for IP initialization, and you should connect to the SPs using the IPMI Tool to investigate.

After reinitializing:

• Configuration - Once re-initializing is complete, the system boots back up. You must run the Connection Utility to reset the Management IP address and then go through the Configuration Wizard as if it were a new installation. The Connection Utility can be downloaded from the Unity product page on the support site. 
• Restore - No one-step restore procedure is available for all Unity configurations that you had before re-initializing. Configuration settings can be backed up from the Service page. A Save Configuration button allows you to save the configuration to a file. However, there is no way to restore this configuration automatically or manually. This save config text file is provided to be a guide for the administrator to manually re-create the setup. By referring to this text file, all configurations should be done manually by the administrator, and all data should be restored from backup media. 
• Licensing - You must reload the license keys during the reconfiguration. You can either go to the license page on Dell Online Support and download the keys or use the saved keys that you might have received from an email earlier.

Requirements for reinit on Dell EMC Unity VSA is unlikely because there are no CNA/hardware changes or D@RE requirements. But, if for any reason, customers want to re-purpose Dell EMC Unity VSA arrays, they can do the reinit. The reinit preserves the Universally Unique Identifier (UUID). Licenses for Unity VSA (Professional Edition) are tied to the UUID, so same license can be re-used after reinit.