Avamar: Gen4S Hardware: Controller Cache Discarded due to Memory or Battery Issue

The following errors appear in the Avamar UI:

<MRMON010> Controller ID: 0  Controller cache discarded due to memory/battery problems  Event ID:10

This occurs if there is sudden power loss, or unresponsive system, when the battery backup unit (BBU) is not fully charged, and Write Back mode is forcefully enabled.

The extended power loss to the system results in the BBU being thoroughly discharged before power recovery. 

1. Log in to the Avamar Utility Node as admin.

2. Elevate to root privilege.

3. Load the root keys per article Avamar: How to Log in to an Avamar Server and Load Various Keys.

4. Using the information from the UI event or the dial-home Service Request:

a. Determine the node that produced the error message.

b. Connect to the node as root using the command below:

ssn 0.# --user=root

(Where 0.# is the physical node number)

5. On the affected node:

a. Review the /var/log/messages file to confirm the error and to verify what occurred before the error message.

b. Review the environment for any issues, such as power outages.

c. Check the current status of the battery:

 CmdTool2 -AdpBbuCmd -GetBbuStatus -a0

See APPENDIX A for a complete sample output.

d. Review the battery state.

If the battery is in a failed state, contact Dell Technologies Technical Support to replace the node using the Disk Transplant procedure.

If the battery is not in a failed state, force a learn cycle on the battery to ensure that it is functioning correctly:  

CmdTool2 -AdpBbuCmd -BbuLearn -a0

If the problem persists, contact Dell Technologies Technical Support for further assistance.

APPENDIX A:

Example output for a good battery:

CmdTool2 -AdpBbuCmd -GetBbuStatus -a0

BBU status for Adapter: 0
 
BatteryType: SuperCaP
Voltage: 9484 mV
Current: 0 mA
Temperature: 32 C
 
BBU Firmware Status:
 
  Charging Status              : None
  Voltage                                 : OK
  Temperature                             : OK
  Learn Cycle Requested                   : No
  Learn Cycle Active                      : No
  Learn Cycle Status                      : OK
  Learn Cycle Timeout                     : No
  I2c Errors Detected                     : No
  Battery Pack Missing                    : No
  Battery Replacement required            : No
  Remaining Capacity Low                  : No
  Periodic Learn Required                 : No
  Transparent Learn                       : No
  No space to cache offload               : No
  Pack is about to fail & should be replaced : No
  Cache Offload premium feature required  : No
  Module microcode update required        : No
 
GasGuageStatus:
  Fully Discharged        : Yes
  Fully Charged           : No
  Discharging             : Yes
  Initialized             : No
  Remaining Time Alarm    : No
  Remaining Capacity Alarm: No
  Discharge Terminated    : No
  Over Temperature        : No
  Charging Terminated     : Yes
  Over Charged            : No
 
  Pack energy             : 94 J
  Capacitance             : 100
  Remaining reserve space : 156
 
 
Exit Code: 0x00