PowerProtect Data Manager 19.17 Storage Array User Guide

Troubleshooting storage array backup and restore issues

The following sections provide guidance for troubleshooting backup and restore issues.

Backup traffic to DD storage using Management network instead of Data network

PowerMax backup jobs might use the Management network instead of the Data network if the DD system was registered in PowerProtect Data Manager using the management IP address.

When registering DD storage in PowerProtect Data Manager, ensure that you use the data network IP address.

PowerMax storage group asset protection level might not reflect the actual value after reconfiguration in Unisphere

When a storage group that was previously discovered in PowerProtect Data Manager is reconfigured in Unisphere, the asset protection level associations in the PowerProtect Data Manager UI Infrastructure > Assets window might not reflect the reconfigured values. For example, if a stand-alone storage group was created and discovered in PowerProtect Data Manager but child storage groups were later added to this storage group, the parent/child protection level is not recognized by PowerProtect Data Manager upon rediscovery.

To troubleshoot this issue:

  1. Temporarily stop scheduled Unisphere asset source discoveries in PowerProtect Data Manager.
  2. Make sure that the impacted storage group does not have a backup copy.
  3. Rename the affected Parent (and dependent Child) storage groups in Unisphere by adding the prefix PPDM_WSG.
    NOTE:The prefix might already be different if it was changed in the sdmng application.yml service configuration file parameter PPDM_WSG_PREFIX.
  4. Discover the PowerMax in PowerProtect Data Manager. This discovery should remove all these storage groups from the PowerProtect Data Manager inventory.
  5. Rename the storage groups to their original names (without the PPDM_WSG prefix).
  6. Discover the PowerMax in PowerProtect Data Manager again.
  7. In the Infrastructure > Assets window, change storage group protection levels as required.
  8. Schedule a discovery of the Unisphere in PowerProtect Data Manager.

Unconfiguring Volume Block jobs automatically start after 2-3 hours of PowerMax unavailability

"Unconfiguring Volume Block -<policy name>" jobs automatically start after 2 to 3 hours of PowerMax unavailability. The jobs fail since no respective array is available. These jobs run for policies which contain PowerMax assets.

If this issue occurs:

  1. Discover the Unisphere for PowerMax once the PowerMax system is available.
  2. Add the assets back to the policies.
  3. Retry the operation.

Configuration job started due to change in protection storage IP address fails

When you change a protection storage system's IP address in PowerProtect Data Manager, the configuration job that is triggered by the protection storage discovery fails to update the remote system on the PowerStore array.

If this issue occurs, run the configuration job again.

Timeout error with /storage-hosts API when restoring using Instant Access

When performing an Instant Access restore, if you select either Map to Host or Map to Host Group on the Location page of the Block Volumes Restore wizard, a timeout error may occur. This issue might happen if there is a network connectivity issue between PowerStore and PowerProtect Data Manager. The issue can also occur if the PowerStore REST API is delayed in fetching the host details and sending a response back to the /storage-hosts API.

If this error occurs, the following message is displayed:

504: Gateway timeout error

As a workaround, close the error message dialog and retry the operation.

Unable to update the remote system after changing the storage unit password

If you try to change the password of an existing storage unit in PowerProtect Data Manager while a backup is in progress, the backup job fails along with the array configuration job that is triggered by the storage unit password change.

The array configuration job fails with the following error even though the remote system is not in use by any protection jobs:

org.springframework.web.client.HttpClientErrorException$BadRequest:
400 Bad Request: "{"messages": [{"code":"0xE02010020086","severity":"Error","message_l10n":"Failed
to modify the remote system protection_policy1-ldpde105-b9ab3 due to associated active remote
sessions.","arguments":["protection_policy1-ldpde105-b9ab3"]}]}"

To resolve this issue, contact Customer Support.

Unable to complete data protection operations when a remote PowerStore system goes into "PARTIAL DATA CONNECTION LOSS" state

Backup, restore, and instant access operations for block volume protection policies cannot complete successfully when a remote PowerStore system goes into PARTIAL DATA CONNECTION LOSS state and does not instantly return to OK state.

This issue occurs primarily due to network connectivity disruptions, and can usually be resolved by maintaining a stable network. The PowerStore system typically returns to OK state without intervention. If the network is stable but the network error under the Connectivity tab of the remote system persists, and the remote system does not return to OK state within 10 to 15 minutes, contact Dell Customer Support.

Instant Access fails for backup and restore sessions that are in an In-Progress, Canceling, or Failed Cleanup Required state

Instant Access fails for backup and restore sessions that are shown as In-Progress, Canceling, or Failed Cleanup Required in PowerStore Manager.

To resolve this issue for Instant Access jobs that are shown as In-Progress or Canceling, wait until the job completes and then retry the Instant Access operation in the PowerProtect Data Manager UI.

To resolve this issue for Instant Access jobs that are shown as Failed Cleanup Required in PowerStore Manager and Failed in PowerProtect Data Manager, complete the following steps in the PowerProtect Data Manager UI:

  1. Click Details in the Details column next to the job.

    PowerProtect Data Manager displays the error details and the steps to resolve the issue.

  2. Complete the steps to resolve the issue, and then retry the Instant Access operation.

If an Instant Access job fails with the error message clean up in progress in PowerStore Manager, wait until the operation completes. Once the operation is complete, retry the Instant Access operation in the PowerProtect Data Manager UI.

During an initial discovery of a PowerStore asset source, PowerProtect Data Manager does not discover or protect empty volume groups

Volume groups that do not contain any volumes cannot be protected and are not discovered as part of the initial discovery of the asset source in the PowerProtect Data Manager UI.

If a volume group was added to a protection policy when it was not empty, but is now empty and a subsequent discovery is performed, the volume group is still shown in the list of assets in the PowerProtect Data Manager UI. Backups that are performed while the volume group is empty will fail with an error.

Empty volume groups can be kept when there are backups that were performed before the volume group was empty that might need to be restored. To restore data that was backed up before the volume group was empty, use Restore to alternate and select a different volume group as the target.

Protection operations fail intermittently when encryption in-flight is enabled from PowerStore

Enabling encryption in-flight on the remote system from PowerStore causes backups and restores to fail intermittently.

To resolve this issue, wait for the completion of the configuration job that was started when the option was enabled, and until the remote system returns to an OK state, and then retry the backup or restore operation.

NOTE:For remote systems that are created in PowerProtect Data Manager, changing the settings of the remote systems from PowerStore is not recommended.

Unable to restore from replicated backups using Instant Access

Instant Access restores from replicated backup copies fail with the following error:

org.springframework.web.client.HttpClientErrorException$BadRequest: 400 Bad Request: "{"messages":[{"code":"0xE0204001002F","severity":"Error","message_l10n":"Instant Access/Retrieve operation failed because remote snapshot 89d298f5-ae7e-485b-934c-63e27b35d6c2 does not belong to the remote system 96c0782a-d228-4bb4-9bfb-1698a10e3d37.","arguments":["89d298f5-ae7e-485b-934c-63e27b35d6c2","96c0782a-d228-4bb4-9bfb-1698a10e3d37"]}]}"

Block volume Restore to Original might fail with error

A Restore to Original of a block volume might fail with the following error:

org.springframework.web.client.HttpClientErrorException$BadRequest: 400 Bad Request... Retrieve session already exists for a given resource

To resolve this issue, contact Dell Customer Support.

Internal Server error due to target protection storage user password or user account issue

During a storage array configuration, backup, or restore job, the following error might occur on the PowerStore system when the target protection storage user password is incorrect or the target protection storage user account has been locked or disabled:

org.springframework.web.client.HttpServerErrorException$InternalServerError: 
500 Internal Server Error: "{"messages":
[{"code":"0xE02010020082","severity":"Error","message_l10n":"Operation on
remote system failed due to internal error, try again. If the error persists,
contact your service provider for assistance."}]}"

If the cause of the error is an incorrect password, change the password using the following steps:

  1. Change the target protection storage user password in the PowerProtect Data Manager UI.
  2. Change the same password in the corresponding credential object.
  3. Retry the job.

If the job continues to fail with the same error, the account might be disabled. Contact Dell Customer support for more information about the steps required to enable the account again.

Protection policy configuration fails due to an invalid X.509 certificate verification code

During protection policy configuration, if the array configuration job fails with the error Unable to import the certificate. Verification code : 3, the CA certificate cannot be imported due to an invalid certificate verification code. As a result, the protection policy configuration cannot complete successfully.

If the CA certificate was generated on a PowerProtect DD system with DDOS version 7.5 or earlier, the certificate does not contain the CA:TRUE extension. PowerStore rejects any CA certificate that lacks the CA:TRUE extension.

To resolve this issue, regenerate the self-signed CA certificate:

  1. On the DD system, open a command prompt and type the following command:

    adminaccess certificate generate self-signed-cert regenerate-ca

    Regenerating the certificate breaks existing trust relationships with peer systems.

  2. To restore the trust relationship, add the new DD certificate to the truststore of the peer system.

Re-create a PowerProtect Data Manager user with the role user that has been accidentally deleted

A new target protection storage user is created as part of the configuration job for PowerStore block volume protection policies. The password for this user is managed by PowerProtect Data Manager, and the name of this user is in the format user_ targetStorageName_UniqueIdentifier. If this user is accidentally deleted, then you can re-create this user with the same username credentials. In the PowerProtect Data Manager UI, go to Administration > Credentials to obtain the username.

  • If the target protection storage is a DDsystem or DDVE system, log in to the DDOS to create a user with the role user.

After re-creating the user, ensure that you change the password in the corresponding credentials object, and maintain the same password in the PowerProtect Data Manager UI. Go to Administration > Credentials, select the user, and click Edit to update the password.

PowerMax restore job does not use all available Protection Engines

A PowerMax discovery job recalculates the list of available Proxies. If a protection engine is temporarily unavailable, a freshly added or updated discovery job removes them from the list of available protection engines. When this occurs, PowerMax backup and restore jobs will not use these protection engines, even when they are available again.

To resolve this issue, run a discovery of PowerMax systems to add these protection engines to the list of available protection engines again. Discovery jobs can be scheduled to run automatically.

Synthetic Full backup runs as full

A synthetic full backup is automatically promoted to a full backup in the following situations:

  • A storage group is added to the protection policy for the first time.
  • A storage group is moved to a different policy.
  • A storage group is expanded with new volumes (in which case only the backup of the new volumes is promoted to full).
  • The last snapshot of the storage group is deleted from the PowerMax.
  • The last backup of a storage group is deleted from PowerProtect Data Manager.
  • There is an issue with communication between the Block Volume protection engine and the Unisphere server.

Data Transferred shows 0 bytes or an amount of data smaller than the asset size

Data Transferred represents the amount of data which is transferred from a PowerMax system for a protected storage group.

Synthetic Full backup only transfers differences between previous and current backup.

Full backup by default transfers allocated blocks from the PowerMax system.

Full backup with the option USE_ALLOCATION_MAP set to false transfers 100% of allocated and unallocated blocks of all volumes from storage group.

Unsuccessful backup after server DR or PowerProtect Data Manager update

When the sdmng service application.yml configuration file is modified, the backup might not work after a server disaster recovery (DR) or PowerProtect Data Manager update.

This configuration file is not retained after a PowerProtect Data Manager update or server DR. Any parameters that were previously changed will require reconfiguration after these operations.

Multiple replication network setups in PowerStore

Issues have been observed with PowerStoreOS version 4.0 that are related to multiple replication network setups. The data connection to DD systems can fail under the following scenarios:

  1. There is a separate storage network for storage/host connectivity and replication/data mobility, and the DD create remote system is unable to find IP interfaces or initiators to create a data connection. This occurs because the data connection logic looks for IP interfaces with both storage (iSCSI) and replication purposes (at both IP port and Storage network level).
  2. When multiple storage networks for replication or storage purposes are available, the data connection logic selects a network device that does not match the selected storage network. This causes the remote system to remain in a complete data connection loss state.

Rate this content

Accurate
Useful
Easy to understand
Was this article helpful?
0/3000 characters
  Please provide ratings (1-5 stars).
  Please provide ratings (1-5 stars).
  Please provide ratings (1-5 stars).
  Please select whether the article was helpful or not.
  Comments cannot contain these special characters: <>()\