PowerProtect Data Manager 19.13 Storage Direct User Guide

Agent registration

On Windows, if the agent fails to establish a connection with the PowerProtect Data Manager server, agent registration might fail with the following error message:

During a network connectivity test, the agent is unable to reach the PowerProtect Data Manager server by using ping. 

1.	If the ping command is blocked in the environment, the agent registration can still complete successfully. 
Review the agent service logs at INSTALL_DIR\DPSAPPS\AgentService\logs to verify that the registration is successful. If the registration is successful, the status of the agent host indicates Registered in the PowerProtect Data Manager UI.
2.	If the ping command is not blocked in the environment, the agent registration might not complete successfully because a network connection cannot be started. If this occurs, complete the following steps to troubleshoot the issue:

On Linux or AIX, if the agent fails to establish a connection with the PowerProtect Data Manager server, agent registration might fail with the following error message:

During a network connectivity test, the agent is unable to reach the PowerProtect Data Manager server by using ping and curl. 

1.	If the ping command is blocked in the environment and curl is not installed, the agent registration can still complete successfully. 
Review the agent service logs at /opt/dpsapps/agentsvc/logs to verify that the registration is successful. If the registration is successful, the status of the agent host indicates Registered in the PowerProtect Data Manager UI.
2.	If the ping command is not blocked in the environment, the agent registration might not complete successfully because a network connection cannot be started. If this occurs, complete the following steps to troubleshoot the issue:

If agent registration fails with these error messages, complete the following operation:

1. Use any network packet tracing tool to trace the packets from the agent system to PowerProtect Data Manager.
2. Start the packet tracing between the source IP of the agent system and the destination IP of PowerProtect Data Manager.
3. Start the network traffic between the agent system and PowerProtect Data Manager.

   Wait 10 to 15 seconds.

4. Analyze the captured packets.
5. Look for SYN and SYN_ACK packets to see if a 3-way handshake is being performed.

   Determine whether the source agent or the destination PowerProtect Data Manager is blocking the connection.

   If network traffic is blocked, contact your network security team to resolve the port communication issue.

Agent host is registered with short name instead of FQDN

Application agent registration using the short name of the agent host is not supported after updating to PowerProtect Data Manager 19.11 or later versions.

If the agent host was previously registered with the short name, and you attempt to re-register the host with the short name after updating PowerProtect Data Manager to version 19.11 or a later version, registration fails with the following error:

Unable to register the agent host <hostName> because short name is not allowed for agent registration.

To resolve the issue, do not remove the host. Configure the host with the Fully Qualified Domain Name (FQDN), and then retry the registration.

If you try to register a new host with the short name after updating PowerProtect Data Manager to version 19.11 or a later version, registration fails with the following error:

Unable to register the agent host <hostName> because short name is not allowed for agent registration.

To resolve this issue, remove the application agent host in the PowerProtect Data Manager UI:

1. Go to Infrastructure > Application Agents.
2. Select the entry for the application agent host and click Remove.

   After the removal, the host is in a "Deleted" state.

   NOTE: The application agent host is automatically removed from the Application Agents window within 24 hours after the deletion.
3. After the removal process is complete, configure the host with the Fully Qualified Domain Name (FQDN), and then retry the registration.

PowerProtect agent service operations

To troubleshoot PowerProtect agent service operations, you can check the PowerProtect agent service log file OpAgentSvc-<timestamp>.log, which is created in <agent_service_installation_location>\logs on Windows and <agent_service_installation_location>/logs on AIX or Linux. To modify the log level and retention of temporary files, you can modify specific parameter settings in the config.yml file.

To modify the log level and retention of temporary files, you can perform the following steps:

1. Stop the agent service.
2. Open the config.yml file in an editor.
3. Modify the log-level settings in the following parameters, as required:
   • DEBUG
   • INFO
   • WARNING
   • ERROR
   • CRITICAL
   NOTE: These parameters are listed in order of decreasing number of messages in the debug information output. The default log-level is INFO.
4. To retain the temporary files, set the keepTempFiles parameter to True in the config.yml file.
   NOTE: The agent service and application agent communicate through the temporary files, which are typically deleted after use but can be useful for troubleshooting purposes. Do not leave the keepTempFiles parameter set to True permanently, or the temporary files can use excessive space on the file system.
5. Start the agent service.

PowerProtect Data Manager UI display of localhost.localdomain hostname

In the PowerProtect Data Manager UI, the Application Agents, Asset Sources, and Protection Jobs windows might list the asset primary hostname as localhost.localdomain instead of the expected FQDN.

The display of localhost.localdomain as the hostname in the PowerProtect Data Manager UI windows might occur when you specify the host's actual FQDN setting for the loopback address in the /etc/hosts file. For example, when you add the following settings in the /etc/hosts file, the first setting value, localhost.localdomain, appears as the hostname in the PowerProtect Data Manager UI windows, instead of the actual FQDN:

127.0.0.1 localhost.localdomain localhost6 localhost6.localdomain6
127.0.0.1 blrv027d233.blr.lab.emc.com blrv027d233

Ensure that the host's actual FQDN is not specified for the loopback address and do not specify hostnames that start with "local" in the /etc/hosts file.

Troubleshooting PowerProtect agent service installations

A PowerProtect agent service installation might fail with the following error message:

Service 'PowerProtect Agent Service' (AgentService) could not be installed. Verify that you have sufficient privileges to install system services.

Possible causes of the installation failure are as follows:

• The installation was attempted on a passive node of a Failover Cluster Instance (FCI).
• The installation was canceled and a rollback left some stale entries of PowerProtect agent services.

As a workaround, clean up the PowerProtect agent service entries, and retry the installation.

Protection policies cannot be created from an imported configuration file that uses VMAX

When trying to import a VMAX storage group protection policy configuration file created by another Storage Direct agent deployment, the protection policy configuration job can fail with the following description:

Configuring Replication Restore vDisks

For the protection policy configuration to succeed, delete and recreate the secondary restore devices before importing the configuration file.

Troubleshooting initiation of SMIS server discovery job

The "Reconfigure Storage Group" job is not initiated after running a discovery of the SMIS server from the PowerProtect Data Manager UI on the first attempt.

The "Reconfigure Storage Group" job is initiated upon running the SMIS server discovery a second time.

Encapsulation fails during policy creation

When a protection policy is created, a process that is called encapsulation occurs, which creates backup and restore FTS devices on the VMAX system and links the DD vDisk with FTS. If a retention lock exists on the VMAX system, or if the SMIS services are not running, encapsulation fails.

To ensure that there is no retention lock on the VMAX system, run the following command as the root user from the SMIS server:

symcfg list -lockn all

The output is similar to the following:

               S Y M M E T R I X      L O C K S

                              Lock       Lock    Lock                Time
    SymmID       Attachment   Status     Number  Usage               Held (Sec)

    000196700638 Local        Locked     15      Config Change       13572 -> Almost 4 hours
    000192604348 Remote       N/A        N/A     N/A                 N/A
    000297000476 Remote       N/A        N/A     N/A                 N/A

If SMIS services are not running, an exception appears in the logs indicating that the storsvd service is not available and a connection to SMIS cannot be established using the SYMAPI calls. If this exception occurs:

1. Run the following command as the root user from the SMIS server to verify the status of storsvd:

   ./stordaemon show storsrvd

   If the service is unavailable, the message *** Daemon storsrvd is not currently running appears.

2. Restart the service by running ./stordaemon start storsrvd.
3. Run the ./stordaemon show storsrvd command again to verify that the status is now Running.
4. To view the remote server details, run ./stordaemon action storsrvd -cmd show server.
5. To view the network configuration, run ./stordaemon action storsrvd -cmd show -netinfo.

Restart of encapsulation fails

When an encapsulation job for the Storage Direct agent does not complete successfully, a restart of the job in the PowerProtect Data Manager UI fails with the message error: Data Domain vDisk pool not provided remediation: null.

Encapsulation fails with error "SYMAPI_C_NET_HANDSHAKE_FAILED"

If the encapsulation of a device fails with the error SYMAPI_C_NET_HANDSHAKE_FAILED

1. Ensure that proper name resolution can occur by verifying that the PowerProtect Data Manager server/SMIS server and SDA host are resolvable by either DNS or the hosts file.
2. Perform the following steps to ensure that the PowerProtect Data Manager server can obtain all the necessary information from the SMIS server:
   1. Log in to PowerProtect Data Manager as an administrator.
   2. Go to the /usr/emc/API/symapi/config/netcnfg directory.
   3. Verify that the server entry exists. For example, <ConnectionName> - TCPIP <hostname> <ipaddress> 2707 -.
   4. Verify whether the Solutions Enabler base daemon is running. For example:

      admin@xxxxx:~> stordaemon list -all
      Available Daemons  ('[*]': Currently Running, '[NI]': Not Installed):
      [*] storapid              EMC Solutions Enabler Base Daemon
          storgnsd              EMC Solutions Enabler GNS Daemon
          storrdfd              EMC Solutions Enabler RDF Daemon
          storevntd             EMC Solutions Enabler Event Daemon
      [*] storwatchd            EMC Solutions Enabler Watchdog Daemon
          storsrmd              EMC Solutions Enabler SRM Daemon
      

   5. Export the environment variables SYMCLI_CONNECT_TYPE=REMOTE and SYMCLI_CONNECT= <ConnectionName>.
   6. Run symcfg list. The command output should display all VMAX and PowerMax systems that have been added to the SMIS server.

Failure of configuration file validation

Configuration file validation fails when multiple storage group assets are selected for policy inclusion if the configuration file is not formatted correctly.

When you select multiple storage group assets as part of a VMAX Storage Group protection policy in PowerProtect Data Manager, validation fails if the configuration file is not formatted correctly. For example, a configuration file with the following format might be pushed to the host:

DDBOOST_USER = 148_78-xxxxx-932c9
DEVICE_HOST = IP address
DEVICE_PATH = /148_78-xxxxx-932c9-SU
DDVDISK_USER = 148_78-xxxxx-932c9
#RESTORE_DEVICE_POOL = 148_78-xxxxx-932c9
#RESTORE_DEVICE_GROUP = R-sdm_xxxxx_SG7-0638
#RESTORE_DEVICE_GROUP = R-sdm_xxxxx_SG8-0638
  # DD_BOOST_FC =
  # DD_PORT =
VMAX_FASTX_RESTORE_SG = R-sdm_xxxxx_SG7-0638
#VMAX_FASTX_RESTORE_SG = R-sdm_xxxxx_SG8-0638

To work around this issue, enclose VMAX_FASTX_RESTORE_SG in a comment and remove RESTORE_DEVICE_POOL and one of the entries for RESTORE_DEVICE_GROUP from the comment.

Failure of MTree replication

MTree replication fails when adding a replication stage for multiple protection policies if assets have the same user or vDisk pool.

If an existing Storage Direct user has updated to the Storage Direct agent for PowerProtect Data Manager and uses the same user or vDisk pool for protected assets to create multiple protection policies with a replication stage, an MTree replication job fails with an error similar to the following: Unable to create DataDomain user xxxx, User xxxx already exists.

To work around this issue, manually add the secondary DD system details in the configuration file.

Troubleshooting replication for assets already configured for a stand-alone agent

Replication is not supported for assets in a storage group protection policy for existing Storage Direct agent users if replication is already configured for a stand-alone agent.

The addition of a replication stage as part of a storage group protection policy is not supported when you use PowerProtect Data Manager to create a Storage Direct agent setup and then try to import it into another PowerProtect Data Manager.

To update the Storage Direct agent, remove the secondary DD system details from the configuration file before importing it.

Clicking Cloud Tier from Add Policy > Objectives when adding a VMAX protection policy causes a busy indicator to display indefinitely

When adding a VMAX protection policy that has a replication objective, the user interface shows a Cloud Tier link. However, a protection policy cannot have both a replication objective and a Cloud Tier objective. If you click Cloud Tier, a busy indicator is displayed indefinitely and the Cloud Tier dialog never appears.

To work around this issue, do not click the link. If you do click the link, refresh the page to clear the busy status and continue configuring the protection policy.