PowerScale: After Provisioning Connectivity, Post Provisioning Steps Failed
Summary: After Dell SupportAssist (Connectivity) provisioning finishes, an error message displays: "Error: Provisioning was successful, however some post provisioning steps failed."
Symptoms
Scenario 1:
Connectivity should automatically disable Secure Remote Services. Sometimes the automation is unable to disable Secure Remote Services, so an error is present, such as the ones listed here:
OneFS 9.9.0.0 and earlier:
Powerscale-1# # isi supportassist provision view
Provision Status:
Current State: COMPLETED
Sub-state: completed
Success or Fail: Fail
Percent Complete: 100%
Error: Provisioning was successful, however some post provisioning steps failed: Failed to disable SRSOr:
Powerscale-1# # isi supportassist provision view
Provision Status:
Current State: COMPLETED
Sub-state: completed
Success or Fail: Fail
Percent Complete: 100%
Error: RuntimeError: Failed to connect to REST server: isi_papi_d service is unavailableIn OneFS 9.10 and later, the command to check is now:
Powerscale-1# isi connectivity provision viewScenario 2:
In this scenario, Secure Remote Services did not stop running, as seen here:
Powerscale-1# # isi esrs view
Warning: Secure Remote Service is at end of service life. Upgrade connectivity to Dell Technologies connectivity services as soon as possible by using 'isi connectivity'. See https://www.dell.com/support/kbdoc/en-us/000152189/powerscale-onefs-info-hubs cluster administration guides for more information.
Enabled: Yes <<<<<<<<<<<<<<<<<<<<<<<<<<<
Primary ESRS Gateway: 1.2.3.4
Secondary ESRS Gateway:
Alert on Disconnect: Yes
Gateway Access Pools: subnet0.pool0
Gateway Connectivity Check Period: 60
License Usage Intelligence Reporting Period: 86400
Download Enabled: No
SRS Download Enabled: Yes
ESRS File Download Timeout Period: 50
ESRS File Download Error Retries: 3
ESRS File Download Chunk Size: 1000000
ESRS Download Filesystem Limit: 80
Offline Telemetry Collection Period: 7200
Gateway Connectivity Status: Disconnected <<<<<<<<<<<<<<<<<<<<<<Cause
Scenario 1:
The daemons that control Secure Remote Services are not in the expected state, and require manual intervention.
Scenario 2:
Secure Remote Services is not in the expected state, and must be disabled manually.
Resolution
Scenario 1:
Generally the issue happens when the two daemons that control Secure Remote Services are in a respected state. Verify that they are in the correct status:
PowerScale-1# isi services -a | egrep 'esrs|rsapi'
isi_esrs_d ESRS Service Daemon Disabled
isi_rsapi_d Remote-Service API Daemon Enabled This is how Secure Remote Services would look after enabling Connectivity:
Powerscale-1# isi esrs view
Warning: Secure Remote Service is at end of service life. Upgrade connectivity to Dell Technologies connectivity services as soon as possible by using 'isi connectivity'. See https://www.dell.com/support/kbdoc/en-us/000152189/powerscale-onefs-info-hubs cluster administration guides for more information.
Enabled: No
Primary ESRS Gateway: 1.2.3.4
Secondary ESRS Gateway:
Alert on Disconnect: Yes
Gateway Access Pools: subnet0.pool0
Gateway Connectivity Check Period: 60
License Usage Intelligence Reporting Period: 86400
Download Enabled: No
SRS Download Enabled: Yes
ESRS File Download Timeout Period: 50
ESRS File Download Error Retries: 3
ESRS File Download Chunk Size: 1000000
ESRS Download Filesystem Limit: 80
Offline Telemetry Collection Period: 7200
Gateway Connectivity Status: DisconnectedThe isi_rsapi_d daemon should be enabled, if it is in the disabled state, enable it with this command:
Powerscale-1# isi services -a isi_rsapi_d enableThe isi_esrs_d daemon should be disabled after provisioning Connectivity, disable it with this command:
Powerscale-1# isi services -a isi_esrs_d disableVerify isi_rsapi_d is now running on every node, and that there are no more isi_esrs_d daemons running with this command:
PowerScale-1# isi_for_array -s "ps -auxww | egrep 'isi_rsapi|isi_esrs' | grep -v grep"The output should return something similar to this:
Powerscale-2# isi_for_array -s "ps -auxww | egrep 'isi_rsapi|isi_esrs' | grep -v grep"
Powerscale-1: root 2869 0.0 0.8 108068 46684 - Ss Thu00 0:03.86 /usr/sbin/isi_rsapi_d
Powerscale-2: root 3015 0.0 0.6 121852 35640 - Ss Wed00 0:05.96 /usr/sbin/isi_rsapi_d
Powerscale-3: root 3056 0.0 0.6 102756 36972 - Ss Wed07 0:04.32 /usr/sbin/isi_rsapi_d
Powerscale-4: root 2743 0.0 0.6 108676 36412 - Ss Tue21 0:04.69 /usr/sbin/isi_rsapi_d
Powerscale-2#If the output does not reflect the example output, initiate a gather and contact Dell Technical Support.
If the output looks the same as the example, provision again using a new Access Key and Pin.
Scenario 2:
Try to disable Secure Remote Services with this command:
PowerScale-1# isi esrs modify --enabled=false --force
If the command returns to the prompt without an error, check the status:
Powerscale-1# # isi esrs view
Warning: Secure Remote Service is at end of service life. Upgrade connectivity to Dell Technologies connectivity services as soon as possible by using 'isi connectivity'. See https://www.dell.com/support/kbdoc/en-us/000152189/powerscale-onefs-info-hubs cluster administration guides for more information.
Enabled: No <<<<<<<<<<<<<<<<<<<<<<<<<<<
Primary ESRS Gateway: 1.2.3.4
Secondary ESRS Gateway:
Alert on Disconnect: Yes
Gateway Access Pools: subnet0.pool0
Gateway Connectivity Check Period: 60
License Usage Intelligence Reporting Period: 86400
Download Enabled: No
SRS Download Enabled: Yes
ESRS File Download Timeout Period: 50
ESRS File Download Error Retries: 3
ESRS File Download Chunk Size: 1000000
ESRS Download Filesystem Limit: 80
Offline Telemetry Collection Period: 7200
Gateway Connectivity Status: Disconnected A new provision can be tried, and use a new Access Key and Pin using the Software ID (SWID) of the cluster.
If the command errors and fails to disable, initiate a gather and contact Dell Technical Support.
This issue is expected to be addressed in a future OneFS code release.