Article Number: 000110933

PowerStore: PowerStore X Performance Best Practice Tuning

Summary: This article recommends best practices to apply to enable maximum performance when configuring a new PowerStore X model appliance.

Article Content

Instructions

Table of Contents
1. PowerStore X Models – Performance Best Practices
2. Manual Procedure
3. Automated Procedure (Prior to PowerStore 1.0.3)
4. Script

1. PowerStore X Models

When configuring a new PowerStore X model appliance, it is highly recommended to apply these best practices to enable maximum performance. This is a non-disruptive procedure but it is recommended to apply these settings prior to provisioning any resources on the appliance.
Starting with PowerStore 1.0.3, a user will be prompted to configure these settings during the cluster initial configuration, which involves providing two additional IPs and setting MTU Size to 9000 (Jumbo Frames) if supported.
Prior to PowerStore 1.0.3, or if not configured during the cluster initial configuration, these settings can either be done entirely manually or for convenience a script is attached to this KB which will automate steps 3.2-3.4. While step 3.1 is still a pre-requisite, along with changing the MTU to 9000 if Jumbo Frames are supported within the environment, it is highly recommended to use this script after completing step 3.1 and 3.3.

2. Manual Procedure

During the initial configuration of a PowerStore X model appliance, best practice settings can be automatically applied to the appliance to enable maximum performance. This new step in the initial Configuration Wizard will:

Ask the user to provide two additional storage IPs, with the wizard automatically:

Mapping the storage IPs as additional internal iSCSI targets
Changing the internal ESXi queue depth according to depth according to best practices

Allow the user to change the cluster MTU to 9000, if supported within the customer’s environment, according to best practices.

If the best practice settings were not applied during the initial configuration of a PowerStore X model appliance, the steps to apply some of the settings after the initial configuration have been automated. However, not all the configuration steps have been automated. It is strongly recommended that you apply the best practices during the initial configuration of a PowerStore X model appliance.

2.1 Optimization during initial Configuration

During Initial Configuration for a PowerStore X cluster, a user will reach an optimization steps to configure best practice settings to enable maximum performance. A user is asked to providing two additional IP addresses, and setting the Storage MTU Size. Although not required, it is recommended to change the MTU to 9000 to enable maximum performance. Please check that all networks used by PowerStore support Jumbo Frames, as using Jumbo Frames on a network that doesn't support it end-to-end may impact functionality.

2.2 Optimization after initial Configuration

If not applied during initial configuration, this release automates some of the steps previously required, but not all. Please follow the steps below:

2.2.1 iSCSI Targets

The Initial Configuration Wizard (ICW) requires a minimum of 6 IP addresses for the Storage Network, but it is highly recommended to supply additional IP addresses to maximize performance. These additional IP addresses are used to enable additional iSCSI targets and sessions, which will significantly increase performance. The number of additional IP addresses depends on the model:
PowerStore 1000X: No additional IP addresses and configuration needed
PowerStore 3000X – 9000X: 2 additional IP addresses (1 per node)

Follow these instructions to configure additional iSCSI targets:

If these additional IP addresses are not added in the ICW, they can also be added later in the Settings -> Network IPs -> Storage page in PowerStore Manager.
Navigate to the Hardware -> Appliance -> Ports -> Virtual Ports page.
Select the NodeA-vFEPort2 and NodeB-vFEPort2 virtual ports and click Map Storage Network.
Open vSphere and navigate to Hosts and Clusters -> PowerStore X model ESXi node -> Configure -> Storage Adapters.
- Click vmhba64 -> Static Discovery and confirm that two targets are listed.
- Click Rescan Storage and then confirm the additional targets are listed.
Repeat steps 4a and 4b on the other PowerStore X model ESXi node.

2.3.1 Jumbo Frames

By default, the Maximum Transmission Unit (MTU) is set to 1500. Using jumbo frames (9000 MTU) increases performance and network efficiency by sending larger payloads in the Ethernet frame. In order to use jumbo frames, all equipment on the network must also be configured for jumbo frames: the switch the PowerStore X model appliance is connected to, and the PowerStore X appliance. After enabling jumbo frames on the switch, follow these instructions to enable jumbo frames on the appliance:

1. In PowerStore Manager, navigate to the Settings -> Cluster MTU page.
a. Change the cluster network’s MTU size to 9000.
b. Navigate to Network IPs -> Storage.
c. Click Reconfigure.
d. Confirm there are no resources in use or actively being migrated
e. Change the storage network’s MTU size to 9000.

3. Automated Procedure starting with PowerStore 1.0.3

This table summarizes the changes made as part of this procedure to significantly increase performance:

Model Targets ESXi Queue Depth MTU Comment
PowerStore 9000X 1 additional target needed per Controller VM (2 additional IPs per appliance) = 2 total targets per Controller VM (8 total Storage IP Addresses) 256 9000 Additional target Queue Depth Change, and MTU change required
PowerStore 7000X 1 additional target needed per Controller VM (2 additional IPs per appliance) = 2 total targets per Controller VM (8 total Storage IP Addresses) 256 9000
PowerStore 5000X 1 additional target needed per Controller VM (2 additional IPs per appliance) = 2 total targets per Controller VM (8 total Storage IP Addresses) 256 9000
PowerStore 3000X 1 additional target needed per Controller VM (2 additional IPs per appliance) = 2 total targets per Controller VM (8 total Storage IP Addresses) 256 9000
PowerStore 1000X 1 target per Controller VM = 6 total Storage IP Addresses (Default) 128 (Default) 9000 Only MTU change required

3.1. iSCSI targets

Follow these instructions to configure additional iSCSI targets:

1. If these additional IP addresses are not added in the ICW, they can also be added later in the Settings -> Network IPs -> Storage page in PowerStore Manager.

2. Navigate to the Hardware -> Appliance -> Ports -> Virtual Ports page.

3. Select the NodeA-vFEPort2 and NodeB-vFEPort2 virtual ports and click Map Storage Network.

4. Open vSphere and navigate to Hosts and Clusters -> PowerStore X model ESXi node -> Configure -> Storage Adapters.

a. Click vmhba64 -> Static Discovery and confirm that two targets are listed.
b. Click Rescan Storage and then confirm the additional targets are listed.

5. Repeat steps 4a and 4b on the other PowerStore X model ESXi node.

3.2. Queue Depth

If running the script in section 4, the queue depth does not need to be manually configured. Skip to section 4.

The iSCSI queue depth setting controls the number of outstanding IOs on each PowerStore X model ESXi node. For maximum performance, this setting should be modified to the following values depending on the model:

PowerStore 1000X: No changes needed
PowerStore 3000X – 9000X: 256

Follow these instructions to configure the queue depth setting:

1. Login to the PowerStore X model ESXi node through SSH using the root ESXi password, which, as recommended, should have been changed from the default below:
Username: root
Password: <DST>_123!

2. Gather a list of the iSCSI devices:
[root@H7047-host-1:~] esxcfg-scsidevs -c |awk '{print $1}' | grep naa.68ccf09800
naa.68ccf098003627e45222d0b308a6486d
naa.68ccf09800c15b9b1c9afe443a367359

3. Verify the queue depth setting for each iSCSI device:
[root@H7047-host-1:~] esxcli storage core device list -d naa.68ccf098003627e45222d0b308a6486d | grep "competing"
No of outstanding IOs with competing worlds: 128

[root@H7047-host-1:~] esxcli storage core device list -d naa.68ccf09800c15b9b1c9afe443a367359 | grep "competing"
No of outstanding IOs with competing worlds: 128

4. Change the queue depth to the appropriate value on each iSCSI device:
[root@H7047-host-1:~] esxcli storage core device set -d naa.68ccf098003627e45222d0b308a6486d -O 256
[root@H7047-host-1:~] esxcli storage core device set -d naa.68ccf09800c15b9b1c9afe443a367359 -O 256

5. Confirm the new queue depth values for each device:
[root@H7047-host-1:~] esxcli storage core device list -d naa.68ccf098003627e45222d0b308a6486d | grep "competing"
No of outstanding IOs with competing worlds: 256

[root@H7047-host-1:~] esxcli storage core device list -d naa.68ccf09800c15b9b1c9afe443a367359 | grep "competing"
No of outstanding IOs with competing worlds: 256

6. Repeat these steps on the other PowerStore X model ESXi node.

3. Jumbo Frames

3.4. Restarting iSCSI sessions

Note: If running the script in section 4, the queue depth does not need to be manually configured. Skip to section 4.

1. Login via SSH into the first internal ESXi host

LinuxLinux$ ssh -l root 10.244.28.42Password:

The time and date of this login have been sent to the system logs.

Caution: All commands run on the ESXi shell are logged and may be included in support bundles. Do not provide passwords directly on the command line. Most tools can prompt for secrets or accept them from standard input. VMware offers supported powerful system administration tools. Please see www.vmware.com/go/sysadmintools for details. The ESXi Shell can be disabled by an administrative user. See the vSphere Security documentation for more information. [root@H6085-host-1:~]

2. The software iSCSI adapter for ESXi is vhba64. Search for sessions on that interface. This will return a list in the format
<adapter>,<iscsi-target>,<session-id>

[root@H6085-host-1:~] esxcli iscsi session list | grep vmhba64,
vmhba64,iqn.2015-10.com.dell:dellemc-powerstore-fnm00182700366-a-0a85ce10,00023d000001
vmhba64,iqn.2015-10.com.dell:dellemc-powerstore-fnm00182700366-a-0a85ce10,00023d000002
vmhba64,iqn.2015-10.com.dell:dellemc-powerstore-fnm00182700366-b-6efc363b,00023d000001
vmhba64,iqn.2015-10.com.dell:dellemc-powerstore-fnm00182700366-b-6efc363b,00023d000002
vmhba64,iqn.2015-10.com.dell:dellemc-powerstore-fnm00182700366-a-3e662d33,00023d000001
vmhba64,iqn.2015-10.com.dell:dellemc-powerstore-fnm00182700366-a-3e662d33,00023d000002
vmhba64,iqn.2015-10.com.dell:dellemc-powerstore-fnm00182700366-b-4f71ed2a,00023d000001
vmhba64,iqn.2015-10.com.dell:dellemc-powerstore-fnm00182700366-b-4f71ed2a,00023d000002
[root@H6085-host-1:~]

3. Verify that there are at least 2 sessions for each target. If there are less than 2, rescan the iSCSI adapter, and check again

[root@H6085-host-1:~] esxcli storage core adapter rescan --adapter vmhba64
[root@H6085-host-1:~]

4. Once the number of sessions is verified, remove, and recreate each session. Rescanning will recreate a removed session. It is necessary to rescan after removing each session to ensure path redundancy.

[root@H6085-host-1:~] esxcli iscsi session remove -A vmhba64 -n iqn.2015-10.com.dell:dellemc-powerstore-fnm00182700366-a-0a85ce10 -s 00023d000001

[root@H6085-host-1:~] esxcli storage core adapter rescan --adapter vmhba64

5. Repeat step 4 for each session.

6. Repeat steps 1 through 5 for the second internal ESXi host.

4. Script

After having provided the additional storage IP addresses and mapping them to the Storage Network (Step 3.1 above) (not required for PowerStore X 1000), and increasing MTU to 9000 on both the switch the PowerStore X model appliance is connected to and on the cluster via the PowerStore manager (Step 3.3 above), this script will change the queue depth to the appropriate values, and restart the iSCSI sessions to apply these changes.

Pre-requisites:

- For PowerStore 3000X through 9000X models, step 3.1 must be completed manually per the above.
8 total IP addresses are provided for the storage network. 6 are the minimum required during cluster creation, but 8 are highly recommended for maximum performance. If 6 were provided during cluster creation, an additional 2 need to be provided by following steps 3.1.1 above
- Per 3.3, in order to use jumbo frames, all equipment on the network must also be configured for jumbo frames.
- There are two versions of the script – Please use the correct one based on your PowerStore X model:
One version of the script exists for PowerStore 1000X model
Another version of the script exists for PowerStore Models: 3000X, 5000X, 7000X, 9000X models
- To download the PowerStore X performance tuning script from Solve Desktop:
PowerStore -> PowerStore X Procedures -> Scripts and Tools -> PowerStore X performance tuning script zip
- To download the PowerStore X performance tuning script from Solve Online:
Solve Online -> PowerStore X Procedures -> Scripts and Tools -> PowerStore X performance tuning script zip

Instructions:

1. Copy the file to the first internal ESXi host.

Linux$ scp PST3000X-9000X-PerformanceBP.sh root@10.244.28.42:
Password:

PST3000X-9000X-PerformanceBP.sh 100% 2277 2.2KB/s 00:00
Linux$

2. Log in to the first internal ESXi host and run the script.

Linux$ ssh -l root 10.244.28.42
Password:

The time and date of this login have been sent to the system logs.

[root@H6085-host-1:~] ./PST3000X-9000X-PerformanceBP.sh

This script will perform the following configuration changes:

*** Restart all iSCSI sessions ***

* Restart iSCSI session 00023d000002 on target iqn.2015-10.com.dell:dellemc-powerstore-fnm00182700366-a-0a85ce10 *

esxcli iscsi session remove -A vmhba64 -n iqn.2015-10.com.dell:dellemc-powerstore-fnm00182700366-a-0a85ce10 -s 00023d000002