Deploy Dell PowerFlex v3.6.x

Configure the SDS for optimal memory usage

For certain system configurations, the SDS needs to be configured to be able to use more than one Non-Uniform Memory Access (NUMA) domain.

About this task

In many cases, memory is split into a few NUMA domains. For example, in a two-CPU-socket server configuration, each of the CPUs is in a different NUMA domain. The SDS is configured by default to have affinity to socket 0 in a server or VM, and therefore, by default, the SDS is connected only to NUMA 0, and only has access to the memory in NUMA 0.

In ESXi in HCI configurations, the NUMA definition requires special attention. When configuring the SVM, there are cases where the SVM includes two vNUMAs. For example, if the number of vCPUs required for the SVM is larger than the number of physical cores in the CPU, ESXi will configure the SVM with two vNUMAs. Other cases are ESXi HCI with replication, where the SVM includes both the SDR and the SDS. The SDS will be configured on vNUMA 0 and the SDR on vNUMA 1. The total memory will be split equally between the two vNUMAs, and therefore, the SDS may not have enough memory on NUMA 0.

The memory size required for the SDS depends on the configuration, and on other factors. In order to determine the required size for the SDS, and whether NUMA configuration is required, follow the size recommendations provided in the Getting to Know Dell PowerFlex Guide.

If the memory supported by NUMA 0 (usually half the total memory) is less than the amount of memory required by the SDS, you need to allow the SDS access to the memory in the other NUMA. For example, if a node has 96 GiB of physical memory, divided into 48 GiB on each CPU, and the SDS requires 58 GiB of memory, this NUMA modification is required, because 48 GiB is not sufficient for the SDS.

This operation will have a small effect on performance (less than 10%), and for this reason, is not the default setting.

NOTE: This setting can also be modified on operational SDSs that are expected to use more memory in the future, for example, due to adding SDS devices. Note that this procedure requires a graceful SDS restart in order to take effect.

Steps

Use one of the following options to edit the NUMA configuration:

Option	Description
Use the echo command	On the node where the SDS is deployed, issue the command: echo "numa_memory_affinity=0" >> /opt/emc/scaleio/sds/cfg/conf.txt
Use a text editor	On the node where the SDS is deployed, navigate to the file: /opt/emc/scaleio/sds/cfg/conf.txt Open the file with a text editor, and add the following line to the file: `numa_memory_affinity=0` Save the file.

Gracefully restart the SDS:
1. Use one of the PowerFlex management interfaces to place the SDS in instant maintenance mode.
2. Restart the SDS process:
```
/opt/emc/scaleio/sds/bin/delete_service.sh
/opt/emc/scaleio/sds/bin/create_service.sh
```
3. Exit instant maintenance mode.
4. Wait for the rebuild to finish before performing other tasks.

Results

Configuration of memory usage for the SDS is complete.

Next steps

Repeat this procedure on all SDSs that require this modification.

Welcome

Welcome to Dell