PowerFlex: SDS MD_cache status is pending result

Two missing scenarios in the official documentation for configuring and enabling MD_cache fail to showcase the above.

Scenario 1:  PD is created > SDSs are added > MD_cache is configured and enabled

When newly created SDSs are added to a newly created PD before MD_cache is configured and enabled, they will have FGL metadata cache state set to "pending result", and applying MD_cache will require a user to run the procedure described below in the Resolution section under "Steps to configuring and enabling MD_cache successfully".

Scenario 2:  Existing PD with SDSs > MD_cache is configured and enabled

When MD_cache is configured and enabled on an existing PD with existing SDSs, the SDSs will have FGL metadata cache state set to "pending result", and applying MD_cache will require a user to run the procedure described below in the Resolution section under "Steps to configuring and enabling MD_cache successfully".

Steps to configuring and enabling MD_cache successfully
Attention: Upon disabling and enabling MD_cache on PD with existing SDSs, performance might be affected.

1) Run the following command to allocate the memory used by MD_cache on each SDS in the Protection Domain:
Note: The maximum amount of memory that can be allocated per SDS is 32GiB.

scli --set_default_fgl_metadata_cache_size (--protection_domain_id <ID> | --protection_domain_name <NAME>) --metadata_cache_size_mb <SIZE>

2) Run the following command to enable MD_cache in the Protection Domain:

scli --enable_fgl_metadata_cache (--protection_domain_id <ID> | --protection_domain_name <NAME>)

3) Run the following command to configure the MD_cache size for each SDS in the PD:

scli --set_fgl_metadata_cache_size (--sds_id <ID> | --sds_name <NAME> | --sds_ip <IP> [--sds_port <PORT>]) (--metadata_cache_size_mb <SIZE> | --use_protection_domain_default_size)

--metadata_cache_size_mb <SIZE>
    Define the default size of the metadata cache for the SDS in MB
--use_protection_domain_default_size
    Use the default size of the Protection Domain

4) Run the following command to disable MD_cache in the Protection Domain:

scli --disable_fgl_metadata_cache (--protection_domain_id <ID> | --protection_domain_name <NAME>)

5) Run the following command to enable MD_cache in the Protection Domain:

scli --enable_fgl_metadata_cache (--protection_domain_id <ID> | --protection_domain_name <NAME>)

Example:

[root@MDM1 ~]# scli --add_protection_domain --protection_domain_name PD1
Successfully created protection domain. Object ID 339ccbae00000000

[root@MDM1 ~]# scli --add_sds --sds_ip 172.1.1.1 --protection_domain_name PD1 --device_name /dev/sdb,/dev/sdc,/dev/sdd --sds_name SDS1 --storage_pool_name SP1
Successfully created SDS SDS1. Object ID 3789dbd500000000

[root@MDM1 ~]# scli --set_default_fgl_metadata_cache_size --protection_domain_name PD1 --metadata_cache_size_mb 32768
Successfully set fine granularity default metadata cache size for Protection Domain

[root@MDM1 ~]# scli --enable_fgl_metadata_cache --protection_domain_name PD1
Successfully enabled fine granularity metadata caching on Protection Domain

[root@MDM1 ~]# scli --query_sds --sds_name SDS1 | grep FGL
FGL metadata cache information:
        FGL metadata cache state is pending result <<<
        FGL metadata cache size: 32768 MB

[root@MDM1 ~]# scli --set_fgl_metadata_cache_size --sds_name SDS1 --use_protection_domain_default_size
Successfully set fine granularity metadata cache size for SDS

[root@MDM1 ~]# scli --disable_fgl_metadata_cache --protection_domain_name PD1
Successfully disabled fine granularity metadata caching on Protection Domain

[root@MDM1 ~]# scli --query_sds --sds_name SDS1 | grep FGL
FGL metadata cache information:
        FGL metadata cache is disabled <<<
        FGL metadata cache size: 32768 MB

[root@MDM1 ~]# scli --enable_fgl_metadata_cache --protection_domain_name PD1
Successfully enabled fine granularity metadata caching on Protection Domain

[root@MDM1 ~]# scli --query_sds --sds_name SDS1 | grep FGL
FGL metadata cache information:
        FGL metadata cache is enabled <<<
        FGL metadata cache size: 32768 MB