PowerFlex: OpenStack Volume Creation Fails Intermittently

Summary: OpenStack experiences issues managing volumes with sporadic behavior when creating and deleting them.

This article applies to This article does not apply to This article is not tied to any specific product. Not all product versions are identified in this article.
Check out other resources

Symptoms

Scenario

OpenStack creates and deletes thousands of small volumes during normal operations. During peak workload hours, creating and deleting volumes may not work.

Errors and screenshots were observed in this scenario where the 2.5 ratio was reached:

Screenshot of OpenStack server list that errored
Additional screenshot of OpenStack server list that errored
Screenshot of actual error message
 

Impact

Unable to create new volumes.

Cause

OpenStack has a cinder.conf file that sets an over-provisioning limit.

The value powerflex_max_over_subscription_ratio=2.5 was reached during high workload hours and not during off hours, hence the sporadic times that the issue was seen. 
[flexos]
backend_host=hostgroup
volume_driver=cinder.volume.drivers.dell_emc.powerflex.driver.PowerFlexDriver
san_login=admin
san_password=********
san_ip=X.X.X.X
powerflex_storage_pools=PD01:PD01_FG01
volume_backend_name=flexos
powerflex_allow_migration_during_rebuild=False
powerflex_allow_non_padded_volumes=False
powerflex_max_over_subscription_ratio=2.5
powerflex_rest_server_port=443
powerflex_round_volume_capacity=True
powerflex_server_api_version=True
powerflex_unmap_volume_before_deletion=False
san_thin_provision=True
driver_ssl_cert_verify=False
driver_ssl_cert_path=

Resolution

The over-provisioning setting is causing the volume errors. Changing the value in the cinder.conf from 2.5 to a higher value, for example 3.5, allows the operations to work smoothly during peak hours.

The powerflex_max_over_subscription_ratio key was first introduced in Cinder version 17.4.1.dev13 (Victoria).


Fix:

Working as expected.


Impacted versions:

  • ScaleIO 2.0.x and later
  • VxFlex OS 3.0.x and later
  • PowerFlex 3.5.x and later
  • PowerFlex 4.x and later

Affected Products

PowerFlex rack, VxFlex Ready Nodes, PowerFlex Appliance, PowerFlex custom node, PowerFlex Software
Article Properties
Article Number: 000226370
Article Type: Solution
Last Modified: 26 Jun 2024
Version:  1
Find answers to your questions from other Dell users
Support Services
Check if your device is covered by Support Services.