VPLEX: How to Set Rebuild Transfer Size
Summary: This article describes how to increase or decrease the rebuild transfer rate, and the effects of doing this.
Instructions
A REBUILD WILL OCCUR WHEN:
- Logging failed, mirror is marked as out of date
- Whenever a new mirror is added to the device, such as a migration
- Whenever an existing device is marked out of date
- A distributed RAID 1 has one leg marked out of date due to storage issues or
interclusterlink issues - When a mirror (leg) of a
distributed-deviceis removed from the source
Command to return transfer rate for all devices:
VPlexcli:/>rebuild show-transfer-size *
Command to return device name and transfer size.
VPlexcli:/>rebuild show-transfer-size --devices *
----------- ------------- ddev_1 2M ddev_10 2M ddev_11 2M ddev_12 2M ddev_13 2M ddev_7 2M ddev_8 2M ddev_9 2M
Default size: 128k
Max size: 2M (A size larger than this can lead to host performance issues)
It is recommended this be set to 128KB as a balance between minimal host impact and acceptable rebuild/migration performance.
Generally speaking, using transfer-sizes larger than 2MB has no proven to show significant rebuild time improvement.
Set the transfer-size on a specified device to 1M:
VPlexcli:/>rebuild set-transfer-size --devices /clusters/cluster-1/devices/testdevice --limit 1M- Alternatively:
VPlexcli:/distributed-storage/distributed-devices/DD_1234>set transfer-size 1M
Set transfer size at component level
VPlexcli:/distributed-storage/distributed-devices/DD_1234/distributed-device-components/device_Symm1234_1234_1>set transfer-size 1M
Set the transfer-size for all devices to 2M:
VPlexcli:/>rebuild set-transfer-size * 2M
Set the transfer-size for all distributed devices to 256k:
VPlexcli:/distributed-storage/distributed-devices>set *::transfer-size 256k
Notice if throughput increases after increasing the transfer size - It is not always the case that increasing the transfer rate speeds up the rebuilds.
Additional Information
- VPLEX rebuild traffic does not do QoS and is not aware of contention /traffic
- If transfer rate is set too high, it affects performance
- The rebuild-type should be set to full, and this cannot be set manually
- If the volumes are thin-provisioned, then the rebuild should be thin
- If the volumes are thick-provisioned, then the rebuild should be thick
- The VPLEX chooses the best fit rebuild type, and this cannot be set manually
- It always tries to rebuild incrementally initially and if this is not possible, then a full rebuild begins
VPLEX has two types of rebuilds: full and logging.
A full rebuild occurs whenever a new mirror is added to device, such as a migration, or whenever an existing device is marked out of date.
A logging rebuild occurs when a mirror (leg) of a distributed-device is removed or added from the source, the VPLEX tracks the changes made on the blocks of the source device from the logging volume, and apply these changes to the mirror once it is reattached.