I have a customer that asked me to submit the following Feature request.

Summary - Enhance the RAID rebalance algorithm to better deal with restriping Fast Track space.

 Background - Fixing suboptimal striping of Fast Track devices is regularly tricky/time consuming due to Fast Track usually being fully allocated. As a result the system usually doesn't have space within the Fast Track to create new optimally striped devices. As a result the system will usually work to empty a device by moving the pages to other devices. Once it completes this it will destroy the existing device and then start the process again on the next device. Unfortunately at this point the system will usually see the empty space in the Fast Track and create a new RAID device that has the same characteristics as the device that was just destroyed. As a result the system won't be able to make any progress without some manual intervention.

The standard approach from CoPilot is to simply disable all the Fast Track classes for a given tier, run a rebalance to move all the data out, and then re-enable the classes. At this point the space will be completely unallocated, so the system will be able to make optimal stripes. Unfortunately this requires free space equal to 20% of the tier size. This is usually possible with tier 1, but becomes increasingly difficult with large tier 3 systems. (Stratacare's 9761/9762 system has ~360TB in tier 3. As a result disabling tier 3 Fast Track would require 72TB of free space on the system.)

The next approach I've seen from CoPilot is to disable the class, find some RAID devices that are [mostly] on separate spindles (e.g. one device is on disks 1-10 and the other is on disks 11-20.), mark those specific devices to prune, let a rebalance empty the space out, and then re-enable the class/run an addspace to create a new device in that space. This process can then be repeated until all the RAID devices are optimally striped. While this process will eventually result in optimal striping it requires lots of manual intervention/time from CoPilot (and typically the business partner and customer). It would be much better for everyone if the system could be smarter about this rebalance and do it in such a way that doesn't need lots of manual work.

Suggestions - I see two options on how to improve the rebalance process to avoid this problem. The first is fairly simple, but will not likely not be as efficient as possible. (It will also still get stuck in some cases.) The second option would be more complicated to implement, but would likely require moving fewer pages multiple times.

 Option 1:

* Summary - Always prune 2 devices before creating a new one in the same tier/track.

* Details - When a RAID rebalance is running keep track of the characteristics (disks with extents, size of each extent, track (Fast or Standard), etc.) of the last RAID device that was destroyed. Do not allow an automatic addspace to create a new RAID device with the same characteristics as the device that was just destroyed. Based on what we've seen the RAID rebalance process typically works through one RAID class before moving onto the next class. Assuming successive devices are in the same class, there are three cases:

** Devices overlap on no disks - In this case the system should be able to combine the devices to make a new device that is more optimally striped. (It's possible that there will be some space left unallocated after this process if the previous devices had different extent sizes.)

** Devices overlap on all disks - In this case the system should be able to combine the devices to make a new device that has the same striping efficiency, but with larger extents. While this doesn't get us a more optimally striped device, it does get us fewer sub-optimally striped devices. Thus this device can be combined in the future with another device on different spindles.

** Devices overlap on some disks - In this case the system would be able to partially combine parts of the RAID devices into one more optimally striped device. Some additional amount of space would be left unallocated. (It could either be put into it's own device, or it could be left unallocated to merge with other future space.)

* If the newly created device(s) are sub-optimally striped they should then be marked to prune.

* This solution is likely to run into problems in the following scenarios:

** Different classes have corresponding RAID devices - If R10 has a device on disks 1-10 and R5 has a device on disks 11-20, this algorithm will not be able to get either of them to be optimally striped.

** Matching devices widely separated in queue - As I understand it the devices that need to be rebalanced are placed in a queue. If corresponding devices are widely separated in the queue then they may take an excessively long time to merge together. It also may result in very large devices once they are finally merged. (Assuming newly created sub-optimal devices are placed at the end of the queue, they will continue to remain separated. This specific scenario could be handled by pulling devices out of the queue randomly rather than always pulling from the front. It isn't clear if that will help in the general case though.)

Option 2

* Summary - Pick corresponding devices and prune them in order to create space for an optimal stripe.

* Details - This is mainly an automated method of what CoPilot is currently doing. When the system is doing a rebalance in a tier/track that's fully allocated, the system should block automatic addspace commands for that tier/track. It should then find a set (two or more) of RAID devices that would be optimally striped if they were merged into a single device. (Ideally they'd have the same (or similar) extent sizes so as to avoid extra space that can't be included in the resultant device.)

Once the system finishes emptying those devices it can then create a new device that is optimally striped. (In the event the source devices are from different classes, it should create multiple new optimally striped devices split between the classes.) If there is any left over space it can either leave it unallocated or create a new sub-optimal device in that space. This process should then be repeated until all devices are optimally striped.

Implementation Notes - Since these options will block space allocation for periods, they should be bypassed if the system is in conservation (or emergency) mode. In those cases it's obviously better to allocate the space sub-optimally than to run out of space.