davestagg
1 Copper

RE: Performance issue / poor performance: VRTX with Fault Tolerant / Dual Shared PERC 8

Jump to solution

Erik,

Thanks for your time, effort and sharing the detail in this forum. It is much appreciated.

Thought you might like to know that my Dell sales rep alerted me to this issue in between purchase and shipping. We had the option to go Single or Dual PERC. Thanks for the information provided here, as I am much better able to understand the issue. At least this is not a surprise nor a headache for us.

No doubt, your efforts have helped this to be a smoother experience for us and other Dell customers. (fingers crossed!)

Dave.

Richard_artes
1 Nickel

RE: Performance issue / poor performance: VRTX with Fault Tolerant / Dual Shared PERC 8

Jump to solution

Dell have now stopped selling the VRTX with dual PERC controller. My order was cancelled and re-issued with a single PERC controller.

Godverdomme
1 Copper

RE: Performance issue / poor performance: VRTX with Fault Tolerant / Dual Shared PERC 8

Jump to solution

It is no longer required to remove the second SPERC controller to be able to use write-back instead of write-through as described in Erik's post. You can now disable the second PERC8 controller from the CMC. In the current firmware (1.35), the second disabled controller can be set as disabled and requires manual intervention to activate if the primary active controller were to fail. The usefulness of this is not really detailed in the patch notes, but this "fix" is meant to allow people to enable Write-Back and get rid of the abysmal performance you have when using both controllers in Write-Through.

Automatic failover (cold failover, causing disconnect) is a feature that will be release later. A lot later will be the actual "active/active" firmware update, which would allow "live" failover without downtime.

Steps:

  1. Download VRTX CMC firmware version 1.35 or higher.
  2. Shut down all your blades.
  3. In the CMC interface, press "Chassis Overview > Update".
  4. Select both checkboxes for the CMC controllers in the "CMC firmware" heading and press "Apply CMC Update".
  5. Enter the location of the CMC firmware file and Apply it.
  6. The CMC will show you its progress. Uploading takes about 8 minutes, applying the update a few minutes more. The CMCs will reboot after applying the update and you will get kicked out of the webinterface.
  7. Once the CMC has rebooted, browse to the "Storage > Controllers > Troubleshooting" section.
  8. For the SPERC of your choice, select the "Disable RAID controller" option and apply. This will reboot your storage component.
  9. Once rebooted, go to "Storage > Virtual Disks > Manage" and select "Edit: Write Policy" and pick "Write Back" instead of "Write Through" for all your virtual disks (unless you have a reason not to).
  10. The change above will be carried out immediately, but it is still suggested to reset the CMC once more under "Chassis Overview > Power > Control" with "Reset CMC (warm boot)".
  11. Boot your shut down blades.

This will allow you to have a second PERC8 installed in your VRTX, in case the other fails. But you will have to do a manual intervention to fail over in case of failure. I suppose this is primarily meant for hard to reach locations (remote offices without IT staff or easy access for Dell support technicians). This is also what we use it for.

Hopefully by end of the year we will have the automatic failover feature and then in the course of next year the true active/active config with write-back enabled (synchronized caches). I'm not going to hold my breath for the synchronized cache firmware fix... I suspect that won't be an easy one for Dell.

Highlighted

RE: Performance issue / poor performance: VRTX with Fault Tolerant / Dual Shared PERC 8

Jump to solution

Finally Dell Support budged. This is the latest update from support:

--
Write Back
Cache Enablement

We are actively driving a plan to deliver a Write-Back option (via firmware) for Dual PERC configurations. We are working this plan with great urgency and will have it implemented as soon as we are able complete our development and Enterprise validation of the firmware.
--

0 Kudos

RE: Performance issue / poor performance: VRTX with Fault Tolerant / Dual Shared PERC 8

Jump to solution

Could you please stop marking the post "Posted by on 13 Jun 2014 11:48 AM" as answer? Because it is not the answer to the issue.

0 Kudos
Moderator
Moderator

RE: Performance issue / poor performance: VRTX with Fault Tolerant / Dual Shared PERC 8

Jump to solution

Erik

My post is the answer to your inquiry. The reason you are experiencing poor performance is because write caching is disabled in the dual PERC configuration. The posts you keep marking as the answer propose that there is a hardware fault. There is not a hardware issue with the controller. This issue is with understanding how cache and RAID function.

This configuration was released with redundancy as a focus. Many people purchased this with the expectation of performance as well. We are PLANNING to update the firmware to allow caching to work with a dual PERC configuration. There is no guarantee that it will be implemented or that it will allow the full performance of a single PERC. Until it has been released we are not going to set the expectation that it will allow a dual PERC configuration to perform as well as a single PERC configuration.

Thanks

Daniel Mysinger
Dell EMC, Enterprise Engineer

Get support on Twitter @DellCaresPRO

0 Kudos
ttoonka
1 Copper

RE: Performance issue / poor performance: VRTX with Fault Tolerant / Dual Shared PERC 8

Jump to solution

Thank you here for great discussion, I would like to publish my result.

I am running VRTX with PERC8 Shared in Active/Passive config with Write Through policy, disk cache is enabled.

RAID 10 from 8x1TB SAS HDD

PERC8 firmware version is 23.8.10-0059

My results are not that bad as you have published above.

0 Kudos
ttoonka
1 Copper

RE: Performance issue / poor performance: VRTX with Fault Tolerant / Dual Shared PERC 8

Jump to solution

0 Kudos
ttoonka
1 Copper

RE: Performance issue / poor performance: VRTX with Fault Tolerant / Dual Shared PERC 8

Jump to solution

the same test with Disk Chache disabled, so  disk chache make big difference and it could be solution, but it increase a risk.

0 Kudos

RE: Performance issue / poor performance: VRTX with Fault Tolerant / Dual Shared PERC 8

Jump to solution

Hi TTOONKA,

Thanks for your additions. Unfortunately for me it is not completely clear what you mean with caching enabled and or disabled. From what I understand if you set the write policy to "Write through" this is equal to having caching disabled. So I don't get how you enabled caching while using the "write through" policy.

Which caching did you enable/disable? In the VRTX Chassis management? In Windows? Both?

What are you running on the VRTX? Windows Servers with shared storage? A HyperV Failover Cluster?

Although you results seem to be better than my thorough testing, you would be surprised how much performance you gain when you use "write back" as caching policy. although to enable "write back" you have to disable the 2nd PERC controller and reconfigure the Virtual Disk to use "write back" as caching policy.

0 Kudos