Performance issue / poor performance: VRTX with Fault Tolerant / Dual Shared PERC 8

Hi,

I have been struggling with a storage performance issue for a few months. I found out that the culprit in this performance hit, is the Fault Tolerant Shared PERC 8 in the VRTX. In this configuration you are bound to the write-through caching policy, which is currently performing very bad.

Because of this big performance hit on the storage (caused by the caching policy) I had the following problems:

• Poor performance of (HyperV) VM's
• Poor copy performance (copying big files within the same volume 70MB/s max)
• Unable to make a backup of the VM's using either Veeam B&R 7 or Backup Exec 2014 RTM (backup and VSS are related)
• VSS time-out errors (0x8000ffff- Catastrophic failure)
• Checkpoint creation of a VM took about 2 minutes

The release notes of the latest firmware (23.8.12-0061) for the SPERC 8 state:

Firmware supports VRTX running with single SPERC8 controller;
- Write-back cache is enabled for VDs.
- Does not have high availability capability

Firmware supports VRTX running with dual SPERC8 controller;
- High availability capability present (Both SPERC8 needs to be connected to expanders)
- One of the controllers is active and the other is passive
- Write-back cache is disabled for VDs; only option for write cache is Write Through.


To downgrade from a dual controller to a single controller, I removed the second SPERC8 controller, removed the second expander, re-cabled the SAS connections and downgraded the chassis to 1.25. Just removing the controller and expander wasn't enough, because then I still was unable to set "write-back" as caching policy. As soon as I downgraded to 1.25 I was able to set write-back as caching policy.

With the single controller configuration, the storage performance is as it should be. All the earlier mentioned problems are gone:

• (HyperV) VM's perform excellent
• copy performance is excellent (copying big files within the same volume around 650MB/s max)
• Backup actually works (backup and VSS are related)
• No more VSS time-out errors (0x8000ffff- Catastrophic failure)
• Checkpoint creation of a VM took about 4 seconds

From this experience I currently strongly advise to not go for the dual SPERC8 configuration and stick with the single configuration. Hopefully a firmware update will solve this performance issue, because at the moment having the dual / fault tolerant SPERC 8 configuration with this very poor storage performance makes no sense at all. Yeah, you're fault tolerant... but no backup and bad performing VM's aren't really a good option in a production environment I think...

Or might there be an essential setting I am missing for the write through setup? (I have tried different settings already...)

Below are the benchmarks made using Atto. The difference is tremendous! Funny fact: while doing the benchmark with the write back policy enabled, the virtual disk was still in its background initialization  process, and with the write through it was not. 

NOTE: Both VD's were configured as RAID6