Michael kainer
1 Copper

RE: Performance issue / poor performance: VRTX with Fault Tolerant / Dual Shared PERC 8

Jump to solution

Hi,

I think in the past, with older biosversions, you had the option for enabling and disabling the write cache. I don't how it was working because I have never seen it live. But I think there was a reason for disabling it. I read the manul of vrtx and they had a screenshot of this point. I tried to activate it, but with the newer biosversions you don't have the possiblity to do that.

When I was talking to DELL months ago with my problem, they said at first, they do not know if it is possible to solve the problm with just a firmwareupdate because at that time they had the big probem of keeping the write cache synchron.

0 Kudos
ttoonka
1 Copper

RE: Performance issue / poor performance: VRTX with Fault Tolerant / Dual Shared PERC 8

Jump to solution

Hi,

I should probably share more details.

running ESXi 5.5, tests were performed on Windows 2012, not really relevant information.

When I was talking about Disk Cache, I meant enabling/disabling Cache Disk Policy you can enable on Virtual Disk

0 Kudos
KDMann
1 Copper

RE: Performance issue / poor performance: VRTX with Fault Tolerant / Dual Shared PERC 8

Jump to solution

Ttoonka,

Regarding "My results are not that bad as you have published above."

That is because you are running RAID-10 vs. RAID-6, and (apparently) you have also enabled the on-disk caches as well.  You should not ever do this unless you are prepared to lose data in the on-disk cache!!

In any case, those results are still very bad, RAID-10 performance on writes should be no less than 50% of reads -- so your results look quite ugly for RAID 10 on 10KRPM disks!!!

You should try something like IOmeter that will also tell you response times.  If you do this you would see that response times are 100x longer than with write-back cache setting.

IMHO; The idea that VRTX designers believed that FOUR server blades could share a >>single<< LSI RAID chip was silly to begin with.  That chip was meant to go in a single-server application.  It is the same LSI chip Dell uses on a standard PERC.

What's worse is that that also believed that they could run without write cache!!  The team that developed this box seems to know very little about storage.

KDMann
1 Copper

RE: Performance issue / poor performance: VRTX with Fault Tolerant / Dual Shared PERC 8

Jump to solution

Re: "I meant enabling/disabling Cache Disk Policy you can enable on Virtual Disk".

One should NEVER enable the Cache Disk Policy at the VDisk level, this is only used for applications that can safely lose data in the cache in the event of a bus reset, etc.

Whatever performance you were seeing with that turned on is only relevant in non-production environments.

ttoonka
1 Copper

RE: Performance issue / poor performance: VRTX with Fault Tolerant / Dual Shared PERC 8

Jump to solution

RAID 10 should minimalize the risk to zero of having disks cache enable and fisk failure.

RAID 10 should be writing simultaneously to two disks. If one disk fails, I still have a data in second disk and RAID 10 will recover from this situation.

For poweroutage, one should always use UPS, and disks itself having small capacitor, which will keep a data in their case for a while.

Am I wrong or what else could happen ?

PS: I do agree that the performance is not the greatest as We all would expect.

 

Thx

T.

 

0 Kudos
Michael kainer
1 Copper

RE: Performance issue / poor performance: VRTX with Fault Tolerant / Dual Shared PERC 8

Jump to solution

Hi,

look at this diskussion: http://en.community.dell.com/support-forums/servers/f/906/p/19587459/20644031#20644031

If you don't see the linkg google: Performance issue / poor performance: VRTX with Fault Tolerant / Dual Shared PERC 8

0 Kudos
Blue407
1 Nickel

RE: Performance issue / poor performance: VRTX with Fault Tolerant / Dual Shared PERC 8

Jump to solution

I will be watching this thread with interest as I too have just purchased a 2nd PERC card for my VRTX for redundancy. I will do some testing next week.

Comments:-

@TTOONKA - Think about what you are saying about RAID10 with caching. Data goes from the O/S through the h/w and into the cache on the active PERC, if that PERC were to fail with data in the cache it would be lost forever. Doesn't matter if you have RAID10 or RAID1, the data is not on the disk, it still in the cache that failed.

I can total understand the issue and why Dell have decided to switch off write cache, what they need to do is work out a way of the 2nd PERC card being also on-line and getting the same data as the active card constantly. That way if the active one fails, the backup one continues. However I would imagine this will be extremely difficult to do, it would require the 2 card to be in constant communication to know how much of the 1Mb cache has already been written and where its got to...

In the meantime, I will test my performance with RAI5, if its not good enough I will have to look at following the detailed post above and disable the spare PERC in the CMC, only re-enabling it if the primary fails. That way I can have write caching enabled Smiley Happy

I presume this problem is only with write caching and that read-caching can be on?

0 Kudos
ttoonka
1 Copper

RE: Performance issue / poor performance: VRTX with Fault Tolerant / Dual Shared PERC 8

Jump to solution

@BLUE407, thanks for jumping into the discussion, but you misunderstood me.

I was talking about havin enabled disk cache (not PER8 controller cache) to improve writing speed if used controller configuration is Fault Tolerance > Active/Passive controller.

Means if one active PERC8 controller dies, simultaneously system switch to passive PERC8 which become Active one, no data lost.

To avoid data lost when we have disk cache enabled I believe is enought to use RAID 10, where writes are going at the same time to two different disks (probability that two disks dies at the same time is close to zero).

This was my thought and the space for a discussion.

In general, Dell needs to work on this topic and mirror controller cache to passive controller if they would allow enable it (which is not the case today with Active/Passive configuration).

 

T.

0 Kudos

RE: Performance issue / poor performance: VRTX with Fault Tolerant / Dual Shared PERC 8

Jump to solution

As it seems it was a question of firmware update afterall! The latest firmware found here (or here) states:

Fixes & Enhancements
Fixes:
- Fixed an issue in which Virtual Disks may become inaccessible when migrated to another Power Edge VRTX.
- Fixed an issue in which Virtual Disk to blade server mapping may be deleted after a failover.
- Fixed an issue in which a cable failure in large configurations could cause the Shared PERC8 to fault.
- Fixed an issue in which a redundant VD may be unable to use the "Copy Back" feature after disabling the second Shared PERC8
- Fixed an issue in which I/O traffic wasn't always sent using the "Fast Path" feature.
- Improved Battery Back Up (BBU) messaging in the storage controller logs.
- Fixed an issue in which the rebuild of a redundant virtual disk starts over after a controller failover.

Enhancements:
- VDs can now be configured with write back caching in dual Shared PERC8 configurations
- Increased hard drive predictive failure polling interval to 5 minutes.
- Firmware TTY logs persist across controller and chassis resets.

I will implement the new firmware and test it right away.

0 Kudos
mrosenecker
1 Copper

RE: Performance issue / poor performance: VRTX with Fault Tolerant / Dual Shared PERC 8

Jump to solution

Erik,

Please let us know if it works for you.  I just found this today, as well, and I have a customer that is having random disconnect issues with their vSphere VMs running in a dual Shared PERC config.  

Thanks for being the guinea pig!!

           --Mark

0 Kudos