UNSOLVED

doug.hairfield

updated

10 years ago

0

110960

March 2nd, 2012 05:00

Tier 1 Full

sometimes I see that all our tier1 is full, RAID5 and RAID10.  Should I expect to see some performance degradation if all my Tier1 is at 100%?  Right now we just run a manual dataprogression cycle to move some stuff down to Tier3 and some times that helps if there is stuff we can move down, sometimes there isn't anything to move down. 

  • 3842

    2

    Posted March 2nd, 2012 06:00

    Doug,

    Yes, you will definitely see a performance degradation when your tier one is full.  In essence, when it gets full your lower tiered drives (the slower ones) will try to accommodate the work load.  

    I'd strongly recommend re-evaluating your data progression profiles.  Try to keep your tier 1 available for your writes, and use lower tiers for your read data.  

    If you need additional help with this, let me know.

  • jsterry01

    2 Posts

    3842

    0

    Posted March 3rd, 2012 08:00

    We were having the same issue and modified our storage profiles which cleaned out all but about 5% in Tier 1 Raid 10 (SSD) according to Enterprise Manager under Storage Types.  However, we noticed that no data is migrating to Raid 5 in Tier 1 additionally under Disks in Enterprise Manager under SLSSD it shows that they are 99% allocated.  Any idea what's happening?  Seems like conflicting info.

  • 3842

    0

    Posted March 14th, 2012 09:00

    Hey, thanks for the reply.  Let me take this question a little further.

    If our Tier1 write is full, for both standard and fast track, but the Tier1 Reply/Read still has some room left, which leaves Tier1 with some free space over all, will the write start to cannibalize the remaining space for the reads to use for writes?

  • jsterry01

    2 Posts

    3842

    0

    Posted March 14th, 2012 09:00

    Copilot helped us figure this out.  When the array was installed it was installed with a Raid 5-9 setup in tier 1, however there wasn't enough SSDs to support that.  Therefore when data progression would run it would just kick the Raid 10 data in tier 1 down to Raid 5 in tier 2.  They changed tier 1 to be Raid 5-5 and that resolved the issue. Thanks.

  • hallidayr

    48 Posts

    3842

    0

    Posted March 15th, 2012 08:00

    I had a similar thing happen to me on several occasions - except it was in Tier 3. We have only 5 active SATA disks and the default was RAID5-9. Tier1 filled up because no data could get migrated down. Annoyingly this appears to be some sort of default, we had the same thing corrected twice on a single storage center.

  • keith_s_l

    5 Posts

    3847

    0

    Posted May 3rd, 2012 09:00

    When performance degradation occurrs because Teir1 gets full, and the lower tiered drives (the slower ones) will try to accommodate the work load for the writes (and the reads...)

    ... just how much degredation are we talking about?   Are we talking about a fractional percentage degradation in the 1% to 100% range?   Or are we talking about a performance degradation of several orders of magnitude, or hundreds of orders of magnitude?

  • hallidayr

    48 Posts

    3847

    0

    Posted May 3rd, 2012 12:00

    It really depends on your disk configuration. In our environment we'd see a significant hit to I/O if Tier 1 filled up. Our Tier 1 has 11 active 15K FC disks. If that filled up then writes would be done to 5 active 7200 RPM SATA disks. Reads would still come from wherever the data lives.

  • keith_s_l

    5 Posts

    3847

    0

    Posted May 3rd, 2012 13:00

    One of our arrays, for example, is all 10K drives, so the differences in the tiers is essentiall just raid10 vs raid5.

    During data progression, we go from writes completing in 2 milliseconds per write, to hundreds and even thousands of milliseconds per writes...  For most types of writes, the array essentially becomes un-writable.

  • keith_s_l

    5 Posts

    3847

    0

    Posted May 7th, 2012 13:00

    What ways can data-progression be configured, so that data progression can occur, but not monopolize the back-end IO while it is occuring?  It is just while data progression is running that our performance degredation is unbearable.  Can data progression be limited to a single thread, instead of 5 or 8 threads, and be allowed to run whenever it needs to, instead of starting a specific time and running as long as it takes?

  • hallidayr

    48 Posts

    1018

    0

    Posted May 8th, 2012 08:00

    I believe data progression can be configured in a few ways, but the settings are hidden. You'll need to get Copilot to make those changes.