Basic questions un-answered by storage scope?

In an effort to get this thread closed out (i'm now on year two of trying for this answer), maybe we, as the customers, can show EMC how to get the the basic numbers required to generate reports and perhaps they can release some query or create a tool that will jive.  There are, of course, several variables, but we'll start with the easiest and work our way up.  I'm going to offer suggestion on how to get the numbers, please, I'm begging, jump in and correct me if I'm off.

Specific to the DMX platforms:

How much usable capacity did I buy (post BIN):

Record the following numbers from Control Center:

Configured Capacity (Control Center properties tab).

Mapped allocated/possibly allocated (Control Center properties tab, export to excel then you can filter for the next two).

Mapped unallocated BCV/RDF

Mapped unallocated regular (non replicated).

Unmapped Regular (Again, Control Center properties tab, export to excel and filter.

Unmapped Replicated.

System devices. (Control Center properties tab, add them up).

So, here is the math:

Usable = (configured) - (system devices)

Allocated = (mapped allocated/Possibly allocated) + (Mapped unallocated BCV/RDF)+(unmapped replicated)

Available = (unmapped regular)+(mapped unallocated regular)

Assumptions and gotchas

You don't make a BCV unless you are using a BCV (this will show as allocated space).

When you export the sheet use the smaller view (not symmetrix devices to avoid double counts by FA).

Watch your meta's and hypers on the output.  You can filter for Meta = N/A and drag all the hypers over so you can calculate with one column.

I take unconfigured (theoritically you can use this space, but more than likely, you never will due to lack of splits) system devices, and spares and throw it into "overhead".

Delete all gatekeepers (they are lost in a rounding error if your using TB unless you have mad amounts).

For percentage allocated, allocated / usable.

For percentage available, available / usable.

The reason I do these last two sperately, is I can check my work.  Since the percentages are calculated seperately, and usable almost never changes, if they don't add up to 100%, you'll know you went wrong.

I've got this down to where i can do 6 DMX's in about 1 1/2 hours.

Thanks for your input.   Can you please define the term allocated and if the definition holds good both in a Symmetrix and a Clariion world?

I must always keep in mind my audience for reports who look at it in business terms, CTO, CFO and Storage manager.  Storage Admins do not run reports for storage admins.  There are three numbers that are critical for planning, staffing, floor space, power, HVAC and budgeting.  If you can visualize a brand new server sitting on the floor next to your brand new frame.

Overhead: Space that customer cannot use.  Systems devices, spares, etc..

Usable capacity: How much total disk space can I allocate to a host, appliance, or function.  This would NOT include hot spares, system devices, etc.

                             Example: When the CTO of the company says, "How much storage is currently on the floor?"  This is the number he needs.  He cares not about system devices.

Allocated: How much space in my frame is used by a host, appliance, or function. This is space that cannot be allocated to a new host.

                              When the CTO asks, "How much space are we using up?" This would be his number.

Available: How much space in my frame can be allocated to the new host, appliance, or function.

                              CTO, CFO, Storage Manager.  "How much space can I use for upcoming projects so I can decide how much more we have to purchase."

Allocated and Availble add up to equal usable capacity.

You had asked for definitions that apply to both the DMX and the Clariion.  Given that the above definitions are not platform (or vendor) specific, I'm not sure the question applies.  I think what we are looking for from the smart folks is how to get those three numbers.

Again, Storage administrators do not run reports for other storage administrators.  Keep in mind the audience.

Another opportunity is  earmarked storage or storage frames part of Tech refresh. Coding outside the box seems to be the only alternative for customized reports. Not properly accounting for these items will  affect overall available storage numbers for provisioning , reporting and trending.

Classifying tiers , availability and cost of storage within a tiered category isn't possible.

Most of all the reporting required to effectively report requires coding outside of ECC.

Multi vendor support isnt consistent  for various flavors of arrays within the sts tables.   Symm, HDS, CLariion , Centera etc.

Oh dont let me forget.  Shared storage requires addtional coding as well.

There are many pieces to the vacuum as stated before depending on customer / business requirements.

These issues are very cumbersome in our large dynamic environments with multiple instances of ECC.

If Host utilization is required . A whole other challenge arises. Host agent management and LDT.   Storagescope will continue to report on Stale data and doesn't sync multi agent deployments.

Yet another area of customized reporting.

Ecc agents cannot truly determine host device utilization. Only FS utilization as a whole.  Certain groups would like to know device utilization by device/tier level for reclamation opportunities. Complex code is required to try an determine these values to get close to real numbers.

Regards,

-j

Joe_M wrote: Classifying tiers , availability and cost of storage within a tiered category isn't possible. (...) Ecc agents cannot truly determine host device utilization. Only FS utilization as a whole.  Certain groups would like to know device utilization by device/tier level for reclamation opportunities. Complex code is required to try an determine these values to get close to real numbers.

The ECC 6.1 UB7 release notes mentioned there is improved tier based reporting now in STS where you can now define tiers. I initially thought it was something to do with FAST only but apparently not according to a colleague. I haven't had a look at UB7 yet to verify.

You can also assign a $ value to a host/lun tiering report by using a custom calculated column in query builder e.g. multiply GB field by X for $X per GB ?

With the device utilization and reclaimation I am not sure what you mean? When I have gone hunting for orphaned luns which are mapped and masked to a server but never used I found the host agent was invaluable to identify this. Other stuff like finding luns which have an empty unused VG on them is also good.

So, here is the math: Usable = (configured) - (system devices) Allocated = (mapped allocated/Possibly allocated) + (Mapped unallocated BCV/RDF)+(unmapped replicated) Available = (unmapped regular)+(mapped unallocated regular)

I have been through some of this before and its admittedly tedious trying to figure out how to get the right outputs

One thing I have found is that the "allocated" flag for DMX devices is very useful in there is a decent amount of intelligence in its algorithm? If its an RDF or BCV device it will do some checks to figure out if its allocated "logically" somewhere even if its not mapped or masked? e.g. for an R2 it will check if the R1 is mapped or masked? or BCV I think it does a similar thing against the source dev or checks symdgs on symm agent host? (cant remember exactly, its in the console online help)

When it comes to finding accurate available (usable) devs I found I had to:

1) use query builder and filter dialogue to filter out any system devs when determining available/usable capacity

2) manually create dev storage pools in ECC console (via SPS) using BIN file knowledge of tiering and device ranges which then will appear in STS

I want to assure everyone that this thread has been of great interest to the StorageScope team.  We're definitely interested in helping you get the answers to these questions, via a Query Builder query in v6 and as part of the built-in reports in SRM7 when it arrives.  I plan to ask the members of the forum, and those on this thread in particular, for help when it comes time to re-work the utilization queries in SRM7 (we aren't to that stage yet).

As for the tiering comments from Joe_M:

UB7 now includes some built-in tiering capability.  You can define tier criteria based on array type, disk technology (Flash/FC/SATA), disk size, and RAID protection.  You can also import SymmTier definitions from a FAST-compatible Symmetrix and map them to tier definitions.  SScope will then scan all your LUNs and map them to the appropriate tier.  Built-in queries show tiered storage by host and storage group for chargeback purposes.  We also provide tiered storage by array or by site for capacity planning purposes.  We're planning a live demo of this capability in January for this forum.

Keep the suggestions coming -- we're listening!

I think this should flow into a "request for enhancement".

After reading some of the posts here, I think we have an issue at the most basic level.

I agree with the EMC posts (and countless conversations) that it is extremely difficult to report "Allocated space" because the definition changes (at a query level) between EMC, HP, HDS, IBM, etc.  Indeed, the queries fall apart even in the comparison of DMX and Clariion's.

Could this be solved by simply changing the basics?

Instead of #arrays.allocated, would it be more appropriate to use:

#array.symmetrix.allocated

#array.clariion.allocated

#array.eva.allocated

I don't believe it is possible to apply a common defenition world wide.

I do plan on wrapping this up into a request for enhancement, so would appreciate any feedback.

nfwtnit

I think you raise some good points

Does anyone know if this https://powerlink.emc.com/nsepn/webapps/btg548664833igtcuup4826/km/live1//en_US/Legacy_Conversion/PL_Support/Doc_Library/300-000-651_a04_C1110.pdf is still valid?

It is from 2004 but does all the decriptions between the different array types still hold true?