We are trying to setup some kind of alerting when cluster/datastores reach a certain level but I need some help to figure out the best way to go about it.
First, the way we work with our environment here, is we create multiple 2TB or 3TB datastore per cluster. Our cluster ratio consist of 12 ESX host and we keep a ratio of 12 VM per host.
Once we get a datastore at around 10-15% of free space we deploy on the next datastore.
The issue I am having with the capacity trending using regular dashboard and report, is that I can get report on every datastore with less than 10-15% free space but it will report every datastore with less than 10-15%, even the one that we are fully aware and that we don't want to be alerted on. Same thing with our 144 VM limit per cluster. Once it reach it, it is fine but we dont need to be alerted unless it goes over, as we should be deploying VM to the next cluster.
To give you an idea of our environment, 15 clusters, 68 ESX, 164 Datastores, 190TB, 908 VM.
Now I tried going to the Capacity Management tab under VMWare Environnement, I used the Reports button to run a template on the Cluster/Datastore capacity trend, but it will report on all the datastore even the local datastore that we dont even use.
To try to simplify my question: Am I able to receive a weekly report and/or alarm saying "HEY this datacenter/cluster/datastore is running out of ressource, take action on it !" or "Here is the remaining quantity of VM you can deploy in X cluster and X cluster if you dont increase storage and RAM"
I don't want our team to log every week to take a look at the nice Capacity Management dashboard, I want vFoglight to inform us when there is a need.
Yes, you can do what you want. The very best part of Foglight is that you can do damn near anything.
Here's one option.
We have a custom row-oriented table which our capacity allocation folks use, which displays all of the clusters, and shows various capacity measures like cpu utilization %, vRAM utilization %, non-local disk free space, etc. 23 columns in all.
It uses custom functions to calculate cluster type, cluster full status, VMs remaining to fill ( based on the average size VM already deployed), vRAM available for expansion ( a different calc than VMs to fill, since it only takes the vRAM into account), disk status, etc. IP ranges are included, based on the IP addresses of VMs already deployed, so that the user can check the IP of the new VM to make sure it will fit on the IP ranges available on the cluster.
Our rules are a bit convoluted, and yours are certainly different, but this is definately possible. And once you have the calculations, you can use the same functions in alerts.
So, does this help you get an idea of where you can go with this?
First, we need to know what rules you want to use.
Let's start with vRAM available for expansion of existing VMs.
How this is calculated depends on:
How much RAM are you willing to allocate to vRAM. Given that you pay licensing on vRAM Granted, not used, in general the amount of vRAM you are willing to grant on a cluster is tied in some fashion to how much you have licensed, and in few cases would you want to over-provision your RAM, at least in production. This influences the next item.
How much RAM are you willing to allow to be utilized? ESX Hosts don't like to run with 100% utilized, and going over 100% = ESX swapping, a very bad thing. We limit a cluster to 70% RAM utilized, so when some VMs need more, they don't push the hosts over 100%. The more you grant, the higher the % utilized goes, in general.
As far as setting up values, we know what will be our rules, but I meant showing where to go and what to create in vFoglight to get a capacity planning table like yours.
As of right now, all of our cluster are running under 65% memory utilization, and licensing/vRam is not an issue as we are using Enterprise Plus. And we are far from overprovisioning our RAM as we keep a really conservative approch to our Virtual Infrastructure (12 ESX/Cluster, running a max of 12 VM per host). It keeps the management happy and our environment is really stable. We are running ESX host with 144GB of RAM.
To answer your question, we would like to stay under 80% of RAM utilization for sure.
Ok, so now, create a table displaying 1 row per VMWCluster, with the standard columns you want to be displayed.
first, create this query:
Module
UHG_IT/Components
Name
UHG_All_Clusters
Reference Id
system:uhg20it_components.48
Last Modified Time
Apr 23, 2012 10:54:56 AM CDT
UI Query
No
Ignore Reference Objects
No
Public
Yes
Deprecated
No
Comments
List of all Clusters
Context Help
List of all Clusters
Relevant Role(s)
(none)
Allowed Role(s)
(all)
Select objects of type VMware Cluster from the "(Default)" datasource (of type foglight-5)
from Root of (foglight-5:default)/VMWModel/virtualCenters/clusterCollection/clusters
Then, build your table. This is what I have as the configuration on my table:
Module UHG_IT/ClusterPerformance Name List of VMware Clusters with data for the past 7 days Component Row-Oriented Table Purpose(s) Dashboard, Home Page, Page, Pagelet, Portal, Portlet, Reportlet Comments Primary Input None Additional Context Key Name Evaluate Once Value ClusterList False Query ('UHG_All_Clusters' from the System module UHG_IT/Components) timeRange False Last 7 days Configuration
Rows Context returning "Localized Value" Columns Column Value Context /name, Renderer: 'Normal' from the System module Dashboard Development/Support/Common/UI Table returning "Localized Value" ID Name Cell Alignment Horizontal Left Column Value Context /UHG_Cluster_Type/latest/value returning "Localized Value" ID Cluster_Type Header Localized string (Cluster Type) Column Value Context /UHG_Cluster_Fill_State/latest/value returning "Localized Value" Export Value Context /UHG_Cluster_Fill_State/latest/value returning "Localized Value" ID Cluster_Full Header Localized string (Cluster Full) Column Value Context /esxServerCount/period/average, Renderer: 'Number - No Decimal Point' from the System module UHG_IT/Components returning "Localized Value" ID Host_Count Header Host Count Cell Alignment Horizontal Right Column Value Context /UHG_Cluster_Core_Count/latest/average returning "Localized Value" ID Cores Header Cores Cell Alignment Horizontal Right Column Value Function ('VAR1 div VAR2' from the System module UHG_IT/Components, Renderer: 'KB to TB - No Units - Scale -9' from the System module UHG_IT/Components) On Null Localized string (n/a) {VAR1} Context /totalHz/period/average returning "Localized Value" {VAR2} Context /UHG_Cluster_Core_Count/latest/average returning "Localized Value" ID Core_Speed Header Localized string (Core Speed GHz) Cell Alignment Horizontal Right Column Value Context /virtualMachinesPoweredOnCount/period/average, Renderer: 'Number - No Decimal Point' from the System module UHG_IT/Components returning "Localized Value" ID VMs_Running Header VMs Running Cell Alignment Horizontal Right Column Value Context /virtualMachinesCount/period/average, Renderer: 'Number: No Decimals, No Unit' from the System module Management Server/System returning "Localized Value" ID VMs_Provisioned Header Localized string (VMs Provisioned) Cell Alignment Horizontal Right Column Value Function ('UHG_VMs_To_Fill_Cluster - Current Load incl Granted' from the System module UHG_IT/UHG_VM_Allocation, Renderer: 'Number - No Decimal Point - Red/Grey/Green' from the System module UHG_IT/Components) {VMWCluster} Context returning "Localized Value" ID VMs_To_Fill Header Localized string (VMs to Fill Cluster, Renderer: 'Text Renderer: Strong' from the System module Management Server/Commons) Column Value Context /totalHz/period/average, Renderer: 'KB to GB - No Units - Scale -6' from the System module UHG_IT/Components returning "Localized Value" ID CPU MHz Total Header Localized string (CPU MHz Total) Cell Alignment Horizontal Right Column Value Context /cpuUtilization/period/average, Renderer: 'Percent Renderer 2 Digits No Units' from the System module UHG_IT/Components returning "Localized Value" ID CPU_Used_% Header CPU Used Avg % Cell Alignment Horizontal Right Column Value Function ('UHG_vRAM_Avail_For_Expansion_on_Cluster' from the System module UHG_IT/UHG_VM_Allocation, Renderer: 'Number - No Decimal Point - Red/Grey/Green' from the System module UHG_IT/Components) {timeRange} Context returning "Localized Value" {VMWCluster} Context returning "Localized Value" ID vRAM_Avail_For_Expansion Header Localized string (vRAM Avail For Expansion, Renderer: 'Text Renderer: Strong' from the System module Management Server/Commons) Column Value Context /memoryUtilization/period/average, Renderer: 'Percent Renderer 2 Digits No Units' from the System module UHG_IT/Components returning "Localized Value" ID vRAM_Used_% Header vRAM Used Avg % Cell Alignment Horizontal Right Column Value Function ('VAR1 div VAR2' from the System module UHG_IT/Components, Renderer: 'No Units - Scale -1' from the System module UHG_IT/Components) {VAR1} Context /memoryGranted/period/average returning "Localized Value" {VAR2} Context /totalMemory/period/average returning "Localized Value" ID vRAM_Granted_Perc Header Localized string (vRAM Granted %) Cell Alignment Horizontal Right Column Value Context /memoryGranted/period/average, Renderer: 'KB to GB - No Units - Scale -6' from the System module UHG_IT/Components returning "Localized Value" ID vRAM_Granted Header vRAM Granted GB Cell Alignment Horizontal Right Column Value Context /totalMemory/period/average, Renderer: 'MB to GB - No Units - Scale -3' from the System module UHG_IT/Components returning "Localized Value" ID Total_vRAM Header Total vRAM GB Cell Alignment Horizontal Right Column Value Context /cpuPercentReady/period/average, Renderer: 'Percent Renderer 2 Digits No Units' from the System module UHG_IT/Components returning "Localized Value" ID CPU_%_Ready Header CPU Ready Delay Avg % Cell Alignment Horizontal Right Column Value Context /memoryBalloon/period/average, Renderer: 'KB to GB - No Units - Scale -6' from the System module UHG_IT/Components returning "Localized Value" ID Balloon_vRAM Header Balloon vRAM Avg GB Cell Alignment Horizontal Right Column Value Context /memorySwapUsed/period/average, Renderer: 'KB to GB - No Units - Scale -6' from the System module UHG_IT/Components returning "Localized Value" ID Swap_vRAM Header Swap vRAM Avg GB Cell Alignment Horizontal Right Column Value Context /storageSpaceTotal/period/average, Renderer: 'KB to GB - No Units - Scale -6' from the System module UHG_IT/Components returning "Localized Value" ID Disk_Total Header Localized string (Disk Total TB) Cell Alignment Horizontal Right Column Value Context /storageSpaceUsed/period/average, Renderer: 'KB to GB - No Units - Scale -6' from the System module UHG_IT/Components returning "Localized Value" ID Disk_Used Header Localized string (Disk Used TB) Cell Alignment Horizontal Right Column Value Function ('VAR1 minus VAR2' from the System module UHG_IT/Components, Renderer: 'KB to GB - No Units - Scale -6' from the System module UHG_IT/Components) {VAR1} Context /storageSpaceTotal/period/average returning "Localized Value" {VAR2} Context /storageSpaceUsed/period/max returning "Localized Value" ID Disk_Free Header Localized string (Disk Free TB) Column Value Context /UHG_Cluster_Guest_IP_Ranges/latest/value returning "Localized Value" ID IPRanges Header Localized string (IP Ranges) Header Alignment Horizontal Left Cell Alignment Horizontal Left Selection Configuration Auto Select Policy None Cell Alignment Horizontal Right Large Data Visible Row Limit 500 Large Data Min Column Natural Width 80 Rows Background Color Configuration Alternate Row Background Color #dff1cf Enable Alternate Row Background Color true Default Sort Column ID Name Searching Configuration Enabled true Simple Search Position Left if no actions Portlet Options Scrollbars Auto
Reference Id system:uhg20it_clusterperformance.1 Last Modified Time Jun 29, 2012 6:53:49 AM CDT Public Yes Deprecated No Priority None Relevant Role(s) (none) Allowed Role(s) (all)
When that's done, we'll work on the custom functions.
When you create a WCF table you can define the timerange statically or allow for it to change. Also the renders he is using are several he created himself and some are from the vBundle, the functions are his own it looks like.
john_s_main
132 Posts
696
0
Posted July 18th, 2012 12:00
Yes, you can do what you want. The very best part of Foglight is that you can do damn near anything.
Here's one option.
We have a custom row-oriented table which our capacity allocation folks use, which displays all of the clusters, and shows various capacity measures like cpu utilization %, vRAM utilization %, non-local disk free space, etc. 23 columns in all.
It uses custom functions to calculate cluster type, cluster full status, VMs remaining to fill ( based on the average size VM already deployed), vRAM available for expansion ( a different calc than VMs to fill, since it only takes the vRAM into account), disk status, etc. IP ranges are included, based on the IP addresses of VMs already deployed, so that the user can check the IP of the new VM to make sure it will fit on the IP ranges available on the cluster.
Our rules are a bit convoluted, and yours are certainly different, but this is definately possible. And once you have the calculations, you can use the same functions in alerts.
So, does this help you get an idea of where you can go with this?