Trending Available VMs based on CPU, Memory and Disk

Hi John,

There are a couple of use case I have for this, one would be to create trended graphs per Cluster for the number of Available VMs based on Storage, CPU, and Memory for the set amount of days I want to see.  The second use case would be to run a daily report to extract the raw data to a CSV file for the current days availability metrics based on the available VMs for Storage, Mem, and CPU.

Thanks,

Joshua

I've done something similar to what you are requesting.

Can you give me more information on what exactly you want to accomplish?

Hmmm.. I have functions created to do things like:

• calculate how many VMs, of a size similar to what's already on a cluster, which will fit on that cluster. We are calculating based on a 7 day average VM usage base, for vCPU Utilization, vRAM Utilization, vRAM Granted, and contiguous disk space available on non-local datastores.  It could easily be expanded to incorporate vCPU to physical core ratio, etc, but we've not seen any real limitation except vRAM in our environment.
• using the same base, calculate the amount of cluster vRAM which can be used to expand VMs before reaching maximum thresholds.

How would you calculate the new VM load on your cluster into the future?

(This is assuming you don't have enough new VMs being provisioned every day to create a statistically valuable trend - we don't, and we have 13,000 VMs deployed.  We can calculate a statistically valid trend based on a week's deployments across our entire environment, but still have to take into account the guy who wants 2 TB of disk added to his VM out of the blue, or the project to deploy 100 new VMs with 8 cores and 32GB ram each.)

Anyway,  answering your questions.

1. Trended graphs per cluster - as with anything in Foglight, you need a data source which includes all of the datapoints to be displayed.  Foglight generally uses data from agents to create historical charts, or will use various statistical methods to create an artificial dataset showing the projected values into the future.  This will take more conversation.

2. Daily report to extract  raw data to .csv:   I'm putting the finishing touches on an easier way to do just what you're looking for, will update this over the next few days as I get it ready for someone else to use.

The functions you have seem to be what I am looking for my environment. From a vCPU/Core Ratio, I find that ESX starts to show contention around a certain ratio point.  This is based on seeing performance issues on a Host that had only ~50% CPU utilization but VM's Experiencing 25-30% CPU Ready.  The tricky part is that this ratio could change depending on how many single processor VMs reside on the host vs multiple processor machines.  As you know when adding an 8vCPU VM it needs 8 logical cores in parallel which definitely could cause greater CPU contention on the host if you have more and more larger vCPU machines.

Trended graphs per cluster - I would take the daily reports in csv format and upload them to our Enterprise Data Warehouse, from there we can use Business Objects and create trended graphs per cluster and possibly per host depending on the level of detail the vFoglight scripts go into.

Would you have some time to chat over the phone and/or share your scripts?

Thanks,

Joshua

I've seen that 1:4 physical to virtual CPU is about the most you want to do, maybe less if you have VMs with more cores than exist in a single chip socket in your ESX host.  It's scheduler contention which creates the % ready issue, making your VM wait for the correct configuration of processors to all be available at the exact same timeslice. 

Another issue not mentioned is core to chip ratio: e.g. if you have dual core CPUs, try to keep the number of VMs with more than 2 vCPU to a minimum; quad-core, keep em 4 cores or less, etc. 

If you put the data into Business Objects, what formula would you use to project out the future data points?  Can do the same in vFoglight, just need to know what method you want to use.

Just finishing up the ability to import a config file, then will post the script.