clintonskitson
3 Argentium

Powershell Scripts for Collecting, Processing and Federating esxtop storage adapter data

These scripts are meant to assist in simplifying and collecting esxtop storage adapter performance information from ESX/i host(s) in a synchronized manner.  The scripts will allow you to run one command to start performance collections from all hosts at one time.   The data is collected in CSVs and can then be leveraged in many ways.  A script that is also included in this set is one that helps to federate and filter the data to the screen to allow for custom views across hosts and for specific metrics.

Highlights

  • One command to start esxtop collections for all available storage metrics from many hosts
  • See performance information from a fan-out perspective on all ESX/i hosts at one time for paths, adapters, or datastores
  • Synchornized collections happen on a "best attempt" basis where hosts do collections on set intervals
  • Federated visibility of data by way of the show_ script
  • Scalability in allowing for many powershell windows viewing live data coming from CSVs
  • VAAI statistics

Updates

03/05/11 - updated to fix XP job bug, job hanging

MD5: 47CC9CE0500815B7D691C01574C1371C

Labels (1)
0 Kudos
34 Replies
JB-Hans
1 Copper

Re: Powershell Scripts for Collecting, Processing and Federating esxtop storage adapter data

its probably my stupidity but if you can help me overcome that i would be very grateful:)


I have both tried this on a clean desktop and my own desktop.

I can excute the script cust_get_esxtop_adapters for each host fine.

But when i excute the script collect_esxtop_adapters i get the following error

[vSphere PowerCLI] C:\Program Files (x86)\VMware\Infrastructure\vSphere PowerCLI> .\collect_esxtop_adapters.ps1 -esxip esx03,esx04,itxesx05,esx06,es
x09,esx10,esx11 -username root -password blaat
26-2-2011 17:29:01    Get-Datastore        Could not find VIServer with name 'esx03'.
+ CategoryInfo          : ObjectNotFound: (esx03:String) [Get-Datastore], VimException
+ FullyQualifiedErrorId : Core_ObnSelector_SelectObjectByNameCore_ObjectNotFound,VMware.VimAutomation.ViCore.Cmdlets.Commands.GetDatastore

26-2-2011 17:29:01    Get-Datastore        Could not find any of the servers specified by name.
+ CategoryInfo          : ObjectNotFound: (VMware.VimAutom...Server[] Server:RuntimePropertyInfo) [Get-Datastore], ServerObnFailureException
+ FullyQualifiedErrorId : Core_ObnSelector_SetNewParameterValue_ServerSpecifiedButNotFound,VMware.VimAutomation.ViCore.Cmdlets.Commands.GetDatastore

Cannot index into a null array.
+ CategoryInfo          : InvalidOperation: (0:Int32) [], RuntimeException
+ FullyQualifiedErrorId : NullArray

26-2-2011 17:29:01    Get-EsxTop        Could not find VIServer with name 'esx03'.
+ CategoryInfo          : ObjectNotFound: (esx03:String) [Get-EsxTop], VimException
+ FullyQualifiedErrorId : Core_ObnSelector_SelectObjectByNameCore_ObjectNotFound,VMware.VimAutomation.ViCore.Cmdlets.Commands.GetEsxTop

26-2-2011 17:29:01    Get-EsxTop        Could not find any of the servers specified by name.
+ CategoryInfo          : ObjectNotFound: (VMware.VimAutom...Server[] Server:RuntimePropertyInfo) [Get-EsxTop], ServerObnFailureException
+ FullyQualifiedErrorId : Core_ObnSelector_SetNewParameterValue_ServerSpecifiedButNotFound,VMware.VimAutomation.ViCore.Cmdlets.Commands.GetEsxTop

No object has been specified to the get-member cmdlet.
+ CategoryInfo          : CloseError: (:) [Get-Member], InvalidOperationException
+ FullyQualifiedErrorId : NoObjectInGetMember,Microsoft.PowerShell.Commands.GetMemberCommand

Cannot bind argument to parameter 'InputObject' because it is null.
+ CategoryInfo          : InvalidData: (:) [Add-Member], ParameterBindingValidationException
+ FullyQualifiedErrorId : ParameterArgumentValidationErrorNullNotAllowed,Microsoft.PowerShell.Commands.AddMemberCommand

Index operation failed; the array index evaluated to null.
+ CategoryInfo          : InvalidOperation: (:) [], RuntimeException
+ FullyQualifiedErrorId : NullArrayIndex

Index operation failed; the array index evaluated to null.
+ CategoryInfo          : InvalidOperation: (System.Collections.Hashtable:Hashtable) [], RuntimeException
+ FullyQualifiedErrorId : NullArrayIndex

Index operation failed; the array index evaluated to null.
+ CategoryInfo          : InvalidOperation: (:) [], RuntimeException
+ FullyQualifiedErrorId : NullArrayIndex

Index operation failed; the array index evaluated to null.
+ CategoryInfo          : InvalidOperation: (:) [], RuntimeException
+ FullyQualifiedErrorId : NullArrayIndex

26-2-2011 17:29:15    Get-EsxTop        Could not find VIServer with name 'esx03'.
+ CategoryInfo          : ObjectNotFound: (esx03:String) [Get-EsxTop], VimException
+ FullyQualifiedErrorId : Core_ObnSelector_SelectObjectByNameCore_ObjectNotFound,VMware.VimAutomation.ViCore.Cmdlets.Commands.GetEsxTop

26-2-2011 17:29:15    Get-EsxTop        Could not find any of the servers specified by name.
+ CategoryInfo          : ObjectNotFound: (VMware.VimAutom...Server[] Server:RuntimePropertyInfo) [Get-EsxTop], ServerObnFailureException
+ FullyQualifiedErrorId : Core_ObnSelector_SetNewParameterValue_ServerSpecifiedButNotFound,VMware.VimAutomation.ViCore.Cmdlets.Commands.GetEsxTop

No object has been specified to the get-member cmdlet.
+ CategoryInfo          : CloseError: (:) [Get-Member], InvalidOperationException
+ FullyQualifiedErrorId : NoObjectInGetMember,Microsoft.PowerShell.Commands.GetMemberCommand

Cannot bind argument to parameter 'InputObject' because it is null.
+ CategoryInfo          : InvalidData: (:) [Add-Member], ParameterBindingValidationException
+ FullyQualifiedErrorId : ParameterArgumentValidationErrorNullNotAllowed,Microsoft.PowerShell.Commands.AddMemberCommand

Index operation failed; the array index evaluated to null.
+ CategoryInfo          : InvalidOperation: (:) [], RuntimeException
+ FullyQualifiedErrorId : NullArrayIndex

Index operation failed; the array index evaluated to null.
+ CategoryInfo          : InvalidOperation: (System.Collections.Hashtable:Hashtable) [], RuntimeException
+ FullyQualifiedErrorId : NullArrayIndex

Index operation failed; the array index evaluated to null.
+ CategoryInfo          : InvalidOperation: (:) [], RuntimeException
+ FullyQualifiedErrorId : NullArrayIndex

Index operation failed; the array index evaluated to null.
+ CategoryInfo          : InvalidOperation: (:) [], RuntimeException
+ FullyQualifiedErrorId : NullArrayIndex

Index operation failed; the array index evaluated to null.
+ CategoryInfo          : InvalidOperation: (:) [], RuntimeException
+ FullyQualifiedErrorId : NullArrayIndex

0 Kudos
clintonskitson
3 Argentium

Re: Powershell Scripts for Collecting, Processing and Federating esxtop storage adapter data

I think there is an easy fix for this one.  The following line, "Get-Datastore        Could not find VIServer with name 'esx03'."  shows that powercli is having a problem finding the host by the name that you referenced.  Can you try running a "ping esx03" first and see if it gets replies?  If you don't have DNS or lookups working from the system that is running powershell, you can enter the IP addresses of the esx servers instead.  You said that running the cust_get_esxtop_adapters worked by itself?  Are you sure that running ".\cust_get_esxtop_adapters.ps1 -esxhost esx03 -esxuser root -esxpassword xx -interval 5" worked?

Thanks

0 Kudos
JB-Hans
1 Copper

Re: Powershell Scripts for Collecting, Processing and Federating esxtop storage adapter data

hi, ok, i`m not that stupid:) no dns works fine, besides that i have the same problem connecting by ip.

BUT i first need to have run connect-viserver for every host...

I just tried installing vsphere 4.1.1 on the vcenter host (its also running the dns server for my vmware network) first thing i tried, pinging which worked.

Then get-datacenter -server esx03.domain.local, "could not find viserver with name 'esx03.domain.local'

Then get-datacenter -vmhost esx03.domain.local "you are not currently connected to any servers. Please connect first using connect-viserver or one of its aliases"

then i did: connect-viserver esx03.domain.local, now get-datacenter -server esx03.domain.local works.

then i did: get-datastore -server esx04.domain.local, "could not find viserver with name 'esx04.domain.local'

so i tried:

connect-viserver viserver01.domain.local

then: get-datastore -server esx04.domain.local, "could not find viserver with name 'esx04.domain.local'

BUT: get-datastore -vmhost esx04.domain.local works...

yes the ".\cust_get_esxtop_adapters.ps1 -esxhost esx03 -esxuser root -esxpassword xx -interval 5" works for every host... BUT!!!! i need to do a connect-viserver to every host first otherwise it wont work...

0 Kudos
clintonskitson
3 Argentium

Re: Powershell Scripts for Collecting, Processing and Federating esxtop storage adapter data

Thanks for hanging with me on this one.  One of the downsides to collecting esxtop information via powershell is that you need to use the "-server" parameter which means that it requires you specify credentials to an esxhost instead of an all encompassing vcenter instance when connecting via connect-viserver.  My problem in the script was that I wasn't checking to make sure the credentials that were specified actually successfuly logged you in to the esxhost.  In your case, by logging in directly via connect-viserver before running the script, the connet-viserver that was kicked off in the script was failing and didn't matter since you were already logged in.  Since I am kicking the cust_get_esxtop_adapters scripts off as jobs, they count as separate instances of powercli and thus need to actually successfully login and cannot use existing connect-viserver sessions.  I have uploaded an updated zip that should error out on you when running the cust_get_esxtop_adapters script as you did before saying the credentials are wrong (since esx denied them).  Once you can successfuly run the script (make sure you -esxpassword is correct), then all should be good under the collection script.  This "should" work 😃

Great info thanks for working through it.

0 Kudos
JB-Hans
1 Copper

Re: Powershell Scripts for Collecting, Processing and Federating esxtop storage adapter data

ok great! np all doing to for the greater good! (read "EQL"!!! :))

i`ll try soon and let you know..

0 Kudos
JB-Hans
1 Copper

Re: Powershell Scripts for Collecting, Processing and Federating esxtop storage adapter data

sorry..

the individual script works, but the collect script gives the error:

C:\Program Files (x86)\VMware\Infrastructure\vSphere PowerCLI> .\collect_esxtop_adapters.ps1 -esxip esx03,esx04,itxesx05,esx06,esx09,esx10,esx11 -username root -password

ESXTOP collection and processing

an EMC vSpecialist Production
Everything VMware at EMC
powershell window will have jobs running in the background, either run get-job | stop-job or close the window to stop get-esxtop collections
Expect to see significantly high CPU utilization on different cores of this system during the first sampling for each host, this is due to loading datastore in
formation
Load will lessen, but will still be noticeable after the first sampling
This may take up to 30 seconds per ESX server to begin samplings
Stopping previously running jobs in this powershell window before continuing


Id              Name            State      HasMoreData     Location             Command
41              esx03        Running    True            localhost            ...

Kicked off job and waiting for first sampling to complete before moving on..
Failed to connect to esx03 using credentials specified.

0 Kudos
JB-Hans
1 Copper

Re: Powershell Scripts for Collecting, Processing and Federating esxtop storage adapter data

btw, shouldnt you get the password as a secure string?

0 Kudos
clintonskitson
3 Argentium

Re: Powershell Scripts for Collecting, Processing and Federating esxtop storage adapter data

Sure, this was an easy change that I added and might fix your issue as well.  I am assuming there was something going on when passing your password from the collect to the cust script.  If you specify a "-securepw"  instead of -username and -password the cust_get_esxtop_adapters script will now prompt you for a password in a popup window and pass the credentails securely.

0 Kudos
JB-Hans
1 Copper

Re: Powershell Scripts for Collecting, Processing and Federating esxtop storage adapter data

ok great! its working!!! How long should i collect?

0 Kudos