Unsolved
This post is more than 5 years old
116 Posts
0
20653
Powershell Scripts for Collecting, Processing and Federating esxtop storage adapter data
These scripts are meant to assist in simplifying and collecting esxtop storage adapter performance information from ESX/i host(s) in a synchronized manner. The scripts will allow you to run one command to start performance collections from all hosts at one time. The data is collected in CSVs and can then be leveraged in many ways. A script that is also included in this set is one that helps to federate and filter the data to the screen to allow for custom views across hosts and for specific metrics.
Highlights
- One command to start esxtop collections for all available storage metrics from many hosts
- See performance information from a fan-out perspective on all ESX/i hosts at one time for paths, adapters, or datastores
- Synchornized collections happen on a "best attempt" basis where hosts do collections on set intervals
- Federated visibility of data by way of the show_ script
- Scalability in allowing for many powershell windows viewing live data coming from CSVs
- VAAI statistics
Updates
03/05/11 - updated to fix XP job bug, job hanging
MD5: 47CC9CE0500815B7D691C01574C1371C
JB-Hans
9 Posts
0
February 26th, 2011 08:00
its probably my stupidity but if you can help me overcome that i would be very grateful:)
I have both tried this on a clean desktop and my own desktop.
I can excute the script cust_get_esxtop_adapters for each host fine.
But when i excute the script collect_esxtop_adapters i get the following error
[vSphere PowerCLI] C:\Program Files (x86)\VMware\Infrastructure\vSphere PowerCLI> .\collect_esxtop_adapters.ps1 -esxip esx03,esx04,itxesx05,esx06,es
x09,esx10,esx11 -username root -password blaat
26-2-2011 17:29:01 Get-Datastore Could not find VIServer with name 'esx03'.
+ CategoryInfo : ObjectNotFound: (esx03:String) [Get-Datastore], VimException
+ FullyQualifiedErrorId : Core_ObnSelector_SelectObjectByNameCore_ObjectNotFound,VMware.VimAutomation.ViCore.Cmdlets.Commands.GetDatastore
26-2-2011 17:29:01 Get-Datastore Could not find any of the servers specified by name.
+ CategoryInfo : ObjectNotFound: (VMware.VimAutom...Server[] Server:RuntimePropertyInfo) [Get-Datastore], ServerObnFailureException
+ FullyQualifiedErrorId : Core_ObnSelector_SetNewParameterValue_ServerSpecifiedButNotFound,VMware.VimAutomation.ViCore.Cmdlets.Commands.GetDatastore
Cannot index into a null array.
+ CategoryInfo : InvalidOperation: (0:Int32) [], RuntimeException
+ FullyQualifiedErrorId : NullArray
26-2-2011 17:29:01 Get-EsxTop Could not find VIServer with name 'esx03'.
+ CategoryInfo : ObjectNotFound: (esx03:String) [Get-EsxTop], VimException
+ FullyQualifiedErrorId : Core_ObnSelector_SelectObjectByNameCore_ObjectNotFound,VMware.VimAutomation.ViCore.Cmdlets.Commands.GetEsxTop
26-2-2011 17:29:01 Get-EsxTop Could not find any of the servers specified by name.
+ CategoryInfo : ObjectNotFound: (VMware.VimAutom...Server[] Server:RuntimePropertyInfo) [Get-EsxTop], ServerObnFailureException
+ FullyQualifiedErrorId : Core_ObnSelector_SetNewParameterValue_ServerSpecifiedButNotFound,VMware.VimAutomation.ViCore.Cmdlets.Commands.GetEsxTop
No object has been specified to the get-member cmdlet.
+ CategoryInfo : CloseError: (:) [Get-Member], InvalidOperationException
+ FullyQualifiedErrorId : NoObjectInGetMember,Microsoft.PowerShell.Commands.GetMemberCommand
Cannot bind argument to parameter 'InputObject' because it is null.
+ CategoryInfo : InvalidData: (:) [Add-Member], ParameterBindingValidationException
+ FullyQualifiedErrorId : ParameterArgumentValidationErrorNullNotAllowed,Microsoft.PowerShell.Commands.AddMemberCommand
Index operation failed; the array index evaluated to null.
+ CategoryInfo : InvalidOperation: (:) [], RuntimeException
+ FullyQualifiedErrorId : NullArrayIndex
Index operation failed; the array index evaluated to null.
+ CategoryInfo : InvalidOperation: (System.Collections.Hashtable:Hashtable) [], RuntimeException
+ FullyQualifiedErrorId : NullArrayIndex
Index operation failed; the array index evaluated to null.
+ CategoryInfo : InvalidOperation: (:) [], RuntimeException
+ FullyQualifiedErrorId : NullArrayIndex
Index operation failed; the array index evaluated to null.
+ CategoryInfo : InvalidOperation: (:) [], RuntimeException
+ FullyQualifiedErrorId : NullArrayIndex
26-2-2011 17:29:15 Get-EsxTop Could not find VIServer with name 'esx03'.
+ CategoryInfo : ObjectNotFound: (esx03:String) [Get-EsxTop], VimException
+ FullyQualifiedErrorId : Core_ObnSelector_SelectObjectByNameCore_ObjectNotFound,VMware.VimAutomation.ViCore.Cmdlets.Commands.GetEsxTop
26-2-2011 17:29:15 Get-EsxTop Could not find any of the servers specified by name.
+ CategoryInfo : ObjectNotFound: (VMware.VimAutom...Server[] Server:RuntimePropertyInfo) [Get-EsxTop], ServerObnFailureException
+ FullyQualifiedErrorId : Core_ObnSelector_SetNewParameterValue_ServerSpecifiedButNotFound,VMware.VimAutomation.ViCore.Cmdlets.Commands.GetEsxTop
No object has been specified to the get-member cmdlet.
+ CategoryInfo : CloseError: (:) [Get-Member], InvalidOperationException
+ FullyQualifiedErrorId : NoObjectInGetMember,Microsoft.PowerShell.Commands.GetMemberCommand
Cannot bind argument to parameter 'InputObject' because it is null.
+ CategoryInfo : InvalidData: (:) [Add-Member], ParameterBindingValidationException
+ FullyQualifiedErrorId : ParameterArgumentValidationErrorNullNotAllowed,Microsoft.PowerShell.Commands.AddMemberCommand
Index operation failed; the array index evaluated to null.
+ CategoryInfo : InvalidOperation: (:) [], RuntimeException
+ FullyQualifiedErrorId : NullArrayIndex
Index operation failed; the array index evaluated to null.
+ CategoryInfo : InvalidOperation: (System.Collections.Hashtable:Hashtable) [], RuntimeException
+ FullyQualifiedErrorId : NullArrayIndex
Index operation failed; the array index evaluated to null.
+ CategoryInfo : InvalidOperation: (:) [], RuntimeException
+ FullyQualifiedErrorId : NullArrayIndex
Index operation failed; the array index evaluated to null.
+ CategoryInfo : InvalidOperation: (:) [], RuntimeException
+ FullyQualifiedErrorId : NullArrayIndex
Index operation failed; the array index evaluated to null.
+ CategoryInfo : InvalidOperation: (:) [], RuntimeException
+ FullyQualifiedErrorId : NullArrayIndex
clintonskitson
116 Posts
0
February 26th, 2011 17:00
I think there is an easy fix for this one. The following line, "Get-Datastore Could not find VIServer with name 'esx03'." shows that powercli is having a problem finding the host by the name that you referenced. Can you try running a "ping esx03" first and see if it gets replies? If you don't have DNS or lookups working from the system that is running powershell, you can enter the IP addresses of the esx servers instead. You said that running the cust_get_esxtop_adapters worked by itself? Are you sure that running ".\cust_get_esxtop_adapters.ps1 -esxhost esx03 -esxuser root -esxpassword xx -interval 5" worked?
Thanks
JB-Hans
9 Posts
0
February 26th, 2011 23:00
hi, ok, i`m not that stupid:) no dns works fine, besides that i have the same problem connecting by ip.
BUT i first need to have run connect-viserver for every host...
I just tried installing vsphere 4.1.1 on the vcenter host (its also running the dns server for my vmware network) first thing i tried, pinging which worked.
Then get-datacenter -server esx03.domain.local, "could not find viserver with name 'esx03.domain.local'
Then get-datacenter -vmhost esx03.domain.local "you are not currently connected to any servers. Please connect first using connect-viserver or one of its aliases"
then i did: connect-viserver esx03.domain.local, now get-datacenter -server esx03.domain.local works.
then i did: get-datastore -server esx04.domain.local, "could not find viserver with name 'esx04.domain.local'
so i tried:
connect-viserver viserver01.domain.local
then: get-datastore -server esx04.domain.local, "could not find viserver with name 'esx04.domain.local'
BUT: get-datastore -vmhost esx04.domain.local works...
yes the ".\cust_get_esxtop_adapters.ps1 -esxhost esx03 -esxuser root -esxpassword xx -interval 5" works for every host... BUT!!!! i need to do a connect-viserver to every host first otherwise it wont work...
JB-Hans
9 Posts
0
February 27th, 2011 08:00
ok great! np all doing to for the greater good! (read "EQL"!!! :))
i`ll try soon and let you know..
clintonskitson
116 Posts
0
February 27th, 2011 08:00
Thanks for hanging with me on this one. One of the downsides to collecting esxtop information via powershell is that you need to use the "-server" parameter which means that it requires you specify credentials to an esxhost instead of an all encompassing vcenter instance when connecting via connect-viserver. My problem in the script was that I wasn't checking to make sure the credentials that were specified actually successfuly logged you in to the esxhost. In your case, by logging in directly via connect-viserver before running the script, the connet-viserver that was kicked off in the script was failing and didn't matter since you were already logged in. Since I am kicking the cust_get_esxtop_adapters scripts off as jobs, they count as separate instances of powercli and thus need to actually successfully login and cannot use existing connect-viserver sessions. I have uploaded an updated zip that should error out on you when running the cust_get_esxtop_adapters script as you did before saying the credentials are wrong (since esx denied them). Once you can successfuly run the script (make sure you -esxpassword is correct), then all should be good under the collection script. This "should" work =)
Great info thanks for working through it.
JB-Hans
9 Posts
0
February 27th, 2011 08:00
sorry..
the individual script works, but the collect script gives the error:
C:\Program Files (x86)\VMware\Infrastructure\vSphere PowerCLI> .\collect_esxtop_adapters.ps1 -esxip esx03,esx04,itxesx05,esx06,esx09,esx10,esx11 -username root -password
ESXTOP collection and processing
an EMC vSpecialist Production
Everything VMware at EMC
powershell window will have jobs running in the background, either run get-job | stop-job or close the window to stop get-esxtop collections
Expect to see significantly high CPU utilization on different cores of this system during the first sampling for each host, this is due to loading datastore in
formation
Load will lessen, but will still be noticeable after the first sampling
This may take up to 30 seconds per ESX server to begin samplings
Stopping previously running jobs in this powershell window before continuing
Id Name State HasMoreData Location Command
41 esx03 Running True localhost ...
Kicked off job and waiting for first sampling to complete before moving on..
Failed to connect to esx03 using credentials specified.
JB-Hans
9 Posts
0
February 27th, 2011 09:00
btw, shouldnt you get the password as a secure string?
clintonskitson
116 Posts
0
February 27th, 2011 13:00
Sure, this was an easy change that I added and might fix your issue as well. I am assuming there was something going on when passing your password from the collect to the cust script. If you specify a "-securepw" instead of -username and -password the cust_get_esxtop_adapters script will now prompt you for a password in a popup window and pass the credentails securely.
JB-Hans
9 Posts
0
February 28th, 2011 08:00
ok great! its working!!! How long should i collect?
clintonskitson
116 Posts
0
February 28th, 2011 09:00
Great news, glad to hear it. We would like to see a 24 hour period if possible.
Thanks
fjacquet1
3 Posts
0
March 1st, 2011 03:00
Hello
I have those exceptions :
Am I supposed to have a specific timezone / regional settings ? I am running ESXi 4.1 on the 7 hosts
benwayj
56 Posts
0
March 1st, 2011 12:00
I'm running from a XP desktop with the CLI, powershell 2, and .net installed.
Running it from the powershell ISE
I'm running :
.\collect_esxtop_adapters.ps1 -esxip 10.30.34.70,10.30.34.45,10.30.34.47,172.18.9.5,172.18.9.10,172.18.9.15,172.18.9.20 -username root -password password
The output on screen is:
clintonskitson
116 Posts
0
March 1st, 2011 13:00
It looks like powershell is having a hard time loading PowerCLI. Try this command in a powershell window "Add-PSSnapin VMware.VimAutomation.Core". How long does it take to load if successful? Have you been able to use PowerCLI at all? Can you do a connect-viserver ?
"Timed out waiting for a program to execute. The command being executed was "C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\csc.exe" /noconfig /fullpaths @"C:\Documents and
benwayj
56 Posts
0
March 1st, 2011 13:00
The PSSnapin only takes about 15 seconds to complete and connect-viserver works it just complains about the cert
I haven’t used PowerCLI on this box for anything. I just set it up to run these test.
clintonskitson
116 Posts
0
March 1st, 2011 14:00
It looks like you are trying to execute the cust_get_esxtop_adapters.ps1 script instead of the collest_esxtop.ps1 script. When executing against many hosts, the collect_esxtop.ps1 will call cust_get_esxtop_adapters.ps1 script once per host. Otherwise it will result in a similar error to what you are seeing.
The below command should work for you.
.\collect_esxtop_adapters.ps1 -esxip ip1,ip2,ip3,ip4 -username root -password xx