Unsolved
This post is more than 5 years old
46 Posts
1
6101
Another Analyzer Question
Good Morning,
I am working on a script to grab analyzer performance stats via CLI and load them into a database. My question goes all the way back to a basic understanding of Analyzer and archiving.
My system is a CX300-i running the latest FLARE code
My goal is to:
- Collect data from both SPs every 4 hours
- Archive the data
- Start a new archive to run for the next 4 hours
- Dump the previously created archive file to CSV
- Execute a script to load the DB
To that end I have a bat file that executes every 4 hours and (amoung others) executes the following commands:
step 2
naviseccli -h 10.x.x.x analyzer -archiveretrieve -file Cx300i_SPA.nar -location c:\path -Overwrite y -Retry 2 -v
naviseccli -h 10.x.x.x analyzer -archiveretrieve -file Cx300i_SPB.nar -location c:\path -Overwrite y -Retry 2 -v
ren Path\Cx300i_SPA.nar "Cx300i_SPA_%date%_%MTime%.nar"
ren Path\Cx300i_SPB.nar "Cx300i_SPB_%date%_%MTime%.nar"
naviseccli -h 10.94.35.2 analyzer -archive -delete -all -o
naviseccli -h 10.94.32.4 analyzer -archive -delete -all -o
Step 3
naviseccli -h 10.94.35.2 analyzer -archive -new
naviseccli -h 10.94.32.4 analyzer -archive -new
Step 4
naviseccli analyzer -archivedump -data C:\San\Analyzer\"Cx300i_SPA_%date%_%MTime%.nar" C:\San\Analyzer\"Cx300i_SPB_%date%_%MTime%.nar" -out tempdump.csv -join -overwrite y
I am not getting the results I expected and I believe it is because of my misunderstanding the process. In the above, I have to sleep for 20 minutes between the deletion of the old archives and creation of the new archives because it doesnt think there are 10 intervals present. That suggests the archive need not be created for the accumulation of data to take place.
My thought was that the following is what really happens:
- Turn analyzer on to always run
- It accumulates data even though an archive file has not been created
- When an archive file is created, it transfers the collected data into the archive file
- When an archive retrieve is executed, the just-created archive file is saved to disk as an NAR
- The old archive files can then be deleted
Is this a correct understanding? If so, that would suggest the order of my script should be (with analyzer already running):
- wait 4 hours
- create archive (analyzer -archive -new)
- grab archive (-archiveretrieve -file Cx300i_SPA.nar or B)
- delete old archives to start fresh accumulation (analyzer -archive -delete -all)
- dump .NAR files to csv (analyzer -archivedump -data path\"Cx300i_SPA_%date%_%MTime%.nar" path\"Cx300i_SPB_%date%_%MTime%.nar" -out tempdump.csv -join -overwrite y)
- call script to import data
Can anyone clarify my thinking on this? I looked in the Anayzer manual but it seemed to be geared more toward the mechanics of the commands than to this type of question.
Thanks for your help,
W
alias23122
139 Posts
1
August 30th, 2010 08:00
We do something like this, but we just let analyzer run all the time, and create the nar file when it wants to. We don’t create or delete archives, we let Clariion take care of that.
So our procedure is:
1) Get a list of nar files every 6 hours.
2) Compare list with files that have already been downloaded.
3) If a new file is found, download it.
We have been doing this for the past year and haven’t had a problem.
BTW, what database are you loading this data into? You will very quickly exceed millions of rows (unless you have a small Clariion environment).
kelleg
4.5K Posts
0
August 30th, 2010 14:00
From Primus emc148982 - see the IMPORTANT note near the begining:
ID: emc148982
Domain: EMC1
Solution Class: 3.X Compatibility
Goal How do I automate collecting .NAR files?
Goal Scripting .NAR file dumps prior to FLARE Release 24
Goal Scripting .NAR file for FLARE Release 24 and later
Fact Product: CLARiiON CX4 Series
Fact Product: CLARiiON CX3 Series
Fact Product: CLARiiON CX Series
Fact Product: CLARiiON FC Series
Fact EMC SW: Navisphere Analyzer (versions prior to Release 24)
Fact EMC SW: Navisphere Analyzer (Release 24 and later)
Symptom Need a script to automate retrieving .NAR file dumps every 24 hours to a Windows server.
Change Customer site has two CX700 arrays and one CX600 array and is running Navisphere Analyzer to collect performance data for analysis.
Fix
With Analyzer Release 24 and later, you can use the Navisphere GUI to start Analyzer using the Periodic Archiving function that will automatically create and store Analyzer archives on the array (Periodic Archiving is enabled by default). These archives can then be manually retrieved using the GUI interface (Tools\Analyzer\Archive\Retrieve) or by using the Navisphere Analyzer CLI commands (see the Navisphere Analyzer Command Line Interface (CLI) manual available on Powerlink). If you need to script this function (retrieving the archives), you can use Navisphere Analyzer CLI archiveretrieve command to perform the same functions (creating and retrieving the current archive in one operation).
IMPORTANT: The function of the archiveretrieve command is really a "create and retrieve" command. When you issue the command, archiveretrieve will create a new archive file based on the contents of the current archive buffer. If the buffer holds 10 or more data points, a new archive will be created and downloaded to your workstation. The name of this file depends on the -File switch - this is the filename that you assign. If the buffer contains less than 10 data points, the command will take the date from the last archive created (if one exists on the array), rename the file to what you gave in the -File and download to your workstation. There are conditions where the data may appear to be from an older archive with a new name. If you just need to use Navisphere CLI to download existing archives from the array, use the "analyzer -list" command to list all the available archives and use the "analyzer -f " to retrieve the files. The -File is the name from the -List command. You can also use the "-analyzer -path " if you want to list all archives stored on the array and retrieve one or more files.
Caution! Remember that if Analyzer is not licensed on the array the archive files will be encrypted for Release 24 and later. The archive filename suffix will be .NAZ. If Analyzer is licensed the suffix will be .NAR. When using the batch program below, make sure you use the correct suffix. If using unlicensed Analyzer and you name the archive .NAR when you try to open the file, you will get an error message saying that the file is encrypted and that you must contact Support to read the file.
With Analyzer Release 22 and earlier, no Periodic Archiving function automatically creates the archives. You will need to manually retrieve the archive using the Navisphere GUI (which creates and retrieves the archive in one operation) or use a CLI script to create and retrieve the archive (the JAVA archiveretrieve.jar command). With Analyzer Release 22 and earlier you must install the optional Analyzer Enabler in order to use Analyzer. It is disabled if not licensed.
In either case, you should review the Navisphere On-Line Help for additional information about using Analyzer and also the Navisphere Analyzer CLI manual (available from Powerlink) for additional information about the different CLI commands. Below are some examples of the methods used to automate retrieving the archives from the array.
Scripting for FLARE releases prior to Release 26
Sometimes it is useful and necessary to automate the collection of Navisphere Analyzer .NAR files in order to review the performance of the array.
By creating a small .BAT file that runs the Navisphere archiveretrieve.jar command, you can set up a Windows scheduler task to run the .BAT file at different times throughout the day. The archiveretrieve.jar command will work for all operating system versions of Navisphere Host Agent/CLI package. The structure of the batch file will differ for each operating system. This example is for Windows.
The following BAT file commands will create and collect the NAR/NAZ files from the array and store them in a subdirectory called \archive off the C:\Programs Files\EMC\Navisphere CLI directory. The .BAT file should be created in the "\Navisphere CLI" directory in order to run correctly. Since the archiveretrieve.jar command is a Java command and requires that you use the security model for Java, you must add the username/password of a global user on the array (or create a security file).
You will need to add the IP address for both SP A and SP B. You should also change "serial#" to the last four digits of the array serial number or some other identifier:
For example, "-4736-CX700-SPA.nar" would identify the NAR file as coming from a CX700 array with the last four digits of the array serial number of 4736.
For /f "tokens=2-4 delims=/ " %%a in ('date /t') do (set date=%%a-%%b-%%c)
java -jar archiveretrieve.jar -User <username> -Password -Scope 0 -Address < IP_address_for_SP_B> -File %date%-%time%-serial#-SPB.nar -Location "c:\program files\emc\navisphere cli\archive"
Scripting for FLARE Release 26 and later
JAVA commands will not work with Release 26.
If you run the -archiveretrieve command before the next 10 Archive Interval data points have been collects, the file retrieved will be the last one created. Remember that the -archiveretrieve command will create and retrieve a new archive, it will not retrieve specific archives already stored on the array.
The files created with the -archiveretrieve command will remain on the array. There is a limit to the number of archives stored and when this limit is reached, the oldest archives will be overwritten.
You can use the Release 24 Java commands for use with Release 24 and earlier arrays, but not with Release 26 arrays.
The analyzer “.jar” files will not be installed when you install Release 26 CLI. If you install over Release 24 and already have the “.jar” files, you will have the option to keep them.
New Analyzer subcommands with Release 26 CLI will work the same as the Java commands now work. You can run these commands against an array running Release 26 and later and with FLARE releases of 19, 22 and 24. (You can now use the same script for all arrays as long as they all have the most current flare release from 19 to 26).
The script below will only work on arrays where analyzer is enabled and running AND (for Release 24 and later) that Periodic Archiving is Disabled AND “Stop Automatically After” is not enabled.
Naviseccli –h xxx.xxx.xxx.xxx -User xxxx -Password xxxx -Scope 0 analyzer -archiveretrieve -File %date%-%time%-serial#-SPx.nar -Location "c:\program files\emc\navisphere cli\archive"
You will need the IP address for SP A and SP B and the username\password of a global admin user on the array that you intend to run the batch file against. Change the -file parameter to give the archive a name that matches the array, maybe the last 4 digits of the serial number.
See if this helps
glen
wolf8963
46 Posts
0
September 3rd, 2010 06:00
Thanks for your help all. Both answers were helpful and I have a system up and running.
Does anyone know why the data captured from SPA includes stats for SPB and vice-versa? I notice that the values for the same object and timestamp between those pulled from SP A and those from SP B are slightly different. IE:
pulled from Object TimeStamp %util
SPA SP B 08/30/2010 23:41 13.82716
SPB SP B 08/30/2010 23:41 13.55102
Why is there seemingly data for the same thing from both sides? and why would it be different? Do I need to include both in the data I analyze or are they essentially the same?
Thanks for the info
kelleg
4.5K Posts
0
September 3rd, 2010 07:00
We run analyzer on both SP's when you start logging - first one SP is started and then shortly following we start the other SP. We do that in case the data from one SP gets corrupted - the data should be almost identical. This is the reason that you do not merge an SPA NAR with and SPB NAR. Think of it as a backup. You only need one to look at performance. If for some reason the data looks strange, try the other.
glen
SKT2
2 Intern
2 Intern
•
1.3K Posts
0
September 5th, 2010 18:00
which file in SP collects did give u this number(% of SP utilization)?
kelleg
4.5K Posts
0
September 6th, 2010 12:00
I think that was from the DUMP of Analyzer NAR files
glen
wolf8963
46 Posts
0
September 7th, 2010 13:00
Correct - NAR Dump - and if I did a good job, it makes me a Nar-do-well
Glen62
2 Posts
0
December 1st, 2010 16:00
Hi alias23122
Would you mind to share your script?
Thanks
Glen
DHoffman2
306 Posts
0
July 15th, 2015 12:00
I'm getting messages on some of our newer VNX arrays as follows.
Error returned from delete archive:
Request failed.Caller not privileged.
I'm using the following (removing sensitive data/passwords from the command).
For /f "tokens=2-4 delims=/ " %%a in ('date /t') do (set date=%%a-%%b-%%c)
For /f "tokens=1-3 delims=: " %%a in ('time /t') do (set time=%%a-%%b-%%c)
Rem 1234 Array
java -jar archiveretrieve.jar -user -password -scope 0 -address 172.x.y.z -File VNX-1234-SPA-%date%-%time%.nar -Location "F:\script\NAR_Files\1234
I have moved these scripts to a new Windows machine and did run the command to add the security file. The script does seem to work for some of the OLDER arrays CX4's. But not for the newer ones
Suggestions?
dynamox
2 Intern
2 Intern
•
20.4K Posts
0
July 15th, 2015 17:00
i thought the java version was deprecated ages ago, switch to naviseccli.
brettesinclair
2 Intern
2 Intern
•
715 Posts
0
July 16th, 2015 05:00
Agree with Dynamox, use naviseccli with the managefiles and -retrieve options
DHoffman2
306 Posts
0
July 16th, 2015 05:00
hmmm.. Ok,..well, I acquired this script and was trying to make it work in our environment. I'll look over the naviseccli commands that will do what we need.
DHoffman2
306 Posts
0
July 16th, 2015 07:00
Thanks. That appears to have worked.