I am working on a script to grab analyzer performance stats via CLI and load them into a database. My question goes all the way back to a basic understanding of Analyzer and archiving.
My system is a CX300-i running the latest FLARE code
My goal is to:
Collect data from both SPs every 4 hours
Archive the data
Start a new archive to run for the next 4 hours
Dump the previously created archive file to CSV
Execute a script to load the DB
To that end I have a bat file that executes every 4 hours and (amoung others) executes the following commands:
I am not getting the results I expected and I believe it is because of my misunderstanding the process. In the above, I have to sleep for 20 minutes between the deletion of the old archives and creation of the new archives because it doesnt think there are 10 intervals present. That suggests the archive need not be created for the accumulation of data to take place.
My thought was that the following is what really happens:
Turn analyzer on to always run
It accumulates data even though an archive file has not been created
When an archive file is created, it transfers the collected data into the archive file
When an archive retrieve is executed, the just-created archive file is saved to disk as an NAR
The old archive files can then be deleted
Is this a correct understanding? If so, that would suggest the order of my script should be (with analyzer already running):
wait 4 hours
create archive (analyzer -archive -new)
grab archive (-archiveretrieve -file Cx300i_SPA.nar or B)
delete old archives to start fresh accumulation (analyzer -archive -delete -all)
Can anyone clarify my thinking on this? I looked in the Anayzer manual but it seemed to be geared more toward the mechanics of the commands than to this type of question.
We do something like this, but we just let analyzer run all the time, and create the nar file when it wants to. We don’t create or delete archives, we let Clariion take care of that.
So our procedure is:
1) Get a list of nar files every 6 hours.
2) Compare list with files that have already been downloaded.
3) If a new file is found, download it.
We have been doing this for the past year and haven’t had a problem.
BTW, what database are you loading this data into? You will very quickly exceed millions of rows (unless you have a small Clariion environment).
Goal Scripting .NAR file dumps prior to FLARE Release 24
Goal Scripting .NAR file for FLARE Release 24 and later
Fact Product: CLARiiON CX4 Series
Fact Product: CLARiiON CX3 Series
Fact Product: CLARiiON CX Series
Fact Product: CLARiiON FC Series
Fact EMC SW: Navisphere Analyzer (versions prior to Release 24)
Fact EMC SW: Navisphere Analyzer (Release 24 and later)
Symptom Need a script to automate retrieving .NAR file dumps every 24 hours to a Windows server.
Change Customer site has two CX700 arrays and one CX600 array and is running Navisphere Analyzer to collect performance data for analysis.
Fix
With Analyzer Release 24 and later, you can use the Navisphere GUI to start Analyzer using the Periodic Archiving function that will automatically create and store Analyzer archives on the array (Periodic Archiving is enabled by default). These archives can then be manually retrieved using the GUI interface (Tools\Analyzer\Archive\Retrieve) or by using the Navisphere Analyzer CLI commands (see the Navisphere Analyzer Command Line Interface (CLI) manual available on Powerlink). If you need to script this function (retrieving the archives), you can use Navisphere Analyzer CLI archiveretrieve command to perform the same functions (creating and retrieving the current archive in one operation).
IMPORTANT: The function of the archiveretrieve command is really a "create and retrieve" command. When you issue the command,archiveretrieve will create a new archive file based on the contents of the current archive buffer. If the buffer holds 10 or more data points, a new archive will be created and downloaded to your workstation. The name of this file depends on the -File switch - this is the filename that you assign. If the buffer contains less than 10 data points, the command will take the date from the last archive created (if one exists on the array), rename the file to what you gave in the -File and download to your workstation. There are conditions where the data may appear to be from an older archive with a new name. If you just need to use Navisphere CLI to download existing archives from the array, use the "analyzer -list" command to list all the available archives and use the "analyzer -f " to retrieve the files. The -File is the name from the -List command. You can also use the "-analyzer -path " if you want to list all archives stored on the array and retrieve one or more files.
naviseccli -h IP_Address_SP -username -password -scope <0|1> analyzer -archive -list -- lists all archives stored on the array
naviseccli -h IP_Address_SP -username -password -scope <0|1> analyzer -archive -all -- this command requires user input - retrieves all archives stored on the array to the directory where the command is run from. Add the -path to specify the download location. Add the -o switch to download ALL archives without requiring user input
naviseccli -h IP_Address_SP -username -password -scope <0|1> analyzer -archive -path -- this command will list all the archives stored on the array and requires user input - allows you to select one or more of the listed archives to download to the location . The -o switch does not work for this command.
naviseccli -h IP_Address_SP -username -password -scope <0|1> analyzer -archive -file -- this command requires user input in the basic mode - allows you to selectively download one archive to the directory where the command is run from. You supply the . Add the -path to specify the download location. Add the -o switch to download without requiring user input. You can use this command for scripting. You must know the name of the archive that you want to download - run the -list command to get list of stored archives.
Caution! Remember that if Analyzer is not licensed on the array the archive files will be encrypted for Release 24 and later. The archive filename suffix will be .NAZ. If Analyzer is licensed the suffix will be .NAR. When using the batch program below, make sure you use the correct suffix. If using unlicensed Analyzer and you name the archive .NAR when you try to open the file, you will get an error message saying that the file is encrypted and that you must contact Support to read the file.
With Analyzer Release 22 and earlier, no Periodic Archiving function automatically creates the archives. You will need to manually retrieve the archive using the Navisphere GUI (which creates and retrieves the archive in one operation) or use a CLI script to create and retrieve the archive (the JAVA archiveretrieve.jar command). With Analyzer Release 22 and earlier you must install the optional Analyzer Enabler in order to use Analyzer. It is disabled if not licensed.
In either case, you should review the Navisphere On-Line Help for additional information about using Analyzer and also the Navisphere Analyzer CLI manual (available from Powerlink) for additional information about the different CLI commands. Below are some examples of the methods used to automate retrieving the archives from the array.
Scripting for FLARE releases prior to Release 26
Sometimes it is useful and necessary to automate the collection of Navisphere Analyzer .NAR files in order to review the performance of the array.
By creating a small .BAT file that runs the Navisphere archiveretrieve.jar command, you can set up a Windows scheduler task to run the .BAT file at different times throughout the day. The archiveretrieve.jar command will work for all operating system versions of Navisphere Host Agent/CLI package. The structure of the batch file will differ for each operating system. This example is for Windows.
The following BAT file commands will create and collect the NAR/NAZ files from the array and store them in a subdirectory called \archive off the C:\Programs Files\EMC\Navisphere CLI directory. The .BAT file should be created in the "\Navisphere CLI" directory in order to run correctly. Since the archiveretrieve.jar command is a Java command and requires that you use the security model for Java, you must add the username/password of a global user on the array (or create a security file).
You will need to add the IP address for both SP A and SP B. You should also change "serial#" to the last four digits of the array serial number or some other identifier:
For example, "-4736-CX700-SPA.nar" would identify the NAR file as coming from a CX700 array with the last four digits of the array serial number of 4736.
For /f "tokens=2-4 delims=/ " %%a in ('date /t') do (set date=%%a-%%b-%%c)
For /f "tokens=1-2 delims=: " %%a in ('time /t') do (set time=%%a-%%b-%%c)
When Navisphere Analyzer is running, it collects data points at the "Archive Interval" that you specify in the Tools/Analyzer/Data Logging screen. Since the log can hold a finite number of data points (155) before it starts to wrap, the time to collect the NAR files will depend on the archive interval (you can determine the time covered by multipling the Archive Interval by 156 time intervals). When Analyzer is first installed, the interval is set to a default of 600 seconds (or 120 seconds for newer arrays), which means that the log can only hold about 26 hours of data and should be collected every 24 hours. If you change the interval to 60 seconds, the log can hold about 2.5 hours before it starts to wrap and you should collect NAR files every 2.5 hours. When you change the interval, there will be an estimate of the length of time for the NAR.
In addition, when you change the Archive Interval (which requires you to be in Engineering Mode for Release 22 and lower), you must also check the "Clear Archive" box (Release 22 and lower only). Failing to do so will result in a .NAR file with incorrect data.
Note: The version of archiveretrieve.jar that you use is important. For CLARiiON arrays running Release 19, 22 or 24, you must use the latest version of Navisphere CLI for Windows Release 24. Install it cleanly - do not install it over an older version. This version (24) will work correctly for Release 19, 22 and 24. Release 26 does NOT include the Analyzer Java commands. The JAVA commands will not work with arrays running Release 26 and later, only with Release 24 and earlier.
For this script to work correctly with Release 24, you must disable the "Periodic Archiving" and "Stop Automatically After" options in Tools/Analyzer/Data Logging. This will allow Analyzer to continuously collect data in the archive (similar to Release 19). With Release 24 and later, once you issue the retrieve command, the archive will be empty and you can not retrieve a new archive until 10 Archive Interval have elapsed.
In future FLARE releases, Java commands will not be supported. The new naviseccli commands will include three new commands that will perform the same function as the archiveretrieve.jar, archivedump.jar,and archivemerge.jar commands.
Scripting for FLARE Release 26 and later
JAVA commands will not work with Release 26.
If you run the -archiveretrieve command before the next 10 Archive Interval data points have been collects, the file retrieved will be the last one created. Remember that the -archiveretrieve command will create and retrieve a new archive, it will not retrieve specific archives already stored on the array.
The files created with the -archiveretrieve command will remain on the array. There is a limit to the number of archives stored and when this limit is reached, the oldest archives will be overwritten.
You can use the Release 24 Java commands for use with Release 24 and earlier arrays, but not with Release 26 arrays.
The analyzer “.jar” files will not be installed when you install Release 26 CLI. If you install over Release 24 and already have the “.jar” files, you will have the option to keep them.
New Analyzer subcommands with Release 26 CLI will work the same as the Java commands now work. You can run these commands against an array running Release 26 and later and with FLARE releases of 19, 22 and 24. (You can now use the same script for all arrays as long as they all have the most current flare release from 19 to 26).
-archiveretrieve
-archivedump
-archivemerge
The script below will only work on arrays where analyzer is enabled and running AND (for Release 24 and later) that Periodic Archiving is Disabled AND “Stop Automatically After” is not enabled.
Use a .BAT file to retrieve archives. In this example, “getnar.bat” is used.
Find a Windows workstation (management workstation) and install the latest Navisphere CLI Release 26.
Create a bat file, getnar.bat (see sample script below), in the c:\program files\emc\navisphere cli\ directory.
Then create a new directory called \archive off the "navisphere cli" directory.
Modify the parameters in the commands to match your system:
You will need the IP address for SP A and SP B and the username\password of a global admin user on the array that you intend to run the batch file against. Change the -file parameter to give the archive a name that matches the array, maybe the last 4 digits of the serial number.
Then create a Scheduled Task in Windows to run the batch file every xx interval where xx is based on the archive interval that you have set on the array.
If the archive interval is 600 seconds, run the program every 24 hours
If the archive interval is 300 seconds, run the program every 12 hours
If the archive interval is 120 seconds, run the program every 5 hours
If the archive interval is 60 seconds, run the program every 2.5 hours
Thanks for your help all. Both answers were helpful and I have a system up and running.
Does anyone know why the data captured from SPA includes stats for SPB and vice-versa? I notice that the values for the same object and timestamp between those pulled from SP A and those from SP B are slightly different. IE:
pulled from Object TimeStamp %util
SPA SP B 08/30/2010 23:41 13.82716
SPB SP B 08/30/2010 23:41 13.55102
Why is there seemingly data for the same thing from both sides? and why would it be different? Do I need to include both in the data I analyze or are they essentially the same?
We run analyzer on both SP's when you start logging - first one SP is started and then shortly following we start the other SP. We do that in case the data from one SP gets corrupted - the data should be almost identical. This is the reason that you do not merge an SPA NAR with and SPB NAR. Think of it as a backup. You only need one to look at performance. If for some reason the data looks strange, try the other.
I have moved these scripts to a new Windows machine and did run the command to add the security file. The script does seem to work for some of the OLDER arrays CX4's. But not for the newer ones
alias23122
139 Posts
1907
1
Posted August 30th, 2010 08:00
We do something like this, but we just let analyzer run all the time, and create the nar file when it wants to. We don’t create or delete archives, we let Clariion take care of that.
So our procedure is:
1) Get a list of nar files every 6 hours.
2) Compare list with files that have already been downloaded.
3) If a new file is found, download it.
We have been doing this for the past year and haven’t had a problem.
BTW, what database are you loading this data into? You will very quickly exceed millions of rows (unless you have a small Clariion environment).