We do something like this, but we just let analyzer run all the time, and create the nar file when it wants to. We don’t create or delete archives, we let Clariion take care of that.
So our procedure is:
1) Get a list of nar files every 6 hours.
2) Compare list with files that have already been downloaded.
3) If a new file is found, download it.
We have been doing this for the past year and haven’t had a problem.
BTW, what database are you loading this data into? You will very quickly exceed millions of rows (unless you have a small Clariion environment).
Goal Scripting .NAR file dumps prior to FLARE Release 24
Goal Scripting .NAR file for FLARE Release 24 and later
Fact Product: CLARiiON CX4 Series
Fact Product: CLARiiON CX3 Series
Fact Product: CLARiiON CX Series
Fact Product: CLARiiON FC Series
Fact EMC SW: Navisphere Analyzer (versions prior to Release 24)
Fact EMC SW: Navisphere Analyzer (Release 24 and later)
Symptom Need a script to automate retrieving .NAR file dumps every 24 hours to a Windows server.
Change Customer site has two CX700 arrays and one CX600 array and is running Navisphere Analyzer to collect performance data for analysis.
Fix
With Analyzer Release 24 and later, you can use the Navisphere GUI to start Analyzer using the Periodic Archiving function that will automatically create and store Analyzer archives on the array (Periodic Archiving is enabled by default). These archives can then be manually retrieved using the GUI interface (Tools\Analyzer\Archive\Retrieve) or by using the Navisphere Analyzer CLI commands (see the Navisphere Analyzer Command Line Interface (CLI) manual available on Powerlink). If you need to script this function (retrieving the archives), you can use Navisphere Analyzer CLI archiveretrieve command to perform the same functions (creating and retrieving the current archive in one operation).
IMPORTANT: The function of the archiveretrieve command is really a "create and retrieve" command. When you issue the command,archiveretrieve will create a new archive file based on the contents of the current archive buffer. If the buffer holds 10 or more data points, a new archive will be created and downloaded to your workstation. The name of this file depends on the -File switch - this is the filename that you assign. If the buffer contains less than 10 data points, the command will take the date from the last archive created (if one exists on the array), rename the file to what you gave in the -File and download to your workstation. There are conditions where the data may appear to be from an older archive with a new name. If you just need to use Navisphere CLI to download existing archives from the array, use the "analyzer -list" command to list all the available archives and use the "analyzer -f " to retrieve the files. The -File is the name from the -List command. You can also use the "-analyzer -path " if you want to list all archives stored on the array and retrieve one or more files.
naviseccli -h IP_Address_SP -username -password -scope <0|1> analyzer -archive -list -- lists all archives stored on the array
naviseccli -h IP_Address_SP -username -password -scope <0|1> analyzer -archive -all -- this command requires user input - retrieves all archives stored on the array to the directory where the command is run from. Add the -path to specify the download location. Add the -o switch to download ALL archives without requiring user input
naviseccli -h IP_Address_SP -username -password -scope <0|1> analyzer -archive -path -- this command will list all the archives stored on the array and requires user input - allows you to select one or more of the listed archives to download to the location . The -o switch does not work for this command.
naviseccli -h IP_Address_SP -username -password -scope <0|1> analyzer -archive -file -- this command requires user input in the basic mode - allows you to selectively download one archive to the directory where the command is run from. You supply the . Add the -path to specify the download location. Add the -o switch to download without requiring user input. You can use this command for scripting. You must know the name of the archive that you want to download - run the -list command to get list of stored archives.
Caution! Remember that if Analyzer is not licensed on the array the archive files will be encrypted for Release 24 and later. The archive filename suffix will be .NAZ. If Analyzer is licensed the suffix will be .NAR. When using the batch program below, make sure you use the correct suffix. If using unlicensed Analyzer and you name the archive .NAR when you try to open the file, you will get an error message saying that the file is encrypted and that you must contact Support to read the file.
With Analyzer Release 22 and earlier, no Periodic Archiving function automatically creates the archives. You will need to manually retrieve the archive using the Navisphere GUI (which creates and retrieves the archive in one operation) or use a CLI script to create and retrieve the archive (the JAVA archiveretrieve.jar command). With Analyzer Release 22 and earlier you must install the optional Analyzer Enabler in order to use Analyzer. It is disabled if not licensed.
In either case, you should review the Navisphere On-Line Help for additional information about using Analyzer and also the Navisphere Analyzer CLI manual (available from Powerlink) for additional information about the different CLI commands. Below are some examples of the methods used to automate retrieving the archives from the array.
Scripting for FLARE releases prior to Release 26
Sometimes it is useful and necessary to automate the collection of Navisphere Analyzer .NAR files in order to review the performance of the array.
By creating a small .BAT file that runs the Navisphere archiveretrieve.jar command, you can set up a Windows scheduler task to run the .BAT file at different times throughout the day. The archiveretrieve.jar command will work for all operating system versions of Navisphere Host Agent/CLI package. The structure of the batch file will differ for each operating system. This example is for Windows.
The following BAT file commands will create and collect the NAR/NAZ files from the array and store them in a subdirectory called \archive off the C:\Programs Files\EMC\Navisphere CLI directory. The .BAT file should be created in the "\Navisphere CLI" directory in order to run correctly. Since the archiveretrieve.jar command is a Java command and requires that you use the security model for Java, you must add the username/password of a global user on the array (or create a security file).
You will need to add the IP address for both SP A and SP B. You should also change "serial#" to the last four digits of the array serial number or some other identifier:
For example, "-4736-CX700-SPA.nar" would identify the NAR file as coming from a CX700 array with the last four digits of the array serial number of 4736.
For /f "tokens=2-4 delims=/ " %%a in ('date /t') do (set date=%%a-%%b-%%c)
For /f "tokens=1-2 delims=: " %%a in ('time /t') do (set time=%%a-%%b-%%c)
When Navisphere Analyzer is running, it collects data points at the "Archive Interval" that you specify in the Tools/Analyzer/Data Logging screen. Since the log can hold a finite number of data points (155) before it starts to wrap, the time to collect the NAR files will depend on the archive interval (you can determine the time covered by multipling the Archive Interval by 156 time intervals). When Analyzer is first installed, the interval is set to a default of 600 seconds (or 120 seconds for newer arrays), which means that the log can only hold about 26 hours of data and should be collected every 24 hours. If you change the interval to 60 seconds, the log can hold about 2.5 hours before it starts to wrap and you should collect NAR files every 2.5 hours. When you change the interval, there will be an estimate of the length of time for the NAR.
In addition, when you change the Archive Interval (which requires you to be in Engineering Mode for Release 22 and lower), you must also check the "Clear Archive" box (Release 22 and lower only). Failing to do so will result in a .NAR file with incorrect data.
Note: The version of archiveretrieve.jar that you use is important. For CLARiiON arrays running Release 19, 22 or 24, you must use the latest version of Navisphere CLI for Windows Release 24. Install it cleanly - do not install it over an older version. This version (24) will work correctly for Release 19, 22 and 24. Release 26 does NOT include the Analyzer Java commands. The JAVA commands will not work with arrays running Release 26 and later, only with Release 24 and earlier.
For this script to work correctly with Release 24, you must disable the "Periodic Archiving" and "Stop Automatically After" options in Tools/Analyzer/Data Logging. This will allow Analyzer to continuously collect data in the archive (similar to Release 19). With Release 24 and later, once you issue the retrieve command, the archive will be empty and you can not retrieve a new archive until 10 Archive Interval have elapsed.
In future FLARE releases, Java commands will not be supported. The new naviseccli commands will include three new commands that will perform the same function as the archiveretrieve.jar, archivedump.jar,and archivemerge.jar commands.
Scripting for FLARE Release 26 and later
JAVA commands will not work with Release 26.
If you run the -archiveretrieve command before the next 10 Archive Interval data points have been collects, the file retrieved will be the last one created. Remember that the -archiveretrieve command will create and retrieve a new archive, it will not retrieve specific archives already stored on the array.
The files created with the -archiveretrieve command will remain on the array. There is a limit to the number of archives stored and when this limit is reached, the oldest archives will be overwritten.
You can use the Release 24 Java commands for use with Release 24 and earlier arrays, but not with Release 26 arrays.
The analyzer “.jar” files will not be installed when you install Release 26 CLI. If you install over Release 24 and already have the “.jar” files, you will have the option to keep them.
New Analyzer subcommands with Release 26 CLI will work the same as the Java commands now work. You can run these commands against an array running Release 26 and later and with FLARE releases of 19, 22 and 24. (You can now use the same script for all arrays as long as they all have the most current flare release from 19 to 26).
-archiveretrieve
-archivedump
-archivemerge
The script below will only work on arrays where analyzer is enabled and running AND (for Release 24 and later) that Periodic Archiving is Disabled AND “Stop Automatically After” is not enabled.
Use a .BAT file to retrieve archives. In this example, “getnar.bat” is used.
Find a Windows workstation (management workstation) and install the latest Navisphere CLI Release 26.
Create a bat file, getnar.bat (see sample script below), in the c:\program files\emc\navisphere cli\ directory.
Then create a new directory called \archive off the "navisphere cli" directory.
Modify the parameters in the commands to match your system:
You will need the IP address for SP A and SP B and the username\password of a global admin user on the array that you intend to run the batch file against. Change the -file parameter to give the archive a name that matches the array, maybe the last 4 digits of the serial number.
Then create a Scheduled Task in Windows to run the batch file every xx interval where xx is based on the archive interval that you have set on the array.
If the archive interval is 600 seconds, run the program every 24 hours
If the archive interval is 300 seconds, run the program every 12 hours
If the archive interval is 120 seconds, run the program every 5 hours
If the archive interval is 60 seconds, run the program every 2.5 hours
Thanks for your help all. Both answers were helpful and I have a system up and running.
Does anyone know why the data captured from SPA includes stats for SPB and vice-versa? I notice that the values for the same object and timestamp between those pulled from SP A and those from SP B are slightly different. IE:
pulled from Object TimeStamp %util
SPA SP B 08/30/2010 23:41 13.82716
SPB SP B 08/30/2010 23:41 13.55102
Why is there seemingly data for the same thing from both sides? and why would it be different? Do I need to include both in the data I analyze or are they essentially the same?
We run analyzer on both SP's when you start logging - first one SP is started and then shortly following we start the other SP. We do that in case the data from one SP gets corrupted - the data should be almost identical. This is the reason that you do not merge an SPA NAR with and SPB NAR. Think of it as a backup. You only need one to look at performance. If for some reason the data looks strange, try the other.
I have moved these scripts to a new Windows machine and did run the command to add the security file. The script does seem to work for some of the OLDER arrays CX4's. But not for the newer ones
hmmm.. Ok,..well, I acquired this script and was trying to make it work in our environment. I'll look over the naviseccli commands that will do what we need.
alias23122
139 Posts
1
August 30th, 2010 08:00
We do something like this, but we just let analyzer run all the time, and create the nar file when it wants to. We don’t create or delete archives, we let Clariion take care of that.
So our procedure is:
1) Get a list of nar files every 6 hours.
2) Compare list with files that have already been downloaded.
3) If a new file is found, download it.
We have been doing this for the past year and haven’t had a problem.
BTW, what database are you loading this data into? You will very quickly exceed millions of rows (unless you have a small Clariion environment).
kelleg
4 Operator
•
4.5K Posts
0
August 30th, 2010 14:00
From Primus emc148982 - see the IMPORTANT note near the begining:
ID: emc148982
Domain: EMC1
Solution Class: 3.X Compatibility
Goal How do I automate collecting .NAR files?
Goal Scripting .NAR file dumps prior to FLARE Release 24
Goal Scripting .NAR file for FLARE Release 24 and later
Fact Product: CLARiiON CX4 Series
Fact Product: CLARiiON CX3 Series
Fact Product: CLARiiON CX Series
Fact Product: CLARiiON FC Series
Fact EMC SW: Navisphere Analyzer (versions prior to Release 24)
Fact EMC SW: Navisphere Analyzer (Release 24 and later)
Symptom Need a script to automate retrieving .NAR file dumps every 24 hours to a Windows server.
Change Customer site has two CX700 arrays and one CX600 array and is running Navisphere Analyzer to collect performance data for analysis.
Fix
With Analyzer Release 24 and later, you can use the Navisphere GUI to start Analyzer using the Periodic Archiving function that will automatically create and store Analyzer archives on the array (Periodic Archiving is enabled by default). These archives can then be manually retrieved using the GUI interface (Tools\Analyzer\Archive\Retrieve) or by using the Navisphere Analyzer CLI commands (see the Navisphere Analyzer Command Line Interface (CLI) manual available on Powerlink). If you need to script this function (retrieving the archives), you can use Navisphere Analyzer CLI archiveretrieve command to perform the same functions (creating and retrieving the current archive in one operation).
IMPORTANT: The function of the archiveretrieve command is really a "create and retrieve" command. When you issue the command, archiveretrieve will create a new archive file based on the contents of the current archive buffer. If the buffer holds 10 or more data points, a new archive will be created and downloaded to your workstation. The name of this file depends on the -File switch - this is the filename that you assign. If the buffer contains less than 10 data points, the command will take the date from the last archive created (if one exists on the array), rename the file to what you gave in the -File and download to your workstation. There are conditions where the data may appear to be from an older archive with a new name. If you just need to use Navisphere CLI to download existing archives from the array, use the "analyzer -list" command to list all the available archives and use the "analyzer -f " to retrieve the files. The -File is the name from the -List command. You can also use the "-analyzer -path " if you want to list all archives stored on the array and retrieve one or more files.
Caution! Remember that if Analyzer is not licensed on the array the archive files will be encrypted for Release 24 and later. The archive filename suffix will be .NAZ. If Analyzer is licensed the suffix will be .NAR. When using the batch program below, make sure you use the correct suffix. If using unlicensed Analyzer and you name the archive .NAR when you try to open the file, you will get an error message saying that the file is encrypted and that you must contact Support to read the file.
With Analyzer Release 22 and earlier, no Periodic Archiving function automatically creates the archives. You will need to manually retrieve the archive using the Navisphere GUI (which creates and retrieves the archive in one operation) or use a CLI script to create and retrieve the archive (the JAVA archiveretrieve.jar command). With Analyzer Release 22 and earlier you must install the optional Analyzer Enabler in order to use Analyzer. It is disabled if not licensed.
In either case, you should review the Navisphere On-Line Help for additional information about using Analyzer and also the Navisphere Analyzer CLI manual (available from Powerlink) for additional information about the different CLI commands. Below are some examples of the methods used to automate retrieving the archives from the array.
Scripting for FLARE releases prior to Release 26
Sometimes it is useful and necessary to automate the collection of Navisphere Analyzer .NAR files in order to review the performance of the array.
By creating a small .BAT file that runs the Navisphere archiveretrieve.jar command, you can set up a Windows scheduler task to run the .BAT file at different times throughout the day. The archiveretrieve.jar command will work for all operating system versions of Navisphere Host Agent/CLI package. The structure of the batch file will differ for each operating system. This example is for Windows.
The following BAT file commands will create and collect the NAR/NAZ files from the array and store them in a subdirectory called \archive off the C:\Programs Files\EMC\Navisphere CLI directory. The .BAT file should be created in the "\Navisphere CLI" directory in order to run correctly. Since the archiveretrieve.jar command is a Java command and requires that you use the security model for Java, you must add the username/password of a global user on the array (or create a security file).
You will need to add the IP address for both SP A and SP B. You should also change "serial#" to the last four digits of the array serial number or some other identifier:
For example, "-4736-CX700-SPA.nar" would identify the NAR file as coming from a CX700 array with the last four digits of the array serial number of 4736.
For /f "tokens=2-4 delims=/ " %%a in ('date /t') do (set date=%%a-%%b-%%c)
java -jar archiveretrieve.jar -User <username> -Password -Scope 0 -Address < IP_address_for_SP_B> -File %date%-%time%-serial#-SPB.nar -Location "c:\program files\emc\navisphere cli\archive"
Scripting for FLARE Release 26 and later
JAVA commands will not work with Release 26.
If you run the -archiveretrieve command before the next 10 Archive Interval data points have been collects, the file retrieved will be the last one created. Remember that the -archiveretrieve command will create and retrieve a new archive, it will not retrieve specific archives already stored on the array.
The files created with the -archiveretrieve command will remain on the array. There is a limit to the number of archives stored and when this limit is reached, the oldest archives will be overwritten.
You can use the Release 24 Java commands for use with Release 24 and earlier arrays, but not with Release 26 arrays.
The analyzer “.jar” files will not be installed when you install Release 26 CLI. If you install over Release 24 and already have the “.jar” files, you will have the option to keep them.
New Analyzer subcommands with Release 26 CLI will work the same as the Java commands now work. You can run these commands against an array running Release 26 and later and with FLARE releases of 19, 22 and 24. (You can now use the same script for all arrays as long as they all have the most current flare release from 19 to 26).
The script below will only work on arrays where analyzer is enabled and running AND (for Release 24 and later) that Periodic Archiving is Disabled AND “Stop Automatically After” is not enabled.
Naviseccli –h xxx.xxx.xxx.xxx -User xxxx -Password xxxx -Scope 0 analyzer -archiveretrieve -File %date%-%time%-serial#-SPx.nar -Location "c:\program files\emc\navisphere cli\archive"
You will need the IP address for SP A and SP B and the username\password of a global admin user on the array that you intend to run the batch file against. Change the -file parameter to give the archive a name that matches the array, maybe the last 4 digits of the serial number.
See if this helps
glen
wolf8963
46 Posts
0
September 3rd, 2010 06:00
Thanks for your help all. Both answers were helpful and I have a system up and running.
Does anyone know why the data captured from SPA includes stats for SPB and vice-versa? I notice that the values for the same object and timestamp between those pulled from SP A and those from SP B are slightly different. IE:
pulled from Object TimeStamp %util
SPA SP B 08/30/2010 23:41 13.82716
SPB SP B 08/30/2010 23:41 13.55102
Why is there seemingly data for the same thing from both sides? and why would it be different? Do I need to include both in the data I analyze or are they essentially the same?
Thanks for the info
kelleg
4 Operator
•
4.5K Posts
0
September 3rd, 2010 07:00
We run analyzer on both SP's when you start logging - first one SP is started and then shortly following we start the other SP. We do that in case the data from one SP gets corrupted - the data should be almost identical. This is the reason that you do not merge an SPA NAR with and SPB NAR. Think of it as a backup. You only need one to look at performance. If for some reason the data looks strange, try the other.
glen
SKT2
2 Intern
•
1.3K Posts
0
September 5th, 2010 18:00
which file in SP collects did give u this number(% of SP utilization)?
kelleg
4 Operator
•
4.5K Posts
0
September 6th, 2010 12:00
I think that was from the DUMP of Analyzer NAR files
glen
wolf8963
46 Posts
0
September 7th, 2010 13:00
Correct - NAR Dump - and if I did a good job, it makes me a Nar-do-well
Glen62
2 Posts
0
December 1st, 2010 16:00
Hi alias23122
Would you mind to share your script?
Thanks
Glen
DHoffman2
2 Intern
•
306 Posts
0
July 15th, 2015 12:00
I'm getting messages on some of our newer VNX arrays as follows.
Error returned from delete archive:
Request failed.Caller not privileged.
I'm using the following (removing sensitive data/passwords from the command).
For /f "tokens=2-4 delims=/ " %%a in ('date /t') do (set date=%%a-%%b-%%c)
For /f "tokens=1-3 delims=: " %%a in ('time /t') do (set time=%%a-%%b-%%c)
Rem 1234 Array
java -jar archiveretrieve.jar -user -password -scope 0 -address 172.x.y.z -File VNX-1234-SPA-%date%-%time%.nar -Location "F:\script\NAR_Files\1234
I have moved these scripts to a new Windows machine and did run the command to add the security file. The script does seem to work for some of the OLDER arrays CX4's. But not for the newer ones
Suggestions?
dynamox
9 Legend
•
20.4K Posts
0
July 15th, 2015 17:00
i thought the java version was deprecated ages ago, switch to naviseccli.
brettesinclair
2 Intern
•
715 Posts
0
July 16th, 2015 05:00
Agree with Dynamox, use naviseccli with the managefiles and -retrieve options
DHoffman2
2 Intern
•
306 Posts
0
July 16th, 2015 05:00
hmmm.. Ok,..well, I acquired this script and was trying to make it work in our environment. I'll look over the naviseccli commands that will do what we need.
DHoffman2
2 Intern
•
306 Posts
0
July 16th, 2015 07:00
Thanks. That appears to have worked.