Highlighted
wolf8963
2 Iron

Another Analyzer Question

Good Morning,

I am working on a script to grab analyzer performance stats via CLI and load them into a database. My question goes all the way back to a basic understanding of Analyzer and archiving.

My system is a CX300-i running the latest FLARE code

My goal is to:

  1. Collect data from both SPs every 4 hours
  2. Archive the data
  3. Start a new archive to run for the next 4 hours
  4. Dump the previously created archive file to CSV
  5. Execute a script to load the DB



To that end I have a bat file that executes every 4 hours and (amoung others) executes the following commands:

step 2

naviseccli -h 10.x.x.x analyzer -archiveretrieve -file Cx300i_SPA.nar -location c:\path -Overwrite y -Retry 2 -v
naviseccli -h 10.x.x.x analyzer -archiveretrieve -file Cx300i_SPB.nar -location c:\path -Overwrite y -Retry 2 -v

ren Path\Cx300i_SPA.nar "Cx300i_SPA_%date%_%MTime%.nar"
ren Path\Cx300i_SPB.nar "Cx300i_SPB_%date%_%MTime%.nar"

naviseccli -h 10.94.35.2 analyzer -archive -delete -all -o
naviseccli -h 10.94.32.4 analyzer  -archive -delete -all -o

Step 3

naviseccli -h 10.94.35.2 analyzer -archive -new
naviseccli -h 10.94.32.4 analyzer  -archive -new

Step 4

naviseccli analyzer -archivedump -data C:\San\Analyzer\"Cx300i_SPA_%date%_%MTime%.nar"  C:\San\Analyzer\"Cx300i_SPB_%date%_%MTime%.nar"  -out tempdump.csv -join  -overwrite y

I am not getting the results I expected and I believe it is because of my misunderstanding the process. In the above, I have to sleep for 20 minutes between the deletion of the old archives and creation of the new archives because it doesnt think there are 10 intervals present. That suggests the archive need not be created for the accumulation of data to take place.

My thought was that the following is what really happens:

  • Turn analyzer on to always run
  • It accumulates data even though an archive file has not been created
  • When an archive file is created, it transfers the collected data into the archive file
  • When an archive retrieve is executed, the just-created archive file is saved to disk as an NAR
  • The old archive files can then be deleted



Is this a correct understanding? If so, that would suggest the order of my script should be (with analyzer already running):

  • wait 4 hours
  • create archive (analyzer -archive -new)
  • grab archive (-archiveretrieve -file Cx300i_SPA.nar or B)
  • delete old archives to start fresh accumulation (analyzer  -archive -delete -all)
  • dump .NAR files to csv (analyzer -archivedump -data path\"Cx300i_SPA_%date%_%MTime%.nar"  path\"Cx300i_SPB_%date%_%MTime%.nar"  -out tempdump.csv -join  -overwrite y)
  • call script to import data

Can anyone clarify my thinking on this? I looked in the Anayzer manual but it seemed to be geared more toward the mechanics of the commands than to this type of question.

Thanks for your help,

W



Labels (1)
13 Replies
alias23122
3 Argentium

Re: Another Analyzer Question

We do something like this, but we just let analyzer run all the time, and create the nar file when it wants to. We don’t create or delete archives, we let Clariion take care of that.

So our procedure is:

1) Get a list of nar files every 6 hours.

2) Compare list with files that have already been downloaded.

3) If a new file is found, download it.

We have been doing this for the past year and haven’t had a problem.

BTW, what database are you loading this data into? You will very quickly exceed millions of rows (unless you have a small Clariion environment).

kelleg
5 Tungsten

Re: Another Analyzer Question

From Primus emc148982 - see the IMPORTANT note near the begining:

The following is a Primus(R) eServer  solution:

ID: emc148982
Domain: EMC1
Solution Class: 3.X  Compatibility

Goal       How do I automate collecting .NAR  files?

Goal       Scripting .NAR file dumps prior to FLARE Release  24

Goal       Scripting .NAR file for FLARE Release 24 and  later

Fact       Product: CLARiiON CX4  Series

Fact       Product: CLARiiON CX3  Series

Fact       Product: CLARiiON CX  Series

Fact       Product: CLARiiON FC  Series

Fact       EMC SW: Navisphere Analyzer (versions prior to  Release 24)

Fact       EMC SW: Navisphere Analyzer (Release 24 and  later)

Symptom    Need a script to automate retrieving .NAR file  dumps every 24 hours to a Windows server.

Change     Customer  site has two CX700 arrays and one CX600 array and is running Navisphere Analyzer  to collect performance data for analysis.

Fix       

With Analyzer Release 24 and later, you can use the Navisphere GUI to start  Analyzer using the Periodic Archiving function that will automatically create  and store Analyzer archives on the array (Periodic Archiving is enabled by  default). These archives can then be manually retrieved using the GUI interface  (Tools\Analyzer\Archive\Retrieve) or by using the Navisphere Analyzer CLI  commands (see the Navisphere Analyzer Command Line Interface (CLI) manual  available on Powerlink).  If you need to script this function (retrieving the archives), you can use  Navisphere Analyzer CLI archiveretrieve command to perform the same functions  (creating and retrieving the current archive in one operation).

IMPORTANT: The function of the  archiveretrieve command is really a "create and retrieve" command. When you  issue the command, archiveretrieve will create a new archive  file based on the contents of the current archive buffer. If the buffer holds 10  or more data points, a new archive will be created and downloaded to your  workstation. The name of this file depends on the -File <filename> switch  - this is the filename that you assign. If the buffer contains less than 10 data  points, the command will take the date from the last archive created (if one  exists on the array), rename the file to what you gave in the -File  <filename> and download to your workstation. There are conditions where  the data may appear to be from an older archive with a new name. If you just  need to use Navisphere CLI to download existing archives from the array, use the  "analyzer -list" command to list all the available archives and use the  "analyzer -f <filename>" to retrieve the files. The -File <filename>  is the name from the -List command. You can also use the "-analyzer -path  <pathname>" if you want to list all archives stored on the array and  retrieve one or more files.

  • naviseccli -h IP_Address_SP<A|B> -username <name> -password  <password> -scope <0|1> analyzer -archive -list  -- lists all  archives stored on the array
  • naviseccli -h IP_Address_SP<A|B> -username <name> -password  <password> -scope <0|1> analyzer -archive -all  -- this command  requires user input - retrieves all archives stored on the array to the  directory where the command is run from. Add the -path <pathname> to  specify the download location. Add the -o switch to download ALL archives  without requiring user input
  • naviseccli -h IP_Address_SP<A|B> -username <name> -password  <password> -scope <0|1> analyzer -archive -path <pathname>  --  this command will list all the archives stored on the array and requires user  input - allows you to select one or more of the listed archives to download to  the location <pathname>. The -o switch does not work for this  command.
  • naviseccli -h IP_Address_SP<A|B> -username <name> -password  <password> -scope <0|1> analyzer -archive -file <filename>  --  this command requires user input in the basic mode - allows you to selectively  download one archive to the directory where the command is run from. You supply  the <filename>. Add the -path <pathname> to specify the download  location. Add the -o switch to download without requiring user input. You can  use this command for scripting. You must know the name of the archive that you  want to download - run the -list command to get list of stored  archives.

Caution! Remember that if Analyzer is not licensed on the  array the archive files will be encrypted for Release 24 and later. The archive  filename suffix will be .NAZ. If Analyzer is licensed the suffix will be .NAR.  When using the batch program below, make sure you use the correct suffix. If  using unlicensed Analyzer and you name the archive .NAR when you try to open the  file, you will get an error message saying that the file is encrypted and that  you must contact Support to read the file.

With Analyzer Release 22 and earlier, no Periodic Archiving  function automatically creates the archives. You will need to manually retrieve  the archive using the Navisphere GUI (which creates and retrieves the archive in  one operation) or use a CLI script to create and retrieve the archive (the JAVA  archiveretrieve.jar command). With Analyzer Release 22 and  earlier you must install the optional Analyzer Enabler in order to use Analyzer.  It is disabled if not licensed.

In either case, you should review the Navisphere On-Line Help for additional  information about using Analyzer and also the Navisphere Analyzer CLI manual  (available from Powerlink) for additional information about the different CLI  commands. Below are some examples of the methods used to automate retrieving the  archives from the array.

Scripting for FLARE releases prior to Release 26

Sometimes it is useful and necessary to automate the collection of Navisphere  Analyzer .NAR files in order to review the performance of the array.

By creating a small .BAT file that runs the Navisphere  archiveretrieve.jar command, you can set up a Windows scheduler  task to run the .BAT file at different times throughout the day. The  archiveretrieve.jar command will work for all operating  system versions of Navisphere Host Agent/CLI package. The structure of the batch  file will differ for each operating system. This example is for Windows.

The following BAT file commands will create and collect the NAR/NAZ files  from the array and store them in a subdirectory called \archive off the  C:\Programs Files\EMC\Navisphere CLI directory. The .BAT file should be created  in the "\Navisphere CLI" directory in order to run correctly. Since the  archiveretrieve.jar command is a Java command and requires that  you use the security model for Java, you must add the username/password of a  global user on the array (or create a security file).

You will need to add the IP address for both SP A and SP B. You should also  change "serial#" to the last four digits of the array serial number or  some other identifier:

For example,  "-4736-CX700-SPA.nar" would identify the NAR file as coming  from a CX700 array with the last four digits of the array serial number of  4736.

For /f "tokens=2-4  delims=/ " %%a in ('date /t') do (set date=%%a-%%b-%%c)

For  /f "tokens=1-2 delims=: " %%a in ('time /t') do (set  time=%%a-%%b-%%c)
java  -jar archiveretrieve.jar -User <username>  -Password <password> -Scope 0  -Address <IP_address_for_SP_A> -File  %date%-%time%-serial#-SPA.nar -Location "c:\program files\emc\navisphere  cli\archive"

java -jar archiveretrieve.jar  -User <username> -Password <password> -Scope 0 -Address  <IP_address_for_SP_B> -File %date%-%time%-serial#-SPB.nar  -Location "c:\program files\emc\navisphere cli\archive"
When Navisphere Analyzer  is running, it collects data points at the "Archive Interval" that you specify  in the Tools/Analyzer/Data Logging screen. Since the log can hold a finite  number of data points (155) before it starts to wrap, the time to collect the  NAR files will depend on the archive interval (you can determine the time  covered by multipling the Archive Interval by 156 time intervals). When Analyzer  is first installed, the interval is set to a default of 600 seconds (or 120  seconds for newer arrays), which means that the log can only hold about 26 hours  of data and should be collected every 24 hours. If you change the interval to 60  seconds, the log can hold about 2.5 hours before it starts to wrap and you  should collect NAR files every 2.5 hours. When you change the interval, there  will be an estimate of the length of time for the NAR.
In addition, when you  change the Archive Interval (which requires you to be in Engineering Mode for  Release 22 and lower), you must also check the "Clear Archive" box (Release 22  and lower only). Failing to do so will result in a .NAR file with incorrect  data.
Note: The version of  archiveretrieve.jar that you use is important. For CLARiiON  arrays running Release 19, 22 or 24, you must use the latest version of  Navisphere CLI for Windows Release 24. Install it cleanly -  do not install it  over an older version. This version (24) will work correctly for Release 19,  22 and 24. Release 26 does NOT include the Analyzer Java commands. The JAVA  commands will not work with arrays running Release 26 and later, only with  Release 24 and earlier.
For this script to work  correctly with Release 24, you must disable the "Periodic Archiving"  and "Stop  Automatically After" options in Tools/Analyzer/Data Logging. This will allow  Analyzer to continuously collect data in the archive (similar to Release 19).  With Release 24 and later, once you issue the retrieve command, the archive will  be empty and you can not retrieve a new archive until 10 Archive Interval have  elapsed.
In future FLARE releases,  Java commands will not  be supported. The new naviseccli commands will include three new commands that will perform the same function as  the archiveretrieve.jararchivedump.jar, and  archivemerge.jar commands.

Scripting for FLARE Release 26 and  later

JAVA commands will not work with Release 26.

If you run the -archiveretrieve command before the next 10 Archive Interval  data points have been collects, the file retrieved will be the last one created.  Remember that the -archiveretrieve command will create and retrieve a  new archive, it will not retrieve specific archives already stored on  the array.

The files created with the -archiveretrieve command will remain on the array. There is a limit to the number of archives  stored and when this limit is reached, the oldest archives will be  overwritten.

You can use the Release 24 Java commands for use with  Release 24 and earlier arrays, but not with Release 26 arrays.

The  analyzer “.jar” files will not be installed when you install Release 26 CLI. If  you install over Release 24 and already have the “.jar” files, you will have the  option to keep them.

New Analyzer subcommands with Release 26 CLI will  work the same as the Java commands now work. You can run these commands against  an array running Release 26 and later and with FLARE releases of 19, 22 and 24.  (You can now use the same script for all arrays as long as they all have the  most current flare release from 19 to 26).

  • -archiveretrieve
  • -archivedump
  • -archivemerge

The script below will only work on arrays where analyzer is enabled and  running AND (for Release 24 and later) that Periodic Archiving is Disabled AND  “Stop Automatically After” is not enabled.

  1. Use a .BAT file to retrieve archives. In this example,  “getnar.bat” is  used.
  2. Find a Windows workstation (management workstation) and install the latest  Navisphere CLI Release 26.
  3. Create a bat file, getnar.bat (see sample script below), in the c:\program  files\emc\navisphere cli\ directory.
  4. Then create a new directory called \archive off the "navisphere cli"  directory.
  5. Modify the parameters in the commands to match your  system:

    Naviseccli –h xxx.xxx.xxx.xxx -User xxxx -Password xxxx -Scope 0  analyzer -archiveretrieve -File %date%-%time%-serial#-SPx.nar -Location  "c:\program files\emc\navisphere cli\archive"

    You will need the IP  address for SP A and SP B and the username\password of a global admin user on  the array that you intend to run the batch file against. Change the -file  parameter to give the archive a name that matches the array, maybe the last 4  digits of the serial number.
  6. Then create a Scheduled Task in Windows to run the batch file every xx  interval where  xx is based on the archive interval that you have set on the  array.
       
    • If the archive interval is 600 seconds, run the program every 24 hours
    • If the archive interval is 300 seconds, run the program every 12 hours
    • If the archive interval is 120 seconds, run the program every 5 hours
    • If the archive interval is 60 seconds, run the program every 2.5  hours

----------------------------------------------------------------------------------------------------------------------------------------------

For  /f "tokens=2-4 delims=/ " %%a in ('date /t') do (set  date=%%a-%%b-%%c)

For /f "tokens=1-3 delims=: " %%a in ('time /t') do  (set time=%%a-%%b-%%c)

Naviseccli -User xxxx -Password xxxx -Scope 0  -Address xxx.xxx.xxx.xxx analyzer –archiveretrieve -File  %date%-%time%-serial#-SPA.nar -Location "c:\program files\emc\navisphere  cli\archive"

Naviseccli -User xxxx -Password xxxx -Scope 0 -Address  xxx.xxx.xxx.xxx analyzer –archiveretrieve -File %date%-%time%-serial#-SPB.nar  -Location "c:\program files\emc\navisphere  cli\archive"

See if this helps

glen

0 Kudos
wolf8963
2 Iron

Re: Another Analyzer Question

Thanks for your help all. Both answers were helpful and I have a system up and running.

Does anyone know why the data captured from SPA includes stats for SPB and vice-versa? I notice that the  values for the same object and timestamp between those pulled from SP A and those from SP B are slightly different. IE:

pulled from      Object     TimeStamp                    %util

SPA              SP B          08/30/2010 23:41           13.82716

SPB              SP B          08/30/2010 23:41           13.55102

Why is there seemingly data for the same thing from both sides? and why would it be different?  Do I need to include both in the data I analyze or are they essentially the same?

Thanks for the info

0 Kudos
kelleg
5 Tungsten

Re: Another Analyzer Question

We run analyzer on both SP's when you start logging - first one SP is started and then shortly following we start the other SP. We do that in case the data from one SP gets corrupted - the data should be almost identical. This is the reason that you do not merge an SPA NAR with and SPB NAR. Think of it as a backup. You only need one to look at performance. If for some reason the data looks strange, try the other.

glen

0 Kudos
SKT2
4 Tellurium

Re: Another Analyzer Question

which file in SP collects did give u this number(% of SP utilization)?

0 Kudos
kelleg
5 Tungsten

Re: Another Analyzer Question

I think that was from the DUMP of Analyzer NAR files

glen

0 Kudos
wolf8963
2 Iron

Re: Another Analyzer Question

Correct - NAR Dump    -   and if I did a good job, it makes me a Nar-do-well

Glen62
1 Copper

Re: Another Analyzer Question

Hi alias23122

Would you mind to share your script?

Thanks

Glen

0 Kudos
DHoffman2
3 Argentium

Re: Another Analyzer Question

I'm getting messages on some of our newer VNX arrays as follows. 

Error returned from delete archive:

Request failed.Caller not privileged.

I'm using the following (removing sensitive data/passwords from the command).

For /f "tokens=2-4 delims=/ " %%a in ('date /t') do (set date=%%a-%%b-%%c)

For /f "tokens=1-3 delims=: " %%a in ('time /t') do (set time=%%a-%%b-%%c)

Rem 1234 Array

java -jar archiveretrieve.jar -user <a global username here> -password <password for that global user> -scope 0 -address 172.x.y.z -File VNX-1234-SPA-%date%-%time%.nar -Location "F:\script\NAR_Files\1234

I have moved these scripts to a new Windows machine and did run the command to add the security file.  The script does seem to work for some of the OLDER arrays CX4's.  But not for the newer ones

Suggestions? 

0 Kudos