Monitoring Symmetrix Performance using symstat CLI

Question

Monitoring Symmetrix Performance using symstatCLI Introduction This article describes how to use the Solutions Enabler command symstat to view the performance statistics of Symmetrix Array. The symstatcommand captures, in real-time, statistics information about a Symmetrix array. Detailed Information The symstat command performs the following features: Retrieves the performance counts for the Symmetrix array as a whole Retrieves the performance counts for a Director or Director Ports Retrieves the performance counts for Symmetrix devices Retrieves the performance counts for Device Group, Composite Group, RDF Group The symstat command provides different types of performance display options: REQUESTS - Reports I/O requests and throughput for selected devices, directors, or SRDF/A sessions. (This is the default type; if no type is specified, REQUESTS is used.) BACKEND - Reports back-end I/O requests and throughput for selected devices. PORT - Reports performance statistics for a director port. ISCSI - Reports GigE network statistics. CACHE - Reports cache activity for selected front-end or remote link directors, or SRDF/A sessions MEMIO - Reports cache memory to disk activity for selected devices. CYCLE - Reports cycle summary information for SRDF-A sessions DISK - Reports back-end I/O requests and throughput for selected disks. PREFETCH - Reports track prefetch disk activity for selected back-end directors only. RDF - Reports SRDF statistics from the perspective of RA groups, devices, or directors. DMSP - Reports dynamic mirroring service policy (DMSP) statistics for the selected device(s). Command Examples: 1. Array Performance Statistics. I/O and throughput statistics can be returned for all devices on a specific array by specifying the Symmetrix ID by -sid, specifying the interval by –iand the number of samples (counts) by –C. ( Symmetrix ID = 250, counts = 100, interval = 60 in following example) symstat-type REQUEST -sid 250 -i60 -c 100 In command output, IO/Sec READ/WRITE are Read/Write IOPS, KB/Sec READ/WRITE are Throughput per second. 2. Device Performance Statistics. symstat can display performance statistics such as I/O requests via front-end adapters and throughput on devices. For example, for sample intervals of 120 seconds and a sample count of 3 on logical device DEV001 in device group prod_r1, use command: symstat -i 250 -c 3 -g prod_r1 -ld DEV001 3. Device BACKEND Statistics. Using symstat with the BACKEND type specified,you can display performance statistics such as I/O requests using a DA (back-end director) and throughput on a device. symstat-type BACKEND -i 60 -c 3 -g prod_r1 -ld DEV001 4. Director I/O requests. symstat can display the I/O requests and throughput activity performance statistics on any or all directors. symstat-type REQUESTS -sid 250 -dir ALL -i60 -c 100 Specifying single director, use following command: symstat-type REQUESTS -dir 7b -sid 250 -i10 -c 100 In command output, IO/sec is IOPS which summarize Host to Front-end directors, disk to Back-end directors, local to Remote link directors. RW is Total read/write cache request rate. 5. Director ports performance statistics. Using symstat with the PORT type specified, you can display performance statistics such as I/O requests (IO/Sec) and throughput (Kbytes/Sec) of director ports for a specific Symmetrix array. For example, for sample intervals of 60 seconds, and a sample count of 3 (for I/O requests and throughput) on Symmetrix 250 all director ports: symstat -i 60 -c 5 -type PORT -sid 250 -dir ALL Specifying single director port, use following command: symstat-type REQUESTS -dir 7b -sid 250 -i10 -c 100 6. MEMIO performance statistics. Using symstat, you can display performance statistics, such as cache memory to disk I/O on particular device group. For example, enter symstat-type MEMIO -sid 250 -i 60 -c 100 In command output, WP Tracks are the count not yet destaged to disk. Prefchd/DestgdTracks/SecareTracks per second pre-fetched from disk to cache upon detection of a sequential read stream and Tracks per second saved into disks. %Dev WPmax is Write-pending device limit percentage which defined on VMAX system level. 7. Disk performance statistics. Using symstat with the DISK type specified,you can display performance statistics, such as I/O requests and throughput on a physical disk. For example, for sample intervals of 60 seconds and a sample count of 3 on disk 02A:C5, enter: symstat-type disk -i 60 -c 3 -sid250 -disk 2a,C,5 For all SCSI IDs on DA 02A, interface C, use command: symstat-type disk -i 60 -c 3 -sid250 -disk 2a,C,ALL 8. SRDF performance statistics. The SRDF type statistics extend the statistical information provided by the SRDF director to external applications providing greater visibility into the performance and behavior of Symmetrix arrays in the field. These statistics can be used to monitor on-going activity and to analyze problematic behavior. To return statistics for all RA group numbers, specify all, use command: symstat -sid 250 -i 60 -c 2 -type RDF -rdfg 11 For example, to return SRDF device-level statistics for device 37, use command: symstat -sid 250 -i 60 -c 2 -type RDF -dev 37 To return SRDF director level statistics, use command: symstat -sid 250 -i 60 -c 2 -type RDF -dir 1d To return link-level details, specify the -rdflink option as shown in the example below command. symstat -sid 250 -i 60 -c 2 -type RDF -dir 2c –rdflink For more command symstatcommand example, you can refer EMC Solutions Enabler Symmetrix Array Management CLI Product Guide on support.emc.com Author: Fenglin Li iEMC APJ Please click here for forall contents shared by us.

pablosanzoni_bn · Answer

Hi, it is a very useful information, Thanks for sharing it into the group.- Regards Pablo

ECN-APJ · Answer

Hi Pablo, we are happy that it helps.

justjoan1 · Answer

What's the best method or command to verify new/additional storage devices are associated with an Oracle database dg?

Fenglin1 · Answer

You need to know the dev name in your oracle dg then apply above commands.

Anonymous User · Answer

Hi ,It is really very useful stuff.......

ECN-APJ · Answer

Hi Pathri and Iavakumar, it is our pleasure. Thanks for your nice feedback.

JRanalli · Answer

With regards to RA port CPU I/O settings: "Synchronous I/O Percent", "Asynchronous I/O Percent", and "Copy I/O Percent".

What should they be set to in a SRDF/A environment? Does setting the "Asynchronous I/O Percent" to 20% as an example,,, limit the RA processors to only devoting 20% of their processing power to Asynchronous I/O ?

PedalHarder · Answer

If there is just Asynchronous SRDF traffic active, it will get 100% of the CPU. The QOS values are enforced only if the traffic type is present. If 2 of the 3 traffic types are present, CPU is shared as a ratio of the QOS values.

For example, with the default values are 70/20/10; if there is synchronous and adaptive copy traffic only, the CPU will be shared 7:1. That is, 87.5% for synchronous and 12.5% for adaptive copy.

ECN-APJ · Answer

Hi Jasonc,Thanks for your sharing. Hi dakota,please follow Jason's suggestion, he is right.

Neesha1 · Answer

Hi Li/Jason/Experts,

Thanks a lot for providing the commands.

I have few queries on these commands. After getting output, how we are going to interpret it???

What are the certain scenarios when we need to take actions based on the visible stats.. For example, one application showing continuous 20ms/40ms response time is experiencing performance issue & now we need to take some action...

In short, what are the maximum value of few of the parameters of the output result of all the above mentioned commands, so that we can say that Ok.. The disk(s), cache memory, FA/DA/RDF directors/ports or the I/O requests and throughput activity is facing Write/Read/Any Latency or Performance issue & will go ahead & perform certain activities for remediation of the same.

Appreciate a quick & elaborate reply from your end.

Regards

Neesha

Rivendell_86 · Answer

Hello, is it possible to get traffic info for a specific RDF port?

I tried:

symstat -type RDF -dir 4e -port 7 -sid 196 -i 60 -c 10

symstat -RA 4e -sid 196 -i 60 -c 10 -port 7

- the above return stats for whole dir 4e, not just for port 7

symstat -type port -dir 4e -port 7 -sid 196 -i 60 -c 10

- returns N/A

12:21:18 DIRECTOR PORT IO/sec Kbytes/sec
12:22:18 RF-4E 7 N/A N/A

WOW Storage Team · Answer

Hello, How do we check the IOPS stats of a remote lun from Prod Site, when there is no GUI and SYMSTAT commands helped out.   Thanks,

DELL-Sam L · Answer

Hello WOW Storage Team, There is not a way to view IOPS information via CLI.

VMAX

Was this post helpful?