Unsolved
This post is more than 5 years old
2.1K Posts
0
1764
Balancing the use of Gatekeeper devices
We are currently investigating performance concerns on a VMAX20K that may be related to syscall activity on shared FAs. There are a few things about this that concern me and I'm trying to get more info on this since I have never seen (or at least remember seeing) any best practices around where you map your GK devices. In investigating this I have also found that although we have GK devices deployed through 4 different FAs to the same management hosts the syscall traffic seems to be much higher on two of those FAs than it is on the other two. I know that syscall traffic doesn't get load balanced the same way data traffic can be, but I'm thinking there must be some way to make sure that the traffic uses the available GKs and paths in some balanced method. Is this totally random? Are there things you can do to make sure it is more balanced? How do other people handle the masking views for GKs?
Any information you have on this could be helpful, so please feel free to jump in with anything you know.
dynamox
2 Intern
2 Intern
•
20.4K Posts
1
June 29th, 2016 13:00
Allen,
I am not sure if you are still running ECC, i still am, monitoring/performance collection for my aging 20Ks. Last year i had to move gatekeepers dedicated to ECC onto its own dedicated set of FAs because syscalls from ECC were driving my FA CPU utilization over 60% and i had some application people complains about the performance. Do you see utilization stats that concern you ?
Allen Ward
2.1K Posts
0
July 13th, 2016 11:00
Yes, we are seeing high utilization stats and Support is pointing toward the syscall traffic as a contributing factor.
Are you running your Solutions Enabler on VMs? Are you doing anything special to balance the traffic over your dedicated management ports? I don't want to have to use four different masking views to balance the GKs out through 4 different ports but I'm struggling with conflicting advice on using the FIXED policy pathing at the ESX level then Physical RDM to get the GKs up to the VM where I can apparently use PowerPath to balance traffic (but only if you read the right reference... the other ones say I can't use PP).
I really don't want to have to force individual paths for individual GKs, but I will have to if I can't figure out a better way to balance things.
dynamox
2 Intern
2 Intern
•
20.4K Posts
0
July 14th, 2016 06:00
yes, ECC and Unisphere for VMAX are running on VMs. I am using the FIXED policy in ESX because i had very problematic experience with either MRU or RoundRobin (can't remember which one i tried). Basically when i tried one of those other policies, syscalls would get lost in la-la land. For example i would run "symclone terminate" on a bunch of clone devices where some devices had 10 meta members. I would end up in situation where 5 members of a meta would terminate but the other 5 would stay in their original state. We are talking about meta members getting out of wack like that, something that i wouldn't be able to do with symcli even if i wanted to do. I don't know if running PowerPath on ESX would have fixed that but we don't have PowerPath on our ESX servers so i went back to using the FIXED policy and all syscall issues were resolved. I am beating the snot out of one of my FAs with all these syscalls but since i am using dedicated ports (and FA CPU is not shared either) i am not impacting other hosts.
Allen Ward
2.1K Posts
0
July 14th, 2016 10:00
OK, that makes sense, but it sounds like you are using multiple FA ports (from your first response) but most of the traffic is going down one FA (from your second response). Are the GKs multipathed out of the array (single masking view with multiple ports in the Port Group)? Or are you using multiple masking views with one port per group?
Based on what I've been seeing I'm thinking of creating four masking views each with a port group limiting things to a single port and dedicated GKs in each one. That way the ESX physical hosts will only see a single path per GK and when I present the GKs up to the VM as physical RDM devices I can pick a group that has diverse paths to maximize the balance of traffic. It's messy to get there, but at least it might allow a balance of the syscall traffic across all four ports that I'm using for those GKs.
dynamox
2 Intern
2 Intern
•
20.4K Posts
0
August 1st, 2016 20:00
sorry, missed this. I have one masking view that consists of two ports and multiple gatekeepers. You are correct i am not doing any type of gatekeeper balancing, my goal was to get them away from my production FAs/ CPU sharing.
I like your idea of creating dedicating masking views, or maybe controlling path selection inside of VMware ? I don't know if blocking specific paths is persistent between reboots.
https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2000552
Allen Ward
2.1K Posts
0
August 18th, 2016 12:00
I'm not sure if I trust things to remain consistent if we rely on the host side to handle this. I'm pretty sure the next time we do this I'm going to go with individual masking views for a balanced set of GKs down each of four paths. Depending on how this current situation works out I might be doing that sooner than later (with the existing environment).
It would be really nice if there were a simplified automated way of handling this without jumping through hoops to get it right.