powermax-node pods in CrashLoopBackOff, with initiator/timeout errors

Question

Hi, I'm new to PowerMax, no admin training, so, please excuse newbie questions. Trying to test `csi-powermax` CSI driver on my OpenShift cluster, I'm happy to see `powermax-controller` pods Running, but `powermax-node` pods start crashing after first 10 minutes. When I check pods' logs, I see error messages like;   - 'nodeStartup could not GetFCHostPortWWNs' - 'GetInitiatorList failed: Get \'https:// :8443/univmax/restapi/91/sloprovisioning/sym metrix/000196800573/initiator\': context deadline exceeded' - 'No valid initiators- could not initialize FC or iSCSI'   early, then later when CLBO;   - 'Couldn't find any ip interfaces on any of the port-groups' - 'No topology keys could be generated'   I'd be grateful if someone here can nudge me in the right direction as to what may be amiss perhaps with the array or the deploy configs.   thanks, - Hiro

ankur.patel · Answer

No problem. Zoning is how the powermax 'sees' your servers. Once this is done the install script logs into the powermax, sees that the host is there registers it.   When the csi driver does its thing it will tell the powermax to create the desired volume and present it to the servers it was zoned/registered to.    The zoning is a one time thing so not something you need to do beyond the initial deployment unless you add additional servers

miyamotoh · Answer

Thanks for the response. I know I want to test with FC, but am not sure what 'zoning a node' means. I have to pass this to my PowerMax admin. Bear with me...

ankur.patel · Answer

did you already zone the nodes to the powermax? or if iSCSI log into the powermax via the host iscsi initiators?

ankur.patel · Answer

yup, this is all your storage team would handle in Unisphere and on the network switches. Nothing you can really see from the host side of things.

miyamotoh · Answer

Thanks for clarifying that. Will wait for the admins, see if I can get through, and report back, next week.

But, actually, if it's not too much to ask, what are the things that need to be done in Unisphere? I can log into Unisphere. I just don't have access to Switches, etc..

miyamotoh · Answer

While I wait for responses from my PowerMax admins who are based in Covid-stricken area, maybe I can ask a few more newbie questions here.

Going back to the installation doc, I see;

Zoning of the Host Bus Adapters (HBAs) to the Fibre Channel port director must be completed.
Ensure that the HBA WWNs (initiators) appear on the list of initiators that are logged into the array.

as prereqs, and I'm not sure how to ensure these are done properly. Are these about creating Host(s) and/or a Host Group in Unisphere? Or, are these something that needs to be done outside Unisphere?

Thanks, - Hiro

YamunadeviS · Answer

Hi, ​ Please follow the below steps and respond with the results    1)If  the zoning has been proper, the HBA initiator should be present in the list of initiators on the array and you can check the same in Unisphere GUI. 2) If Step 1 is proper, please record the response time of the RESTAPI call for getting the list of initiators.   Get call of this RESTAPI:   Get \'https:// :8443/univmax/restapi/91/sloprovisioning/symmetrix/000196800573/initiator We will analyze the results and respond to you back. Thanks

miyamotoh · Answer

Sorry, it took a long time to circle back. I finally got zoning done, and I do see the value in the `/sys/class/fc_host/host1/port_name` on my VMs shown in the Hosts > Initiator page of the array on Unisphere. And I made sure my Port Group that I specify in the YAML for driver install has what the Junctions from zoning shows, though the doc seems to say `portGroups` is for iSCSI only, while I need FC with this PowerMax.

Anyhow, my error persists, "no topology key could be generated," and the powermax-node pods keep crashing/restarting.

The Hosts > Initiators page takes a long time to load on Unisphere. When it loads, it'd show 25000+ initiators for the array. When I `time curl` the GET call for the Initiators list, sometimes I get the response (=huge JSON) after 3 minutes, other times, I never get response for 15min... Is this (slowness) the problem? Any suggestions?

thanks, - Hiro

Containers

Was this post helpful?