Maybe I broke MEM so let me give you a little background. I know my setup is not ideal but I have to make do with what I have.
My host has one 10gig connection (shared with regular network traffic on DVS with network io control) and one 1gig connection as a backup on a standard switch. The 10gig and 1gig are both added to a software iscsi initiator. In short, I'm sending everything over the 10gig and only using 1gigs as backups.
On the eql side, I have one array connected at 1x10gig and another with 2x1gig.
Per the MEM rules and from what I saw originally, MEM made 1 connection to the 10gig array ( I forget what it did for the 1gig array). The problem with that is I did not have an active connection on the backup 1gig connection. VMware was reporting only 1 path to the 10gig array. To fix this, I change the policy to fixed (VMware) and set the 10gig as active. In testing this seemed to work as I wanted: when I disabled the 10 gig, the 1gig showed as active (I/O). When turning 10gig back on, it switched back to 10gig.
Could having changes the policy on a few of the luns messed up MEM/ESXi? I tried changing it back the the eql policy but it didn't seem to help (I rebooted twice but didn't see a difference unless having my VC was down didn't allow something to update correctly on the host?)
Forgot to mention that when I put the policy back to what it should be, the 10gig to 10gig was still showing 2 paths. Technically that's what I want but that's not what MEM should be doing as far as I know.
> When the 10GbE is active it's connecting to the GbE arrays
Yes but only to read ISOs and clone from templates (which are on the 1gig array). The 10gig hosts are only running VMs from a 10gig array.
> Are you using VLANs segment out the traffic?
Yes. All iSCSI traffic is on it's own vlan
I'm tempted to reinstall the host and see if that error was there before I changed the policies. I don't think it was but I can't be 100% sure. Even if I don't see 1gig connection originally, maybe a failure of the 10gig connection will magically make it discover the 1gig connection and use it. I'm very tempted to find some time to try it :)
A little more info... I went back in the esxi logs (vmkwarning.log and shell.log) and as best as I can tell, the error was there on the first reboot after installing the plugin, before i configured anything. Maybe the error is normal and nothing to worry about?
Checking the vmkwarning.log file for the file I did the install, it seems that error shows up when the plugin is installed. There's a few other errors but they seem to go away on reboot, only one comes back
ALERT: Jumpstart dependency error: Initialization barrier "vmkeventd" is not provided by any plugin but the following plugins require it: "psp-eql"
ALERT: Jumpstart dependency error: Initialization barrier "vmkernel vmkapi compatibility" is not provided by any plugin but the following plugins require it: "psp-eql"
ALERT: Jumpstart dependency error: Initialization barrier "NMP" is not provided by any plugin but the following plugins require it: "psp-eql-load"
ALERT: Jumpstart dependency error: Initialization barrier "restore system configuration" is not provided by any plugin but the following plugins require it: "advanced-user-configuration-options"
I opened a ticket with Dell and it got all the way to level 4 but they say they can't reproduce it and tell me i need to ask vmware if this error is something to worry about.
I don't think it has anything to do with your upgrade or path as I can reproduce the problem by diong a clean install of ESXi (dell customized ISO), then doing esxcli software vib install -d ... to install MEM. There is zero iSCSI configured and the system is about as clean as you can get (only thing i did was assign an IP so i could connect and upload the installer to the local drives)
Jumpstart dependency error: Initialization barrier "vmkeventd" is not provided by any plugin but the following plugins require it: "psp-eql"
I am in the process of upgrading to vSphere 6, did vCenter first, and just got around to installing the Dell customized ESXi 6 image to the hosts.
Prior to upgrading, I changed my path policy to VMware Round Robin and did not uninstall MEM. I am wondering if this shows up because I didn't uninstall MEM 1.2, only changed the policy to RR, upgrade host and going to install MEM 1.3.
Keep me posted if you hear anything - not going to open a support case to chase my tail. Has there been any impact to your system so far? Besides the glaring red message that will cause me to lose sleep, have you been able to ignore it?
This is my host I'm using to rebuild vcenter on version 6 so the only thing running is vcenter and the PSC so it's basically idle (and with all the distractions at work, I haven't been able to get much done).
What hardware are you installing this on, what network cards are in the machine and is this the dell customized version of esxi you're using? If I can reproduce this easily with a clean install and Dell can't, i'm wondering if this is hardware related (ie. maybe it doesn't like broadcom nics although i don't see what MEM really caring about what nic is used)
I am using 3x Dell R620's with 1x PS6100X. I am upgrading from 5.5 running MEM 1.2.
I upgraded EQL to V 8-0-3 and rebuilt a clean vCenter 6 Windows Server with vUM. Just started remediating the hosts to ESXi 6 today. I am also using the Dell Customized ESXi 6 image (VMware-VMvisor-Installer-6.0.0-2494585.x86_64-Dell_Customized-A00.iso).
The Dell R620's have 2 NIC's - 1 Broadcom NetXtreme BCM5720 and 1 Broadcom NetXtreme BCM5719.
Of course I have everything configured for best practice. I follow my build doc so I don't screw things up :)
Eventually I'm going to try VVols but in my environment (dev/qa) where people are snapshot happy on their VMs, I'm just wondering if reverts are gong to take a long time now because of how vvol snapshots work (basically the opposite of the way things work now).
Are there are other settings that need adjustment for VVols? I've looked around and I don't think I've seen that setting mentioned anywhere.
I don't suppose you can look for my ticket on the Dell side and see if you can get the guys to take another kick at the can to reproduce it. We've got confirmation now that it's not just a "me" issue :)
Edit: I'm not very bright. Apparently I can't see the little 1 and 2 buttons at the bottom of the thread to change pages :)
Anyway, I wanted to ask Don if there is any harm setting the NOOP timeout now, regardless of array firmware and MEM version so I don't have to worry about forgetting to switch later. Does this setting work for existing connections or is it like the delayed ack (?) setting which has to be set before discovering volumes as changing it after the fact doesn't work as easily. Can this setting be used if there is a mix of regular and vvols?
crackedup
72 Posts
0
June 24th, 2015 18:00
Hi Don,
Maybe I broke MEM so let me give you a little background. I know my setup is not ideal but I have to make do with what I have.
My host has one 10gig connection (shared with regular network traffic on DVS with network io control) and one 1gig connection as a backup on a standard switch. The 10gig and 1gig are both added to a software iscsi initiator. In short, I'm sending everything over the 10gig and only using 1gigs as backups.
On the eql side, I have one array connected at 1x10gig and another with 2x1gig.
Per the MEM rules and from what I saw originally, MEM made 1 connection to the 10gig array ( I forget what it did for the 1gig array). The problem with that is I did not have an active connection on the backup 1gig connection. VMware was reporting only 1 path to the 10gig array. To fix this, I change the policy to fixed (VMware) and set the 10gig as active. In testing this seemed to work as I wanted: when I disabled the 10 gig, the 1gig showed as active (I/O). When turning 10gig back on, it switched back to 10gig.
Could having changes the policy on a few of the luns messed up MEM/ESXi? I tried changing it back the the eql policy but it didn't seem to help (I rebooted twice but didn't see a difference unless having my VC was down didn't allow something to update correctly on the host?)
Thanks
crackedup
72 Posts
0
June 24th, 2015 18:00
Forgot to mention that when I put the policy back to what it should be, the 10gig to 10gig was still showing 2 paths. Technically that's what I want but that's not what MEM should be doing as far as I know.
crackedup
72 Posts
0
June 25th, 2015 11:00
> When the 10GbE is active it's connecting to the GbE arrays
Yes but only to read ISOs and clone from templates (which are on the 1gig array). The 10gig hosts are only running VMs from a 10gig array.
> Are you using VLANs segment out the traffic?
Yes. All iSCSI traffic is on it's own vlan
I'm tempted to reinstall the host and see if that error was there before I changed the policies. I don't think it was but I can't be 100% sure. Even if I don't see 1gig connection originally, maybe a failure of the 10gig connection will magically make it discover the 1gig connection and use it. I'm very tempted to find some time to try it :)
crackedup
72 Posts
0
June 25th, 2015 15:00
i reinstall the host and the error is still there. sometimes the screen changes so fast you don't see it but if I press alt+f11 and I still see it.
crackedup
72 Posts
0
June 25th, 2015 15:00
A little more info... I went back in the esxi logs (vmkwarning.log and shell.log) and as best as I can tell, the error was there on the first reboot after installing the plugin, before i configured anything. Maybe the error is normal and nothing to worry about?
Checking the vmkwarning.log file for the file I did the install, it seems that error shows up when the plugin is installed. There's a few other errors but they seem to go away on reboot, only one comes back
ALERT: Jumpstart dependency error: Initialization barrier "vmkeventd" is not provided by any plugin but the following plugins require it: "psp-eql"
ALERT: Jumpstart dependency error: Initialization barrier "vmkernel vmkapi compatibility" is not provided by any plugin but the following plugins require it: "psp-eql"
ALERT: Jumpstart dependency error: Initialization barrier "NMP" is not provided by any plugin but the following plugins require it: "psp-eql-load"
ALERT: Jumpstart dependency error: Initialization barrier "restore system configuration" is not provided by any plugin but the following plugins require it: "advanced-user-configuration-options"
crackedup
72 Posts
0
July 2nd, 2015 11:00
I opened a ticket with Dell and it got all the way to level 4 but they say they can't reproduce it and tell me i need to ask vmware if this error is something to worry about.
I don't think it has anything to do with your upgrade or path as I can reproduce the problem by diong a clean install of ESXi (dell customized ISO), then doing esxcli software vib install -d ... to install MEM. There is zero iSCSI configured and the system is about as clean as you can get (only thing i did was assign an IP so i could connect and upload the installer to the local drives)
LR_Brian
20 Posts
0
July 2nd, 2015 11:00
I've got this error too.
Jumpstart dependency error: Initialization barrier "vmkeventd" is not provided by any plugin but the following plugins require it: "psp-eql"
I am in the process of upgrading to vSphere 6, did vCenter first, and just got around to installing the Dell customized ESXi 6 image to the hosts.
Prior to upgrading, I changed my path policy to VMware Round Robin and did not uninstall MEM. I am wondering if this shows up because I didn't uninstall MEM 1.2, only changed the policy to RR, upgrade host and going to install MEM 1.3.
Edit - still there after installing MEM 1.3.
LR_Brian
20 Posts
0
July 2nd, 2015 12:00
Thanks crackedup.
Keep me posted if you hear anything - not going to open a support case to chase my tail. Has there been any impact to your system so far? Besides the glaring red message that will cause me to lose sleep, have you been able to ignore it?
crackedup
72 Posts
0
July 2nd, 2015 12:00
This is my host I'm using to rebuild vcenter on version 6 so the only thing running is vcenter and the PSC so it's basically idle (and with all the distractions at work, I haven't been able to get much done).
What hardware are you installing this on, what network cards are in the machine and is this the dell customized version of esxi you're using? If I can reproduce this easily with a clean install and Dell can't, i'm wondering if this is hardware related (ie. maybe it doesn't like broadcom nics although i don't see what MEM really caring about what nic is used)
LR_Brian
20 Posts
0
July 2nd, 2015 12:00
I am using 3x Dell R620's with 1x PS6100X. I am upgrading from 5.5 running MEM 1.2.
I upgraded EQL to V 8-0-3 and rebuilt a clean vCenter 6 Windows Server with vUM. Just started remediating the hosts to ESXi 6 today. I am also using the Dell Customized ESXi 6 image (VMware-VMvisor-Installer-6.0.0-2494585.x86_64-Dell_Customized-A00.iso).
The Dell R620's have 2 NIC's - 1 Broadcom NetXtreme BCM5720 and 1 Broadcom NetXtreme BCM5719.
crackedup
72 Posts
0
July 2nd, 2015 13:00
Hi Don,
Of course I have everything configured for best practice. I follow my build doc so I don't screw things up :)
Eventually I'm going to try VVols but in my environment (dev/qa) where people are snapshot happy on their VMs, I'm just wondering if reverts are gong to take a long time now because of how vvol snapshots work (basically the opposite of the way things work now).
Are there are other settings that need adjustment for VVols? I've looked around and I don't think I've seen that setting mentioned anywhere.
I don't suppose you can look for my ticket on the Dell side and see if you can get the guys to take another kick at the can to reproduce it. We've got confirmation now that it's not just a "me" issue :)
LR_Brian
20 Posts
0
July 2nd, 2015 14:00
Not that I expected much to be different, just wanted to confirm this exact error occured on all 3 of my hosts.
LR_Brian
20 Posts
0
July 6th, 2015 19:00
Thanks for the update Don!
Glad to hear you were able to reproduce the issue. Even happier that it doesn't impact the operation of MEM.
crackedup
72 Posts
0
July 7th, 2015 05:00
Edit: I'm not very bright. Apparently I can't see the little 1 and 2 buttons at the bottom of the thread to change pages :)
Anyway, I wanted to ask Don if there is any harm setting the NOOP timeout now, regardless of array firmware and MEM version so I don't have to worry about forgetting to switch later. Does this setting work for existing connections or is it like the delayed ack (?) setting which has to be set before discovering volumes as changing it after the fact doesn't work as easily. Can this setting be used if there is a mix of regular and vvols?
Thanks