VAAI for NAS and Isilon

Hi Jason,

I'm assuming you're using OneFS 7.0.1 and vSphere 5.x in conjunction with the VAAI-NAS plugin. Have you tried cloning a VM to the same OneFS7 datastore? Then clone the same VM to a different datastore to compare performance times. That would be a better indicator of whether VAAI is functioning than a simple file copy.

As for the Hardware Acceleration question, I've seen a similar glitch on a pre-release version of the NAS plugin, and was told at the time there was a glitch in how the plugin communicated with vCenter. I've  forwarded your post to Isilon's internal engineers to see whether it's still an issue, and I'll let you know when I hear back from them.

James Walkenhorst

PS. Sorry Jase, I didn't see you'd replied earlier.

Jason,

When you run /usr/lib/vmware/nas_plugins/isi_plugin_credential/auth_gen does it give an error, or simply complete?

If it completes, then it HAS been enabled.  If I recall correctly, the documentation states that the host does not need to be rebooted.  I have not been successful getting it to work without rebooting a host.

I have only been able to get it to work after rebooting. 

You can see a screen capture of it here: http://www.jasemccarty.com/blog/wp-content/uploads/2012/11/ISIVASA-11.png

Additionally, it only works on OneFS 7.0, not previous versions.

If I can be of any further help, let me know.

Jase McCarty

EMC Sr. vSpecialist

NP James... You always bring additional value to the conversation...

Cheers!

Thanks for your reply Jase (and James). I think you were on the right track when you said that the ESX host needed to rebooted.

When I ran /usr/lib/vmware/nas_plugins/isi_plugin_credential/auth_gen it completed successfully. I am running OneFS 7.0.1.2. ESX servers are running 5.0.0, 469512.

So I tried rebooting one of the ESX hosts in the cluster and it wouldn't reconnect back to the cluster. After some head scratching and searching I found a VMware KB (http://kb.vmware.com/kb/2042590) that matches what I am seeing. The resolution though says to uninstall the EMC Isilon NAS VAAI plugin. Am I missing something here? How do I get the Isilon NAS Plugin work with ESX?

Thanks!

That's interesting.  I haven't seen that error before.  Also I haven't used the plugin with anything less than 5.0 Update 1.

The build number that you are reporting is the GA version of vSphere 5.0, released on 24 AUG 11.  I've used 5.0 Update 1 (build 623860 dated 12 MAR 12), as well as 5.0 Update 2 (build 914586 dated 20 DEC 12) and 5.1/5.1.0a (builds 799733 & 838463 dated 10 SEP 12 & 25 OCT 12 respectively).

Is there something keeping you from upgrading 5.0 GA to 5.0 Update 1 or Update 2? Also, make sure all IP addresses in the SmartConnect zone you are mounting are not being blocked in the NFS Client for the host. I would at least check the NFS Client settings.

Obviously an ESXi upgrade may or may not be inline with your organization, but I would seriously consider an updated build (even 5.0) for additional reasons (security/etc).

Good luck.

Jase

Isilon's engineering dept (them's that wrote the plugin) confirmed that they'd seen the same error/incompatibility that VMware is reporting, but only on older versions of ESXi. I think that's probably consistent with your environment, so if it's at all possible (and realizing that it may very well not be) I'm also curious as to what the outcome would be if you could upgrade your host(s)...?

Sweet! 

It looks like you are on your way.

Cheers,

Jase

After uninstalling the VAAI plugin so I could get the host talking to vCenter again I updated two of the hosts in the cluster to ESX 5.0u2 and then reinstalled the VAAI plugin. It is now showing Supported under Hardware Acceleration as expected. So apparently ESXi5.0 is not supported with the VAAI plugin. I don't see anywhere in the documentation where that is explicitly called out so it might be good to clarify the version requirements in future docs to help prevent pain to others. Then again I may just be the only person left on the planet running ESXi 5.0. Thanks for your help!

Perhaps I spoke too soon. On the third host I uninstalled the VAAI plugin, then updated it to ESXi5.0u2. When I installed the VAAI plugin again it did the same thing with the same symptoms in KB2042590.

I have uninstalled and reinstalled it several time and each time I install the VAAI plugin, run auth_gen and reboot it does the same thing.

Jason,

With that being said, I'd suggest you engage EMC Support.  They should be able to help with better (and documented) troubleshooting than here on ECN.

I don't have any 5.0.x hosts in my lab now, but I plan on standing up some today and seeing if I experience the same issues.

Thanks,

Jase

Hi again Jason.

I passed this over to the engineering group again to see if I can find out the oldest build of ESXi that DOESN'T exhibit the connection failure, and I'll get back to you when I hear from them again. In the meantime, I'm gonna echo Jase's recommendation that you open a support ticket with EMC so they can collect diagnostic and configuration info and get to the bottom of what you're seeing.

Jason,

I experienced this same issue with vSphere 5.0 Update 1.  A base install of 5.0 Update 1 behaved similarly (build 623860).

After rebuilding the host and updating it with all the relevant 5.0 patches (now at build 914586), it behaved properly.

With that being said, 5.0 Update 1, plus patches, is probably a more accurate requirement.  Just wanted to report back some of what I found.

Thanks,

Jase

Update: A fresh install of 5.0 Update 1, followed by an install of the plugin, and then update to 914586, has made vCenter lose connectivity to my host, and some functions from the command line are unavailble (like esxcli).  I'd suggest updating to build 914586 before loading the plugin.

Hi Jase,

It's very random. I am running build 914586 on all 6 of the hosts in the cluster. Just now when I rebooted one of the hosts that was working before it wouldn't boot up and exhibited the same problem.

I think we may have some attention from EMC/Isilon on this issue so maybe something will come of that.

Jason

Are your issues still persisting? Or have they cleared up?

We are still waiting for some help. David Vitera and Eric Sirles were the people we were put in touch with but it sounded like David was the one we needed and was out of office and then busy or something so we haven't actually talked to him yet.

We ended up having to uninstall VAAI for Isilon from all the ESX hosts as it would randomly cause them not to boot. We are still interested in getting it working at some point though.