OneFS 8.2 changes to SSIP

Hi,

Try these troubleshooting steps https://support.emc.com/docu93184_Isilon-Customer-Troubleshooting-Guide:-Troubleshoot-Issues-with-the-OneFS-Web-Administration-Interface.pdf?language=en_US

Thanks Josh, there's nothing wrong with the authentication service.

It's just that, it seems that disabling the SSIP to be used for accessing SSH, webgui, sftp, etc.. is one of the changes in 8.2.

Here's the pop up image:

Isilon Administration
403: Forbidden
Accessing OneFS web administration interface over a configured SmartConnect service IP address is forbidden.

Web:www.isilon.com Support:1-800-782-4362 | Worldwide: +1.508.497.7901 or http://support.emc.com

And I am able to access ssh, webgui just fine using any System zone pool IP.

But as mentioned before, I need to be using the SSIP, like I have been for years.

Thanks!

Agreed, we've just hit the same thing. We access the cluster with

https://smartconnect.clustername.example.com:8080/

and have an SSL certificate created for smartconnect.clustername.example.com. Now apparently we need a pool to access the Web UI/SSH?

An additional thought, how will the host SSH keys work if each time you attempt to connect to the cluster you get a different node, and therefore a different SSH host key?

A much worse problem: if you use a pool address, sessions break as the sessions are node local. So, as soon as you end up on a new node your session gets broken.

After upgrading to 8.2, I have an issue with the Service IP as well.  In Active Directory, the Isilon was joined to the domain using the DNS name of the Service IP (let's call that DNS name "sip").  Now, when I try to apply NTFS security to a folder (using advanced settings), or change ownership of a folder, I get "The program cannot open the required dialog box because it cannot determine whether the computer named "smartconnect" is joined to a domain"

"Smartconnect" not the name that the Isilon is joined to the domain as - it's joined using the service IP DNS name of "sip".  No DNS entries were changed - just the upgrade happened.

yep, I setup virtual Isilon 8.2 and noticed that I was no longer able to map to SMB share using \\SSIP\sharename. While I know it's not best practice to connect to specific node IP, we did have a couple of use cases where we had to point applications to specific node IP and since SSIP will move during a node going offline event ...we would use that IP to ensure connectivity for CIFS clients would not be lost. 

This is indeed a very annoying and porrly thought-out change.   While I can understand that the SSIP is intended purely to reach the delegated DNS server on port 53, the problems that this has created are legion.

The pop-up message comes from a VirtualHost definition in  /usr/local/apache24/conf/webui_httpd.conf:

ServerAdmin support@isilon.com
DocumentRoot "/usr/local/www/static"
SSLEngine on
SSLProxyEngine on
SSLCertificateFile /usr/local/apache2/conf/ssl.crt/server.crt
SSLCertificateKeyFile /usr/local/apache2/conf/ssl.key/server.key

AliasMatch ^/(.*)$ /usr/local/www/static/httpd/SSIPText.html

Require all granted

So if you connect to the SSIP and port 8080, you get the static HTML box telling you not to.  I thought that I would have to setup a separate SSL certificate for every node (!) as the certificate and key are stored locally in /usr/local/apache24/conf/ssl.key and /usr/local/apache24/conf/ssl.crt.   Swapping in new key & certificates was tedious but not hard.

However the section in webui_httpd.conf that handles connections to port 8080 on all IPs except the SSIP uses a certificate from the cache; you may have seen in KB323665 the steps to change the SSL certificate use the 'isi certificate import' command.  Again, no problem importing a certificate for every node.

BUT only one of those can have the name 'default' attached to it.  I have modified the SSLCertificateFile in each node's webui_httpd.conf to point to the respective local certificate, but on stopping and restarting Apache, that is re-written with the value from the cache of the 'default' certificate.

In effect every node MUST use the same SSL certificate, the one named as 'default'.  So you must connect using a single cluster-wide name for this to work, but you cannot do this any more with 8.2.

So far as I can see it is totally broken, as what you MUST do you CANNOT do.  Thanks Dell EMC.

So there seem to be 2 choices - (1) Make do that you can only connect securely to the 1 node that is set up as the 'default' SSL certificate (2) connect insecurely without SSL, risking your storage system passwords to eavesdropping.

In conclusion I think that the Isilon team need to come up with a new class of 'floating IP address' that if need be is only used for everything EXCEPT port 53.  We have a small cluster, but for a big cluster this would be a severe problem.

check https://community.emc.com/docs/DOC-79731 - 

OneFS 8.2: Smartconnect Service IP Address (SSIP/VIP) does not allow connections beyond DNS traffic.

@isi_tim 

Just as an aside for the copy/paste issue: Expand the editor menu (three horizontal dots . . .  from the menu bar), then in the expanded menu choose for "Insert/Edit code sample" 

(Wierd enough, these menu items might be missing sometimes. With a narrow browser window or a large font size, some menu items are simply dropped from the super-dynamic layout, sigh.)

-- Peter

Thanks Peter,

yes I was doing that. Also tried "Spoiler tags". The code functionality on this site is broke. It doesn't support e.g. shell and so syntax-highlighting is broken. But even worse, the parsing on the site is broken and so, depending on what you try to paste, posts will fail silently or give a bogus error about unable to find https://.../post/post/post.

The last time I did this, I ended up posting Snipping Tool images of the code in question

Tim

Hi,

@isi_tim 

Any chance I could trouble you for those example over email please? We just upgraded to v9.1 and its been nothing but a massive pain with all the "hardening" which isn't actual hardening. More like a facade of hardening held up by duct tape.

Thank you.