OneFS 8.x and Twinax cables

DellEMC hasn't certified them but they still work.

I'm referring to the Isilon Supportability and Compatibility Guide - docu44518.pdf, published May 16, 2017

Ed,

I am sure you have been around the block and know where that will go. Any issue i have in the future will be blamed on Twinax, poor performance - Twinax, raining outside- Twinax.  Can't afford to be in an unsupported config for prod environment.

That would presume you could actually get someone to work on your tickets.

dynamox, can you share your current config?

Isilon is investigating the discrepancies and will work to circle back soon.

Peter,

what specifically are you looking for ?

Cluster size: 17 x X410 nodes (SED)

Networking: 10g interfaces connected to Nexux5k/FEX using Cisco Twinax cables (3 meters). Using LACP, trunking multiple VLANs

OneFS: 7.2.1.4, would like to upgrade to 8.x this summer

Thank you

Yeah, since you say you are using LACP, DO NOT UPGRADE to 8. All my LACP broke with 8 using twinax. Twinax would mostly work without twinax (but reported incorrect speed), but would never form a LACP. I had to order a whole set of optics to get my LACP back

Actually, no, they won't work in all situations

Specifically I had twinax and using LACP, lost all connectivity post upgrade when Isilon refused to like the twinax and would not form a bond

Had to buy a whole set of optics

thank you glennsb.

So DellEMC, what was the logic of dropping support for Twinax cables ?  You are letting your customers down with decisions like that.

dynamox

I think glennsb sounds like a customer, not a Dell EMC employee.  Anyway I helped get that section into the document in the first place back when I worked for Isilon corporate, because everyone kept asking "what works". (and I no longer do, so take this response with that in mind)  But what to think about, as you read the document, is that Isilon, nor any storage vendor can "support" optics that they didn't sell you.  They'll support right up to the slot in the NIC card.  There are hundreds of different SFP+ optics on the market, and it's impossible to test them all. That's why in the doc it was listed as "known working", but not "supported".  SFP+ is a cabling standard, but not everything or everyone conforms properly.  For instance, many of the 10G-Base-T optics on the market consume more power than the SFP+ spec allows for, which means they're very hit and miss on different NIC cards, or in different switches.  And as Isilon nodes evolve, so do the NIC cards themselves.

Ultimately the 10Gb-SR Finissair optics that Isilon has historically used have always been bulletproof, and if the network team wants Cisco Optics or something else on the switch-side that's totally fine too, which makes it really easy.  Plus 10Gb-SR optics are really cheap these days, as are fiber patch cables.

In any networking or telecomm world there is always the term of a point of demarcation, or a 'demarc' point as it's commonly referred to, where the responsibility changes hands from say an ISP to your own networking team.  Honestly the only question here is where does that point lie?

As to moving between versions of OneFS and having issues with optics after that fact, that's just plain strange; assuming no hardware or firmware has changed, someone from Dell EMC Support or engineering may be able to comment on that.

If you read at the bottom of that document it solicits feedback on what optics you have or have not been able to get to work, and where to send that information.  That request was made genuinely.  The only way that anyone knows what will or won't work, is to share information from their own experiences.  If I were an enterprise Storage Admin these days, I would probably just use what came with the nodes, and call it good, because the network team is happy, the storage team is happy, and the fiber cable in the middle costs 20-30 bucks and can be easily replaced if it is somehow the problem.

~Chris Klosterman

Principal SE, Datadobi

chris.klosterman@datadobi.com

Chris,

We are using industry standard Cisco Twinax, not some obscure brand and been using them since we bought our first Isilon back in 2010.  So the fact that all of a sudden it's busted is very disappointing.  SFP+  are $300 a pop , when you have 2 x 17 nodes cluster it adds up.

Indeed Cisco is pretty standard, I'm a former CCNA myself.  But on Cisco Twinax there are different lengths, there are active and passive cables, etc.  In fact, Cisco alone lists 30 different 10GB SFP+ options on their own site.

Cisco 10GBASE SFP+ Modules Data Sheet - Cisco

Isilon has had 3 different models of NIC card in the last few years, Chelsio, Intel, and a third that I forget off the top of my head.  So to test all of those, that's 90 different sets of tests that have to be conducted.  Now add in Arista, Brocade, and others and you can see how it quickly gets out of hand.  Again I'm not defending hardware engineering here, just explaining the logic behind not "certifying, or supporting" any optics other than those sold with the nodes.  I personally would prefer to have at least a couple of the 30 that were tested with each release and documented as known working, and probably a couple other options from a few other vendors as well.  Add in 40GbE and the problem gets worse, not better.  Anyway, just some perspective on it is all I was trying to provide.

~Chris

Fair enough Chris but EMC/Isilon teams know what their customers are using at least you hope local teams do.  I am sure we are not the only customer using Twinax with Isilon so why would developers intentionally drop support for Twinax.  Reason i say intentionally is because no previous OneFS upgrades did Twinax validation during pre-upgrade pre-checks and we have been upgrading going back to 6.4.  This was the very first pre-check where Twinax came up as an issue. So why remove support ?

EMC historically has been very good at compatibility, Isilon has been very aggressive to deprecate MS commands while mid-tier VNX File support older commands which has been very helpful to customers with older operating systems, older scanner where firmware cannot be upgraded yet need scan-to-file functionality. So again question to Dell/EMC Isilon team, why remove support for Twinax ?

dynamox,

There was no explicit decision to remove Twinax support. Due diligence is being done to understand the extent of the gap either in documentation, or via any regression in Twinax support from previous releases. People have been dedicated to sorting out the answer to ensure the documentation is clear and any unexpected regressions are identified and addressed.