So are you planning to change any actual IPs in use on the Isilon,

or just the netmask resp. prefix size?

Btw, are you feeling comfortable with all connections sitting on node 1?

You may want to check wether SmartConnect has been set up correctly,

so you get reasonable load balancing;  even more important,

will node-failovers do no harm to the running playout services?

Cheers

-- Peter

Hi Peter

All we want to do is change the subnet mask from 255.255.255.0 to 255.255.254.0.

Smartconnect had not been set up correctly, but it is now, and I do have connections on all the other nodes, however the core devices have permanent isilon mounts and havent switched over - I imagine they will just stay connected until such time as they are rebooted or we force them to change some how.

In respect to the failovers and playout, well, this is the question, I have no idea. I was hoping someone would know.

Is there a way to gracefully fail a node in order to force connections to move to another node? I havent been able to find a way. And if we do fail a node, is the failover seamless, or will it drop the connection? We have TV playout playing video files of up to an hour in length (4GB or so) live off Isilon. Ideally I'd like to be able to tell it ok, you can keep playing this one, but after that you need to get your file from a different node...

> All we want to do is change the subnet mask from 255.255.255.0 to 255.255.254.0.

OneFS allows this to be done on the fly.

Interestingly, testing it on a somewhat stressed virtual Isilon node running OneFS 8.0.1

I found it to cause NFS service stalls of few tenths of a second up to one second.

> In respect to the failovers and playout, well, this is the question, I have no idea. I was hoping someone would know.

Migrating an IP address to a different physical interface (on a different node)

needs to be tracked by the ARP protocol which has certain timeouts

that can cause stalls from several seconds to minutes. That's basic

networking functionality and not specific to Isilon; the Internet has

good articles on ARP caching, and you'll find the arping tool for flushing the cache.

(We are using NFS here, so I never bothered wether the Isilon

takes care of flushing the ARP cache during failovers.)

In the end you will need to find out how your environment behaves,

I'd recommend setting up a separate test subnet and test clients to experiment,

using your Isilon, or better, a test Isilon. NFS v3 connections will

stall but not fail or disconnect, SMB2 connection will disconnect

and usually re-connect, while ongoing SMB2 transfers will fail.

It depends on your playout systems how they can manage

stalls or disconnects. I would assume they buffer at least a couple of

seconds of a video stream and can handle certain transient outages.

It's also noteworthy that the newest versions of OneFS and SMB support HA.

> - I imagine they will just stay connected until such time as they are rebooted or we force them to change some how.

Have there been set up multiple IP addresses per node, as recommended?

If so, this command will redistribute the addresses across the cluster (OneFS 7.x syntax):

(Expect the aforementioned ARP cache timeouts here).

hth

-- Peter

Hi Peter

Thanks for your help so far.

We arent using NFS, only samba - its a largely windows environment here, and NFS is only used for backups from our linux servers.

I've done the subnet change, that was painless thanks for that.

Fortunately we are not having to migrate any IP's only the subnet mask needs to be changed (I made this decision because of ARP and a few other obvious reasons - like having to renumber 100 servers as well)

The rebalance command has not seemed to make much of a difference. We do not have multiple ip addresses per node (EMC did not ask for them), we have one lagg per node with one IP per lagg. I still have about 100 connections on node1 and only 7-8 on the other each (each).

Glad to hear the subnet change has worked well.

As for the re-balancing, it cannot work with only one IP address per node.

Recommended read:

https://support.emc.com/docu58740_Isilon-External-Network-Connectivity-Guide---Routing,-Network-Topologies,-and-Best-Practices-for-SmartConnect.pdf?language=en_US

Cheers

-- Peter