2x2 vs 4x1 Link Aggregation for Block and File traffic.

Question

The goals:

* Migrate our existing CIFS/NFS services from our existing server+DAS setup to our shiny new NS20s.
* Learn and Develop an initial iSCSI infrastructure for VMware/Xen.

The context:

We're not sure how our block vs file level network traffic will evolve during our roll-out of our 2x NS20s. Our current setup has 8x dedicated (old) Solaris file servers each with a 1Gb Ethernet serving CIFS and NFS. From our analysis so far it seems that only one or two of these servers are saturating their ethernet links at peak times. Our estimates suggest that the traffic will get close to 200MB/s only at peak times (average at peak is 150MB/s). Some of the traffic to the existing servers will not be transferred onto the NS20s are those services are going elsewhere. i.e. the total heading for the NS20s should be less than the 8 existing (ageing) servers.

The problem:

Our 2x NS20s in active/passive setup each have 4x 1Gb active ethernet ports each. The data from the existing servers is to be split across the 2x NS20s. This gives a total network bandwidth similar to our existing 8x 1Gb.

The catch is we'd like to experiment with some iSCSI in this mix too. From this point of view creating an LACP aggregation of 2x1Gb for the block traffic and another LACP aggregation of 2x 1Gb for the file traffic seems like a useful think to do. However, I've noticed that it doesn't seem possible to add additional devices to an existing LACP link after creation. This would seem to prevent us from being able to go for a 2x2 block/file now, and later migrate to a 4x1 file only later if required without having to re-define all the interfaces on top. This would be useful to be able to do as we could shift the iSCSI off elsewhere and let the NS20s serve CIFS/NFS only.

The questions:

1. Have I lost the plot?
2. Should I be worried about letting the block and file type traffic share the same LACP aggregation?
3. Is there a better approach to this that I'm missing (that doesn't involve spending any more money).

dynamox · Accepted Answer

i've seen blades lock up on big Catalyst boxes ..if you go with 3x1 ...maybe put cge interface on a separate blade to get at least some kind of physical redundancy, obviously another physical switch is better ..but at least it will keep you running (in degraded mode).

dynamox · Answer

if i understood correctly, you would want to create an LACP trunk with 2 interfaces but that will leave you in a vulnerable state since LACP members have to be on the same switch ..if that switch happens to fail ..your clients lose access. Since NS20 has 4 ethernet ports you could do 3x1 ..with 3 connections to switch A and 1 connection to switch B. The 3 connections would form and LACP trunk and forth connection would simply be a cge device going to another switch. Combine LACP trunk with cge device to form an FSN device. At that point you could assign different VLAN for iSCSI traffic and use another VLAN for CIFS/NFS traffic all on the same FSN device.

Rainer_EMC · Answer

Besides dynamox good comments about switch failure its really a judgement call

Yes, its best practice to separate ISCSI traffic from file traffic - if you have Apps that dont take failure lightly like Exchange, SQL, ... I would do that
If dedicating interfaces is too inflexible at least put it into an extra VLAN on top of the LACP
In this case you should also put it into an extra subnet so that you dont get into routing trouble

How many interfaces you use for what really depends on how much traffic you expect

Yes, you cant currently expand an LCAP device on the Celerra
Could you please open a product enhancement request for that through Powerlink ?
But deleting and recreating isnt too difficult and if you know what your're doing or have EMC support help it can be done with relatively little user downtime

carwynedwards · Answer

Thank you very much for your replies. I have to say I got much better answers here than via the support request

Something else that came to mind is our VMD/CIFS Server configuration. As we are splitting data across VDMs to allow migration of data (and service) by type we have the option of going for the 2x2 setup and moving the CIFS servers to the other LCAP aggregate later rather than keep them in one big 4x1. Trade-off I know.

As for the FSNs - we don't currently have the network infrastructure for that (from what I understand of it, different team). We have a nice expensive Cisco 6500 with dual supervisors and the LACP aggs. are split across different blades but this is as much as we've been able to do so far.

Part of the reason for the experimenting and the reason for opting for the FC version of the NS20 is to allow us to learn and compare the options we have.

carwynedwards · Answer

We have some physical redundancy as the different links in the aggregation are on different blades. I do see where you are coming from though.

Unfortunately in our case if the 6500 all goes phut our routes to the clients are gone anyway so the FSN would only be used inside the datacenter. Still this might be worth it for the iSCSI.

Good thinking.

Rainer_EMC · Answer

In that case I would go for one 4-port LCAP but create an extra VLAN and subnet for the ISCSI traffic

Thats best practice - if you do it right then you dont get the normal other protocol chatter and broad/multicast traffic there and have the ISCSI as undisturbed as possible.
If you are using it just as a local IP SAN then you dont even have to make it routeable, which decreases the chances someone else using it.

Celerra

2x2 vs 4x1 Link Aggregation for Block and File traffic.

Was this post helpful?