Hi Don,

Thanks for the answer and I have read the document as well.

My current idea is to have the esxi hosts equipped with a 10GbE card, that connects to a 10GbE switch (Dell 8164). The 10GbE switch would have 10GbE EQL arrays (PS6110E) and 1GbE EQL arrays (PS6100E) connected, with 2 pools , 1 pool with 1GbE members and 1 pool with 10GbE members.

Is this a valid approach, or do you see possible issues with this?

Thanks,

Bruno

Hi Don,

Thanks for the feedback provided, however a key idea here resides on the migration path.

We will not stop using the 1GbE arrays, since they are fairly new and fully functional , within the next few years therefore I need to have a solution that would allow the esxi hosts to access the 1GbE and 10GbE arrays without issues.

In short we need to maintain access to 1GbE and 10GbE from esxi hosts equipped only with 10GbE network cards, where 1GbE and 10GbE arrays and the esxi hosts are connected to the 10GbE switches.

Having said that my idea is based on the following :

Network

• install 10GbE network cards on the esxi hosts
• connect the 1GbE and 10GbE arrays to the 10GbE switches
• connect the 10GbE esxi hosts to the 10GbE switches

EQL configuration

• Add the 10GbE arrays to the existing Group Name -- EQL_Prod
• Create a new pool , let's say EQL_10GbE , where all 10GbE members are placed

Based on this my EQL Group -- EQL_Prod would look like :

• Pool EQL_10GbE with 2 members (this is the new pool)
  • 2 x PS6110E
• Pool EQL_1GbE with 2 members (this is the existing pool)

• 1 x PS6100E
• 2 x PS6100 XV

From the VMware side I would have :

• 1 Dual Port 10GbE network card , with port 01 connected to Switch #01 and port 02 connected to Switch [tag:02] 
• 2 vswitches , each with 1 physical link associated
• EQL Hit installed for MPIO
• iSCSI connection to the Group EQL_Prod IP
• access to multiple volumes, based on the 1GbE and on the 10GbE arrays

With this layout I have the following :

• different pools based on the EQL arrays network speed (1GbE and 10GbE)
• usage of 1 single type of switch and network cards on the esxi hosts - 10GbE on both
• access to volumes residing on the 1GbE based arrays and on the 10GbE based arrays

Is this a valid configuration, or am I getting myself into problems ? ;)

Thank you,

Bruno

Don, thanks for the clarification and indeed the iSCSI aren't teamed, therefore all problems above mentioned would arise.

Having said that the best way to keep 1GbE and 10GbE access would be to have the esxi hosts configured with 1GbE and 10GbE NIC's and each NIC would connect to it's respective switch in terms of speed (1GbE nic connected to the 1 GbE switch, and the 10GbE nic connected to the 10GbE switch).

Regarding the ISL between 1 and 10GbE switches a minimum of 4 x 1GbE in LAG or a preferable 2 x 10GbE LAG would be accepted. I assume that EQL replication between a 10GbE array and a 1GbE would use only 1 x 1 GbE . Can you confirm if this okay?

Bruno

Hi Don,

I have been thinking on a solution based on the usage of 1GbE NICs and 10GbE NICs on the ESXi hosts connected to different groups, and I guess this is what you are recommending.

In this case I could have , for example, a Group EQL_Prod_1GbE with IPs configured on the 192.168.100.0/24 and another Group EQL_Prod_10GbE  with IPs configured on the 192.168.110.0/24 .

Regarding the connection between 1GbE and 10GbE i don´t see the need to create any sort of connection with each other, do you agree?

Thanks

Bruno

Indeed i would need ISL to achieve data replication from the volumes hosted in the 10GbE Group to the 1GbE Group. 

The disadvantage on this setup is that I have 4 switches to manage and also several vswitches/port groups to manage within the VMware.

Regarding the 10GbE access to the 1GbE arrays, I would like to explore a bit deeper this situation.

At our environment I don't expect VM's able to saturate 4 x 1 GbE since they will be mainly test/development machines, and any production based VMs would be placed on the 10GbE based datastores.

Since the 1GbE arrays are 4x1GbE , as long as the performance demand from the VM's hosted on the 1GbE based datastores don't go beyond 80% of 4x1GbE, the environment should not have issues related to  packet loss/retransmissions and any other type of network issues.

Is this a fair line of thought or am I missing something here?

Bruno

Don, 

The EQL arrays/switches/ESXi nics would all belong to the same subnet, based on their speed.

In my case I would have the following:

 Group EQL_Prod_1GbE with EQL arrays/switches/ESXi nics IPs configured on the 192.168.100.0/24  with VMkernel ports to the SW iSCSI adapter

 Group EQL_Prod_10GbE  with EQL arrays/switches/ESXi nics  IPs configured on the 192.168.110.0/24 with VMkernel ports to the SW iSCSI adapter

The ISL would be probably 4 x 1GbE and is okay to have replication tasks based on 1GbE speeds .

Thanks for the help provided !

Bruno

As a final question, instead of having 4 switches (2 switches 1GbE and 2 swithes 1or 10GbE with ISL) would it be possible advisable to connect all ESXi iscsi initiators (1 and 10GbE) only to the 10GbE switches as long as initiators and arrays run on different subnets ?

With this I would avoid the need to maintain 2 different types of switches , lowering the management costs and so on.

What would be your take on this?

Thanks,
Bruno

Hi Don,

Thanks for your valuable feedback, but in this case the ESXi servers already have the 10Gb cards..

The point here is why to keep using 4years old 1Gbe switches rather than using the newer 10Gbe switches and make use of the bigger packet buffer size ?

In our case the 1Gb arrays would be used for test/dev, and the migration of the current production would be done by means of VMware Storage vMotion.

• All esxi hosts with 2 network cards
  • Network card A is a Quad-Port 1Gb
  • Network card B is a Dual port 10Gb
• Connection between the esxi hosts and the EQL arrays (6100E and 6110E , for example) to the 8164 switch
• Each type of EQL array, would run on different subnets as well as the iscsi initiators
  • The 1Gb arrays would run on the 192.168.100.xxx /24 subnet as well as the initiators based on the 1Gb Quad-Port network cards
  • The 10Gb arrays would run on the 192.168.110.xxx /24 subnet as well as the initiators based on the 10Gb Dual-Port network cards

This setup , as I see it, meets the following requirements:

• 10Gb initiators do not connect to the 1Gb based arrays , avoid possible oversubscription on the ports and sub consequent packet loss, retransmissions, etc..
• The 1Gb and 10Gb environments are split by the means of usage of different subnets (if needed , we can use VLANS on the switches/arrays/hosts)
• The replication from the 10GbE to the 1GbE based array/pool is possible, and all traffic would be handled within the switch, without the need to cross LAG’s across the 1Gb and 10Gb switches
• The settings of jumbo frames will be configured to all ports, since all ports will carry iscsi traffic
• The settings of traffic flow, and others, can be done port per port , or per group of ports , meeting the requirements of each type of array behind the port, either the 1Gb or the 10Gb array

 

- Bruno

Hi Don,

Thanks for your valuable feedback, but in this case the ESXi servers already have the 10Gb cards..

The point here is why to keep using 4years old 1Gbe switches rather than using the newer 10Gbe switches and make use of the bigger packet buffer size ?

In our case the 1Gb arrays would be used for test/dev, and the migration of the current production would be done by means of VMware Storage vMotion. ◦All esxi hosts with 2 network cards ◾Network card A is a Quad-Port 1Gb ◾Network card B is a Dual port 10Gb

◦Connection between the esxi hosts and the EQL arrays (6100E and 6110E , for example) to the 8164 switch ◦Each type of EQL array, would run on different subnets as well as the iscsi initiators ◾The 1Gb arrays would run on the 192.168.100.xxx /24 subnet as well as the initiators based on the 1Gb Quad-Port network cards ◾The 10Gb arrays would run on the 192.168.110.xxx /24 subnet as well as the initiators based on the 10Gb Dual-Port network cards

This setup , as I see it, meets the following requirements: ◦10Gb initiators do not connect to the 1Gb based arrays , avoid possible oversubscription on the ports and sub consequent packet loss, retransmissions, etc.. ◦The 1Gb and 10Gb environments are split by the means of usage of different subnets (if needed , we can use VLANS on the switches/arrays/hosts) ◦The replication from the 10GbE to the 1GbE based array/pool is possible, and all traffic would be handled within the switch, without the need to cross LAG’s across the 1Gb and 10Gb switches ◦The settings of jumbo frames will be configured to all ports, since all ports will carry iscsi traffic ◦The settings of traffic flow, and others, can be done port per port , or per group of ports , meeting the requirements of each type of array behind the port, either the 1Gb or the 10Gb array

 

 

 

Bruno

Hello,

I understand your point and usually I also prefer to keep things split however in this case despite the added complexity to setup I think the advantages overcome the disavantages.

To be fair the 6224 are not recent switches, and having hosts with 1 and 10GbE based initators would avoid the issue of having 10Gb initators connecting to 1Gb targets and vice-versa.

Since I would need to maintain the 1Gb together with the 10Gbe environment , it would be good if at least the switching environment is kept to a single type of switch.

So bottom line from a technical side it would be possible the mentioned configuration , and I fully agree that IP subnet would be combined with VLANs.

From Dell engineering, what do you normally see in the field ? Customers do keep 1Gb and 10Gb switches separated while using 1Gb and 10Gb initiators as well?

 

Bruno