iSCSI - HBAs or NICs

---

@highermath wrote:
I am awash in conflicting information regarding the relative benefits ot NICs vs HBAs to connect servers to iSCSI SANs. Obviously, I would rather spend less and get more performance. Any thoughts on this?

This is what I am setting up to start with:

2 x 2950 w/dual quad-core Xeons and 32 Gb RAM, running Xen 3.x over CentOS
1 AX150i dual processor w 12x250 Gb drives

To start with, the virtual servers will run CentOS 5.x. I would like the option to run Windows 2003 (and 2008) if possible.

The servers will be connected directly to the SAN.

The SAN just arrived, so I need to make a choice now.

---

"I would rather spend less and get more performance" - This doesn't work obviously.

You want performance; you spend more (up to a degree). If you're looking for performance I'm not sure an iSCSI solution with 7200rpm drives is really the way to start.

That being said, working within the parameters you have, a full fledged HBA offloads all of the TCP/IP work from the OS to the HBAs. They actually show up as SCSI adapters and you use specific software (in Qlogic's case this is SANsurfer) to define the settings for the HBA (IP address, subnet mask, gateway and target information).

I'd recommend you check the EMC support matrix at http://www.emc.com/interoperability. It only has a very short list of supported HBAs and CentOS isn't supported. Only 'real' Red Hat Enterprise Linux is supported (and some other commercial distributions of Linux). EMC's thinking is probably that if they have an issue that needs addressing they need to be able to 'pick up the phone and call the vendor of the OS for assistence/support'.

Also, if you can afford an AX150, 1 or more servers, HBAs, and any other needed hardware (e.g. one or more switches), I'd imagine you should also be able to afford a Red Hat license (Dell offers RHEL5 for as little as US$244 when purchasing a Poweredge 2950).

I guess that I gave too much information. I know what an HBA is, I know what a SAN is, and I have a basic understanding of iSCSI.

The idea of using a NIC over and HBA has been widely touted by Intel, and somebody at Dell must like the idea, as you have a link on your site to the Intel sponsored Principled Technologies report on the subject, which asserts that the Intel PRO/1000 PT Dual Port Server Adapter outperforms the QLogic QLA4052C Dual Port iSCSI TOE HBA. I can't find that link now, but you can also get it at http://download.intel.com/network/connectivity/products/iSCSIcomp0407.pdf. This is the price and performance benefit that I mentioned. I was first told about this by a Dell presales engineer.

My question reduces to: Is this for real? Is anyone taking that approach? Is there a counterpoint?

If you look at the Intel PDF, it shows higher performance numbers on sequential reads and writes. If you're going to be backing up or doing streaming audio/video, it would seem the Intel solution would indeed be faster, however, what about if you're going to use the iSCSI storage for random IO? Intel's document doesn't seem to discuss that part.

Also, as you can see in the pdf on page number 2, the Intel solution takes a lot more processor power away from your system compared to the Qlogic solution. The Intel solution seems to take 3 to 5 times as much processor power. With dual quad cores this may not affect you though, as you probably have processor power to spare.

Also, with an AX150/AX150i on sequential reads and writes (large file copies) I have seen speeds between 30MB/s and 50MB/s. When you start doing random IO, these numbers will drop obviously. Therefor, when looking at these numbers it's hard to compare them to what you will experience if you were to try it yourself.

Given the power of your server, and relatively low performance of the AX150i as an iSCSI target, using an HBA to connect is probably overkill. The area where HBA's and/or TOE NICs really shine is when used in underpowered servers, when the server is CPU bound, or when you are connecting to multiple fast iSCSI targets.