Poor Network Performance on 2950

I'm implementing a D2D2T solution for my company. It involves a 2950, an MD1000 with 13X 500GB SATA-II drives and a PowerVault 136T with LTO2 drives. I just installed and configured our equipment (the network, and tape library were pre-existing) and i was running some prelim tests against our exchange server and I noticed that my speed to disk was much less than my speed to tape. This was unreal to me because I benchmarked my disk speed at about 400MB p/s and when Backup Exec was doing the pre-allocation of the disk files, I was averaging about 300MB p/s. So I knew something wasn't right. There is also the fact that going straight to tape, i halfway saturate my GigE link. So I fired up iperf and decided to do some raw network testing involving 3 servers on the same Gigabit ethernet network (Cisco 6509) with standard 1500 MTU. I have a 2950, a 2850 and a 2650. All servers have 4 NICs, the two integrated NICS and a PCI-X Intel Dual Port MT1000 card.  The 2850 and the 2650 are connected to the backup network via a single port off of the PCI-X Intel card. The 2950, I decided to connect one of the BroadCom Nics with TOE enabled to the backup network (this setup is preexisting. I didn't need to re-configure anything for my iperf testing).

 

So, from testing to the 2650 to the 2850 on Intel NICS with the default 8KB window, this is what I get:

 

------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 8.00 KByte (default)
------------------------------------------------------------
[1884] local 129.5.6.2 port 5001 connected with 129.5.6.1 port 8593
[ ID] Interval       Transfer     Bandwidth
[1884]  0.0-10.0 sec   237 MBytes   199 Mbits/sec

 

 

Tuning that window up to 100K (my sweet spot), I get this:

 

------------------------------------------------------------
Server listening on TCP port 5001
TCP window size:  100 KByte
------------------------------------------------------------
[1884] local 129.5.6.2 port 5001 connected with 129.5.6.1 port 10648
[ ID] Interval       Transfer     Bandwidth
[1884]  0.0-10.0 sec   766 MBytes   643 Mbits/sec

 

Now, that is where I expect to be for my network with 1500 byte frames. But the 2950 gives much different results:

 

No difference when using the default window size.

------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 8.00 KByte (default)
------------------------------------------------------------
[1896] local 129.5.6.156 port 5001 connected with 129.5.6.1 port 10774
[ ID] Interval       Transfer     Bandwidth
[1896]  0.0-10.0 sec   237 MBytes   199 Mbits/sec

 

But at what is probably the best window size for my network, my 2950 Broadcom nics are outperformed by

almost 50% (43% to be exact)

 

C:\>iperf -s -w 100k
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size:  100 KByte5
------------------------------------------------------------
[1896] local 129.5.6.156 port 5001 connected with 129.5.6.1 port 10838
[ ID] Interval       Transfer     Bandwidth
[1896]  0.0-10.0 sec   532 MBytes   446 Mbits/sec

 

This is with TOE turned on and the latest drivers. All Machines are running Windows 2003 SP1.

 

So what could be the deal here? Any takers?