Exchange 2010 LUN and DAG Design for ASM/ME backups

 To add some further info, I want to have a 2 server Exchange 2010 deployment (physical servers) with HIT 3.4.2 installed on each (undecided whether to use 2 or 4 iSCSI NICs or a Qlogic adapter to handle the iSCSI traffic). I have a PS6000 that has all 4 ports dedicated to iSCSI traffic.

I have run the exchange storage calculator and HP Exchange sizer, both recommend that I use a single LUN and single database (this is for 500 users). This seems a bit drastic to me, in case database maintenance should ever be required it would take a long time on a single large Database. I would like to go with a single large LUN but with either multiple DBs (or multiple volumes within the LUN, each with its own DB) and LUN for Logs. I am really wondering if ASM would work with those configs though.

Another possible config for me would be a dedicated database lun and log lun per database (I would use 4-5 databases if I go this route). That would require 8 to 10 Luns per server.

Be aware ASM+HIT does not truncate logs.(And therefore is not a fully VSS compliant as detailed here: http://support.microsoft.com/kb/822896)

 If you are relying on native Equallogic tools to do a proper job on Exchange, it won't.

We tried Qlogic, introduced its own management issues, and gave no real advantage. (We wanted boot from iSCSI as well)

We just use standards NICs with the MS iSCSI initiator.

 Joe,

How do you respond to hwangeruk's assertion that ASM doesn't properly snapshot Exchange stores?

I am looking for ASM to provide brick-level restore capability for snapshots taken 3 times a day (Mon to Fri) ... I plan to keep about 10 days worth max. Netbackup will be used to do full nightly backups to tape.

Also, I have some questions around iSCSI in this scenario:

I was planning on using 2 NICs (NC382i HP NICs in an HP server) for iSCSI (the iSCSI network is isolated to its own switches with no non-iSCSI traffic allowed). Would there be any I/O gain in using 4 NICs instead? I know that would double the number of iSCSI connections to the PS6000 I have (and there are a finite number of connections allowed, 128 I think as a total for all volumes?).

Any benefit in using a QLogic iSCSI HBA instead of Gigabit NICs? In terms of IOPs/performance? These are a lot more costly to purchase though.

I am struggling to design a balance between:

- Number of iSCSI nics for best storage performance/redundancy (more NICs means more CPU use on the server as well).

- Number of Database/Log LUNs (to allow for ASM to work, but to reduce database size to manageable amounts in case of eseutil recovery/defrag requirement)

My current thought was 2 DB Luns + 2 Log Luns, with 2 database and logs each, but as you point out, Joe, this wont work with ASM. So I wonder if 4 DB LUNs (1 DB each) and 4 Log Luns is what it might have to be. With 2 NICs, that would be 16 iSCSI connections (32 conns with 4 NICs).

I need to assemble a test environment in the next 1-2 weeks, so any advice is appreciated.

Echange,

Hi, I’m Joe with Dell EqualLogic.  To answer your first question, no, you cannot do a brick level restore if you have 2 data stores on the same volume.  In fact, we recommend keeping each data set on separate volumes, i.e., in your case, you would need 4 volumes, 2 each for the data stores, and 2 each for the logs.  See the release notes and user guide for ASM/ME at support.dell.com/equallogic (login required).

For additional information on our best practices, check the release notes for ASM/ME and this link: http://bit.ly/fLvH8P

Regards,

Joe

Hello,

You can see my full response here on the forum. 

http://en.community.dell.com/support-forums/storage/f/3775/t/19365165.aspx

His complaint was that ASM/ME won't truncate Exchange logs. That's by design, since ASM/ME isn't a replacement for backup and truncating the logs would impact incremental/differencial backups.

The maximum connections supported depends on whether it's a 4000 and the firmware revision.  Since you're is a 6000, the 4.x FW has a limit of 512 connections per pool x 4 pools = 2048 connections.  The 5.0.x firmware doubles that to 1024 per pool x 4pools = 4096 connections. 

re: 2x. NICs.  it's not often that 2x GbE NICs would be maxed out.  Especially in an Exchange environment.   It's about IOs/sec not MB/sec.  Aligning partitions, setting 64K cluster size these are important factors.

re: Qlogic.  When iSCSI was new, CPU processors were much more limited.   The HBA offloaded network/iSCSI cycles from the host.  In environments like high end SQL installations where every CPU cycle was needed, these clearly benefited.   Much less so now.  Also, HBA's can't be a NIC and HBA at the same time.  Current generation of NICs already offload some network features anyway.

-don

Thanks Don,

For my case, I plan to have ASM snapshots augmenting daily/weely and monthly backups to tape, so not having logs truncated is a good thing.

Thanks for the advice on the NICs, these NC382i's are capable of iSCSI offload. I plan on using Windows 2008 R2 for the mailbox server OS. I am a little concerned after seeing this blog:

http://iscsi-hp-proliant.blogspot.com/

Which seems to state that iSCSI offload is not suitable for production use on Broadcom NICs in 2008, but perhaps there are updated drivers. I have these NICs working fine in 2003 server and in ESXi when connecting to the 6000e.

If you have any data on whether its good/bad to use iSCSI offload or not.

So, for the LUNs then, I currently have (in Exchange 2003) 4 mailbox stores (110 GB, 78GB, 78GB, 75GB). This is a migration project to move to Exchange 2010 (still a single mailbox server for now). I think I will now do:

OS: 2008 R2 x64

NICs: 2 x NC382i

iSCSI + TOE offload: to be tested in enabled/disabled configuration.

4 LUNs for databases (500 GB each) - hosted on PS6000e

4 LUNs for logs (80 GB each) - hosted on PS6000e

Configuration to be tested with IOMeter, sqlio, and JetStress 2010 prior to Exchange installation.

The only concern I have is with I/O, as the PS6000e is running RAID50 (needed the space for large file servers). The PS6000e has 2 file servers connected to it, but SANHQ shows a reasonably light load :

Avg Read IOPs: 3.5

AVg Write IOPs: 10.1

Avg Latency (weighted avg): 6.6ms

Avg Queue Depth: 0.7

Est. I/O Load: Low

So, I am hopeful that I should be OK. Exchange 2010 Storage sizing tools (the spreadsheet from the Exchange team blog and HP Exchange 2010 sizer) show that for my number of users and user profile, total database IOPs of 149 are required.

I did previously run JetStress on a Win 2003 server connected with 2  NC382i NICs to the SAN and there were no issues found.

Re: Offload.  There's allot of discussion on that on various boards.   In the VMware forums, I've seen people posting results that were better without the offload than with it. 

On these types of things, my personal preference is to turn them off.  If the CPU becomes bound then the offload feature could prove useful.  10GbE adds allot more load on the host so Jumbo Frames and offload might have more effect. 

In the Vmware posting I saw, Jumbo Frames without offload worked better for them .

Normal 0 false false false EN-US X-NONE X-NONE MicrosoftInternetExplorer4

 http://www.vmadmin.co.uk/vmware/35-esxserver/252-esxihwswiscsijumbo

Regards,

-don