Ask the Expert: Changing The Database Rules For Performance With XtremIO

Should you use ASM redundancy on all-flash?

The latest I have seen shows 3 diskgroups:

• Data
• Redo
• FRA

Plus you need space for the CRS stuff, but that is not able to be in ASM until 12c.

Regards,

Jeff

Matt,

I'm interested in how many ASM Disk Groups DBAs should use? I believe Oracle recommends at least two diskgroups by default (keep in mind I haven't researched this lately):

1. A Data disk group
   1. data
   2. index
   3. undo
   4. temp files
2. A FRA disk group
   1. archive logs
   2. backups

What approach would you recommend? I know you touched upon this in the video at about the 26 minute 30 second mark. The message was with XtremIO the DBA doesn't have to worry about the architecture of ASM Disk groups. Perhaps I'm wrong, but it does seem like some thought should go into ASM disk group design.

For example, if you plan to use replication with RecoverPoint then temporary data should be isolated into a separate disk group, correct? This might also be the case for the cluster voting disk, OCR and ASMSPFile too?

The base ASM disk group config I recommend to our customer DBAs is +data, +redo, +fra. 

I have worked with many customer DBAs that still feel the need to multiplex their redo log groups, so we talk thru either creating another redo log disk group or putting the 2^nd copy in the +fra.

Now, as far as ASM disk group design with XtremIO, to get great performance for files like redo logs, we do not have to create R10 raid groups just for these files and make sure we present each ASM disk/LUN up thru different front-end storage ports. We just don’t have to focus on RAID group design and assigning the best RAID config for each ASM disk group to minimize performance impact to the host & minimize the back-end IOPS to the array. And we don’t have to worry about changing any ASM AU size or coarse/fine grain striping defaults.

If we talk about ASM disk group design with DR in mind, and we are using storage based replication (ie. RecoverPoint), then we need to consider a few things. Keep in mind, storage based replication is the most holistic solution to protect the Application landscape for DR purposes (its not just protecting the DB).  Also, storage based replication creates a crash consistent DB in the DR site. And the contents of the temp tablespace are not used in crash recovery. So to minimize DR replication overhead, one might want to create a +temp asm disk group. I say might because if +temp has minimal writes & reads then why create a separate ASM disk group to eliminate it from replication. Keep things simple by minimizing the number of “things”. If temp tablespace has a higher level of writes & reads, then I would eliminate it from replication by creating its own ASM disk group. In addition we would want to create a separate +grid or +vote disk group and eliminate it from DR replication.  If on Oracle 11.2 or beyond, it’s recommended since the +grid/+vote disk group contains cluster, host, ip specific info and we don’t want to replicate this to a separate DB and its cluster in the DR site.

Lastly, if you clone your RAC DB using XtremIO snapshots (not sure why you wouldn’t use this 90% of the time to Clone the DB), then creating a +grid or +vote disk group is recommended. This allows you to mount the DB on to an existing configured RAC on another set of servers.  btw - I say 90% of the time because there is value in using RMAN Duplicate to create a copy of a DB back to a specific date & time per end user requests. Especially using RMAN Duplicate when the backup is stored on Data Domain .

Maybe it's not directly related to this ATE, but I was wondering if the block size used by the XtremIO is good for all types of databases (MSSQL, Oracle, MySQL, Postgress, whatever)? Or are there specific databases that work better with XtremIO than others?

From an Oracle perspective, the default DB blocksize lines up nicely with XtremIO internals. So we get great & predictable performance for Reads(DBWR), Writes(LGWR, DBWR) for the random I/O and the sequential I/O types Oracle generates.

So No worries for an Oracle DB.

Recently, I was involved with a customer who was doing proof-of-concept testing with DB2 for Linux, UNIX, and Windows and XtremIO and we discovered that as long as 512 byte block sizes were used, there were no issues with database deployment or performance. At this time, DB2 does not support 4K block sizes, but we are having ongoing conversations with the DB2 development team in Toronto and if that functionality becomes available, we will become heavily involved in early testing. It is interesting to note that if adaptive compression is used with XtremIO, deduplication ratios are extremely small or nonexistent. That's because DB2's adaptive compression essentially eliminates all duplicate data before it is written to storage.

Matt:

Yes, I have said many times that EMC XtremIO is a transactional workload accelerator. It was clearly designed with an Oracle database in mind.

Regards,

Jeff

If only all database vendors used the same block sizes!

that's exactly the answer I was looking for!

Here is a recent explanation I sent to a customer concerning block size, XtremIO, and VPLEX:

Traditionally, storage arrays and the hard drives behind them advertised a 512 byte sector size (or physical block size, or logical block size).  This was the SCSI standard. However, most storage arrays use a much larger block size internally – often 4k or larger.  Unaligned writes or writes smaller than the internal block size would carry a performance penalty.  Some time ago, the SCSI standard was modified to allow “advanced format” drives and arrays to advertise a 4k sector size to the initiator.  Only recently have leading operating systems and multi-pathing software packages become widely capable of dealing correctly with this new sector size.  XtremIO and some other all-flash arrays have been the first to embrace this new standard.  Using a 4k sector size eliminates performance issues due to misaligned I/Os as all I/Os are now guaranteed to be aligned to a 4k boundary.  VPLEX has not chosen to implement support for 4k sector sizes yet and as such, VPLEX can only virtualize XtremIO volumes if those volumes advertise a 512-byte sector size.

Customers using VMware to virtualise Oracle on XtremIO have been using 512 byte sectors and receiving very good performance. This is because VMware does not presently support 4k sector size with VMFS datastores either. EMC, VMware, and Cisco have published a performance paper showcasing near linear scalability of IOPS using XtremIO across virtual machines with consistently low latency.

DBAs and many others are very aware of having to match requirements to enable broader solutions. In this case the high availability or continuous availability offered by VPLEX outweighs using a 4k sector size.

Hi,

Microsoft SQL Server has also shown itself to require little to no tuning on XtremIO. Here's 2 papers that discuss performance, one written by Microsoft and another more recent by EMC:

Performance Evaluation of XtremIO Flash Array in a SQL 2012 BI Environment

EMC Extreme Performance and Efficiency for Microsoft SQL

"Should you use ASM redundancy on all-flash?"

    This thread somehow naturally presumed that the original question was whether to use ASM redundancy on XtremIO when it was in fact a generic question about "all-flash."

    Others have rightfully pointed out that for XtremIO the ASM deployment method should be EXTERNAL redundancy. That said, you'll find that there are competitive all-flash products that have no intrinsic data protection and those products do in fact need to employ ASM redundancy for protection against certain failures.

    This topic will need to be revisited when DSSD goes into general availability later this year as well.

In Oracle 12c world there are Oracle recommendations to create seperate disks for below requirements: +grid/+vote, +temp,+data01,+data02,+data03+,...,+redoa/+redob for multiplexing,+fra,+rman_duplication,application OH,Application SW binaries and fs1,fs2,fs_clone. After initial provisioning with 25%-33% free space, during growth additional LUNs are required to be added.

I have 1 Prod with 5 snapped test/dev with 12 different LUNs used on each. Now snapped nonprods make total 60 luns.

At this pace, I can only create 7 or 8 snapped folders for backup or unmounted snapps for future recovery and reach to 512 volumes for the snapped folder. are there any upgrade packs for 2048 snaps etc other than documented for each code version.

Thanks,

Apologies cpcore, looks like we missed your message on Apr 3rd.

You make a couple of great points with your comments and question - I see two main areas to discuss there; 1. number of LUNs per Oracle database, and 2. Number of Snapshots that can be created on XtremIO.

1. much of Oracle's preferred layout is/was related to getting the best from spinning disk, with XtremIO best practice considerations change to maximizing controller throughput, being tuning queue depth and balancing I/O across available controllers - EMC recommends external redundancy and balancing ASM diskgroups across available XtremIO controllers, eg. 2+ LUNs for +DATA. This is by no means any kind of limiting factor, but why not take advantage of multiple active/active controllers in an XtremIO cluster being able to read/write to different LUNs in a diskgroup at the same time.

2. I'm hoping you are confusing the 512 initiator limit for an XtremIO cluster, being the maximum number of supported hosts. There's no such limit on the number of snapshots that can be taken with XtremIO that I am aware of. Do let me know if you've already "hit" a limit that we may need support to look into for you.

"The maximum number of supported hosts depends on the number of EMC X-Bricks in the configuration. VCE recommends that while the maximum number of initiators per EMC XtremIO cluster is 512, the recommended limit of initiators is 32 per FC port for performance."