Hi Quincy,

Thank for your response. Could you please tell me where exactly this setting is as I checked with VMware Team and we not able to find the setting option.

Thanks

Are the datastores on the VMAX using FAST VP? Are these LUNs perhaps still sitting on the lowest tier, like SATA? It takes a while for data to settle on the right tier.

Since it seems you have already ruled out the backend, I would look at the front-end.  I would have said to look there first even if you had not ruled out the backend.

You may need to add FA CPUs and/or Symm devices (striped meta volumes).

Also if you are using the native path manager, there is a parameter which sets the number of IOs per path before a switch.  It is set to 1000 by default.  It should be set to something much lower on Symm, like 1-8.

Could you make use of more smaller host LUNs?  A 40 way meta is probably too wide.  16 way is plenty for performance.  Maybe 4 1TB LUNs rather than 1 4TB LUN?

Also you should try and keep your IO sizes smaller than 256K on the Symm.  I would make your block size 64k or 128k.

I am not sure where the round robin policy setting is, but I can try and find out.

We have 8MB block size on CX disk and 1 MB block size on the VMax Disk. Does this make any diference.

Here is what we have for the configurations:

ESX UCS blades are with 64GB RAM,  5 UCS Blades. Each UCS Blade has 12CPU X2.925Ghz Intel xion Processor

VM thin Pool is of 898 Striped Datadevices made out of 236 Fiber channel Disks. Cores per socket is 6.

ESXi5.

Each USC blade is mapped 4 FA ports on Vmax.

4 TB MetaLun made out of 40X10GB  thin Devices.

We dont have tiering on this pool.

No Fast Polciy attached to the Devices or the Storage groups on the VM side.

We have migrated the TDEV from one pool to another just verify it as the Pool is not issue here.  Even then we have issues. The VM is moved to Datastore back on CX and the Performance just improved.

oops, totally missed that. I thought you only migrated it back to CX

Check this document.

http://www.emc.com/collateral/hardware/white-papers/h8119-tuning-vmware-symmetrix-wp.pdf

What was the size of your datastores and how many VM's were on a datastore on the CX compared to the 4TB new datastore on the VMAX?

This may not be the case but i'm thinking you had smaller datastores with less VM's so you queue depth from the VMware host was sufficient for the smaller datastores.  Now that you have more VM's and more I/O the performance is degraded. 

Are you using SIOC?  If so what is your response time threshold and if you look in the storage performance is VMware throttling your queue depth on that datastore?

is there any FA  flag setting special to vmware