Yes, in file systems with lots of small files and high churn, this can result in very large replays because an entire 2 MB block will be captured in a replay when only a small amount of data has changed. For example, corporate file servers and email servers. This is one of the best practices for Exchange systems and can significantly reduce replay size.
That's correct, the Compellent will only care about the 2 MB blocks. Windows will continue writing out in 16 KB chunks and have no concept of what the SAN's doing on the back end. If you're storing a lot of small files you can create a new storage type that uses 512 KB blocks, which will reduce churn from data progression as changing a single line in a 1 KB file will result in 25% less data being moved around on the SAN. Of course the opposite is to use a filesystem like XFS or VMFS which allows you to write 2 MB blocks, but that's pretty specialized.
Thanks for the reply. As long as the Compellent isn't struggling with replays...ie they're huge replays or we're scheduling a lot of them, is there any reason why 512 KB would be beneficial over 2MB?
I wanted to give a little more depth to the great answer that haliday already gave.
If you check the best practices for the application that you are seeing a lot of churn on chances are the best practice allocation unit size recommendation will be for 64k unit sizes at the NTFS layer. This is the case with almost all database volumes and any high churn applications.
It should be noted though that there is an advantage to 512KB page size in terms of performance and replay retention sizes on these high churn volumes. The piece I wanted to add was that you must use caution when creating and deploying a 512KB page pool. First and foremost it will take space away from your primary 2MB page pool so make sure you have enough space in all of your tiers to avoid filling one up and thus jamming up data progression. The other important thing to consider when creating a smaller page pool size is the addressable page limit associated with a 32bit OS. If you are utilizing series 40 or later controllers and are on a 6.x or later version of firmware (which is a 64bit OS revision recently released) then you will have no issues with addressable page limitations simply because the limit is so high with a 64bit OS. But if you are using a 5.x code revision and are on series 30 or earlier controllers you must take this into consideration. Provision volumes conservatively from the 512KB page pool to avoid hitting an addressable page limitation. A smaller page size will increase performance and reduce overall replay size but will drastically increase the number of pages used.
Of course that limitation will never be hit if your SAN only has a few dozen TB on it. Just food for thought as hitting an addressable page limitation is a show stopper, and if your on series 30 or earlier controllers a controller upgrade will be required to go to 64bit :)
hallidayr
48 Posts
1
April 23rd, 2012 13:00
Yes, in file systems with lots of small files and high churn, this can result in very large replays because an entire 2 MB block will be captured in a replay when only a small amount of data has changed. For example, corporate file servers and email servers. This is one of the best practices for Exchange systems and can significantly reduce replay size.
hallidayr
48 Posts
0
April 23rd, 2012 12:00
That's correct, the Compellent will only care about the 2 MB blocks. Windows will continue writing out in 16 KB chunks and have no concept of what the SAN's doing on the back end. If you're storing a lot of small files you can create a new storage type that uses 512 KB blocks, which will reduce churn from data progression as changing a single line in a 1 KB file will result in 25% less data being moved around on the SAN. Of course the opposite is to use a filesystem like XFS or VMFS which allows you to write 2 MB blocks, but that's pretty specialized.
RGWLP2
3 Posts
0
April 23rd, 2012 13:00
Thanks for the reply. As long as the Compellent isn't struggling with replays...ie they're huge replays or we're scheduling a lot of them, is there any reason why 512 KB would be beneficial over 2MB?
RGWLP2
3 Posts
0
April 23rd, 2012 13:00
Thanks again. I appreciate it.
ToyRacing
1 Message
1
June 22nd, 2012 11:00
Hi,
I wanted to give a little more depth to the great answer that haliday already gave.
If you check the best practices for the application that you are seeing a lot of churn on chances are the best practice allocation unit size recommendation will be for 64k unit sizes at the NTFS layer. This is the case with almost all database volumes and any high churn applications.
It should be noted though that there is an advantage to 512KB page size in terms of performance and replay retention sizes on these high churn volumes. The piece I wanted to add was that you must use caution when creating and deploying a 512KB page pool. First and foremost it will take space away from your primary 2MB page pool so make sure you have enough space in all of your tiers to avoid filling one up and thus jamming up data progression. The other important thing to consider when creating a smaller page pool size is the addressable page limit associated with a 32bit OS. If you are utilizing series 40 or later controllers and are on a 6.x or later version of firmware (which is a 64bit OS revision recently released) then you will have no issues with addressable page limitations simply because the limit is so high with a 64bit OS. But if you are using a 5.x code revision and are on series 30 or earlier controllers you must take this into consideration. Provision volumes conservatively from the 512KB page pool to avoid hitting an addressable page limitation. A smaller page size will increase performance and reduce overall replay size but will drastically increase the number of pages used.
Of course that limitation will never be hit if your SAN only has a few dozen TB on it. Just food for thought as hitting an addressable page limitation is a show stopper, and if your on series 30 or earlier controllers a controller upgrade will be required to go to 64bit :)