If there are several VMs on the same LUN backing up at the same time, that could certainly cause drive contention. That is more likely to be the culprit than the I/O usage of the proxies themselves, in my mind. I won't rule out the proxies using a certain amount of I/O (as I said, I haven't tested it myself) but correlation (the proxy VMs are on the busy LUNs) doesn't imply causation (the proxy VMs are causing the LUNs to be busy).
There is a point at which CBT reaches diminishing returns because the overhead from merging the CBT information outweighs the performance gain of backing up fewer sectors. This is generally a function of the change rate on the disks, rather than their size, however. It might be worthwihle to have somebody from the non-Windows client team take a look at the VM backup performance to see if you're reaching the point of diminishing returns.
The snap itself will generally be very fast but the client software on the proxy has to read any sectors that have changed since the last backup out of the snap and that will generate quite a bit of I/O.
I believe you can also set up stats counters for individual VMs in VMware. It would be interesting to see how much I/O the proxies themselves are actually using, though you might have to put the proxy VM on its own LUN to see whether the I/O is for the proxy's VMDK or the client's -- I'm not sure how granular the stat counters are. If you have the SAN space available, it certainly couldn't hurt to put the proxies on their own LUN(s).
That is my point if you have several vm's and proxies on the same lun trying to read (or snap) and write at the same time, doesn't that cause drive contention?
So if you had lets say 2 dedicated luns for 20 proxies they would not try to read and write at the same time on same lun correct?
I have acouple of large servers with cbt enabled that seem to take longer than should to finish.
And in analizer on the array at the time of backups the luns that have proxies on them have higher utilization and latency.
But the snap happens quick just throwing this out there, having to recreate proxies after SP1.
In general, the performance of a client is going to be bottlenecked by the rate at which the Avamar client software can read the data from the client system's disks. In the case of VMware Image backups, the proxy mounts a snapshot of the target VM's disks to the proxy and the client software reads from that. This means that the bottleneck is likely to be the datastore where the client's VMDKs are stored rather than the datastore where the proxy's VMDKs are stored.
I haven't tested this myself but based on the architecture design, I don't think you will get much of a performance boost (if any at all) from putting the proxy VMs on their own LUNs.
Good to hear the problem is no longer happening. It would be interesting to find out how many proxies you are able to put on the LUN without degrading the performance.
I setup 10 proxies on one lun and the same for the other 10.
The backup jobs are completing faster than before.
But I hit a big snag when recreating the proxies from 6.0.1.66 to 6.1.0.402. The proxy os has changed from rhel to suse, and it takes a reinstall. Before you could rename the proxies, but with suse you need the -proxy-1 at the end when you recreate them.
I was in autopilot mode and followed the same way as always, some proxies would work and some would not. The ones that weren’t rebooted were working and the others were not, the application lost connection to the server and needed to be reregistered. The upgrade pdf shows to mount iso reboot and it will upgrade, but this is not so. I hope this saves someone from chasing their tail like I did. Some might need the same name or like a certain proxy name.
ionthegeek
2 Intern
•
2049 Posts
677
0
Posted November 9th, 2012 17:00
If there are several VMs on the same LUN backing up at the same time, that could certainly cause drive contention. That is more likely to be the culprit than the I/O usage of the proxies themselves, in my mind. I won't rule out the proxies using a certain amount of I/O (as I said, I haven't tested it myself) but correlation (the proxy VMs are on the busy LUNs) doesn't imply causation (the proxy VMs are causing the LUNs to be busy).
There is a point at which CBT reaches diminishing returns because the overhead from merging the CBT information outweighs the performance gain of backing up fewer sectors. This is generally a function of the change rate on the disks, rather than their size, however. It might be worthwihle to have somebody from the non-Windows client team take a look at the VM backup performance to see if you're reaching the point of diminishing returns.
The snap itself will generally be very fast but the client software on the proxy has to read any sectors that have changed since the last backup out of the snap and that will generate quite a bit of I/O.
I believe you can also set up stats counters for individual VMs in VMware. It would be interesting to see how much I/O the proxies themselves are actually using, though you might have to put the proxy VM on its own LUN to see whether the I/O is for the proxy's VMDK or the client's -- I'm not sure how granular the stat counters are. If you have the SAN space available, it certainly couldn't hurt to put the proxies on their own LUN(s).