We newly implemented avamar 6.1 for backing up vmware clients, We are backing up more then 50 clients without any problem but 3 of this clients backup fails with below error. There isn`t any permission issue for our backup user. It is local admin and have administrator role on vcenter. Also there isn`t any snapshot seems for this client. I can`t find any similar issue on powerlink`s knowledgebase. Have you any idea about this issue ?
2012-05-30 12:15:31 avvcbimage Warning <16002>: Too many extra snapshot files (1) were found on the VMs datastore. This can cause a problem for the backup or restore.
2012-05-30 12:15:31 avvcbimage FATAL <16018>: The datastore information from VMX '[EMC-Datastore03] TRISTTRMS05/TRISTTRMS05.vmx' will not permit a restore or backup. (Log #1)
2012-05-30 12:15:31 avvcbimage Error <9759>: createSnapshot: snapshot creation failed (Log #1)
Solved! Go to Solution.
In the past we ibackup this virtual machine with another avamar 5 server. Can this issue cause this problem ?
I also had this issue on a few VM's. I went to the summary tab on the VM on vSphere and found the message below that it needed disk consolidation. Then right click the VM... Snapshot and select consolidate. It may take a while depending on the size of the VM. After that the Avamar VM backup worked. I hope this works for you.
To add to the above
I have found this issue to be caused by VM snapshots leaving 'orphaned' vmdk delta files in the data store. These vmdk's end in 00000x.vmdk and will cause the error message seen above.
This VMWare KB article explains further:
In my case, since the vmdk's weren't referenced by the VM as active disks, I could simply delete them and the backup ran as normal.
We find that if VM snapshots get timed out (that is are still running at the end of a backup window) they quite often get left attached to the Avamar proxy and that can lead to these problems.
Thanks for your helps, issue resolved after shutting down vm then we deleted all 'orphaned' vmdk delta files in the data store and problem resolved,
This problem only presents itself when you are trying to snap a VMware image from a Windows 2008R2 virtual machine.You can call support and they will provide a fix that stops the quiesing of the VM in question.
This advanced attribute: [avvcbimage]quiesce_fs=false will disable quiesced snapshots for VMware image backups.
You can also create a new snapshot and delete it to trigger snapshot consolidation. A storage vMotion to a new datastore will clean up any orphaned snapshot files. You can then delete the files on the old datastore after the vMotion completes.
During a quiesced snapshot, VSS writers in the guest VM are invoked to create an application-consistent snapshot of the VM. If the snapshot is non-quiesced, the snapshot process holds off IO for the VM long enough to create the snapshot but it will only be crash consistent. In practice, non-quiesced snapshot-based backups are easier and more reliable (in my opinion) because you don't have to worry about VSS errors and you don't have to hot add additional disks to the guest VM as you do when creating a quiesced snapshot. That process can lead to weird errors and running out of SCSI addresses, particularly if you have VMs with many disks but few virtual SCSI controllers. I'd try both and see what works best in your environment.