Error 10007 when backing up VMs

The problem with VM backup could be with umlauts characters like (ü, ä, ö) or (~!@$^%(){}[]|,`;#\/:*?<>'"&) or path limitation to the 255 character length.

regards,

Thanks for the reply.  I've checked and everything seems to be pretty standardized with no "unusual" characters or excessive path lengths.

Could you post the backup log for a job with this error or at least the portion of a log that shows this error?  Events alone can be a little misleading but the logs usually tell more of the story.  If you don't want to post the log, search it for "error" and "warning" to find possible areas of interest.

I am experiencing the same issue I believe.  After upgrading to 6.0.1-66 to 6.1.0-402 I am experiencing errors in vSphere 5.0 U1 & Avamar.  It looks like if fails quiescing the server and then runs the snapshot again with quiescing off?

Vsphere error during backup:

Warning message from

esxserver.domain.net: The guest OS has

reported an error during quiescing. The error

code was: 5 The error message was:

'VssSyncStart' operation failed: IDispatch error #

8449 (0x80042301)

warning

8/9/2012 8:00:57 AM

Avamar Log:

2012-08-09 08:00:19 avvcbimage Info <16001>: Found 2 disk(s), 0 snapshots, and 0 snapshot ctk files, on the VMs datastore.

2012-08-09 08:00:19 avvcbimage Info <9692>: a VM snapshot has been requested

2012-08-09 08:00:19 avvcbimage Info <14627>: Creating snapshot 'Avamar-134451361931aa2cdd1597cd8a6ecc0c66ea996ef7f4416edb', quieceFS=1

2012-08-09 08:00:19 avvcbimage Info <14631>: create snapshot task still in progress, sleep for 2 sec.

2012-08-09 08:00:21 avvcbimage Info <14631>: create snapshot task still in progress, sleep for 2 sec.

2012-08-09 08:00:23 avvcbimage Info <14631>: create snapshot task still in progress, sleep for 2 sec.

2012-08-09 08:00:25 avvcbimage Info <14631>: create snapshot task still in progress, sleep for 2 sec.

2012-08-09 08:00:27 avvcbimage Info <14631>: create snapshot task still in progress, sleep for 2 sec.

2012-08-09 08:00:29 avvcbimage Info <14631>: create snapshot task still in progress, sleep for 2 sec.

2012-08-09 08:00:31 avvcbimage Info <14631>: create snapshot task still in progress, sleep for 2 sec.

2012-08-09 08:00:33 avvcbimage Info <14631>: create snapshot task still in progress, sleep for 2 sec.

2012-08-09 08:00:35 avvcbimage Info <14631>: create snapshot task still in progress, sleep for 2 sec.

2012-08-09 08:00:37 avvcbimage Info <14631>: create snapshot task still in progress, sleep for 2 sec.

2012-08-09 08:00:39 avvcbimage Info <14631>: create snapshot task still in progress, sleep for 2 sec.

2012-08-09 08:00:41 avvcbimage Info <14631>: create snapshot task still in progress, sleep for 2 sec.

2012-08-09 08:00:43 avvcbimage Info <14631>: create snapshot task still in progress, sleep for 2 sec.

2012-08-09 08:00:45 avvcbimage Info <14631>: create snapshot task still in progress, sleep for 2 sec.

2012-08-09 08:00:47 avvcbimage Info <14631>: create snapshot task still in progress, sleep for 2 sec.

2012-08-09 08:00:49 avvcbimage Info <14631>: create snapshot task still in progress, sleep for 2 sec.

2012-08-09 08:00:51 avvcbimage Info <14631>: create snapshot task still in progress, sleep for 2 sec.

2012-08-09 08:00:53 avvcbimage Info <14631>: create snapshot task still in progress, sleep for 2 sec.

2012-08-09 08:00:56 avvcbimage Info <14631>: create snapshot task still in progress, sleep for 2 sec.

2012-08-09 08:00:58 avvcbimage Info <14631>: create snapshot task still in progress, sleep for 2 sec.

2012-08-09 08:01:00 avvcbimage Warning <0000>: vSphere Task failed (application quiesce): 'An error occurred while quiescing the virtual machine. See the virtual machine's event log for details.'.

2012-08-09 08:01:00 avvcbimage Info <14627>: Creating snapshot 'Avamar-134451361931aa2cdd1597cd8a6ecc0c66ea996ef7f4416edb', quieceFS=0

2012-08-09 08:01:00 avvcbimage Info <14631>: create snapshot task still in progress, sleep for 2 sec.

2012-08-09 08:01:02 avvcbimage Info <14631>: create snapshot task still in progress, sleep for 2 sec.

2012-08-09 08:01:04 avvcbimage Info <14631>: create snapshot task still in progress, sleep for 2 sec.

2012-08-09 08:01:06 avvcbimage Info <14632>: snapshot create completed

2012-08-09 08:01:06 avvcbimage Info <14634>: New snapshot 'Avamar-134451361931aa2cdd1597cd8a6ecc0c66ea996ef7f4416edb', moref is 'snapshot-38298'.

2012-08-09 08:01:06 avvcbimage Info <14661>: Disk Info:

I'm going to leave this thread open because it appears that Brett is having similar issues, but I've been able to sort out the majority of the issues.  It appears that the 10007 message is fairly generic in nature.  Going to the Administrator / Administration Console and viewing the Event Management log appears to provide some helpful details (you will need to open the error and scroll down to the details section at the bottom).

I've found several things in my case.

1. VM snapshots may be present and this will prevent the backup.  Consolidate them, or select Delete All, which will consolidate the snaps and then delete them.

2. Create another snapshot within VMWare and then select the Delete All button.  For some reason, this deletes snapshots that don't really appear to be there.  Yeah, I'm a bit confused too, but it seemed to work.

3. Make sure there are no Physical RAW device mappings.  Sadly, we have a ton of these. 

4. Check the free snapshot space and size of any .vmdk files.  I had a 400 GB file that picked a fight between Avamar and our EQ SAN.  No winner on this one yet.

5. If the snapshot can't be quiesced, check the I/O.   There just might be too much activity for the snap to take place, or it could indicate an underlying performance problem with a physical drive / SAN.

Hope this help and thanks to all for the feedback.

Eric Marriott

I will try the manual create/delete snapshot sometime today.  I am noticing this in the Windows event log, but while I do not have a floppy presented to teh VM, it still shows up in Device Manager:

Log Name:      Application

Source:        VSS

Date:          8/9/2012 8:00:35 AM

Event ID:      12289

Task Category: None

Level:         Error

Keywords:      Classic

User:          N/A

Computer:      THE_VM_NAME

Description:

Volume Shadow Copy Service error: Unexpected error DeviceIoControl(\\?\fdc#generic_floppy_drive#6&2bc13940&0&0#{53f5630d-b6bf-11d0-94f2-00a0c91efb8b} - 00000000000004A8,0x00560000,0000000000000000,0,00000000003483F0,4096,[0]).  hr = 0x80070001, Incorrect function.

.

I believe the quiese is enabled for this particular VM, if quiese snapshot is enabled, VMWARE will talk to VSS inside the VM to take the VSS snapshot, but if for any reason VSS inside the VM fails creating the snapshot then Vmware snapshot also fails.

If you are running 6.1 proxy add the below flag and run on-demand backup and tell us the result

--quiesce_fs=false

Screenshot for your reference

The plugin will retry the snapshot if the first one fails due to some issue with quiescing (VSS failure is the most common cause for Windows guests). This retry will cause your backup to complete with exceptions.

Ideally, it would be best to address the cause of the VSS failure but if this is not possible you can use Amol's suggestion to disable quiescing, meaning the backup is reported as successful instead of completing with exceptions.

I had this same problem with the Avamar jobs completing with exceptions and vcenter throwing some unable to quiesce drive alerts.  Our problem was that there was a small builtin partition on each of the vm's that had issues.  This partition had to drive letter assigned to it.  All I had to do was give it a drive letter.  Now, during a backup, there are no errors or alerts in Avamar or vcenter.

i am running into the same issue. I have tried all flavors of the quiesce = false options and keep getting

invalid flag or Invalid boolean value 'false

In the dataset options panel i  select: windows vmware image

and in the attribute window I enter:    quiesce_fs

then in the 'attribute value panel' i enter:    false

What version of Avamar are you using? This flag is not available in older versions of the Avamar VMware Image plug-in.

We are running 6.1.00-402

Vmware vsphere is at 5.0

Thank you,

Debbie

It would be helpful to see the complete error message you are receiving. If you can copy and paste the whole line, that may provide more information.

This run completes with exceptions

2-11-26 16:24:53 avvcbimage Info <16001>: Found 1 disk(s), 0 snapshots, and 0 snapshot ctk files, on the VMs datastore.
2012-11-26 16:24:53 avvcbimage Info <9692>: a VM snapshot has been requested
2012-11-26 16:24:53 avvcbimage Info <14627>: Creating snapshot 'Avamar-1353947093a4ea180cfba89977016fd22d51b1b37a307e0289', quieceFS=1
2012-11-26 16:24:53 avvcbimage Info <14631>: create snapshot task still in progress, sleep for 2 sec.
2012-11-26 16:24:55 avvcbimage Info <14631>: create snapshot task still in progress, sleep for 2 sec.
2012-11-26 16:24:57 avvcbimage Info <14631>: create snapshot task still in progress, sleep for 2 sec.
2012-11-26 16:24:59 avvcbimage Info <14631>: create snapshot task still in progress, sleep for 2 sec.
2012-11-26 16:25:01 avvcbimage Info <14631>: create snapshot task still in progress, sleep for 2 sec.
2012-11-26 16:25:03 avvcbimage Info <14631>: create snapshot task still in progress, sleep for 2 sec.
2012-11-26 16:25:05 avvcbimage Info <14631>: create snapshot task still in progress, sleep for 2 sec.
2012-11-26 16:25:07 avvcbimage Info <14631>: create snapshot task still in progress, sleep for 2 sec.
2012-11-26 16:25:09 avvcbimage Info <14631>: create snapshot task still in progress, sleep for 2 sec.
2012-11-26 16:25:11 avvcbimage Info <14631>: create snapshot task still in progress, sleep for 2 sec.
2012-11-26 16:25:13 avvcbimage Info <14631>: create snapshot task still in progress, sleep for 2 sec.
2012-11-26 16:25:15 avvcbimage Warning <0000>: vSphere Task failed (application quiesce): 'An error occurred while quiescing the virtual machine. See the virtual machine's event log for details.'.
2012-11-26 16:25:15 avvcbimage Info <14627>: Creating snapshot 'Avamar-1353947093a4ea180cfba89977016fd22d51b1b37a307e0289', quieceFS=0
2012-11-26 16:25:15 avvcbimage Info <14631>: create snapshot task still in progress, sleep for 2 sec.
2012-11-26 16:25:17 avvcbimage Info <14632>: snapshot create completed
2012-11-26 16:25:17 avvcbimage Info <14634>: New snapshot 'Avamar-1353947093a4ea180cfba89977016fd22d51b1b37a307e0289', moref is 'snapshot-164'.
2012-11-26 16:25:17 avvcbimage Info <14661>: Disk Info:

Disk '2000': file(base):'[VMwareDS01] Citrix Xenapp01_1/Citrix Xenapp01.vmdk', backItUp=1

Then this one of the attempts for what I set it too in the dataset:

Log #1: avvcbimage log 2012-11-26 22:28:24 UTC

2012-11-26 22:28:24 avvcbimage Info <5008>: Logging to /usr/local/avamarclient/var-proxy-1/mex-VM-Daily -1353961772866-a4ea180cfba89977016fd22d51b1b37a307e0289-3016-vmimagew.log

2012-11-26 22:28:24 avvcbimage Info <5174>: - Reading /usr/local/avamarclient/var-proxy-1/avvcbimage.cmd

2012-11-26 22:28:24 avvcbimage Info <6673>: CTL listening on port 45139

2012-11-26 22:28:24 avvcbimage Info <10684>: Setting ctl message version to 3 (from 1)

2012-11-26 22:28:24 avvcbimage Info <16136>: Setting ctl max message size to 268435456

2012-11-26 22:28:24 avvcbimage Error <5062>: Invalid boolean value '=false ' for 'quiesce_fs' flag

Another attempt

2012-11-26 22:32:36 avvcbimage Error <5062>: Invalid boolean value 'false ' for 'quiesce_fs' flag

Another attempt

Thank you,

Debbie