Unsolved
This post is more than 5 years old
2 Intern
•
146 Posts
0
8653
Too many snapshot files found on this VM's datastore.....
Networker 8.2.1.4
Vcenter 5.5
DataDomain backup target
Hello all.
Relative EMC DPS newb here running into another issue. On a few VM's, I get a failure that has the following job details.
Extracted 9 lines, suppressed 532 lines.
See file "E:\Program Files\EMC NetWorker\nsr\logs\policy\VM_PROD\257222-PDC00DNSA802W-2015-8-27-17-13-46" for complete logs.
Extracted Lines:
2015-08-27 22:13:07 avvcbimage Error <16006>: vSphere Task failed: 'Unable to access file since it is locked'.
2015-08-27 22:13:07 avvcbimage Error <14635>: VM Disk Consolidation for '[DS-EMC1261_15AE] PDC00DNSA802W/PDC00DNSA802W.vmx' task creation encountered a problem
2015-08-27 22:13:07 avvcbimage Warning <0000>: VM needs consolidation failed
2015-08-27 22:13:09 avvcbimage Warning <18652>: Too many extra snapshot files (3) were found on this Windows VMs datastore. This can cause a problem for the backup or restore.
2015-08-27 22:13:09 avvcbimage FATAL <0000>: [IMG0002] The datastore information from VMX '[DS-EMC1261_15AE] PDC00DNSA802W/PDC00DNSA802W.vmx' will not permit a restore or backup.
2015-08-27 22:13:09 avvcbimage Error <0000>: [IMG0009] createSnapshot: snapshot creation or pre/post snapshot script failed
2015-08-27 22:13:09 avvcbimage Error <0000>: [IMG0009] createSnapshot: snapshot creation/pre-script/post-script failed
2015-08-27 22:13:09 avnwcomm Warning <0000>: Plug-in returned non-zero exit code (206). Aborting NetWorker connection.
2015-08-27 22:13:20 VM_PROD-1440712803418 Warning <0000>: Workorder "VM_PROD-1440712803418" non-zero exit status 'code 3: No such process'
I am still transitioning over from Netbackup, where I never saw a vm backup failure due to an existing snapshot in place. When i look in the snapshot manager in vcenter, there are no snapshots to remove or consolidate, but within the vm's datastore, I do see some that our VM engineer does not want to, and will not let me, remove. I do not know what affect that will have. Plus, there are just going to be times when we need a snapshot to remain for a while, for maintenace or whatnot.
Is there anything I can do to force it to back these up, or to not look for a snapshot of these? There are perhaps 8 or 10 of the 1000 or so vm's that have this issue. Of course, I could load the Networker agent onto these few vm's and back them up like physical servers, but id prefer not to do that. Not trying to be lazy...just looking for better options.
Thanks all.
Todd
LilianaW
40 Posts
0
August 29th, 2015 03:00
Hello Todd,
1. Does this VM PDC00DNSA802W ( and other few ) have ever had a successful backup?
Assuming not and the fact you are not seeing any snapshots in the snapshot manager, are there any older vmdk delta files leftover that are not in use by PDC00DNSA802W.vmx ?
If yes, you need to ask the your VMware colleagues to remove and consolidate, then try a new backup.
2. For the vSphere Task failed: 'Unable to access file since it is locked' you can try and follow:
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=10051
3. When you saying "when we need a snapshot to remain for a while, for maintenance" you mean snapshots that are not taken from any backup software , but outside ,at the VMware level right? In that case is legitimate to keep for any maintenance task as part of VMware people administration.
All the snapshots used during VBA backup have different names and are supposed not to be leftover.
If leftover, then a troubleshooting of why should be done.
4. I am quite surprised when you say you never saw this behavior with NetBackup using VADP.
I worked for years with NetBackup and space was not clean of status 156 , 13, or 40 errors while backup VMware environment. We are not backup into the vacuum and problems could be elsewhere , we just need to cooperate
5. You need to have consistent backups so any other way of backup will not probably be recommended by any other backup software vendor.
6. Are those errors common to all 8-10 VM's ?
Before you are going to install NetWorker agent inside you may want to troubleshoot every VM failure separately.
Can you upload the LogBundle from the VBA and give the names of the 8-10 failed VMs ?
Regards,
--Liliana
Toddman214
2 Intern
2 Intern
•
146 Posts
0
October 28th, 2015 07:00
Liliana, I'm very sorry I havent responded. I didnt know anyone had replied to my issue.
1. Does this VM PDC00DNSA802W ( and other few ) have ever had a successful backup?
No, it has never backed up cleanly with networker, but will every time with Netbackup.
Assuming not and the fact you are not seeing any snapshots in the snapshot manager, are there any older vmdk delta files leftover that are not in use by PDC00DNSA802W.vmx ?
If yes, you need to ask the your VMware colleagues to remove and consolidate, then try a new backup.
Here are screen shots of what I see.
2. For the vSphere Task failed: 'Unable to access file since it is locked' you can try and follow:
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=10051
Going through this document now.
3. When you saying "when we need a snapshot to remain for a while, for maintenance" you mean snapshots that are not taken from any backup software , but outside ,at the VMware level right? In that case is legitimate to keep for any maintenance task as part of VMware people administration.
All the snapshots used during VBA backup have different names and are supposed not to be leftover.
If leftover, then a troubleshooting of why should be done.
Correct. I was referring to snapshots taken manually, outside of a backup application.
4. I am quite surprised when you say you never saw this behavior with NetBackup using VADP.
I worked for years with NetBackup and space was not clean of status 156 , 13, or 40 errors while backup VMware environment. We are not backup into the vacuum and problems could be elsewhere , we just need to cooperate
Yes, I can take any of these 10 or 12 vm's that will not back up with DPS, and successfully back them up with Netbackup, and even get a clean restore. Its very odd to me. Now, I've always run Netbackup with no quiescing, but when these issues started with Networker, I was running quiescing, but have turned that off since then.
5. You need to have consistent backups so any other way of backup will not probably be recommended by any other backup software vendor.
Agreed
6. Are those errors common to all 8-10 VM's ?
Before you are going to install NetWorker agent inside you may want to troubleshoot every VM failure separately.
Can you upload the LogBundle from the VBA and give the names of the 8-10 failed VMs ?
Yes, these 10 or so VM's all fail with the same type of error message. This is one of the others.
Extracted 6 lines, suppressed 527 lines.
See file "E:\Program Files\EMC NetWorker\nsr\logs\policy\VM_PROD\708161-PDC00SYNC303W-2015-10-27-19-31-54" for complete logs.
Extracted Lines:
2015-10-27 19:31:01 avvcbimage Warning <18652>: Too many extra snapshot files (7) were found on this Windows VMs datastore. This can cause a problem for the backup or restore.
2015-10-27 19:31:01 avvcbimage FATAL <0000>: [IMG0002] The datastore information from VMX '[DS-EMC1261_165F] PDC00SYNC303W/PDC00SYNC303W.vmx' will not permit a restore or backup.
2015-10-27 19:31:01 avvcbimage Error <0000>: [IMG0009] createSnapshot: snapshot creation or pre/post snapshot script failed
2015-10-27 19:31:01 avvcbimage Error <0000>: [IMG0009] createSnapshot: snapshot creation/pre-script/post-script failed
2015-10-27T19:31:02.009+05:00 avnwcomm Warning <0000>: Plug-in returned non-zero exit code (206). Aborting NetWorker connection.
2015-10-27 19:31:12 VM_PROD-1445985002290 Warning <0000>: [ctl_spawn] Workorder "VM_PROD-1445985002290" non-zero exit status 'code 3: No such process'
I am not sure how to upload a log bundle from the vba. I do know how to ssh into the vba, and access the log location. root@pdc00nwkv801:/usr/local/avamarclient/var/#:, but not sure how to upload a logbundle.