Solved!
Go to SolutionC
Clarkson14
1 Rookie
•
89 Posts
0
4059
July 27th, 2016 18:00
Storage VMotion and failed backups
I'm thinking this is probably by design or just a normal occurrence maybe. I guess we haven't done many Storage VMotions since we've been on Avamar, only a year, mostly they would have been a VMotion/StorageVM at once.
Anyways, coincidentally we just setup an XtremIO and StorageVM'ed (only SVM) a number of test VM's over, some to test Avamar backups with. All failed, I had originally forgot to add the new data stores to the proxies, so fixed that and restarted backups and then all the jobs failed again.
Looking at the logs in Avamar I noticed that it was complaining about "...opening path '[Old Data Store]...". So scratching my head, I just tried a normal host VMotion only to another host and restarted the backups and they worked, for the hosts that I did the normal VMotion for.
I thought it might be the XIO or the fact that migrating the VM's over to it we switched them from Thin Provision to Thick Eager, but I tried the same procedure with the Stroage VMotion only without the XIO or Thick Eager switch and still got the failures.
So, I'm probably thinking this is normal in some way and possibly VM related and not necessarily Avamar, but wanted to post here and to the VMware side of things and get some different perspectives. I really appreciate if anyone has come across this and if there is anything else to do other than do a storageVM and then follow up with a normal VM to get the backups working.
We've got 500 VM's we have to StroageVM from an old VNX to this new XIO so I'm wanting to learn this possible caveat before the VM team goes crazy and StorageVM's everything over night and nothing ends up getting backed up. Thanks again.
Clarkson14
1 Rookie
•
89 Posts
2385
0
Posted July 28th, 2016 08:00
Not much to go on but these are the three error lines, without attaching the whole log.
2016-07-27 20:24:09 avvcbimage Error <0000>: [IMG0014] Problem opening vCenter:'FSU', path:'[Old_Data_Store] vmnamehere.local/vmnamehere.local.vmx'. (Log #1)
2016-07-27 20:24:09 avvcbimage Error <0000>: [IMG0014] Problem opening vCenter:'FSU', path:'[Old_Data_Store] vmnamehere.local/vmnamehere.local.vmx'. (Log #1)
2016-07-27 20:24:11 avvcbimage Error <0000>: [IMG0009] createSnapshot: snapshot creation/pre-script/post-script failed (Log #1)
Clarkson14
1 Rookie
•
89 Posts
2385
0
Posted July 28th, 2016 13:00
That did it, thanks dynamox, I guess Avamar wasn't aware of the new stores yet. How often does (or should) Avamar do a "sync" with vCenter so it picks those types of changes up?
By default Avamar syncs the information 60 minutes prior to the backup window, It is controlled by the following flag in mcserver.xml
root@avamar:~/#: grep "auto_syn" /usr/local/avamar/var/mc/server_data/prefs/mcserver.xml
If for some reason Avamar has not synced the information prior to backup window please open a support ticket, we need to investigate this issue.
Regards
Amol Powar
Amol,
thank you, that explains why Avamar discovers new VMs at exactly 7pm in my environment. Is there a command line option to "trigger" this discovery ? My VM admins deploy new VMs in vCenter and want to get into Avamar an hour later to run an on-demand backup. Of course they don't see the VM in Avamar until after 7pm. We use "container" based VM backups.
Clarkson14
1 Rookie
•
89 Posts
2385
0
Posted July 29th, 2016 10:00
In my case I was doing on-demand backups in the middle of the day, hours before any scheduled jobs were to be kicked off, so that may have been my problem I suppose. I know I ran a test yesterday on some newly created stores about the time, based on your info, that would have caused a synch, about an hour before the nightly scheduled backups.
At least I know it's an easy fix if I get a bunch of failed jobs because of this issue.
dynamox
11 Legend
•
20419 Posts
•
87439 Points
2385
0
Posted July 28th, 2016 12:00
In Avamar, have you tried to go to Administrator, right click on vCenter and select "Sync with vCenter"