Unsolved
This post is more than 5 years old
14 Posts
1
5315
GLR restore fails with "unknown error" in NMM GUI and Event ID: 51 in Windows System Event Log.
Dear All,
we are facing problem with NMM GLR restore wich fails with "unknown error please try later" message and eventID 51 in Windows system event log. GLR never worked before in this environment.
Environment:
- Windows server 2008R2 x64
- Exch. 2007 SP3
- NW Client 8.0.2.3
- NMM 2.4.1 b122
- NW Server 8.1.1.4
About one minute after starting up GLR recover in NMM GUI it fails with "unknown error. please try later" error message. Following information is displayed in NMM GUI Monitor:
Networker Recover Options:
General Options:
Diagnostic Output Level = 5
Networker Options:
Restore Type = Conventional
Stop Item Recover On Error = True
Security Options:
Pass Phrases:
Exchange Recover Options:
Granular Level Recovery Options:
Transaction Log Replacement Option = Include only logs from this restore (Point-in-time recovery)
Recover start time: 25.06.2014 14:38:10
Cleaning up GLR resources...
Creating NWFS volume...
Creating GLR RSG...GLR20140625143815
Starting Exchange recover using C:\Program Files\EMC NetWorker\nsr\bin\nsrsnap_vss_recover.exe -D 5 -A RESTORE_TYPE_ORDER=conventional -A BR_ELEVATED_WARNING=true -s spng027.DUBDPNG.local -c SPNG108 -R SPNG108 -A NSR_SNAP_TYPE=vss -A NSR_MOVE_TRANSLOGS=off -A NSR_MOUNT_DB=on -A NSR_MOUNT_RETRY_COUNT=0 -A NSR_EXCHANGE_RECOVER_MODE=glr -I -
The following 2 marked items are being recovered:
Der Objektverweis wurde nicht auf eine Objektinstanz festgelegt.
There was an unknown error while starting the recovery process. Please try again.
Restore destination has been reset for storage group SG_Franzis_1
The options specified in the Recover Options Dialog have been reset back to their safe default values.
Recover end time: 25.06.2014 14:40:05
Recover elapsed time: 1 minutes, 54 seconds.
In Windows system event log the EventID 51 is reported around the problem time:
(it is Event ID 51 - An error was detected on device \Device\xxx\xxx during a paging operation)
Protokollname: System
Quelle: Disk
Datum: 25.06.2014 14:40:04
Ereignis-ID: 51
Aufgabenkategorie:Keine
Ebene: Warnung
Schl sselw rter:Klassisch
Benutzer: Nicht zutreffend
Computer: SPNG108.dubdpng.local
Beschreibung:
Bei einem Auslagerungsvorgang wurde ein Fehler festgestellt. Betroffen ist Ger t \Device\Harddisk3\DR4.
Ereignis-XML:
http://schemas.microsoft.com/win/2004/08/events/event ">
51
3
0
0x80000000000000
18403
System
SPNG108.dubdpng.local
\Device\Harddisk3\DR4
03008000010000....
Yesterday we upgraded NW Client to v8.1.1.4 and NMM to v3.0.1 b245.
So actual configuration is Networker client 8.1.1.4 / Networker NMM 3.0.1 b245 / NW Server 8.1.1.4
Unfortunately the problem is not resolved. The same error occurs on GLR restore. Hence the problem does not seem to be related to Networker/NMM.
Here are the messages from nwfs.raw log file on Exchange Server from latest GLR run (NMM 3.0.1 and NW 8.1.1.4)
Any idea what/where to look for the root cause?
Thanks
BR,
German
Attached:
nmm.raw from Exch. Server
nwfs.raw.zip from Exch. Server
NMM_CC_Report - NMM check result file from Exch. Server.
CarlosRojas
1.7K Posts
1
July 7th, 2014 03:00
Hello German,
I see in the NMM Config Checker report the following:
Unable to get the status of the service msexchangeantispamupdate. Make sure that the service is "Started" and in "Automatic" mode.
Unable to get the status of the service msexchangeedgesync. Make sure that the service is "Started" and in "Automatic" mode.
Unable to get the status of the service msexchangefds. Make sure that the service is "Started" and in "Automatic" mode.
The Microsoft Exchange-Informationsspeicher service (MSExchangeIS) is in the "Running" state and set to "Auto" configuration mode.
The Microsoft Exchange-Mailübergabe service (MSExchangeMailSubmission) is in the "Running" state and set to "Auto" configuration mode.
The Microsoft Exchange-Postfach-Assistenten service (MSExchangeMailboxAssistants) is in the "Running" state and set to "Auto" configuration mode.
The Microsoft Exchange-Replikationsdienst service (MSExchangeRepl) is in the "Running" state and set to "Auto" configuration mode.
The Microsoft Exchange-Systemaufsicht service (MSExchangeSA) is in the "Running" state and set to "Auto" configuration mode.
Unable to get the status of the service msexchangetransport. Make sure that the service is "Started" and in "Automatic" mode.
So looks like there are some problems with those services? Did you find anything in the Application and/or System event logs?
Also looks like the backup previous to the restore failed, can you please make sure that the writers are in status [1] Stable before attempting any other operation?
Thank you,
Carlos
gnworker
14 Posts
0
July 7th, 2014 05:00
Hello Carlos,
thank you for looking at this problem.
As per customer some of Exchange Server roles are implemented on other physical servers. Hence the messages about not running exch. services should be related to particular customer's exch. configuration.
The failed backup must be one started by Networker on schedule, while NMM has being upgraded on Exch. Server. The GLR restore fails with the same error after successful backup as well (tried it on last Friday to see if problem persists while restoring backups created with the same NMM3.0.1. The status of previous Backup was OK at that time). So, in current config (NMM 3.0.1) GLR restore fails while restoring from backups created with NMM2.4.1 and 3.0.1 with the same windows error/waring message 51.
In the list of installed software on Exch. server in question I have seen some application from Trend Micro:
Just curious if this scanner can cause problem. As per customer this appl. should not perform any disk activity.
Do you have any experience with Trend Micro ScanMail for Exch?
Thanks.
Thanks.
CarlosRojas
1.7K Posts
1
July 7th, 2014 05:00
Hi,
Sometimes anti-virus can be a problem, so if you can, disable temporarily the AV and give it another try, however that scan should go to the MB's, and not really to this operations, but yes, please disable AV completely and give it another try.
What happens if you go back to the same version and build as before (2.4.1), are you then able to restore?
Thank you,
Carlos
gnworker
14 Posts
0
July 7th, 2014 05:00
GLR did not work in 2.4.1 either. The same windows error 51.
Then NW Client and NMM were upgraded due to seen problem with GLR restore.
Let you know if GLR works with disabled AV.
Thanks.
gnworker
14 Posts
0
July 17th, 2014 08:00
The customer has disabled Trend Micro MailScan services and tried another GLR restore.
Unfortunately the problem is not resolved. Also no change in problem behavior.
As per nwfs.raw the Eldos CFS is able to mount filesystem (4:07:56 PM) but 20 sec. later due to unknown event the filesystem is dismounted (4:08:14 PM):
0 7/17/2014 4:07:54 PM 1 0 0 9276 7896 0 spng108.dubdpng.local (pid7896) NSR notice NWFS mount media is starting
0 7/17/2014 4:07:54 PM 0 0 0 9276 7896 0 spng108.dubdpng.local (pid7896) NSR info cbfs_mount: storage has been mounted and it is available to the system.
0 7/17/2014 4:07:56 PM 0 0 0 9276 7896 0 spng108.dubdpng.local (pid7896) NSR info Eldos CFS successfully mounted virtual drive to c:\Program Files\EMC NetWorker\nsr\tmp\nwfs\NetWorker Virtual File System\ with ELDOS Callback timeout 120000
0 7/17/2014 4:07:56 PM 1 0 0 9276 7896 0 spng108.dubdpng.local (pid7896) NSR notice NWFS mount media is complete
0 7/17/2014 4:08:14 PM 1 0 0 9276 7896 0 spng108.dubdpng.local (pid7896) NSR notice NWFS server disconnect is starting
0 7/17/2014 4:08:14 PM 1 0 0 9276 7896 0 spng108.dubdpng.local (pid7896) NSR notice NWFS server disconnect is complete
0 7/17/2014 4:08:14 PM 1 0 0 9276 7896 0 spng108.dubdpng.local (pid7896) NSR notice NWFS unmount media is starting
0 7/17/2014 4:08:14 PM 0 0 0 8652 7896 0 spng108.dubdpng.local (pid7896) NSR info cbfs_unmount: storage has been un-mounted and it is no longer available to the system.
0 7/17/2014 4:08:14 PM 0 0 0 9276 7896 0 spng108.dubdpng.local (pid7896) NSR info Eldos CFS successfully un-mounted virtual drive from c:\Program Files\EMC NetWorker\nsr\tmp\nwfs\NetWorker Virtual File System
In Windows Event log the event 51 (An error was detected on device \Device\Harddisk3\DR3 during a paging operation.) is reported.
Any idea ?
Thanks
alexb6
8 Posts
1
July 25th, 2014 04:00
Hello!
We had same errors... the reason was in credentials.
After we starts NMM recovery console from right user (has Organization Admin in Exch, RecieveAS, SendAS, all necessary rigths on Networker server) all errors has gone.
gnworker
14 Posts
0
July 28th, 2014 08:00
Hello Alexb,
thank you for suggestion. We will check permissions. In the meanwhile I have received update from on-site: the GLR restore has succeeded once (with no changes in configuration). Hence it does not seem to be a permission problem. We do some more tests and I let you know the status. Until then will put this thread on-hold. Tanks for assistance.
Amr_Khamis
1 Message
0
February 12th, 2016 14:00
what was the solution for the problem ?