UNSOLVED
Networker backup error without message
I use Windows 2012 R2 server with the networker client. The server is VMware VM with two disks: C: and E:.. I can't make any full backup on DD160. Backups started and after some time stopped. At the client configuration the saveset was set to "ALL". So I made another tests:
1. Set saveset to :
C:\
DISASTER_RECOVERY:\
and full backup was succesfull. Below is the part of the log:
NetWorker savegroup: (notice) helios2 completed, Total 1 client(s), 1 Succeeded. See group completion details for more information.
Succeeded: helios2
Start time: Mon Jun 06 09:36:04 2016
End time: Mon Jun 06 11:43:25 2016
--- Successful Save Sets ---
* helios2:savefs savefs helios2: succeeded.
* helios2:DISASTER_RECOVERY:\ libDDBoost version: major: 2, minor: 6, patch: 5, engineering: 0, build: 449492
* helios2:DISASTER_RECOVERY:\ libDDBoost version: major: 2, minor: 6, patch: 5, engineering: 0, build: 449492
helios2: \\?\VOLUME{1131222B-E7E9-11E3-80B4-806E6F6E6963}\ level=full, 286 MB 00:00:54 136 files
* helios2:DISASTER_RECOVERY:\ Completed savetime=1465198655
* helios2:DISASTER_RECOVERY:\ 90015:save: The backup of VSS emit save set '\\?\VOLUME{1131222B-E7E9-11E3-80B4-806E6F6E6963}\' succeeded.
helios2: C:\ level=full, 56 GB 01:57:01 249316 files
* helios2:DISASTER_RECOVERY:\ Completed savetime=1465198656
* helios2:DISASTER_RECOVERY:\ 90015:save: The backup of VSS emit save set 'C:\' succeeded.
* helios2:DISASTER_RECOVERY:\ libDDBoost version: major: 2, minor: 6, patch: 5, engineering: 0, build: 449492
helios2: WINDOWS ROLES AND FEATURES:\ level=full, 10 MB 00:06:03 31 files
* helios2:DISASTER_RECOVERY:\ Completed savetime=1465205689
* helios2:DISASTER_RECOVERY:\ 90011:save: The backup of the disaster recovery save sets succeeded.
* helios2:DISASTER_RECOVERY:\ OSSRInfo XML Version:3.0
* helios2:DISASTER_RECOVERY:\ 80889:save: Created disaster recover logical object.
helios2: DISASTER_RECOVERY:\ level=full, 81 KB 02:04:08 5 files
* helios2:DISASTER_RECOVERY:\ completed savetime=1465206054
* helios2:DISASTER_RECOVERY:\ Unable to find any full backups of the save set 'helios2:DISASTER_RECOVERY:\' in the media database. Performing a full backup.
nws: index:helios2 level=9, 75 MB 00:02:14 8 files
* helios2:index completed savetime=1465206071
2. Set saveset to: E:\
Backup failed. Below is the part of the log ( I can't find any more detailed information):
NetWorker savegroup: (alert) helios2 completed, Total 1 client(s), 1 Failed. See group completion details for more information.
Failed: helios2
Start time: Mon Jun 06 12:25:58 2016
End time: Mon Jun 06 13:53:25 2016
--- Unsuccessful Save Sets ---
* helios2:E:\ libDDBoost version: major: 2, minor: 6, patch: 5, engineering: 0, build: 449492
* helios2:E:\ Unable to find any full backups of the save set 'helios2:E:\' in the media database. Performing a full backup.
helios2:E:\: retried 1 times.
--- Successful Save Sets ---
* helios2:savefs savefs helios2: succeeded.
nws: index:helios2 level=9, 38 MB 00:02:11 6 files
* helios2:index completed savetime=1465213874
All other client options were the same. I checked disk E:\ using Microsoft tools and everything was OK.
I have other two Windows 2012 R2 and one Windows 2012 servers, also VMs, and there is no problem to backup them.
I have no idea what is wrong.
Networker version is 8.1.3.7 build 563
Marek
MoinMansuri
2 Intern
•
147 Posts
2378
0
Posted June 7th, 2016 04:00
Go to client configuration >>> Apps & Modules >>> Backup Command
save -c helios2
and run the backup again.
I put "save -c helios2" into client configuration and also change
"Client retries" in "group properties" to 0. Backup failed after 2h 45m
with the following message:
bDDBoost version: major: 2, minor: 6, patch: 5, engineering: 0, build:
449492
76677:save: RPC send operation failed; peer = 192.168.129.30:8607, errno
= An existing connection was suddenly closed by the remote host.
74209:save: Quit signal received.
99123:save: Handling an abort while processing Windows backup.
Unable to find any full backups of the save set 'helios2:E:\' in the
media database. Performing a full backup.
The errno was translated from polish: Istniejące połączenie zostało
gwałtownie zamknięte przez zdalnego hosta.
---
Marek Cierniak
W dniu 2016-06-07 13:46, moin napisał(a):
ECN
NETWORKER BACKUP ERROR WITHOUT MESSAGE
reply from Moin Mansuri in NetWorker - View the full discussion
BTW - there is no reason to change the backup configuration for test purposes.
The risk is high that such might be forgotten later.
To test the backup of specific clients, use the command line (it also allows you to increase verbosity):
savegrp [-v] [-l level] -c client1 [-c client2 ... -c client#] -G groupname
With respect to the issue:
- It looks as if the connection will be dropped so that NW cannot even gather status info any more.
Maybe there is another reason why this would happend.
- I would check whether the backup always fails at a certain position/file. To do that ...
- go to the client and run the following command
save [-s server] [-b pool] E:\ > C:\temp\backup.log 2>&1
- verify the output
StuartWhitby
45 Posts
1685
0
Posted June 9th, 2016 06:00
Anything that's got "existing connection was forcibly closed by the remote host" and you want to check out your tcp_keepalive settings. Set this down to under 1 hour on your backup server from the default of 2 hours and "refresh" your networking to pick up the change.
Historically there was a recommendation not to bring this timeout below 15 minutes. However, I think that with the significant advances in network and computing performance since that recommendation was made, I'd personally put this at 10 minutes (or a rounder number of 900 secs/900000ms). This is to come in below the half-close timeout that I think is 11 minutes 45 seconds or some similarly strange number.
Setting this on the NetWorker server should cover your environment for these kind of backup issues, though potentially you'd want it on your storage nodes as well.
I have dane four backups using suggested command:
save E: > C:tempbackup.log 2>&1
and found that backups stopped at randomly files.
I also stopped TCP chimney, TCPIP offload engine and TCP segmentation
offload at client side but this had no effect.
I checked registry tcpip KeepAliveTime key and it is set to 900000.
My client and Networker server belog to two different subnets and all
transmission is passed via our software router ( Vyos system).
When I moved client to the Networker's server subnet backup was
succesfull.
It is very strange because all other servers which I can backup without
any problem also belongs to different subnet.
---
Marek Cierniak
W dniu 2016-06-09 07:42, bingo napisał(a):
ECN
NETWORKER BACKUP ERROR WITHOUT MESSAGE
reply from bingo in NetWorker - View the full discussion
ble1
6 Operator
•
14354 Posts
•
56186 Points
2374
0
Posted June 7th, 2016 04:00
Because log is cut/parsed, you do not see the error (nor you see where/when it breaks as you use retries). Try to save E drive using save command from client. If that works, it might be idle control session which gets killed (idle timeout) which can be addressed easily.