Unsolved
This post is more than 5 years old
86 Posts
0
6165
NetWorker "recover" not working: 53362:recover: Cannot start session with server Server: RPC server is unavailable
Starting the NetWorker CLI command on the client leads to the error message:
[root@Client ~]# recover -c Client -s Server
53362:recover: Cannot start session with server Server: RPC server is unavailable
76677:recover: RPC server is unavailable
NW Client release: 8.2.3.8 (RHEL 7.2)
NW Server release: 8.0.3.6 (RHEL 6.2)
Communication between NW client and NW server seams to be OK:
Client -> Server:
[root@Client ~]# nsradmin -s Server -p nsrexec
NetWorker administration program.
Use the "help" command for help, "visual" for full-screen mode.
nsradmin>
rpcinfo -p Server
program vers proto port service
100000 4 tcp 111 portmapper
100000 3 tcp 111 portmapper
100000 2 tcp 111 portmapper
100000 4 udp 111 portmapper
100000 3 udp 111 portmapper
100000 2 udp 111 portmapper
[root@Client ~]# telnet Server 7937
Trying x.x.x.x...
Connected to Server.
Escape character is '^]'.
Server -> Client
root@Server:/root $ nsradmin -s Client -p nsrexec
NetWorker administration program.
Use the "help" command for help, "visual" for full-screen mode.
nsradmin> ^C
root@Server:/root $ rpcinfo -p Client
program vers proto port service
100000 4 tcp 111 portmapper
100000 3 tcp 111 portmapper
100000 2 tcp 111 portmapper
100000 4 udp 111 portmapper
100000 3 udp 111 portmapper
100000 2 udp 111 portmapper
root@Server:/root $ telnet Client 7937
Trying x.x.x.x...
Connected to Client.
Escape character is '^]'.
^]
telnet> Connection closed.
According to the EMS's NetWorker Compatibility Guide NetWorker server 8.0 and Client 8.2 should be compatible.
How to solve the problem?
Thanks and regards
Michael
Karthik_N1
25 Posts
0
February 5th, 2017 23:00
Please check for portmapper 7938 ports as well and next two ports, i.e 7937-7940
Check for peer information conflicts.
Update here and we'll provide suggestions accordingly.
Kleinenbroich
86 Posts
0
February 6th, 2017 03:00
Hi Karthik,
via telnet I'm able to access port7938 and 7940, but not 7939.
Can you please explain me how to check "check for portmapper 7938 ports"?
Thanks a lot for your help :-)
Regards
Michael
ble1
14.3K Posts
1
February 6th, 2017 05:00
On client running 8.2, you should use nsrrpcinfo and not rpcinfo. Make sure also that iptables is not "blocking the view".
Karthik_N1
25 Posts
1
February 6th, 2017 05:00
You can try disabling firewall at both the endpoints, server and client.
Check output of nsrports, it should be 7937-9936, else logon to client and issue the below commands one by one.
nsradmin -p nsrexecd
p type: NSR system port ranges
And paste the output. Also, check peer information conflicts
Kleinenbroich
86 Posts
0
February 6th, 2017 05:00
Hi Hrvoje,
thanks for your help. Following the nsrrpcinfo output (client -> server, client -> client and nsrrpcinfo with -n -t option) . Server is using NW 8.0.3, therefor no nsrrpcinfo test server -> client.
Between client and server a Firewall is in place. The tcp ports 22,111 and 7937-7990 have been requested to be opened bidirectional. Backups are working fin nrsports was used to configure the port range on the client (server is using the same port range).
[root@client ~]#nsrrpcinfo -p server
PROGRAM VERSION PROTOCOL PORT SERVICE
100000 2 tcp 7938 nsrportmapper Port Mapper
100000 2 udp 7938 nsrportmapper Port Mapper
390436 1 tcp 7958 nsrexecd GSS Authentication
390435 1 tcp 7980 nsrexecd Resource Mirror
390113 1 tcp 7937 nsrexecd Remote Execution
390103 2 tcp 7959 nsrd Server
390109 2 tcp 7959 nsrd Remote Monitoring
390110 1 tcp 7959 nsrd Jukebox Management
390120 1 tcp 7959 nsrd License Manager
390109 2 udp 7982 nsrd Remote Monitoring
390105 5 tcp 7962 nsrindexd Index Database
390105 6 tcp 7962 nsrindexd Index Database
390438 1 tcp 7979 nsrlogd Audit Logging
390107 5 tcp 7953 nsrmmdbd Media Database
390107 6 tcp 7953 nsrmmdbd Media Database
390111 1 tcp 7970 nsrsnmd Storage Node Management
390104 605 tcp 7955 nsrmmd Media Multiplexor
390104 205 tcp 7946 nsrmmd Media Multiplexor
390433 1 tcp 7942 nsrjobd Job Management
[root@client ~]# nsrrpcinfo -p client
PROGRAM VERSION PROTOCOL PORT SERVICE
100000 2 tcp 7938 nsrportmapper Port Mapper
100000 2 udp 7938 nsrportmapper Port Mapper
390436 1 tcp 7961 nsrexecd GSS Authentication
390435 1 tcp 7940 nsrexecd Resource Mirror
390113 1 tcp 7937 nsrexecd Remote Executio
[root@client~]# nsrrpcinfo -n 7938 -t server 100000
Program 100000 version 2 ready and waiting
[root@client ~]# nsrrpcinfo -n 7937 -t server 390113
Program 390113 version 1 ready and waiting
Kleinenbroich
86 Posts
0
February 6th, 2017 06:00
Hi Hrvoje,
still one thing to mention: The tcp ports 22,111 and 7937-7990 have been requested, but not the udp ports.
Regards
Michael
Kleinenbroich
86 Posts
0
February 6th, 2017 06:00
Hi Kathrink,
nsrports on cllient and server:
Service ports: 7937-7990
Connection ports: 0-0
The backup, "browse wizzard" and directed recovery are working. I've no messages in the daemon.raw file regarding peer information conflicts. I'm not sure what exactly what kind of "peer information conflicts" I should look for. Can you please explain?
Thanks and regards
Michael
ble1
14.3K Posts
1
February 7th, 2017 03:00
Port wise it looks to be ok. I believe recover has debug switch so you can try something like following:
recover -D9 -s -a -d /tmp /etc/hosts
This will create bunch of data (just tested it), but it would be interesting to see where it breaks. Also, when you run it, check netstat output on both client and server (and also check /etc/hosts itself to make sure that perhaps some faulty entry is not driving connection sideways).
Kleinenbroich
86 Posts
0
February 7th, 2017 07:00
Hi Hrvoje,
following the output of the recover command and at the end some lines from the netstat output .
Thanks for your help.
Regards
Michael
[root@NW_client ~]# recover -D9 -s NW_server -a -d /tmp /etc/hosts
02/07/17 14:34:21.691215 Creating tcp RPC client handle with host NW_client (::1, prog 390113, vers 1)
02/07/17 14:34:21.691564 Creating TCP/IPv6 RPC client handle prog 390113 vers 1
02/07/17 14:34:21.692494 Found NetWorker portmapper port, setting to 7938.
02/07/17 14:34:21.692587 the service nsrrpc was set, ignoring the system portmapper02/07/17 14:34:21.692642 Creating TCP/IPv6 RPC client handle prog 100000 vers 2
02/07/17 14:34:21.692783 Attempting to bind IPv6 socket descriptor 3
02/07/17 14:34:21.692966 Socket bound to OS determined port
02/07/17 14:34:21.935177 Setting default RPC socket send buffer size to 262144
02/07/17 14:34:21.935224 Setting default RPC socket receive buffer size to 262144
02/07/17 14:34:21.935242 Setting RPC socket send buffer size to 262144
02/07/17 14:34:21.935262 Setting RPC socket recv buffer size to 262144
02/07/17 14:34:21.935541 Bound TCP/IPv6 socket descriptor 3 to port 7938
02/07/17 14:34:21.935586 Auth methods not set, using default
02/07/17 14:34:21.935816 Attempting to bind IPv6 socket descriptor 3
02/07/17 14:34:21.935858 Socket bound to OS determined port
02/07/17 14:34:21.935875 Setting RPC socket send buffer size to 262144
02/07/17 14:34:21.935891 Setting RPC socket recv buffer size to 262144
02/07/17 14:34:21.936000 Bound TCP/IPv6 socket descriptor 3 to port 7937
02/07/17 14:34:21.936043 Auth methods not set, using default
02/07/17 14:34:21.937577 User's total groups = 1, max groups set in environment or calculated = 512 and max groups buffer size = 10914
02/07/17 14:34:21.937670 Creating tcp RPC client handle with host localhost (::1, prog 390436, vers 1)
02/07/17 14:34:21.937712 Creating TCP/IPv6 RPC client handle prog 390436 vers 1
02/07/17 14:34:21.937773 Creating TCP/IPv6 RPC client handle prog 100000 vers 2
02/07/17 14:34:21.937850 Attempting to bind IPv6 socket descriptor 4
02/07/17 14:34:21.937902 Socket bound to OS determined port
02/07/17 14:34:21.937931 Setting RPC socket send buffer size to 262144
02/07/17 14:34:21.937964 Setting RPC socket recv buffer size to 262144
02/07/17 14:34:21.938066 Bound TCP/IPv6 socket descriptor 4 to port 7938
02/07/17 14:34:21.938123 Auth methods not set, using default
02/07/17 14:34:21.938331 Attempting to bind IPv6 socket descriptor 4
02/07/17 14:34:21.938381 Socket bound to OS determined port
02/07/17 14:34:21.938404 Setting RPC socket send buffer size to 262144
02/07/17 14:34:21.938420 Setting RPC socket recv buffer size to 262144
02/07/17 14:34:21.938489 Bound TCP/IPv6 socket descriptor 4 to port 7961
02/07/17 14:34:21.938527 Auth methods not set, using default
02/07/17 14:34:21.956871 RPC Authentication: Client successfully obtained RPCSEC_GSS credentials
02/07/17 14:34:21.957219 RPC Authentication: Client successfully obtained RPCSEC_GSS credentials
02/07/17 14:34:21.957518 Adding range 7937-7990 to "service" port type.
02/07/17 14:34:21.958170 Setting DNS cache TTLs to 1800 secs for positive lookups and 1800 secs for negative lookups
02/07/17 14:34:21.958363 clu_init:ENTRY
02/07/17 14:34:21.958404 calling clu_init_lc()
02/07/17 14:34:21.958440
clu_init_lc(): ENTRY...
02/07/17 14:34:21.958492
get_lc_fspath_vhost_map(): ENTRY ...
02/07/17 14:34:21.958654 No access to file /usr/sbin/lcmap: No such file or directory
02/07/17 14:34:21.958686
clu_init_lc(): Can't build fspath_vhost_map...
02/07/17 14:34:21.958703
dump_map_lc(): ENTRY ...
02/07/17 14:34:21.958742 Lc_use_local_vhost_list = FALSE
02/07/17 14:34:21.958758 MOUNTED filesystems
02/07/17 14:34:21.958772
dump_map_lc(): EXIT ...
02/07/17 14:34:21.958828 calling clu_is_cluster_host_lc()
02/07/17 14:34:21.958850
clu_is_cluster_host_lc(): ENTRY ...
02/07/17 14:34:21.958873
clu_init_hp(): ENTRY ...
02/07/17 14:34:21.958910 LGMutex and LGList creation succeeded
02/07/17 14:34:21.958930
get_user_delimiter_char(): ENTRY ...
02/07/17 14:34:21.958955 Searching for user-specified delimiter in /opt/cmcluster/conf/.nsr_cluster
02/07/17 14:34:21.958980
destroy_cmap_hp(): ENTRY ...
02/07/17 14:34:21.959011
destroy_pkglist(): ENTRY ...
02/07/17 14:34:21.959033
destroy_local_vost(): ENTRY ...
02/07/17 14:34:21.959072
destroy_local_vost(): EXIT ...
02/07/17 14:34:21.959090
destroy_cmap_hp(): EXIT ...
02/07/17 14:34:21.959138
get_cmap_hp(): ENTRY ...
02/07/17 14:34:21.959161
destroy_cmap_hp(): ENTRY ...
02/07/17 14:34:21.959206
destroy_pkglist(): ENTRY ...
02/07/17 14:34:21.959225
destroy_local_vost(): ENTRY ...
02/07/17 14:34:21.959249
destroy_local_vost(): EXIT ...
02/07/17 14:34:21.959266
destroy_cmap_hp(): EXIT ...
02/07/17 14:34:21.959306
dump_cmap_hp(): ENTRY ...
02/07/17 14:34:21.959327
dump_cmap_hp(): EXIT ...
02/07/17 14:34:21.959352 Entering build_local_vhost_tab function
02/07/17 14:34:21.959375 Exitting build_local_vhost_tab function
02/07/17 14:34:21.959396
dump_pkglist(): ENTRY ...
02/07/17 14:34:21.959419
dump_pkglist(): EXIT ...
02/07/17 14:34:21.959438 clu_init_hp(): returning void
02/07/17 14:34:21.959461 clu_is_native_cluster_host:ENTRY
02/07/17 14:34:21.959475
clu_is_cluster_host_lc(): ENTRY ...
02/07/17 14:34:21.959499
clu_is_cluster_host_hp(): ENTRY ...
02/07/17 14:34:21.959522 this_phostname=NW_client
02/07/17 14:34:21.959548 NetWorker is not configured for cluster system, cmviewcl will be bypassed ...
02/07/17 14:34:21.959580 clu_is_native_cluster_host:EXIT:returning value=0
02/07/17 14:34:21.959608 clu_init:EXIT
02/07/17 14:34:21.959649 Reverse DNS lookup failed for address fe80::250:56ff:fe01:a5e
02/07/17 14:34:21.959684 Reverse DNS lookup failed for address fe80::250:56ff:fe01:a5f
02/07/17 14:34:21.959700 Reverse DNS lookup failed for address fe80::250:56ff:fe01:a60
02/07/17 14:34:21.959748 Creating tcp RPC client handle with host NW_server (10.105.x.x, prog 390433, vers 1)
02/07/17 14:34:21.959773 Creating TCP/IPv4 RPC client handle prog 390433 vers 1
02/07/17 14:34:21.959790 Creating TCP/IPv4 RPC client handle prog 100000 vers 2
02/07/17 14:34:21.959854 Attempting to bind IPv4 socket descriptor 3
02/07/17 14:34:21.959908 Socket bound to OS determined port
02/07/17 14:34:21.959932 Setting RPC socket send buffer size to 262144
02/07/17 14:34:21.959947 Setting RPC socket recv buffer size to 262144
02/07/17 14:34:21.960694 Bound TCP/IPv4 socket descriptor 3 to port 7938
02/07/17 14:34:21.961408 Attempting to bind IPv4 socket descriptor 3
02/07/17 14:34:21.961490 Socket bound to OS determined port
02/07/17 14:34:21.961521 Setting RPC socket send buffer size to 262144
02/07/17 14:34:21.961558 Setting RPC socket recv buffer size to 262144
02/07/17 14:34:21.962036 Bound TCP/IPv4 socket descriptor 3 to port 7942
02/07/17 14:34:21.962167 User's total groups = 1, max groups set in environment or calculated = 512 and max groups buffer size = 10914
02/07/17 14:34:21.962241 Creating tcp RPC client handle with host localhost (::1, prog 390436, vers 1)
02/07/17 14:34:21.962285 Creating TCP/IPv6 RPC client handle prog 390436 vers 1
02/07/17 14:34:21.962360 Attempting to bind IPv6 socket descriptor 4
02/07/17 14:34:21.962410 Socket bound to OS determined port
02/07/17 14:34:21.962437 Setting RPC socket send buffer size to 262144
02/07/17 14:34:21.962467 Setting RPC socket recv buffer size to 262144
02/07/17 14:34:21.962678 Bound TCP/IPv6 socket descriptor 4 to port 7961
02/07/17 14:34:21.979838 RPC Authentication: Client successfully obtained RPCSEC_GSS credentials
02/07/17 14:34:22.003916 RPC Authentication: Client successfully obtained RPCSEC_GSS credentials
02/07/17 14:34:22.004894 Registered session id:1, fd:3, idx:0, ssn_max_pollfd:1, 1(512)
02/07/17 14:34:22.030317 Creating tcp RPC client handle with host NW_server (10.105.x.x, prog 390109, vers 2)
02/07/17 14:34:22.030390 Creating TCP/IPv4 RPC client handle prog 390109 vers 2
02/07/17 14:34:22.030424 Creating TCP/IPv4 RPC client handle prog 100000 vers 2
02/07/17 14:34:22.030497 Attempting to bind IPv4 socket descriptor 4
02/07/17 14:34:22.030540 Socket bound to OS determined port
02/07/17 14:34:22.030584 Setting RPC socket send buffer size to 262144
02/07/17 14:34:22.030617 Setting RPC socket recv buffer size to 262144
02/07/17 14:34:22.031162 Bound TCP/IPv4 socket descriptor 4 to port 7938
02/07/17 14:34:22.031844 Attempting to bind IPv4 socket descriptor 4
02/07/17 14:34:22.031940 Socket bound to OS determined port
02/07/17 14:34:22.031973 Setting RPC socket send buffer size to 262144
02/07/17 14:34:22.032007 Setting RPC socket recv buffer size to 262144
02/07/17 14:34:22.032602 Bound TCP/IPv4 socket descriptor 4 to port 7959
02/07/17 14:34:22.032853 User's total groups = 1, max groups set in environment or calculated = 512 and max groups buffer size = 10914
02/07/17 14:34:22.033016 Creating tcp RPC client handle with host NW_server (10.105.x.x, prog 390109, vers 2)
02/07/17 14:34:22.033210 Creating TCP/IPv4 RPC client handle prog 390109 vers 2
02/07/17 14:34:22.033377 Creating TCP/IPv4 RPC client handle prog 100000 vers 2
02/07/17 14:34:22.033578 Attempting to bind IPv4 socket descriptor 5
02/07/17 14:34:22.033793 Socket bound to OS determined port
02/07/17 14:34:22.033955 Setting RPC socket send buffer size to 262144
02/07/17 14:34:22.034129 Setting RPC socket recv buffer size to 262144
02/07/17 14:34:22.034811 Bound TCP/IPv4 socket descriptor 5 to port 7938
02/07/17 14:34:22.035538 Attempting to bind IPv4 socket descriptor 5
02/07/17 14:34:22.035620 Socket bound to OS determined port
02/07/17 14:34:22.035660 Setting RPC socket send buffer size to 262144
02/07/17 14:34:22.035705 Setting RPC socket recv buffer size to 262144
02/07/17 14:34:22.036207 Bound TCP/IPv4 socket descriptor 5 to port 7959
02/07/17 14:34:22.036300 User's total groups = 1, max groups set in environment or calculated = 512 and max groups buffer size = 10914
02/07/17 14:34:22.036367 Creating tcp RPC client handle with host localhost (::1, prog 390436, vers 1)
02/07/17 14:34:22.036407 Creating TCP/IPv6 RPC client handle prog 390436 vers 1
02/07/17 14:34:22.036473 Attempting to bind IPv6 socket descriptor 6
02/07/17 14:34:22.036525 Socket bound to OS determined port
02/07/17 14:34:22.036553 Setting RPC socket send buffer size to 262144
02/07/17 14:34:22.036581 Setting RPC socket recv buffer size to 262144
02/07/17 14:34:22.036793 Bound TCP/IPv6 socket descriptor 6 to port 7961
02/07/17 14:34:22.050673 RPC Authentication: Client successfully obtained RPCSEC_GSS credentials
02/07/17 14:34:22.053375 RPC Authentication: Client successfully obtained RPCSEC_GSS credentials
02/07/17 14:34:22.054438 lgto_auth for `nsrd' succeeded
02/07/17 14:34:22.054534 User's total groups = 1, max groups set in environment or calculated = 512 and max groups buffer size = 10914
02/07/17 14:34:22.054597 Creating tcp RPC client handle with host localhost (::1, prog 390436, vers 1)
02/07/17 14:34:22.054633 Creating TCP/IPv6 RPC client handle prog 390436 vers 1
02/07/17 14:34:22.054707 Attempting to bind IPv6 socket descriptor 5
02/07/17 14:34:22.054779 Socket bound to OS determined port
02/07/17 14:34:22.054812 Setting RPC socket send buffer size to 262144
02/07/17 14:34:22.054840 Setting RPC socket recv buffer size to 262144
02/07/17 14:34:22.055037 Bound TCP/IPv6 socket descriptor 5 to port 7961
02/07/17 14:34:22.071776 RPC Authentication: Client successfully obtained RPCSEC_GSS credentials
02/07/17 14:34:22.076016 RPC Authentication: Client successfully obtained RPCSEC_GSS credentials
02/07/17 14:34:22.080877 Recovering files into /tmp
02/07/17 14:34:22.081030 read filename of /etc/hosts
02/07/17 14:34:22.081151
clu_is_cluster_host_lc(): ENTRY ...
02/07/17 14:34:22.081377 clu_pathl_gethost:ENTRY
02/07/17 14:34:22.081598 input path=/etc/hosts
02/07/17 14:34:22.081760 clu_pathl_gethost:EXIT:Clu_cluster_type is undefined
02/07/17 14:34:22.082598 clu_pathl_free:ENTRY
02/07/17 14:34:22.082796 clu_pathl_free:EXIT
02/07/17 14:34:22.082946 Creating tcp RPC client handle with host NW_server (10.105.x.x, prog 390103, vers 2)
02/07/17 14:34:22.083134 Creating TCP/IPv4 RPC client handle prog 390103 vers 2
02/07/17 14:34:22.083295 Creating TCP/IPv4 RPC client handle prog 100000 vers 2
02/07/17 14:34:22.083501 Attempting to bind IPv4 socket descriptor 4
02/07/17 14:34:22.083675 Socket bound to OS determined port
02/07/17 14:34:22.083845 Setting RPC socket send buffer size to 262144
02/07/17 14:34:22.084005 Setting RPC socket recv buffer size to 262144
02/07/17 14:34:22.084653 Bound TCP/IPv4 socket descriptor 4 to port 7938
02/07/17 14:34:22.085327 Attempting to bind IPv4 socket descriptor 4
02/07/17 14:34:22.085475 Socket bound to OS determined port
02/07/17 14:34:22.085631 Setting RPC socket send buffer size to 262144
02/07/17 14:34:22.085798 Setting RPC socket recv buffer size to 262144
02/07/17 14:34:22.086356 Bound TCP/IPv4 socket descriptor 4 to port 7959
02/07/17 14:34:22.086529 User's total groups = 1, max groups set in environment or calculated = 512 and max groups buffer size = 10914
02/07/17 14:34:22.086694 Creating tcp RPC client handle with host localhost (::1, prog 390436, vers 1)
02/07/17 14:34:22.086866 Creating TCP/IPv6 RPC client handle prog 390436 vers 1
02/07/17 14:34:22.087081 Attempting to bind IPv6 socket descriptor 5
02/07/17 14:34:22.087237 Socket bound to OS determined port
02/07/17 14:34:22.087395 Setting RPC socket send buffer size to 262144
02/07/17 14:34:22.087556 Setting RPC socket recv buffer size to 262144
02/07/17 14:34:22.087868 Bound TCP/IPv6 socket descriptor 5 to port 7961
02/07/17 14:34:22.102232 RPC Authentication: Client successfully obtained RPCSEC_GSS credentials
02/07/17 14:34:22.104576 RPC Authentication: Client successfully obtained RPCSEC_GSS credentials
02/07/17 14:34:22.105554 User's total groups = 1, max groups set in environment or calculated = 512 and max groups buffer size = 10914
02/07/17 14:34:22.106740 lgto_auth for `nsrindexd' succeeded
02/07/17 14:34:22.106918 User's total groups = 1, max groups set in environment or calculated = 512 and max groups buffer size = 10914
02/07/17 14:34:22.107061 Creating tcp RPC client handle with host localhost (::1, prog 390436, vers 1)
02/07/17 14:34:22.107235 Creating TCP/IPv6 RPC client handle prog 390436 vers 1
02/07/17 14:34:22.107426 Attempting to bind IPv6 socket descriptor 5
02/07/17 14:34:22.107584 Socket bound to OS determined port
02/07/17 14:34:22.107742 Setting RPC socket send buffer size to 262144
02/07/17 14:34:22.107927 Setting RPC socket recv buffer size to 262144
02/07/17 14:34:22.108259 Bound TCP/IPv6 socket descriptor 5 to port 7961
02/07/17 14:34:22.122861 RPC Authentication: Client successfully obtained RPCSEC_GSS credentials
02/07/17 14:34:22.123045 Creating tcp RPC client handle with host NW_server (10.105.x.x, prog 390105, vers 6)
02/07/17 14:34:22.123120 Creating TCP/IPv4 RPC client handle prog 390105 vers 6
02/07/17 14:34:22.123156 Creating TCP/IPv4 RPC client handle prog 100000 vers 2
02/07/17 14:34:22.123232 Attempting to bind IPv4 socket descriptor 5
02/07/17 14:34:22.123279 Socket bound to OS determined port
02/07/17 14:34:22.123308 Setting RPC socket send buffer size to 262144
02/07/17 14:34:22.123332 Setting RPC socket recv buffer size to 262144
02/07/17 14:34:22.123877 Bound TCP/IPv4 socket descriptor 5 to port 7938
02/07/17 14:34:22.124521 Attempting to bind IPv4 socket descriptor 5
02/07/17 14:34:22.124593 Socket bound to OS determined port
02/07/17 14:34:22.124627 Setting RPC socket send buffer size to 262144
02/07/17 14:34:22.124652 Setting RPC socket recv buffer size to 262144
02/07/17 14:34:22.125128 Bound TCP/IPv4 socket descriptor 5 to port 7962
02/07/17 14:34:22.141196 RPC Authentication: Client successfully obtained RPCSEC_GSS credentials
02/07/17 14:34:22.363141 calling clntnsr_start_pools_2_2
53362:recover: Cannot start session with server NW_server: RPC server is unavailable
02/07/17 14:34:22.364943 Updating the Total Recover size: 0 and Total Files: 0 to jobdb
02/07/17 14:34:22.366123 Unregistered session id:1, fd:3, idx:0, ssn_max_pollfd:0, 0(512)
02/07/17 14:34:22.366273 indexbr_entry_done()
02/07/17 14:34:22.366426 exit indexbr_entry_done()
root@NW_server:/root $ netstat
Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 NW_server:7942 NW_client:57048 ESTABLISHED
tcp 0 0 NW_server:7942 NW_client:40047 TIME_WAIT
[admin@NW_client ~]$ netstat
Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 NW_client-br:43656 NW_server:nsrrpc TIME_WAIT
tcp 0 0 NW_client-br:ssh deratvbr02:45905 ESTABLISHED
tcp 0 0 NW_client-br:35796 NW_server:nsrrpc TIME_WAIT
tcp 0 0 NW_client-br:37281 NW_server:nsrrpc TIME_WAIT
tcp 0 0 NW_client-br:35046 NW_server:7959 TIME_WAIT
tcp 0 0 NW_client-br:37316 NW_server:7942 ESTABLISHED
tcp 0 0 NW_client:53461 ldap14:ldap ESTABLISHED
tcp 0 0 NW_client-br:38978 NW_server:nsrrpc TIME_WAIT
tcp 0 0 NW_client-br:43004 NW_server:7959 TIME_WAIT
tcp 0 0 NW_client-br:54392 NW_server:7959 ESTABLISHED
tcp 0 0 NW_client-br:51686 NW_server:nsrrpc TIME_WAIT
tcp 0 0 NW_client-br:44438 NW_server:nsrrpc TIME_WAIT
tcp 0 0 NW_client-br:40796 NW_server:nsrrpc TIME_WAIT
tcp 0 0 NW_client-br:40062 NW_server:nsrrpc TIME_WAIT
tcp 0 0 NW_client-br:55054 NW_server:nsrrpc TIME_WAIT
tcp 0 0 NW_client-br:ssh NW_server:34360 ESTABLISHED
tcp6 0 0 localhost:41265 localhost:7961 TIME_WAIT
tcp6 0 0 localhost:46364 localhost:7961 TIME_WAIT
tcp6 0 0 localhost:40040 localhost:7961 TIME_WAIT
tcp6 0 0 localhost:37759 localhost:7961 TIME_WAIT
tcp6 0 0 localhost:41995 localhost:7961 TIME_WAIT
tcp6 0 0 localhost:59891 localhost:7937 TIME_WAIT
tcp6 0 0 localhost:51083 localhost:7961 TIME_WAIT
Kleinenbroich
86 Posts
0
February 7th, 2017 13:00
Hi Hrovje,
the hostname is NW_client. The server has two interfaces (two IPs). In the /etc/hosts file I've assigned two different IPs to two different hostnames: One IP for NW_client and one IP for NW_client-br (the dedicated BnR interface).
On the NetWorker server (and the NW storage node and NW console server) I've added the BnR IP address of the client with NW_client-bnr hostname and all host names used on the client.
(e. g. 10.192.17.4 NW_client-br NW_client NW_client.domain.com).
We've also several interfaces on our NW server, storage nodes and console server. Here we use the same approach: On the client we add IP-backup-interface, hostname, hostname-VLANid, hostname.VLANid.domain.com to the clients /etc/hosts file.
This is the approach we've been using in the past years without problems. From my point of view, I did this time the same tasks as I did in the past.
At the moment I would guess that NW server 8.0.3. and NW client 8.2.3. are not that compatible as they should be or maybe there is a "software glitch" on the NetWorker server. But recover works using other (older) NetWorker clients.
Thanks and regards
Michael
ble1
14.3K Posts
0
February 7th, 2017 13:00
I see NW_client and NW_client-br mentioned. Do you also have NW_server-br? (I would assume so if -br is separate segment on which traffic should go).
ble1
14.3K Posts
0
February 8th, 2017 04:00
I'm using the same (both SP2 and SP3) and also multihomed in pretty much similar way as you have it and it works just fine. If I would replicate your setup to mine, it means I use clients defined in NW as NW_client-br and in each client server backup interface I place NW_server-br and for storage node NW_sn-br. That way I make sure -br VLAN is only used. I suspect some issue in translation happens somewhere due to some config issue most likely.
Kleinenbroich
86 Posts
0
February 8th, 2017 04:00
Hi Hrvoje,
what kind of configuration issue could this be (beside of IP wrong hostnames / IP assignment)?
Thanks and regards
Michael
ble1
14.3K Posts
0
February 8th, 2017 05:00
I would split it to NW config issue and network config issue. Both are broad in some terms.
I did my test again and compared debug outputs... after the point where you get failure I get:
[..]
02/08/17 14:39:27.408682 RPC Authentication: Client successfully obtained RPCSEC_GSS credentials
02/08/17 14:39:27.770639 calling clntnsr_start_pools_2_2
02/08/17 14:39:28.044155 createentries(/etc/hosts)
02/08/17 14:39:28.044334 fullcanon(name=/etc/hosts)
02/08/17 14:39:28.044396 exit fullcanon(fullname=/etc/hosts)
02/08/17 14:39:28.044497 initializing ROOT entry
02/08/17 14:39:28.044529 makeentry()
02/08/17 14:39:28.044563 lookupentry()
02/08/17 14:39:28.044597 exit lookupentry: name=/, savetime=0, ignoreflags=0 -> 0x0000000000000000
02/08/17 14:39:28.044681 lookupentry()
02/08/17 14:39:28.044769 exit lookupentry: name=/, savetime=0, ignoreflags=0 -> 0x0000000000000000
02/08/17 14:39:28.044800 lookupentry()
02/08/17 14:39:28.044829 exit lookupentry: name=/, savetime=0, ignoreflags=0 -> 0x0000000000000000
02/08/17 14:39:28.044871 direntname(): name=/
02/08/17 14:39:28.044900 exit direntname(): normal [/]
02/08/17 14:39:28.044999 makeentry: / not found e_name /
02/08/17 14:39:28.045032 exit makeentry()
02/08/17 14:39:28.045067 lookup `/', fsid = 0xffffffffffffffff, fid = 18446744073709551615
02/08/17 14:39:28.047466 expand_check(/)
02/08/17 14:39:28.047640 User's total groups = 7, max groups set in environment or calculated = 512 and max groups buffer size = 10914
02/08/17 14:39:28.049153 lgto_auth falling back: Operation not supported
[...]
That alone means nothing... so I suspect that block before where you have:
02/07/17 14:34:22.123045 Creating tcp RPC client handle with host NW_server (10.105.x.x, prog 390105, vers 6)
...things get hairy. I assume when you called recover command you did by using -s NW_server-bk interface?
Kleinenbroich
86 Posts
0
February 8th, 2017 08:00
Hi Hrvoje,
yes, we use only the NW_server-bk interface and this also the only interface having IP connectivity between client and server (and storage node and console).
Regards
Michael