Skip to main content
Sign Out

Welcome to Dell

My Account
  • Place orders quickly and easily
  • View orders and track your shipping status
  • Enjoy members-only rewards and discounts
  • Create and access a list of your products
  • Manage your Dell EMC sites, products, and product-level contacts using Company Administration.
Sign In Create an Account Dell Financial Services Premier Sign In Partner Program Sign In
Contact Us

Your Dell.com Carts

Article Number: 000019968

NetWorker: How to Recover the NMC database?

Summary: You can use the NetWorker recoverpsm command to restore the NMC database to a backup of the nmcdb.

This article may have been automatically translated. If you have any feedback regarding its quality, please let us know using the form at the bottom of this page.

Article Content

Instructions

If you have identified NMC database corruption. You can use the NetWorker recoverpsm command to restore the NMC database to a backup of the nmcdb. By default there should be a NMC server backup workflow inside the default Server Protection policy. Ensure that this is scheduled and configured correctly to complete backups of the NMC database. For more information on NMC server protection see the NetWorker Administration Guide as well as the NetWorker Command Reference Guide for both savepsm and recoverpsm options. These documents can be found on https://www.dell.com/support/home/product-support/product/networker/docs

1) First ensure that there is a backup of the NMC database using the mminfo command:
Syntax: mminfo -avot -q client=nmc_server_name 
Example: 
[root@linux1 ~]# mminfo -avot -q client=linux1.amer.lan -t "today" | grep nmcdb_stage
ClientBackupPool.001 Data Domain linux1.amer.lan 08/15/2022 02:00:17 PM 53 MB 687518690 cb full /nsr/nmc/nmcdb_stage


Note: If you do not have the above saveset you cannot recover the NMC and the only option is to reinstall it as a fresh instance.
 

2) On the NMC server open an Administrative/Root command prompt stop the NMC GSTD service (if running):

Windows: net stop gstd
Linux: systemctl stop gst

Note: use tasklist/Task Manager or ps commands to ensure that the gstd service has stopped
 

3) Use the cd command to navigate to the NMC's installation bin directory. Check to ensure that the recoverpsm command is listed in this directory. This command will be used to recover the NMC database.

Windows: [NMC Installation Dir\Management\GST\bin
Linux: /opt/lgtonmc/bin

4) Run the recoverpsm command on the NMC server:

Syntax:
recoverpsm [-s server] [-c client-name] [-d destination] [-p pass-phrase] [-t time] [-hfO] Staging_Directory

Options:

-c client-name

Specifies the name of the source NMC server. See recover for more details.

 d destination

Specifies the directory to which recoverpsm recovers the NMC database. When you use this option, you must manually copy the database files from the destination directory to the database directory defined for the NMC server. Ensure that you retain the same ownership and permissions on the database files and the credential files after the copy completes.

  f

Use this option to delete the current database files that exist in the database directory before starting the recovery operation.

 h

Display the usage information about the recoverpsm command.

 O

Use this option to omit the recovery of the database credential file. Only use this option in the rare case where the credential file is still present and you do not want to use the credential file from the recover.

 p pass-phrase

Use this option to specify an additional pass phrase when you attempt to recover AES encrypted database files. See recover(1m) for more details.

 s server

Specifies the name of the NetWorker server that performed the NMC database backup.

 t time

Use this option to recover the database as of a date specified in nsrgetdate format.

Staging Directory

Specifies the staging directory used by the backup operation. For an automatic backup, specify the save set name

 Example: 

[root@linux1 bin]# ./recoverpsm -f -s linux1.amer.lan -c linux1.amer.lan -p Pa$$w0rd01 /nsr/nmc/nmcdb_stage
108157:recover: Restoring database files...
179040:recover: Step (1 of 11) for PID-6264: Recovery has been started on the client 'linux1.amer.lan'.
179733:recover: Step (2 of 11) for PID-6264: Checking if the set of exclusion files is given for the recovery of the selected files for the client 'linux1.amer.lan'.
179738:recover: Step (3 of 11) for PID-6264: Setting the server 'linux1.amer.lan' for the media database calls for the recovery of the selected files for the client 'linux1.amer.lan'.
179745:recover: Identified a file level recovery with PID 6264 to relocate to the new path '/nsr/nmc/nmcdb' for the client 'linux1.amer.lan'. Updating the total number of steps from 11 to 8.
179748:recover: Step (4 of 8) for PID-6264: Initializing a recovery job with the nsrjobd for the recovery of the selected file(s) for the client 'linux1.amer.lan'.
179749:recover: Step (5 of 8) for PID-6264: Validating and setting the relocation path '/nsr/nmc/nmcdb' for the recovery of selected file(s) for the client 'linux1.amer.lan'.
179754:recover: Step (6 of 8) for PID-6264: Initializing the file level recovery of the client 'linux1.amer.lan'.
179730:recover: Step (7 of 8) for PID-6264: Building the recovery tree of the client 'linux1.amer.lan'.
Recovering 1500 files within /nsr/nmc/nmcdb_stage/ into /nsr/nmc/nmcdb
Requesting 1500 file(s), this may take a while...
179728:recover: Initializing the recovery session with the NetWorker server 'linux1.amer.lan' to recover the selected files.
Recover start time: Mon 15 Aug 2022 02:20:37 PM PDT
179718:recover: Initializing to recover multiple streams of data from the NetWorker server 'linux1.amer.lan'.
Requesting 1 recover session(s) from server.
179714:recover: Initializing the child threads to recover the given recovery stream from the NetWorker server 'linux1.amer.lan'.
179713:recover: Initializing to set up a connection to start reading the data from the recover stream.
Enabling compressed restore for save set ID '687518690' with 'Data Domain' device path '/linux1/ClientBackupDevice01'.
Successfully established the direct file retrieval session for save set ID '687518690' with 'Data Domain' volume 'ClientBackupPool.001'.
./pgdata/backup_label
...
./pgdata/
179716:recover: Coordinating all the child recovery processes.
179717:recover: Releasing all the resources held by the recovery process.
179721:recover: All recovery streams from the NetWorker server 'linux1.amer.lan' are read successfully.
Received 1500 file(s) from NSR server `linux1.amer.lan'
Recover completion time: Mon 15 Aug 2022 02:20:39 PM PDT
179068:recover: Step (8 of 8) for PID-6264: The recovery completed successfully on the client 'linux1.amer.lan'.
108161:recover: Setting permissions for the restored database files

 

5) On the NMC server open an Administrative/Root command prompt stop the NMC GSTD service:
Windows: net start gstd
Linux: systemctl start gst

Note: use tasklist/Task Manager or ps commands to ensure that the gstd service has started.
 

6) Ensure that you can connect to the NMC.


If your NMC recovery fails using recoverpsm please contact Dell EMC support for further assistance. Note that if a recovery is not possible a fresh installation of the NMC will need to be performed.

Additional Information

The NMC server's ..\Management\GSTD\logs\gstd.raw or /opt/lgtonmc/logs/gstd.raw can be used to confirm if there is NMCDB corruption:

Example:

31581 1501512324 2 0 0 5632 6204 0 NMC-server gstd NSR warning 23 *** gstd starting at %s 1 36 24 Mon Jul 31 10:45:24 2017
NMC-server gstd NSR notice 5 %s %s 2 0 24 MM/DD/YYY HH:MM:SS.3852 0 50 Internal error: could not create database process.
NMC-server gstd NSR notice 5 %s %s 2 0 24 MM/DD/YYY HH:MM:SS.643852 0 50 Internal error: could not create database process.
NMC-server gstd NSR notice 5 %s %s 2 0 24 MM/DD/YYY HH:MM:SS.643852 0 50 Internal error: could not release database handle.
NMC-server gstd NSR notice 5 %s %s 2 0 24 MM/DD/YYY HH:MM:SS.675017 0 48 Internal error: could not stop database process.
NMC-server gstd NSR notice 5 %s %s 2 0 24 MM/DD/YYY HH:MM:SS.675017 0 54 FATAL ERROR: could not stop 'EMC GST Database Service'
NMC-server gstd NSR notice 5 %s %s 2 0 24 MM/DD/YYY HH:MM:SS.675017 0 48 Internal error: could not stop database process.
NMC-server gstd NSR notice 5 %s %s 2 0 24 MM/DD/YYY HH:MM:SS.675017 0 48 Internal error: could not stop database process.
NMC-server gstd NSR notice 5 %s %s 2 0 24 MM/DD/YYY HH:MM:SS.675017 0 54 The most recently called subfunction returned error 2.
NMC-server gstd NSR warning 2 %s 1 0 27 library error 22 os error 0
NMC-server gstd NSR warning 23 *** gstd stopping at %s 1 36 24 MM/DD/YYY HH:MM:SS

 

Article Properties

Affected Product

NetWorker

Product

NetWorker

Last Published Date

15 Aug 2022

Version

3

Article Type

How To

Back to Top