Microsoft Windows: Error 1988 Appears in the Directory Service event log of an Active Directory domain controller
Summary: Error 1988 may appear in the Directory Service event log of an Active Directory (AD) domain controller (DC). This error indicates that AD replication failed because at least one lingering object is detected on a partner domain controller. This article discusses the causes of lingering objects and tells how to remove them. ...
Symptoms
Further text in the event description indicates the source DC, the lingering object that is detected, and a recommended course of action. The full event description is long.
Other warnings and errors may appear in the Directory Service event log of an affected DC.
Cause
Suppose that there are three DCs in a domain; DC1, DC2, and DC3. Strict replication consistency is enabled on all three DCs, but all three are allowed to replicate with divergent partners. (See below for more information about these concepts.) Their AD databases are synchronized. The following events then occur in chronological order:
- DC1 is taken offline, either deliberately or by some sort of malfunction.
- A preexisting object with the name User1 is deleted on DC2. DC3 replicates the deletion from DC2 as a tombstone object. DC1 remains offline and does not receive the tombstone object.
- The tombstone lifetime (TSL) passes, and DC1 remains offline. The User1 tombstone object is permanently deleted ("garbage collected") from DC2 and DC3. User1 no longer exists on these two DCs.
- DC1 is brought online. It contains a lingering object (User1), although this has no effect for now.
- An edit is made to an attribute of User1 on DC1.
- DC2 and DC3 are notified of a change (the edit to User1).
- When DC2 and DC3 attempt to replicate the changes from DC1, they do each discover that User1 does not exist in their local databases.
- The replication attempt from DC1 is prevented on DC2 and DC3, and error 1988 is logged in their Directory Service event logs.
This error implies that strict replication consistency is enabled on the local DC, since the attempted replication does not occur. Strict replication consistency is enabled by default. It is intended to detect lingering objects and prevent them from being reintroduced into the domain after deletion.
The error also implies that divergent replication is enabled. When this is enabled, a DC may replicate from a partner that is unreachable for longer than the TSL of the forest. Divergent replication is not enabled by default, but an administrator may enable it manually.
Resolution
Attempting to resolve the issue by disabling strict replication consistency is not recommended. This allows lingering objects to replicate, potentially reintroducing permanently deleted objects into the domain.
Instead, lingering objects should be removed from the DCs on which they exist. The event description of error 1988 contains instructions for removing them using repadmin commands. However, these commands can be cumbersome, as the Globally Unique Identifier (GUID) of at least one DC must be known. Further, only one naming context can be scanned at a time. Microsoft has published a tool, the Lingering Object Liquidator (LOL) , that greatly simplifies this process. It provides a graphical interface for locating and removing lingering objects throughout an AD forest.
The link above provides an overview of the tool, its requirements, and a link to its download page. The following walkthrough assumes the LOL and its prerequisites are already installed on a DC.
- Ensure that the Remote event log Management (RPC) firewall rule is enabled on all DCs that are scanned.
- Launch the LOL.
- Click Detect AD Topology. The tool gathers information about the DCs in the AD forest.
- From the Naming Context dropdown, select the naming context in which one or more lingering objects exist. This is obtained from the description of error 1988. Alternatively, select [Scan All NCs] to scan all naming contexts for lingering objects.
- From the Reference DC dropdown, select the DC which has error 1988 in its Directory Service log.
- From the Target DC dropdown, select a DC which contains lingering objects. This is determined from the description of error 1988, but it mentions the source DC by its hexadecimal GUID rather than its name. You may select each replication partner of the reference DC in turn.
- Once a reference DC and target DC are chosen, click Detect Lingering Objects. All lingering objects detected are listed and automatically selected in the upper pane. A log of the scan is produced in the lower pane and written to a file. If no lingering objects are detected, check the firewall rule in step 1.
- If any lingering objects are not removed, clear them.
- Click Remove Selected Lingering Objects. All selected lingering objects are removed.
- Perform further scans as needed.
repadmin /regkey * -allowdivergent