Duplicate CMDB CI records - Support and Troubleshooting

Duplicate CMDB CI records Issue The "Identification and Reconciliation" (IRE) module provides a centralized framework for identifying and reconciling data from different data sources. It helps maintain the integrity of the CMDB when multiple data sources are used to create and update CI records. The IRE can be tailored to better meet the needs of your organization via identification rules, reconciliation rules, relationships, etc. For more information on the IRE please see:

CMDB Identification and Reconciliation Depending on the configuration, "duplicate" records could be created. It is important to understand what a "duplicate" record is as per the IRE. The IRE uses identification rules to find CIs according to the CMDB hierarchy; therefore a duplicate would be a record created when, per the identification rules, a CI should have been updated instead. Navigate to "Configuration > CI Class Manager" to see what rules would be used for a specific class.

For example, in the following image we see the OOB identifier used for Windows Servers. In the following we see:

The Identifier is "Derived", meaning inherited from a parent class (cmdb_ci_hardware in this case) Is independent, meaning it does not depend on relationships to other CIs Has "Correlation ID" as the first matching criteria

Given the above identifier, two windows servers created by the identification engine with the same "Correlation ID" would be considered duplicates.

Note : The CMDB supports domain separated. Therefore servers created on their respective domains with the same Correlation ID would be considered unique.

Domain separation in Configuration Management Database (CMDB) In the following image we see a custom identifier created for Windows Server where:

"Derived" is no longer present Only serial number will be used to determine uniqueness

Two windows servers with same Correlation ID would not be considered duplicates given the configuration from the above image.

Note : Creating new identifiers in some cases will also require related entries to be created. If a custom identifier is necessary for a child class of cmdb_ci_hardware, please see following link as an example before creating your custom identifier:

How to copy the Hardware Rule CMDB Identifier for specific Sub-Classes In a nutshell, the IRE will follow below logic for independent CIs:

A "payload" will be passed to it the IRE Validate the payload CIs and relationships. Is the payload valid? Yes: Continue No: Error out and log errors Loop through the identification rules and query the CMDB Is a CI found? Yes: Is the CI active given the inclusion rules for this class? Yes: Are there reconciliation rules for this class? Yes: Do the reconciliation rules allow this source to update the CI? Yes: Update CI fields No: Do not update CI No: Update CI fields No: Create CI No: Are there more identification rules to attempt? Yes: Continue to next identification rule No: Create CI Note : As an exception, discovery of vcenter will not use the IRE. See following link for vCenter Discovery for more information:

vCenter Discovery For more details on how the IRE works see:

How the CMDB Identification and Reconciliation Engine works when passing a CI (as payload) to the createOrUpdateCi() Dependent CIs Dependent CIs also use the identification rules. However, dependent relationship rules define the dependency structure of the CI types and the relationship types in these service definitions, helping in CI identification and in the construction of business service maps. During CI identification, the pair of CIs that are being examined, should satisfy at least one hosting rule.

For example, from "Configuration > Identification/Reconciliation > Metadata Editor" we see following rule for Kubernetes Pod:

From "Configuration > CI Class Manager" we see the following identification rule for Kubernetes Pod:

From combination of identification rule and hosting rules we see that for a kubernetes pod to be unique it must:

Have a unique "Kubernetes UID" Be the only kubernetes pod with a "Containd by" relationship to the kubernetes cluster Thus, following two kubernetes pod would be unique:

Kubernetes Pod A with UID Y and "Contained by" relationship to kubernetes cluster A Kubernetes Pod B with UID Y and "Contained by" relationship to kubernetes cluster B In the above, both pods have the same UID however are contained by different kubernetes clusters.

CMDB dependent relationship rules Create dependent relationship rules Release All releases.

Cause CIs which are in different domains would not be considered duplicates even if their data is the same.

In some cases duplicates may be created when, but not limited to, any of the following factors "hide" the CI from the identification engine:

Inclusion rules Query business rules ACL The IRE will query the CMDB in the context of the user calling the IRE. A CI will not be found and a new one will be created when any of the above factors "hide" the CI from the IRE. One example symptom of the above would be a duplicate CI created every time the same server is discovered or a call is made to the IRE API.

Additionally, depending on the release the IRE will not see CIs when:

duplicate_of field is not empty discovery_source is "Duplicate" Slow business rules can contribute to the creation of duplicate dependent CIs when multiple payloads with same dependent CIs are processed. This could happen when discovering the same CI in separate discoveries at the same time, or when the payloads in the same discovery contain such dependent CI in more than one payload. This would be a race condition and not common.

Resolution Inclusion rules Navigate to "Configuration > Identification/Reconciliation > Identification Inclusion Rules" Find the inclusion rule which applies to the class or one of its parent tables Adjust the inclusion rule accordingly Note : Creating an inclusion rule for a parent class, like cmdb_ci_hardware, and having conditions which include "class" is a sure way to create duplicates. The inclusion rule will be used for any child class; therefore any child class passed to the IRE which is not included in the condition will not be "seen". It is best to avoid "Class" as a condition to an inclusion rule.

For more information on health inclusion rules see:

Create health inclusion rule Query BR and ACL Find custom query BRs and ACLs for the affected table and disable them Try to reproduce the issue Issue resolution by disabling BR or ACL confirms root cause Review custom configurations and adjust accordingly Note : Out of box there are no Query BRs or ACLs in the CMDB tables which would affect the IRE.

Following links may be of help in troubleshooting BR and ACL issues:

Debugging business rules ACL debugging tools Slow business rules - dependent CIs Improve performance of such business rule If possible, run the business rule asynchronously In the pattern, add a step to remove duplicates before they are sent back to the instance DISCOVERY ONLY: glide.discovery.multi_page_serial_mode controls if multipage payloads are processed in parallel Set it to "true" so that payloads are processed in serial. This would avoid duplicates creation of duplicates by discovery Related Links Tools Generate and simulate payload execution using identification simulation CMDB APIs (CMDB SDK) IdentificationEngineScriptableApi - Global Troubleshooting tips Have clear steps to reproduce issue Use smallest payload set in order to simplify troubleshooting (only necessary CIs) Test payload with different users if possible (To tests different permissions) Via CI Class Manager check the identification rules, to confirm the CI should indeed be considered a duplicate Via CI Class Manager check for inclusion rules Debug IRE execution as described in following KB via scripts background to see if any BR could be affecting CI queries Debugging Identification and Reconcilliation Engine using scripts in scripts background