Handle Duplicate CIs in your CMDB

Get Well Playbook

A guide for how to manage and remediate duplicate CI records

Table of Contents

Summary . 3

Audience . 3

Problem Overview .. 3

Executive Summary . 3

How this playbook can help you achieve your business goals . 3

How this playbook is structured . 3

Problem Analysis 4

Upstream Causes: 4

What are downstream consequences: 4

What is the Impact to Business Outcome: 5

Engagement Questions: 6

Remediation Plays 7

Summary . 7

Analysis Play . 8

Fix Play – Remediate Duplicates using the Wizard . 9

Data Governance . 17

Summary

Goal for the Playbook (Problem or target this play will address):

The goal of this playbook is to help you manage and resolve (remediate) duplicate configuration items (CIs) in your configuration management database (CMDB).

Details about this playbook

Author Michael Walker, Callan Bond Date 04/28/2025 Addresses HSD # HSD0003784, HSD0006973 Applicable ServiceNow Releases All Releases Time required Approximately 1 to 8 hours

Audience

Configuration Manager or Configuration Management team

ServiceNow Admin

Problem Overview

You don't want duplicate CIs in your CMDB. Duplicate CIs can:

Take up valuable space in your CMDB

Create inaccurate data

Erode confidence in the quality of the data

Lead to data loss

Executive Summary

How this playbook can help you achieve your business goals

You need to resolve (remediate) any duplicate CIs. Resolving duplicates ensures that your platform processes are accurate. To help you resolve the duplicates, and ensure their accuracy, use the wizard included in the base system. This wizard merges duplicate records into a single record.

How this playbook is structured

This playbook guides you through three plays.

The first play (an analysis or validation play) shows you how many duplicate CI records you need to remediate.

The second play (a fix play) helps you use the built-in wizard to merge the CI records.

The third play (a governance play) shows you how to use the CMDB dashboard to see the health of the CMDB and make changes, as needed. Problem Analysis

Upstream Causes

Duplicate CIs can be caused by the following:

Discovery patterns fail to gather key data (such as serial numbers, names, and configuration files). These failures may be due to the logic used in the pattern or a credential failure.

CI Identifiers are configured incorrectly.

Data is entered incorrectly (such as typographical errors, failure to check if a CI already exists)

CI naming conditions, or processes for retiring hardware, aren't followed.

Failures occur when importing or integrating data Remediation tasks that you need to address can occur due to a lack of:

CMDB data ownership

Enough people on the Configuration Management team

Diligence for remediating duplicate CI tasks

Knowledge of the CMDB Dashboard

CMDB Dashboard governance

Downstream Consequences

Data Consequence:

Duplicates appear as identical, but separate, records in list view. This can be confusing.

You need to manually remediate any duplicate records (or records that appear empty).

You may update and then compress the wrong CI. If that happens, and the serial number is changed, you lose all history for that CI.

Operation Consequence:

You may need to use special queries to resolve these duplicates. Using special queries adds to your overhead.

Resolving duplicate CIs can be frustrating.

Services you want to use can be adversely affected when processes break down or operate slowly

App Consequence:

Binding and CI selection for Incident Management could be affected.

CI matching for Vulnerability Response (VR) could fail

Change Management or other ServiceNow applications may require correction

Asset Management performance may be adversely affected by broken links to CIs or by having to remediate duplicate CIs.

Impact on Your Business

Resolving duplicate CIs can have an positive impact on the following areas:

Lower MTTR Data validation

CI binding in Event Management

Increase Operational Visibility Data accuracy

Lifecycle management

Audit/Compliance System integrity

Data governance

Process Automation Data accuracy

Integrity of relationships

Better User Experience Platform adoption

Increased confidence in the CMDB Engagement Questions:

Consider the answers to these questions:

Is the owner of the CMDB clearly identified? Who is accountable? Every practice needs to have an owner.

Who can add records? Is it only trained staff?

How large is your Configuration Management team? The CMDB requires maintenance and governance. Who is responsible for those remediation tasks?.

To get the most benefit from your CMDB, someone needs to maintain the data. If there's no one do that, think about what you really need.

Can the Configuration Manager resolve duplicates, which may require deleting, updating, and editing records? If not, who can?

How many hours a week are you allocating to reconciling duplicate CIs? Is this a priority for your company?

What do you do with the duplicate CIs after they are merged to a single CI? Delete them? Archive them? Remediation Plays

Summary

The table below lists and summarizes each of the remediation plays in the playbook. Details are included later.

Play

Analysis

What is this play about

Lets you see if you have any duplicate CIs.

Required tasks

Use list view to see if you have any duplicate CIs.

Fix

What is this play about

Explains how to use the built-in wizard to resolve (remediate) the duplicate CIs.

Required tasks

Use the built-in wizard to update (or remove) inaccurate data

Data Governance

What is this play about

Explains the importance of monitoring the health of your CMDB, and resolving duplicate CIs as soon as you are aware of them.

Required tasks

Use the CMDB Health dashboards to view the duplicate CIs and take action to remediate them. Then run the CMDB Health Dashboard – Correctness Score Calculation job to update the CMDB.

Analysis Play

What this Play is about

This play helps you identify any duplicate CIs you may have in your CMDB. You can see the duplicate CIs on the Remediate Duplicate Tasks list view.

Required tasks

Option 1:

This play helps you identify duplicate CIs in your CMDB. To first find your duplicate CI’s CMDB health score and a list of details, use the CMDB and CSDM Data Foundations Dashboards .

Step 1: If you are on Xanadu family release or higher, the CMDB and CSDM Data Foundations Dashboards are available out of box. If you are on a version previous to Xanadu, navigate to the ServiceNow Store, search for “CMDB Data Foundations”, and install the free Store app.

View the ServiceNow docs site to learn more about installing and configuring the Dashboards if you are setting them up for the first time.

Step 2: Navigate to the CMDB Data Foundation Dashboard

Step 3: Navigate to the “Data Management Practices” tab and review the “Handled Duplicate CIs” metric.

This metric looks only at Operational records in the Hardware and VM instance classes. If your score shows 100%, there are no duplicate Cis in those classes and so no further action is required. However, this metric should be reviewed monthly.

Step 4: Click on the “Unhandled Duplicate CIs” historical graph widget below to review individual records.

Click on “Show Records” in the upper right to view the individual duplicate CI records. Note: the Data Foundation dashboards ‘Show Records’ only shows CIs marked with “Duplicate Of” as true. The CMDB Workspace (below), will show both the ‘main’ CI and its duplicate(s).

Option 2: Using CMDB Workspace

To View duplicate CIs, navigate to the “Duplicate CIs” panel on the front page of the CMDB Workspace. Click on the pane to see a list of Duplicate CIs. If you see no CIs here you don’t need to do anything else. Note: the CMDB Workspace will show both the ‘main’ CI and its duplicate(s), whereas the Data Foundation dashboards (above) ‘Show Records’ feature only shows CIs marked with “Duplicate Of” as true. Fix Play

What is Play is about

This play helps you remediate duplicate CI records by using the built-in Duplicate CI Remediator wizard. For more information about this wizard and how duplicates are remediated, see Remediate a de-duplication task .

To navigate to the Duplicate Dashboard, go to Workspaces > CMDB Workspace > Management and select De-duplication Dashboard in the right pane:

You can also navigate to the dashboard directly from the “De-duplication” tasks widget on the front page of CMDB Workspace:

Duplicates can be remediated individually, or deduplicate templates can be used in order to batch process large numbers of duplicates with similar properties.

Required tasks

Use Duplicate Templates to Remediate in bulk

De-duplication templates allow you to apply consistent remediation to multiple, similar CIs in a single action. To view all Duplicate Tasks, click on the “View de-duplication tasks” button from within the De-duplication Dashboard.

If you already have a Template Library and Template created for the CIs you wish to Remediate, skip to step 4. Otherwise create a Library and Template. A Library is a means of organizing templates. For example, you may create a “Hardware” library which has templates to remediate various Hardware CI classes. From the De-duplication dashboard click “View template libraries” Click the “Create a library” link and create a category for templates. For example, “Hardware”. Create your De-duplication template. Refer to docs for step-by-step instructions to walk through the wizard. At a high level you will: Fill out general information on the template Create a filter to narrow down which CIs should be deduplicated Select the method for identifying which CI would be the main CI (e.g., the one with the most recent update date, or the one from a certain Discovery source) Configure how you will merge attributes, relationships, and related items Select Duplicate tasks that have the same CI class and click the “Assign to template” button in the upper right. These CIs will be de-duplicated using the criteria you specified in the template. Choose the Library and Template that match the class of the Duplicate CIs you have selected: Click “Add Selected” in the upper right to add the Duplicate CIs to the template Run the template to remediate the Duplicates. To Manually Remediate a single Duplicate task

Manually remediating a single task is not done through the De-duplication dashboard, but from the

Go into the De-duplication task list by searching “duplication” in the left navigation menu and select De-duplication tasks Click on the Duplicate Task number of one task from the list. Click the Remediate button and then Remediate Manually Choose to use the Duplicate CI Remediator Follow the steps in the Wizard. First select which CI to keep The Wizard provides Recommendations for which CI you may want to keep. Some guidelines to consider: CIs with a known Discovery source are better vs those without Most recent Discovery date: generally more recent is better CIs with a greater number of Related items are generally better After choosing the Main CI, choose Remediate Manually In the next choose to keep whether to keep the suggested Attributes or override with options from the other CI(s). Next choose how to merge Relationships, and Related Items Next you may choose to Delete the Duplicate CIs, or to set attributes. If not deleting, you should consider setting the lifecycle fields your organization uses so that the CI is not Operational. E.g., Operational Status to Retired or the Life Cycle Stage to End of Life and Life Cycle Stage Status fields to Retired. You can then use Data Manager to create an Archive policy which will Archive the duplicate CIs and remove them from the CMDB, without Deleting. Finally Review and Confirm your choices. You have successfully remediated the duplicate CIs.

Data Governance Play

What this Play is about

There are two methods to monitor duplicate CIs. Either of these methods should be used at least monthly to confirm Duplicate CIs have not reappeared in your CMDB.

Required tasks

Use the CMDB Workspace Widget

In “Important Actions” section of the CMDB Workspace front page, there are two widgets that should be reviewed regularly: De-duplication tasks, and Duplicate CIs.

Ensure these widgets are reviewed on a monthly basis. If duplicates continue to be populated, investigate the sources that are populating them into the CMDB. Use the CMDB Data Foundation Dashboard

Navigate to Workspaces > CMDB Workspace > Management > CMDB Data Foundations Dashboard. On the Data Management Practices tab, look at the “Handle Duplicate CIs” score. If this is less than 100, check the “Unhandled Duplicate CIs” widget below. Use the De-duplication Dashboard as described above to handle De-duplication tasks. After all De-duplication tasks are Complete, the score for this metric should return to 100. Ensure this metric is reviewed monthly. If duplicates continue to be populated, investigate the sources that are populating them into the CMDB. Use the CMDB Workspace Duplicate widget

Use this chart to see the number of duplicate CIs in each class. Click the name of the class you want to view.

Congratulations

You have completed this Playbook.