SGC-AWS: How to Identify and Remediate Server Merge Issue - Support and Troubleshooting

SGC-AWS: How to Identify and Remediate Server Merge Issue Summary .ns-kb-css-body-editor-container { p { font-size: 12pt; font-family: Lato; color: #000000; } span { font-size: 12pt; font-family: Lato; color: #000000; } h2 { font-size: 24pt; font-family: Lato; color: black; } h3 { font-size: 18pt; font-family: Lato; color: black; } h4 { font-size: 14pt; font-family: Lato; color: black; } a { font-size: 12pt; font-family: Lato; color: #00718F; } a:hover { font-size: 12pt; color: #024F69; } a:target { font-size: 12pt; color: #032D42; } a:visited { font-size: 12pt; color: #00718f; } ul { font-size: 12pt; font-family: Lato; } li { font-size: 12pt; font-family: Lato; } img { display: block; max-width: ; width: auto; height: auto; } } This article explains a known issue in which server Configuration Items (CIs) in the CMDB incorrectly link to multiple Virtual Machines (VMs) due to a server merge problem in the AWS Discovery process via SGC-AWS. It provides steps to identify and clean up these inaccurate relationships and introduces improvements to prevent the issue from recurring.

Issue In some scenarios, duplicate cmdb_rel_ci relationship records were created, linking a single server Configuration Item (CI) to multiple Virtual Machines (VMs). This occurred due to server merge operations that introduced inaccurate relationship data into the CMDB. As a result, the affected server records experienced frequent changes in their install status, caused by conflicting updates from the multiple connected VMs.

Cause (Why This Happens) :

Previously, We were using private DNS name as the server name in EC2 data source of SGC-AWS. However, since this name can be reused either across different VPCs or even within the same VPC when an old VM is retired and a new one is provisioned, it led to unintended server merges in the CMDB.

Impact:

Duplicate or incorrect cmdb_rel_ci (relationship) records. CMDB integrity is compromised. IRE (Identification and Reconciliation Engine) cannot maintain accurate relationships.

Resolution :

We need to run a one-time cleanup to bring CMDB data in consistent state. After remediation, improved logic in SGC-AWS will prevent recurrence.

Server Record Creation Logic Changes: Server record creation is now handled via the SG-AWS-Get-Inventory data source. In this data source, server records are created using the hostName returned by the GetInventory API, ensuring accurate and unique identification.

Additionally, server records in SGC-AWS are now created only if AWS Systems Manager (SSM) is enabled on the instance.

Optional Override:

If you wish to create server records even when SSM is disabled, set the following property in ServiceNow:

sn_aws_integ.create_server_record_even_if_SSM_disabled = true Release .ns-kb-css-body-editor-container { p { font-size: 12pt; font-family: Lato; color: #000000; } span { font-size: 12pt; font-family: Lato; color: #000000; } h2 { font-size: 24pt; font-family: Lato; color: black; } h3 { font-size: 18pt; font-family: Lato; color: black; } h4 { font-size: 14pt; font-family: Lato; color: black; } a { font-size: 12pt; font-family: Lato; color: #00718F; } a:hover { font-size: 12pt; color: #024F69; } a:target { font-size: 12pt; color: #032D42; } a:visited { font-size: 12pt; color: #00718f; } ul { font-size: 12pt; font-family: Lato; } li { font-size: 12pt; font-family: Lato; } img { display: block; max-width: ; width: auto; height: auto; } } The following Service Graph Connector versions and higher:

SR - CMDB - Service Graph Connector for AWS v2.10 Instructions .ns-kb-css-body-editor-container { p { font-size: 12pt; font-family: Lato; color: #000000; } span { font-size: 12pt; font-family: Lato; color: #000000; } h2 { font-size: 24pt; font-family: Lato; color: black; } h3 { font-size: 18pt; font-family: Lato; color: black; } h4 { font-size: 14pt; font-family: Lato; color: black; } a { font-size: 12pt; font-family: Lato; color: #00718F; } a:hover { font-size: 12pt; color: #024F69; } a:target { font-size: 12pt; color: #032D42; } a:visited { font-size: 12pt; color: #00718f; } ul { font-size: 12pt; font-family: Lato; } li { font-size: 12pt; font-family: Lato; } img { display: block; max-width: ; width: auto; height: auto; } } Clean Up Incorrect Relationships Of Server In CMDB:

To resolve the issue, apply the provided update set and run the one-time fix script specific to your cloud connector. The fix script will:

Download the Update Set [ Attach the update set here for download ] Navigate to System Update Sets > Retrieved Update Sets . Import and commit the provided update set. Navigate to: System Definition > Scripts - Background Ensure the scope is set to Global Run the following script to trigger the one-time cleanup process: gs.eventQueue("server_cleanup_event", "", "", ""); This will initiate cleanup in batches by inserting an event to handle stale relationships. This can take hours depending upon the volume of the data. We can monitor the event status in sysevent table by filtering with event name server_cleanup_event .

7.Verify Cleanup Completion

Run the following script to check whether any server is still associated with more than one VM:

var count = 0; var ga = new GlideAggregate('cmdb_rel_ci'); ga.addQuery('parent.sys_class_name', 'IN', ['cmdb_ci_server', 'cmdb_ci_linux_server', 'cmdb_ci_win_server']); ga.addQuery('parent.discovery_source', 'SG-AWS'); ga.addQuery('child.sys_class_name', 'cmdb_ci_vm_instance'); ga.groupBy('parent'); ga.addAggregate('COUNT', 'child'); ga.query(); while (ga.next()) { var vmCount = ga.getAggregate('COUNT', 'child'); if (vmCount > 1) { count++; } } gs.info("Server with VM count more than 1 : " + count); 8. Now run full import by clearing last_run_datetime field of datasources

9. Once import is done, run this fix script Cleanup orphan network adapter data to cleanup the orphan record.

Fix Script Details: This script performs cleanup of incorrect or stale relationships between Virtual Machines (VMs) and server CIs in the CMDB, specifically for the SG-AWS discovery source. The goal is to retain only accurate and most recent relationships, and allow IRE to correctly re-establish them during the next load.

Logic Overview: 1.Identify Target Servers:

Query the sys_object_source table to find servers discovered via SG-AWS-EC2-Instance. Filter records for specific target tables: cmdb_ci_server , cmdb_ci_linux_server , and cmdb_ci_win_server . Limit processing to a maximum of 10000 servers, chunked into batches of 500 for efficiency. 2. Detect Redundant VM Relationships:

For each server, query cmdb_rel_ci to find relationships of type "Virtualized by:Virtualizes". If more than one relationship exists, flag the server for cleanup. 3. Clean Up Stale or Incorrect Relationships:

For each flagged server:

Remove all non-VM relationships ( cmdb_rel_ci ) to eliminate outdated connections. Remove VM relationships where the parent.object_id differs from the child.object_id , marking them as orphaned. Clear the cmdb_ci field from any associated cmdb_ci_network_adapter records. Delete related entries from cmdb_key_value for orphan VMs using their SNK prefix. Delete corresponding entries in sys_object_source to trigger reprocessing by the IRE engine. 4.Post-Cleanup:

The sys_object_source entry deletion ensures that during the next run, the correct relationship is re-created. This ensures only one valid and up-to-date VM-to-server relationship exists per record