How to build an Identification and Reconciliation Engine payload - Support and Troubleshooting

How to build an Identification and Reconciliation Engine payload Description This knowledge base article provides an overview of the CMDB model, IRE, and what a payload passed to the IRE should contain.

Table of Contents CMDB Model What is the CMDB Model How can we check the configuration for a class IRE What is the IRE Payload Using Identification Simulation to determine what a payload should contain Payload Errors Summary Additional Information / Troubleshooting CMDB Model What is the CMDB Model? The class descriptions, identification rules, identifier entries, inclusion rules, reconciliation rules, and dependent relationships if applicable Class description Identification rules and identifier entries The set of rules the IRE utilizes to determine if a CI already exists in the database Reconciliation rules Reconciliation rules specify which discovery sources can update class attributes, and the precedence order among these discovery sources Dependent relationships The identifier for a class determines if this class is independent or dependent. Independent classes can be sent to the IRE without any relationships. Dependent classes need a relationship and information on the CI they depend on to be sent in the same payload. Inclusion Rules During duplication detection of independent CIs, the IRE processes only the CIs that satisfy the identification inclusion rules. How can we check the configuration for a class? Navigate to “Configuration > CI Class Manager” Search for the class of interest, in the following example the class is "Windows Server" The UI will then load the options for this class Selecting "Identification Rule" displays the identification rules for this CI Click on "Reconciliation Rule" to see the reconciliation rules for this class To see required relationships for a dependent class follow previous steps and click on "Dependent relationships", in the following example the class is "Network Adapter" IRE What is the IRE? The Identification and Reconciliation Engine (IRE) is an underlying key component in Identification and Reconciliation, providing a centralized framework to perform identification and reconciliation processes across different data sources. IRE uses identification rules, reconciliation rules, and IRE data source rules when processing incoming data before inserting that data to the CMDB. IRE processes help maintain data integrity in the CMDB.IRE prevents duplicate CIs by uniquely identifying CIs. IRE reconciles CI attributes by allowing only authoritative data sources to write to CMDB. Payload Using Identification Simulation to determine what a payload should contain Navigate to “Configuration > Identification/Reconciliation > Identification Simulation” Select “Start” from the “Start with CI Class” box Populate the Data Source, Class, Attributes, and container Click on “Generate Script”, see example below On the right tab we can see the payload built by the identification simulation with the necessary data At this point, clicking on "Run Simulation" would pass this payload to the IRE and process the payload Note that when using the identification simulation no inserts or updates will take place Payload Errors The following document contains the errors and solutions for Identification Engine errors

IRE Error Messages Set identification engine to debugVerbose Following system property may to be added to log the payload to system logs (syslog table):

Property name = glide.cmdb.logger.source.identification_engine Property value = info,warn,error,debug,debugVerbose Warning: Revert the property to OOB setting of info,warn,error when done with debugging.

Issue 01 - Payload Error Lets run the following payload through the identification engine:

{ "items": [ { "className": "cmdb_ci_network_adapter", "lookup": [], "values": { "mac_address": "08:00:27:66:50:99", "name": "ETH8" } }, { "className": "cmdb_ci_win_server", "lookup": [], "values": { "serial_number": "ABC123!ABC123!" } } ] } The following error is returned:

In payload no relations defined for dependent class [cmdb_ci_network_adapter] that matches any containment/hosting rules: [cmdb_ci_network_adapter >> Owned by >> cmdb_ci_hardware]. Add appropriate relations in payload for '{\"className\":\"cmdb_ci_network_adapter\",\"values\":{\"discovery_source\":\"ServiceNow\",\"mac_address\":\"08:00:27:66:50:99\",\"name\":\"ETH8\"},\"internal_id\":\"8e099f85a04881905c4bda8445bd7eee\",\"sys_object_source_info\":{\"source_name\":\"ServiceNow\"},\"settings\":{},\"sys_ire_info\":{\"ire_received_time\":\"2021-11-24 14:05:13\"}}' The first part of the error message explains that the payload could not be process because the dependent class does not have the necessary relationships (containment/hosting rules) and also lists the relationships which could be passed in the payload in order to properly process it: In payload no relations defined for dependent class [cmdb_ci_network_adapter] that matches any containment/hosting rules : [cmdb_ci_network_adapter >> Owned by >> cmdb_ci_hardware] . The second part of the message suggest how to fix this issue: Add appropriate relations in payload for Based no the information/error returned by the identification engine we can either adjust the payload ourselves or use the identification simulation to build a proper payload. Using the identification engine simulation we build the following payload, now with the missing relationship:

{ "items": [ { "className": "cmdb_ci_network_adapter", "lookup": [], "values": { "mac_address": "08:00:27:66:50:99", "name": "ETH8" } }, { "className": "cmdb_ci_win_server", "lookup": [], "values": { "serial_number": "ABC123!ABC123!" } } ], "relations": [ { "type": "Owns::Owned by", "parent": 1, "child": 0 } ] }

Errors are no longer returned. As this network adapter did not exist, a new one would be created:

{ "className": "cmdb_ci_network_adapter", "operation": "INSERT", "identificationAttempts": [ { "info": "sys_object_source SKIPPED", "identifierName": "", "attemptResult": "SKIPPED", "attributes": [], "hybridEntryCiAttributes": [] }, { "identifierName": "Network Adapter", "attemptResult": "NO_MATCH", "attributes": [ "mac_address", "name" ], "searchOnTable": "cmdb_ci_network_adapter", "hybridEntryCiAttributes": [] } ], "info": [], "errorCount": 0, "mergedPayloadIds": [], "warningCount": 0, "inputIndices": [ 0 ], "markers": [] } Issue 02 - CI field not updated In this example we are trying to update the field "name" of a windows server. However this field is not updated when we call the IRE with a payload.

We first check that the field was populated in the payload passed to the identification engine. This can be done by setting the identification engine to debugVerbose. Having the IRE set to debug will also help us determine why the field was not updated.

In syslog we see:

The above confirms that the field "name" was populated, in the case above the value was "NameToChangeTo". However, we can see that the record name was not updated. In the above image we also see the source of this payload was "ServiceWatch"

With the identification engine still set to debug we review the system logs with source "identification_engine at the time of the failed update. We see the following:

The above explains that the IRE decided not to update this field given the data source, in this case ServiceWatch. Because of the above message we should check the configuration for this class, the reconciliation rules more specifically.

The above image shows that there is a reconciliation rule for class Windows Server which will only allow data source "ServiceNow" to update this class. To allow this data source payload to update the target CI we can either:

Review this reconciliation rule Create/Add a rule to allow this data source to update this table Issue 03 - Not IRE related If the payload contains the required data and no reconciliation rules "block" the update, then the issue is likely not due to identification engine but to business rules or ACLs. In such cases it may be required to directly call the identification engine via scripts background to further investigate. See following documentation on how to do so:

Debugging Identification and Reconciliation Engine using scripts in scripts background Summary We can see what would be required in a payload for a CMDB class by understanding the basic components of the CMDB, CI Class Manager, and how to use the Identification Simulation.

Additional Information / Troubleshooting Debugging Identification and Reconciliation Engine using scripts in scripts background Duplicate CMDB CI records Identification Rules Inclusion Rules Reconciliation Rules