[Nov 2024] Azure Change Processing Table of Contents Feature OverviewRelease VersionChanges in 1.21.0 and above VersionsConfiguring Azure Change ProcessingSupported Resource TypesHigh Level OverviewSystem PropertyData Collected by ACPTables used by ACPACP Pull Changes Scheduled JobACP Process Changes Scheduled JobTable CleanupUpgrade ScenarioTag Governance Feature Overview Azure Change Processing(ACP) regularly retrieves resource changes from Azure, creating shallow CIs (configuration items) in CMDB, which are later updated through full discovery. Release Version 1.21.0 (Nov 2024) of Discovery and Service Mapping Patterns app. Changes in 1.21.0 and above Versions Resource Changes Payload Info (sn_cmp_resource_changes_payload_info) table is now not used in ACP flow. Its functionality is now replaced by ACP Resource Changes (sn_itom_pattern_acp_resource_change) table.Scheduled Jobs have been renamed meaningfully. 'Azure Process Changes' is renamed to 'ACP Pull Changes'.'CPG Build Azure Changes IRE Payload' is renamed to 'ACP Process Changes'. mid.cmp.azure.event.supported_resource_types property is deprecated. Configuring Azure Change Processing For guidance on migrating to Azure Change Processing, refer to the following link: After completing the configuration, ensure to verify the system property: “sn_cmp.azure.disable_new_alert_configs” by navigating to All -> System Properties -> All Properties -> search for property “sn_cmp.azure.disable_new_alert_configs” Supported Resource Types Azure Change Processing supports the following resource types: Microsoft.Compute/virtualMachineMicrosoft.Compute/disksMicrosoft.Network/networkSecurityGroupsMicrosoft.Network/networkinterfacesMicrosoft.Network/publicIPAddresses Azure Change Processing tracks a select set of resource properties for each supported resource type. When a change is detected in one of these properties, Azure Change Processing retrieves an updated snapshot of the resource from Azure and processes it in the CMDB to reflect the change. Resource TypeResource PropertyMicrosoft.Compute/virtualMachine CreateOrDeleteVm : properties.changeType in ('Create','Delete')ChangeStatus: (properties.extended.instanceView.powerState.code) in ('PowerState/running', 'PowerState/stopped', 'PowerState/deallocated'))ChangeDisk: properties.storageProfileChangeTag: tags.*ChangeNetworkInterface : properties.networkProfile.networkInterfaces Microsoft.Compute/disks CreateOrDeleteDisk : properties.changeType in ('Create','Delete')ChangeState : properties.diskStateChangeSize : properties.storageProfileChangeTag : tags.* Microsoft.Network/networkSecurityGroups CreateOrDeleteNsg : properties.changeType in ('Create','Delete')ChangeTag : tags.* Microsoft.Network/networkinterfaces CreateOrDeleteNic : properties.changeType in ('Create','Delete'),changePrimary : properties. properties.primaryChangeIp : properties.ipConfigurations[0]'ChangeTag : tags.* Microsoft.Network/publicIPAddresses CreateOrDeleteIp : properties.changeType in ('Create','Delete'),ChangeIpAddress : 'properties.ipaddress',ChangeFqdn : properties.dnsSettings.fqdnChangeIpConfig : properties.ipConfigurationChangeTag : tags.* High Level Overview Azure Change Processing uses two main triggers (scheduled jobs): ACP Pull Changes : Periodically, retrieves change-related information into the ServiceNow instance.ACP Process Changes : Periodically, process the changes and updates the CMDB. The following image illustrates the high-level overview of Azure Change Processing. System Property sn_itom_pattern.acp_event_queue_batchsize_per_node: The batchsize for the number of ACP changes that can be added to the event queue for each active node. Default value is 100.sn_itom_pattern.acp_processing_time_range_hours: Retrieves the maximum time range (in hours) for selecting records to process. Default value is 24. Data Collected by ACP Azure Change Processing populates the CI attributes based on the response mappings defined in the sn_cmp_response_mapping table for each resource type. cmdb_ci_vm_instance CI attributes: object_idnameinstall_statusoperational_statusstatecpusmemorydisksdisk_sizeguest_os_fullnamevm_inst_id CI Relationships: ParentRelationship TypeChildcmdb_ci_resource_groupContains::Contained bycmdb_ci_vm_instancecmdb_ci_vm_instanceProvisioned From::Provisionedcmdb_ci_compute_templatecmdb_ci_vm_instanceProvisioned From::Provisionedcmdb_ci_cloud_hardware_typecmdb_ci_vm_instanceUse End Point To::Use End Point Fromcmdb_ci_endpoint_vniccmdb_ci_vm_instanceUse End Point To::Use End Point Fromcmdb_ci_endpoint_block cmdb_ci_storage_volume CI Attributes: object_idnameinstall_statusoperational_statusstatesize_bytes CI Relationships: ParentRelationship TypeChildcmdb_ci_resource_groupContains::Contained bycmdb_ci_storage_volumecmdb_ci_endpoint_blockImplement End Point To::Implement End Point Fromcmdb_ci_storage_volumecmdb_ci_vm_instanceUse End Point To::Use End Point Fromcmdb_ci_endpoint_block cmdb_ci_nic CI Attributes public_ipmac_addressoperational_statusprimaryinstall_statusnamestateobject_idprivate_ip CI Relationships ParentRelationship TypeChildcmdb_ci_resource_groupContains::Contained bycmdb_ci_niccmdb_ci_endpoint_vnicImplement End Point To::Implement End Point Fromcmdb_ci_niccmdb_ci_vm_instanceUse End Point To::Use End Point Fromcmdb_ci_endpoint_vnic cmdb_ci_cloud_public_ipaddress CI Attributes object_idinstall_statusoperational_statusnamepublic_dnspublic_ip_addressstate CI Relationships ParentRelationship TypeChildcmdb_ci_resource_groupContains::Contained bycmdb_ci_cloud_public_ipaddresscmdb_ci_nicContains::Contained bycmdb_ci_cloud_public_ipaddress cmdb_ci_compute_security_group CI Attributes object_idinstall_statusoperational_statusnamestate CI Relationships ParentRelationship TypeChildcmdb_ci_resource_groupContains::Contained bycmdb_ci_compute_security_group Tables used by ACP In the ACP flow, four newly introduced tables help manage the collection and processing of changes from Azure resources. These tables serve as a key part of the feature for debugging and tracking status. Below is a detailed breakdown of each table and its role. ACP Resource Type (sn_itom_pattern_acp_supported_resource_types) Purpose : This table holds the list of supported resource types for Azure. It leverages Azure Resource Graph queries to detect changes for the defined resource types in the Azure cloud.Key Fields Resource Type: The specific Azure resource type (e.g., NIC, VM).Query: The resource graph query used to retrieve changes from Azure.Active: Indicates if the resource type is actively being queried. ACP Order Status(sn_itom_pattern_acp_order_status) Purpose : This table tracks the status of event pull orders for each resource type, based on the associated credentials used for querying Azure. Key Fields Resource Type: The type of Azure resource.Credential: The specific credential used for pulling changes.Last Processed Time: The timestamp of the changes were pulled.Status: Tracks the success or failure of the event pull process. After the next scheduled job, this status will be updated based on the CMP Order status.CMP Order: Reference to the CMP Order (sn_cmp_order), which provides details about the current order's progress and outcome. ACP Resource Change (sn_itom_pattern_acp_resource_change) Purpose : This table records changes detected in Azure resources.Key Fields Resource ID: The unique identifier for the changed resource in Azure.Change Payload : A snapshot of the attributes of interest for the resource.Order: A reference to the CMP Order (sn_cmp_order) that pulled the change.Resource Status: A reference to the ACP Resource Status table to track the processing status of the change.State: The current processing state of the change. If the state is marked as 'Error', the associated Resource Status record should be checked for further details. ACP Resource Status (sn_itom_pattern_acp_resource_status) Purpose : This table tracks the overall status of resource processing.Key Fields Resource ID: The identifier for the resource being processed.Status: Tracks the processing state (e.g., Ready, In Progress, Failure,Success).Error Message: If processing failed, this field contains error details. ACP Pull Changes Scheduled Job Overview of ACP Pull Changes Scheduled Job The ACP Pull Changes is a scheduled job designed to create orders in the sn_itom_pattern_acp_order_status table for each credential per supported resource type. By default, this job triggers every 5 minutes. Order Creation Orders are generated in the sn_itom_pattern_acp_order_status table, where each order is submitted with a specific query and window size. The actual orders are then created in the sn_cmp_order table, with references to these orders included in the CMP Order column of the sn_itom_pattern_acp_order_status table. Query Structure Queries for each resource type are stored in the sn_itom_pattern_acp_supported_resource_types table. We utilize a single query for each resource type. For Azure Change Processing, the query is designed to capture changes only when relevant properties are modified. The query consists of three main components: Filter Changes: This retrieves changes from the resourcechanges table if they are included in a specified list. Detailed explanations for each CI are provided in subsequent sections.Summarise: This step condenses the changes into a single record per CI.Snapshot: This pulls data from the resources table for the relevant records Example VM Query For virtual machines (VMs), we filter changes from the resourcechanges table in Azure based on the following properties: CreateOrDeleteVm : properties.changeType in ('Create','Delete')ChangeStatus: (properties.extended.instanceView.powerState.code) in ('PowerState/running', 'PowerState/stopped', 'PowerState/deallocated'))ChangeDisk: properties.storageProfileChangeTag: tags.*ChangeNetwork: properties.networkProfile.networkInterfaces These change records are summarized to create one record per CI, with a snapshot obtained by joining the resources table. When a change occurs in any of the specified properties, we project several attributes for the VM. Since multiple disks and network interfaces may exist, we only include these arrays if there are changes to the disks or network interfaces specifically. An additional project operation is applied to filter these changes for VMs. Sample Output The query construction pattern outlined here is similarly applied to other supported resource types. The relevant properties for supported resources have already been documented. Note: For tag changes, we only capture the change-related information to minimize data retrieval. Tag governance requires processing changes rather than fetching all current tags and calculating differences for updating the CMDB. Query Window Calculation The query window is calculated based on the minimum value of (last trigger time - current time, max default window). The maximum default window out-of-the-box (OOTB) is set to 4 hours. Persisting Changes The payload received from executing the queries is stored in the sn_itom_pattern_acp_resource_change table. The Azure_Persist_Resource_Changes_Response_Processor in the sn_cmp_rb_resourcescript table processes the query response, adding one record per resource ID into the sn_itom_pattern_acp_resource_change table. Flow Diagram ACP Process Changes Scheduled Job Overview of ACP Process Changes Scheduled Job The ACP Process Changes scheduled job is designed to process payloads and persist them into the CMDB. It operates with a default frequency of every minute. Job Functionality Resource ID Retrieval: The job fetches resource IDs that are in a ready state from the sn_itom_pattern_acp_resource_status table.Batch Compilation: It compiles these resource IDs into batches to streamline processing.Queue Insertion: The compiled batches are inserted into the acp_event_queue.Event Processing: An event queue processor then processes these events to update the CMDB. Configuration Details Default Frequency: Every 1 minutesn_itom_pattern.acp_event_queue_batchsize_per_node system property : Default value is 100 and configurable. Response Mapping Response Mapping defines the rules that determine how attributes and relationships within a Configuration Item (CI) are populated. These mappings are stored in the sn_cmp_response_mapping table. Data Source For the ACP process, the data source utilized is Azure Resource Changes. This mapping ensures that data is accurately reflected in the CMDB by specifying how incoming information should be interpreted and applied to existing CIs. Flow Diagram Table Cleanup sn_itom_pattern_acp_order_status: This table has a cleanup policy that automatically removes data every 7 days.sn_itom_pattern_acp_resource_status: This table utilizes 7 partitions for data storage, with cleanup managed through partition rotations.sn_itom_pattern_acp_resource_change: This table also consists of 7 partitions, and data cleanup is similarly handled through rotations. Upgrade Scenario This is applicable for customers moving to 1.21.0 (Nov 2024) or above version from a version lower than 1.21.0 (Nov 2024) of Discovery and Service Mapping Patterns app. Deactivate the "CPG Build Azure Changes IRE Payload" Scheduled Job : Before beginning the upgrade, deactivate the "CPG Build Azure Changes IRE Payload" scheduled job. Monitor Active Transactions : Go to the v_cluster_transaction table and filter for records where the URL is set to "CPG Build Azure Changes IRE Payload." Wait for these transactions to finish processing. If the backlog of records in a "ready" state within the sn_cmp_resource_changes_payload_info table is substantial, it may take time to clear. Alternatively, you may right-click on a transaction and select "kill" to stop it if necessary. Upgrade the Application: After the upgrade, Azure Change Processing will automatically retrieve and process changes from the past 4 hours. Run Cloud Discovery Post-Upgrade: One cloud discovery post-upgrade is recommended to ensure no change processing is missed from backlog during upgrade. Tag Governance Note : Tag Governance is primarily handled by Tag Governance Store App. Part of it is facilitated by Azure Change Processing and Azure Alert Driven Discovery. Tag Governance is a store app which aids customers in effective tag management. Tag governance supports CI audits via tag policies which can be set to run on cloud events. Tag Governance processes tag received configured with with Azure Alert Driven discovery or Azure Change Processing, updates CMDB & triggers tag audits. As mentioned in Azure alerts documentation, Azure Change Processing is better than Azure alerts for getting tag changes. So to summarise Azure Change Processing will get tag changes OOTB which are processed by tag governance. This processing on tag governance side needs some fixes with new changes introduced on Azure Change Processing side(for scaling to heavy load) in this release(Nov 2024) and will be released by tag governance product OOTB in coming releases. Reference : DEF0583362 (KB will be shared for this defect with update set to support tags processing for customers already using tag governance or planning to get tag governance)