Incident creation delayed or incident not created from alertIssue This KB provides troubleshooting steps for when an incident is not created from an alert or the incident creation is delayed. In some scenarios it may provide clear steps to resolution and in others it may provide a direction in the troubleshooting effort. The scheduled job "Event Management - Evaluate Alert Management Rules" processes the alerts and create incidents as a result of processing such alert, if the alert action rule or alert management rule is configured to do so. Note: The sys_trigger table "name" field for this job is "Event Management - Evaluate Alert Manage" as the sys_trigger.name max length is 40. Also, on older versions the job creating incidents as "Event Management - create/resolved incidents by alerts" (truncated to 40 leters). Alert Action Rule Alert action rules are legacy and event management has moved to alert management rules for processing of alerts. If there are still any alert action rules in the instance see following documentation on how to migrate to alert management rules: Migrate alert action rule to an alert management rule Incidents are created by the "Event Management - Evaluate Alert Management Rules" job when using alert action rules. Alert Management Rules With alert management rules the "Event Management - Evaluate Alert Management Rules" job will trigger the necessary flow. The flow will create the incident. The flows are processed by the "Flow Engine Event Handler" jobs. OOB there is a subflow provided to create incidents from alerts, however custom ones could be created as well. For creating custom subflows for event management see Create a custom subflow.Cause01 Sys_trigger job "Event Management - Evaluate Alert Manage" is stuck To confirm if this is an issue: Find the sys_trigger job: /sys_trigger_list.do?sysparm_query=nameSTARTSWITHEvent%20Management%20-%20Evaluate%20Alert%20Manage&sysparm_view= Check the "Next action" timeThis job should run every eleven seconds. If this time is far in the past, this is an indication this job is stuck 02 Issues with subflow 03 Poor performance by OOB or custom business rules 04 BR aborting insert of incident 05 Alert management rules not creating incidents for all domains The schedule job "Event Management - Evaluate Alert Management Rules" runs in the global domain with user "System Administrator". The user is part of the Global domain and all the system jobs generally run in the global domain. If you view the xml you can see the user that runs the following job. OOB this would be run_as display_value="System Administrator"If the domain of the "System Administrator" user or of the scheduled job is tampered with and changed into some other domain, it will evaluate alerts only from that domain 06 Incident not created when using Alert Action Rule Alert does not match the condition specified in the Alert Action Rule. 07 Incident/Task will not be created for an alert which has a valid alert management rule This situation would occur when there are legacy alert action rules still active on the instance and not all of them have been converted into an alert management rule yet. There would be a conflict when an alert gets evaluated against an alert action rule but the alert is expected to be processed by an alert management rule. This conflict would prevent successful application of alert management rules, and in turn, the create incident workflow would not be triggered.ResolutionNote: Some of the steps for resolution may need to be performed by servicenow support. For such, please create a case for assistance. 01 Sys_trigger job "Event Management - Evaluate Alert Manage" is stuck Find the sys_trigger job: /sys_trigger_list.do?sysparm_query=nameSTARTSWITHEvent%20Management%20-%20Evaluate%20Alert%20Manage&sysparm_view= Add columns "Claimed by" and "System ID"Get thread dumps from the running jobReview node logs to look for reasons that the job could be stuck and provide resolution 02 Issues with subflow Open the alert which should have created an incidentReview the "Alert Executions" for the alertCheck if the flow was created, if the flow was executed, and if the flow had any errors Alert executions information Check if the "Flow Engine Event Handler" jobs are running and not stuck sys_trigger_list.do?sysparm_query=name%3DFlow%20Engine%20Event%20Handler&sysparm_view= Resolve issues with subflow or with the "Flow Engine Event Handler" jobs 03 Poor performance by OOB or custom business rules Review the node where the incident was create and look for any logs referring to slow business rules when creating the incident 04 BR aborting insert of incident Look for BR which could be aborting the insert of the incident 05 Alert management rules not creating incidents for all domains Make sure the user which runs job "Event Management - Evaluate Alert Management Rules" and the job are in the global domain 06 Incident not created when using Alert Action Rule Fix alert filter conditions to properly match the alert. Note that they are case sensitive. 07 Incident/Task will not be created for an alert which has a valid alert management rule Set system property: evt_mgmt.alert.management.enable_legacy_alert_action_rules to false. Note: On an instance, if we are expected to have alert management rules working, there should not be a conflict in the form of legacy alert action rules being active. The above property will disable the application of legacy alert action rules thus avoiding the conflict. If are any alert action rules which still must be used, please look into migrating them to alert management rules.Related LinksAlerts were created sometime back, then all of a sudden, incidents are created for them hours/days after the alerts were created