Flow are getting queued or delayed execution - Support and Troubleshooting

Flow are getting queued or delayed execution For the latest information, see https://docs.servicenow.com/csh?topicname=design-considerations-consolidated&version=latest

If you see latency in flow execution then this article is helpful.

Flows which run in background they get executed by flow jobs. When there is request to start/resume a flow in async manner, it creates an event and flow job (which run every 2 sec) will pickup the work and execute the flow.

Why my flows are delayed in process ?

There could be one or more reason of delay in picking up the event.

1. Burst of Event : Sometimes due to mass import/update or any event which affect lot of record and if flow is attached on those table records. It will create mass request for flow execution. By default, there are 3 flow designer job on each node which is processing the work and it might take some time for them to go through.

You can confirm this issue by querying sysevent table and put filter name=flow.fire and state starts with "ready". If you see many record, then you should expect delay in processing flow.

Consider:

Using import sets rather than flows to process large imports Adding more flow capacity will lesson the impact of this, but it should always be considered as the last option. Do this either by increasing the number of nodes or the number of Flow Engine Event Handler jobs 2. Long running flows : Some flows take longer time to complete. Since by default we have 3 Flow Engine Event Handler jobs per node, if all of those threads are executing long running flow then that node will not able to process any new flow event unless any of long running flow will reach to waiting/finish stage.

You can look at sys_flow_context.runtime value (sort desc) to find out if you have long running flow context. It's worth looking at sysevent table order by processing_duration and name = flow.fire to see which event is taking longer time. (sysevent.instance is flow context_id)

Consider:

Avoid long running flows Run long running flows the system has less load Adding more flow capacity will lesson the impact of this, but it should always be considered as the last option. Do this either by increasing the number of nodes or the number of Flow Engine Event Handler jobs 3. Scheduler is busy : Flows get executed via flow designer jobs which is on mercy of scheduler. If Scheduler is overwhelmed then flow designer job may not get slot to execute for some time, leads to flow queue backing up. It's worth checking when sys_trigger and filter with name STARTSWITH flow engine event and parent != null. Note next_action for each row, those shouldn't be > 10mins in past.

If there are many scheduled jobs (sys_trigger has many rows) then flow designer job might not run for some time.

Consider:

Lowering the priority of some jobs to a priority > 100, this will avoid contention with Flow Engine Event Handlers which also run at priority 100. Changing some Async BRs to Synch BRs (before or after record updates) Adding a flow job which runs at a priority less than 100. Consult with Dev-Flow Engines regarding this option 4. Next Action on Jobs : Verify that next_action on jobs in sys_trigger with filer nameSTARTSWITHflow engine event^parent!=NULL. This should not be way in past.

Consider

Consult with performance team and consider reseting next action as current on sys_trigger for flow jobs 5. There isn't enough Flow Engine Event Handler capacity: There was an old PRB which had a workaround to reduce the number of Flow Engine Event Handlers to 1 per node. This is not advised as if a single flow consumes thread capacity all other flows on that node are blocked. Consider adjusting com.glide.hub.flow_engine.event_handler.workers to 3 and then removing the property.

NOTE : From Quebec onwards, flow event get pinned on the node on which they got produced. So If due to any above reason node is not able to process the event, it will take com.glide.nowmq.events.max_ready_wait_seconds seconds (60 seconds from Xanadu) to delegate event from that node to another node.

see xml stats on how many events are getting processed on a particular node https:// /xmlstats.do?include=flowstats