Events not being processed and stuck in ready state. - Support and Troubleshooting

Events not being processed and stuck in ready state. Issue  Symptoms Issue: Events stuck in ready state and the scheduled jobs are stuck waiting for a response. The stack trace is as follows waiting for the RestResponse.

main,glide.scheduler.worker.0,4,Event Management - process events - 2 (18760340 ms) java.lang.Thread.sleep(Native Method) com.glide.ecc.ECCResponsePoller.poll(ECCResponsePoller.java:50) com.glide.rest.outbound.ecc.ECCRESTResponse.fetchAndProcessEccResponse(ECCRESTResponse.java:232) com.glide.rest.outbound.ecc.ECCRESTResponse.getBody(ECCRESTResponse.java:124) com.glide.rest.outbound.scriptable.ScriptableRESTResponse.jsFunction_getBody(ScriptableRESTResponse.java:82) sun.reflect.GeneratedMethodAccessor695.invoke(Unknown Source) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) java.lang.reflect.Method.invoke(Method.java:498) org.mozilla.javascript.MemberBox.invoke(MemberBox.java:138) org.mozilla.javascript.FunctionObject.doInvoke(FunctionObject.java:670) org.mozilla.javascript.FunctionObject.call(FunctionObject.java:614) org.mozilla.javascript.ScriptRuntime.doCall(ScriptRuntime.java:2582) org.mozilla.javascript.optimizer.OptRuntime.callProp0(OptRuntime.java:85) org.mozilla.javascript.gen.sys_script_2cc85296dbe9b3c087aa34f668961901_script_561._c_executeRule_1(sys_script.2cc85296dbe9b3c087aa34f668961901.script:12) org.mozilla.javascript.gen.sys_script_2cc85296dbe9b3c087aa34f668961901_script_561.call(sys_script.2cc85296dbe9b3c087aa34f668961901.script) org.mozilla.javascript.ScriptRuntime.doCall2(ScriptRuntime.java:2650) org.mozilla.javascript.ScriptRuntime.doCall(ScriptRuntime.java:2590) org.mozilla.javascript.optimizer.OptRuntime.call2(OptRuntime.java:42) org.mozilla.javascript.gen.sys_script_2cc85296dbe9b3c087aa34f668961901_script_561._c_script_0(sys_script.2cc85296dbe9b3c087aa34f668961901.script:1) org.mozilla.javascript.gen.sys_script_2cc85296dbe9b3c087aa34f668961901_script_561.call(sys_script.2cc85296dbe9b3c087aa34f668961901.script) org.mozilla.javascript.gen.sys_script_2cc85296dbe9b3c087aa34f668961901_script_561.exec(sys_script.2cc85296dbe9b3c087aa34f668961901.script) com.glide.script.ScriptEvaluator.execute(ScriptEvaluator.java:279) com.glide.script.ScriptEvaluator.evaluateString(ScriptEvaluator.java:118) com.glide.script.ScriptEvaluator.evaluateString(ScriptEvaluator.java:82) com.glide.script.fencing.GlideScopedEvaluator.evaluateScript(GlideScopedEvaluator.java:309) com.glide.script.fencing.GlideScopedEvaluator.evaluateScript(GlideScopedEvaluator.java:214) com.glide.script.fencing.GlideScopedEvaluator.evaluateScript(GlideScopedEvaluator.java:201) com.glide.script.BusinessRule.runScript(BusinessRule.java:403) com.glide.script.BusinessRule.runSyncBusinessRule(BusinessRule.java:293) com.glide.script.BusinessRule.runBusinessRule(BusinessRule.java:204) com.glide.script.BusinessRule.run(BusinessRule.java:180) com.glide.script.GlideRecordBusinessRules.run(GlideRecordBusinessRules.java:95) com.glide.script.GlideRecord.runScripts(GlideRecord.java:1348) com.glide.script.GlideRecord.runScriptsAndEngines(GlideRecord.java:1282) com.glide.script.GlideRecord.updateWithoutResettingOperation(GlideRecord.java:4437) com.glide.script.GlideRecord.update(GlideRecord.java:4365) com.glide.script.GlideRecord.update(GlideRecord.java:4284) com.glideapp.itom.snac.common.AlertState.updateAlert(AlertState.java:613) com.glideapp.itom.snac.common.AlertState$2.execute(AlertState.java:116) com.glideapp.itom.snac.processor.EventTransformationHandler.createOrUpdateAlert(EventTransformationHandler.java:119)

Release All releases

Cause This can be caused if there is an insert/update business rules that sends out a REST call to an endpoint. As everything is done as part of a same schedule worker, it will wait until the response for this REST call is returned. This will cause backlog of Jobs and add additional delay in processing events. Resolution It is not recommended to have business rule on em_event table as it adds additional delay. If you are writing a business rule to send a REST call, make sure it is an ASYNC business rules so that a new sys_trigger record is created. Make sure you define a time out on the REST call so that there is no delay and scheduled workers are freed up