Outbound Web Services — Timeouts & Slow Responses<!-- /*NS Branding Styles*/ --> .ns-kb-css-body-editor-container { p { font-size: 12pt; font-family: Lato; color: var(--now-color--text-primary, #000000); } span { font-size: 12pt; font-family: Lato; color: var(--now-color--text-primary, #000000); } h2 { font-size: 24pt; font-family: Lato; color: var(--now-color--text-primary, black); } h3 { font-size: 18pt; font-family: Lato; color: var(--now-color--text-primary, black); } h4 { font-size: 14pt; font-family: Lato; color: var(--now-color--text-primary, black); } a { font-size: 12pt; font-family: Lato; color: var(--now-color--link-primary, #00718F); } a:hover { font-size: 12pt; color: var(--now-color--link-primary, #024F69); } a:target { font-size: 12pt; color: var(--now-color--link-primary, #032D42); } a:visited { font-size: 12pt; color: var(--now-color--link-primary, #00718f); } ul { font-size: 12pt; font-family: Lato; } li { font-size: 12pt; font-family: Lato; } img { display: ; max-width: ; width: ; height: ; } } Contents 1. Issue2. Timeout Layers — What Can Fire and When3. execute() vs executeAsync() via MID Server4. Key Relationships Between Timeout Layers5. Root Causes 5.1 ECC Queue 30-second timeout — synchronous MID Server call5.2 MID Server 175-second probe timeout5.3 Timeout properties — common misconfigurations5.4 Transaction quota (REST/JSON catch-all) exhausted5.5 Endpoint slow — high Response Time but call eventually succeeds5.6 Large payload timeouts5.7 Intermittent timeouts at predictable times5.8 504 Gateway Timeout from endpoint or proxy 6. MID Server — Additional Diagnostics7. Diagnostic Steps8. Resolution Summary9. Related Articles 1. Issue An outbound REST or SOAP call from ServiceNow does not complete within the expected time window. The failure may appear as an HTTP -1 with no response body, an explicit timeout error, a 504 from the endpoint, or a call that eventually succeeds but takes far longer than expected. Symptoms this article covers: Response status is HTTP -1 in sys_outbound_http_log with no response body.Error message contains: No response for ECC message request after waiting for 30 seconds.Error message contains: com.glide.ecc.ECCResponseTimeoutException.Error message contains: SocketTimeoutException: Read timed out or ReadTimeoutError.Error message contains: Transaction cancelled: quota rule exceeded.The call succeeds but the Response Time field in sys_outbound_http_log is unexpectedly high.Endpoint returns HTTP 504 Gateway Timeout.Timeouts occur intermittently at predictable times (early morning, batch windows). Connect timeout vs. read timeout — check this first A ConnectTimeoutException means the TCP connection never established — the endpoint was unreachable before any data exchange. That is a connection failure, not the timeout pattern covered here. See KB3067060 — Outbound Web Services — Connection Failures & TLS Errors. This article covers read timeouts and ECC timeouts — where connectivity was established but the endpoint or MID Server did not respond in time. 2. Timeout Layers — What Can Fire and When Multiple independent timeout limits apply to every outbound call. They operate at different layers and any one of them can terminate the call. Identifying which layer fired is the essential first step. Timeout properties reference The following system properties control timeout behaviour for outbound calls. All can be reviewed at Filter Navigator → sys_properties.list. Property Default Units What it controls glide.http.connection_timeout 10000 milliseconds Max time to establish a TCP connection. Overridable per request with setHttpTimeout(ms). glide.http.timeout 175000 milliseconds Socket (read) timeout — max time to wait for data on an established connection. Applies on both instance side (direct) and MID Server side (via MID). Overridable per request with setHttpTimeout(ms). glide.http.outbound.max_timeout.enabled true boolean When true, enforces the 30-second cap on ECC response waiting (both direct executeAsync() and MID Server execute()). When false, uses glide.rest.outbound.ecc_response.timeout instead. glide.http.outbound.max_timeout 30 seconds Maximum ECC response wait time when glide.http.outbound.max_timeout.enabled=true. Hard upper bound of 30 seconds — cannot be set higher. waitForResponse() calls cannot override this cap. glide.rest.outbound.ecc_response.timeout 300 seconds ECC response timeout when glide.http.outbound.max_timeout.enabled=false. Only active when the max_timeout property is disabled. glide.soap.outbound.ecc_response.timeout 300 seconds Same as above, specifically for SOAP outbound calls using executeAsync(). 3. execute() vs executeAsync() via MID Server When outbound calls route through a MID Server, the choice between execute() and executeAsync() determines which timeout limits apply and how failures surface. The 30-second ECC cap only fires when execute() is used, or when executeAsync() + waitForResponse() is used — both of which block the initiating thread. 4. Key Relationships Between Timeout Layers Understanding how the layers interact prevents misdiagnosis and wasted effort on the wrong fix. ECC 30s (instance) vs MID 175s (MID Server): These are independent. When execute() is used via MID Server, the instance stops waiting at 30 seconds and throws the ECC timeout error — but the MID Server continues processing the probe for up to 175 seconds. The call may succeed on the MID Server side and a response may arrive in the ECC Queue Input after the instance has already moved on. Check the ECC Queue for a completed Input probe record even when the instance threw a timeout error. Transaction quota wraps everything: The 300-second transaction quota is an outer boundary that applies regardless of which HTTP timeout is configured. A script that spends 250 seconds on database queries before making a 60-second outbound call will hit the 300-second quota during the HTTP call, even though neither the connection timeout nor the socket timeout has fired. Check for quota cancellation in the system log before concluding the HTTP timeout is the cause. executeAsync() + waitForResponse() is not actually asynchronous: Calling waitForResponse() after executeAsync() re-blocks the initiating thread, making the call effectively synchronous again — but with the overhead of a scheduled job and the same 30-second ECC cap. This pattern freezes two threads instead of one. Use execute() directly if you need an inline response. See KB0694711 for the full analysis. glide.http.outbound.max_timeout is in seconds, not milliseconds: Unlike most timeout properties which are in milliseconds, this property takes seconds. The default value of 30 means 30 seconds. Its hard upper bound is 30 seconds — setting it higher has no effect while glide.http.outbound.max_timeout.enabled=true. 5. Root Causes 5.1 ECC Queue 30-second timeout — synchronous MID Server call The most common MID Server timeout pattern in case data. When execute() is called on a message configured to use a MID Server, the instance thread sleeps and polls the ECC Queue for a response. If 30 seconds pass with no Input probe arriving, the instance throws: Error: REST call failed: No response for ECC message request with sysid=<guid> after waiting for 30 seconds in ECC Queuecom.glide.ecc.ECCResponseTimeoutException: No response for ECC message request The 30-second cap is enforced by glide.http.outbound.max_timeout and cannot be raised above 30 seconds on cloud instances. Common causes: The MID Server is down, not validated, or not polling the ECC Queue. Check MID Server Status at Filter Navigator → MID Server → MID Servers.The endpoint takes more than 30 seconds to respond, so the MID Server is still waiting when the instance gives up.The MID Server thread pool is saturated processing other probes, causing this probe to queue behind them. Only a certain number of probes are fetched at a time, proportional to the MID Server thread count. Check for a late-arriving Input probe After the instance throws the 30-second error, navigate to ecc_queue.list and filter for Input records for this MID Server around the time of the failure. If a Processed Input record exists, the MID Server completed the call after the instance gave up. The call succeeded on the endpoint side — the fix is to switch to executeAsync(), not to investigate the endpoint. Resolution: Confirm the MID Server is Up and validated.Check the ECC Queue Output probe State. Ready means the MID Server never picked it up. Processing means the MID Server has it but has not finished. Processed means it completed (check the Input record for the response).If the MID Server is healthy but the endpoint takes more than 30 seconds, switch to executeAsync() with setEccCorrelator() + a business rule on ecc_queue to handle the response. See KB0694711 for the pattern.If the MID Server thread pool is saturated, review the MID Server thread count configuration and consider adding a second MID Server instance. 5.2 MID Server 175-second probe timeout The MID Server's socket timeout for outbound HTTP calls is controlled by the same glide.http.timeout property that governs the instance-side socket timeout, but it is evaluated on the MID Server JVM, not the instance. The default is 175,000 milliseconds (175 seconds). When this fires, the MID Server inserts an error response into the ECC Queue Input, which the instance surfaces as an HTTP -1 or read timeout error. This is distinct from the ECC 30-second timeout — by the time the MID 175s timeout fires, the instance is no longer waiting (the 30-second ECC cap would already have fired if using execute()). This timeout only matters in practice for executeAsync() flows, where the instance is not blocking. Resolution: Confirm the endpoint response time by testing from the MID Server host: curl -w "%{time_total}" -o /dev/null -s <endpoint_url>.If the endpoint genuinely needs more than 175 seconds, restructure to a fire-and-forget pattern where the endpoint calls back to ServiceNow via inbound REST when its processing completes.Check the MID Server agent log (agent0.log) for SocketTimeoutException entries referencing the probe Sys ID to confirm this is the layer that fired. 5.3 Timeout properties — common misconfigurations Several timeout property misconfigurations generate recurring cases. glide.http.outbound.max_timeout appears to have no effect: When glide.http.outbound.max_timeout.enabled=true (the default), the max timeout is hard-capped at 30 seconds regardless of any value set. Calling waitForResponse(60000) does not extend the wait beyond 30 seconds while this property is enabled. To allow longer waits, set glide.http.outbound.max_timeout.enabled=false and use glide.rest.outbound.ecc_response.timeout to set the desired limit — but note that this allows threads to block for much longer, which can exhaust the instance semaphore pool. glide.http.timeout vs glide.http.connection_timeout confusion: Both are in milliseconds. glide.http.connection_timeout (default 10,000ms) is how long to wait to establish the TCP connection. glide.http.timeout (default 175,000ms) is how long to wait for data once connected. A slow endpoint that connects quickly but responds slowly exhausts glide.http.timeout, not the connection timeout. Both can be overridden per request using setHttpTimeout(ms) on the RESTMessageV2 or SOAPMessageV2 object. Transaction quota fires before the HTTP timeout: If the total transaction duration (script + HTTP call) exceeds the transaction quota (300 seconds by default) before the HTTP timeout fires, the quota cancels the transaction. This appears as a timeout but is not an HTTP timeout — it is a server-side quota enforcement. Check the system log for Transaction cancelled: quota rule exceeded to distinguish this from an HTTP timeout. 5.4 Transaction quota (REST/JSON catch-all) exhausted The transaction quota under System Definition → Quota Rules applies to the entire transaction context. The default REST/JSON catch-all quota is 300 seconds. This quota wraps everything: script execution before the call, the HTTP call itself, and any processing after the response returns. Error indicator: Transaction cancelled: quota rule exceeded in the system log. Note that the outbound call may show HTTP -1 in sys_outbound_http_log with no obvious error — the cancellation event is in the system log, not the outbound log. Resolution: Search the system log (syslog.list) around the time of the failure for the exact quota rule name that fired.Optimise the script to reduce total transaction time. Move heavy queries outside the outbound call flow or break work into smaller asynchronous chunks.For legitimate cases where the quota needs to be extended, open a Support case with the quota rule name and a description of the business requirement. Quota increases on cloud instances require ServiceNow review — they are not self-service. 5.5 Endpoint slow — high Response Time but call eventually succeeds When the call succeeds but Response Time in sys_outbound_http_log is high, the endpoint is processing slowly. No ServiceNow timeout has fired — ServiceNow is waiting. The impact: slow outbound calls hold API_INT semaphore slots, reducing concurrent capacity for other inbound and outbound calls on the same node. Resolution: Confirm the pattern is endpoint-side by testing the same call from Postman or curl. If also slow externally, engage the endpoint team.Check whether the slowness correlates with specific time windows. If so, shift the integration calls to off-peak hours.Switch to executeAsync() without waitForResponse() to avoid blocking the initiating thread during slow endpoint periods. See KB0694711 for implementation patterns. 5.6 Large payload timeouts Large request or response payloads are disproportionately affected by timeouts. They take longer to transmit and can exhaust the transaction quota even when the endpoint is healthy. Common patterns from case data: Outbound REST calls uploading attachments or files above 10MB timing out before the upload completes.Outbound calls with large response bodies that the MID Server must buffer and return via ECC Queue. The ECC Queue payload size limit (mid.eccq.max_payload_size) may cause the MID Server to write the payload as a file in the output_oversize folder rather than send it to the instance.MID Server JVM running low on memory when processing very large files, causing probe processing to slow dramatically. Resolution: For large file uploads, split the payload if the endpoint supports chunked or multipart uploads.For large response bodies, use pagination at the endpoint to retrieve data in smaller pages.Check the MID Server output_oversize folder at <MID_HOME>/agent/work/monitors/ECCSender/output_oversize for payloads that exceeded the size limit and were not sent to the instance.If the MID Server is memory-constrained, review the JVM heap allocation (-Xmx in the MID Server wrapper configuration). 5.7 Intermittent timeouts at predictable times Timeouts that occur reliably at specific times of day are almost always caused by endpoint-side or network-side events, not ServiceNow configuration. Common causes: The endpoint runs batch jobs or maintenance at predictable intervals.A network device enforces connection limits or resets connections during its own maintenance.The ServiceNow instance is running resource-intensive scheduled jobs at the same time, which slows the MID Server's ECC Queue polling (the MID Server queries the instance SOAP Table API to fetch jobs — a busy instance responds more slowly). Resolution: Correlate the failure times with known endpoint maintenance windows. If they align, stagger the integration schedule or implement retry logic.Implement client-side retry with exponential backoff. Transient endpoint-side timeouts typically resolve on retry within seconds.Check the ServiceNow instance scheduled job log for heavy jobs running at the same time as the timeouts. 5.8 504 Gateway Timeout from endpoint or proxy HTTP 504 is issued by a gateway or proxy sitting in front of the endpoint when the endpoint fails to respond within the gateway's own timeout window. A 504 confirms that connectivity to the gateway was successful — the failure is endpoint processing speed or gateway timeout configuration, not ServiceNow. Resolution: Check response headers for a Server or Via header identifying the gateway. This confirms the 504 is gateway-issued, not endpoint-issued.Engage the endpoint team to investigate why the endpoint is not responding within the gateway timeout.If the gateway timeout is configurable and the endpoint legitimately needs more time, request an increase from the endpoint team. 6. MID Server — Additional Diagnostics For MID Server timeout failures, the ECC Queue record and MID Server agent log together tell the complete story. ECC Queue probe states Navigate to ecc_queue.list, filter by Agent = your MID Server and Topic contains REST or SOAP. The Output probe State tells you exactly where in the pipeline the failure occurred: Output State What it means Next step Ready MID Server never picked up the probe Check MID Server status and polling. See Section 5.1. Processing MID Server picked it up and is working on it (or it is in the MID Server's internal queue) Check MID Server agent log for Worker starting and Worker completed entries with the probe Sys ID. Processed MID Server completed the call. Check the Input probe for the response. If instance threw a timeout but Output is Processed, the MID Server succeeded after the instance gave up. Switch to executeAsync(). Error MID Server encountered an error executing the probe Check the Error String field on the Input probe. Check MID Server agent log. MID Server agent log Search <MID_HOME>/logs/agent0.log for the probe Sys ID (from the ECC Queue Output record). Key log entries: Worker starting: RESTProbe source: <endpoint_url> — probe execution beganWorker completed: RESTProbe source: <endpoint_url> time: H:MM:SS.mmm — execution finished; time: shows how long the HTTP call tookECCSender.1 Sending ecc_queue.<sysid>.xml — result being sent back to instanceSocketTimeoutException — MID Server hit the 175s read timeout Note — ECC Queue backlog A single slow endpoint call can back up subsequent probes. The MID Server holds probes in an internal queue until a worker thread is free. When a slow probe monopolises a worker thread, probes arriving behind it wait. This causes the 30-second ECC instance timeout to fire on queued probes that have not yet started executing. If you see multiple simultaneous ECC timeout errors, check the MID Server agent log for Worker starting timestamps to confirm whether probes were delayed in the internal queue rather than being slow at the endpoint. 7. Diagnostic Steps Check the Response Time field in sys_outbound_http_log. Non-zero high value with success status = endpoint slow (Section 5.5). Zero or null with failure = timeout fired before any response.Read the error message in the Response tab. Match it to the root causes in Section 5 and the table in Section 8.Check the system log (syslog.list) for Transaction cancelled: quota rule exceeded. If present, the quota fired — not an HTTP timeout. See Section 5.4.For MID Server calls: check the ECC Queue Output probe State (see Section 6). If State = Processed despite the instance throwing a timeout, the MID Server completed successfully after the 30-second cap — switch to executeAsync().For direct calls: test from Postman or curl. If also slow externally, the endpoint is the bottleneck. If fast externally, check the timeout property configuration (Section 5.3).Check for time-correlation. Failures at predictable times = endpoint or network maintenance window. See Section 5.7.Before opening a case, capture: Full error message from the Response tab in sys_outbound_http_log.Response Time value for failing and succeeding calls.Whether the call uses execute() or executeAsync().ECC Queue Output probe State and Sys ID if via MID Server.MID Server agent log excerpt showing Worker starting / Worker completed for the probe.Endpoint response time from Postman or curl on the MID Server host.Time pattern of failures. 8. Resolution Summary Error / symptom Root cause Fix No response for ECC message request after waiting for 30 seconds MID Server not polling, overloaded, or endpoint > 30s Check MID Server health and ECC Queue backlog; switch to executeAsync() + setEccCorrelator() if endpoint is slow ECC Output = Processed but instance threw 30s error Endpoint responded after the 30-second cap fired Switch to executeAsync() — endpoint is working correctly MID Server timeout; SocketTimeoutException in agent log Endpoint response > 175,000ms (MID socket timeout) Optimise endpoint; use callback/fire-and-forget pattern glide.http.outbound.max_timeout appears to have no effect Hard upper bound of 30s enforced; property units are seconds not ms Set max_timeout.enabled=false to use ecc_response.timeout instead; understand thread blocking implications Transaction cancelled: quota rule exceeded Total transaction time exceeded quota (default 300s) Optimise surrounding script; open Support case for quota increase if needed Call succeeds; Response Time very high Endpoint slow; proxy latency; peak load Confirm with Postman; shift to off-peak; switch to executeAsync() Timeouts on large file uploads or responses Payload exceeding HTTP client or MID Server limits Chunk uploads; paginate reads; check output_oversize folder; increase MID JVM heap Timeouts at predictable times Endpoint or network maintenance window Stagger schedule; implement retry with exponential backoff HTTP 504 from endpoint Gateway/proxy timed out waiting for endpoint Engage endpoint team; check gateway timeout configuration 9. Related Articles KB0659194 - Outbound Web Services — Troubleshooting GuideKB3067060 - Outbound Web Services — Connection Failures & TLS ErrorsKB3067106 - Outbound Web Services — Authentication & Authorization FailuresKB3067183 - Outbound Web Services — Endpoint Errors (4xx/5xx)KB0694711 — RESTMessageV2 and SOAPMessageV2 execute() vs executeAsync() best practicesKB0855595 — How the ECC Queue table records get processed: from output ready to input processedKB0960404 — MID Server Landing Page