<script custom-tag="" type="application/ld+json">{"@context":"https://schema.org","@type":"TechArticle","headline":"ATF Tests fails to report test result when excessive console logs are generated","image":"http://support.servicenow.com/29f1d2661baf6010cafa53d8624bcb43.iix","author":{"name":"ServiceNow Support","url":"http://support.servicenow.com/now"},"keywords":"ServiceNow, NowSupport public Knowledge Base articles, Known Error,KB0819732,","wordcount":"871","publisher":"ServiceNow","url":"https://support.servicenow.com/kb?id=kb_article_view&sysparm_article=KB0819732","datePublished":"2020-07-13","dateCreated":"2020-03-30","dateModified":"2024-05-23","description":"<h2>ATF Tests fails to report test result when excessive console logs are generated</h2><br/><div style=\"overflow-x:auto\"><h2>Description</h2><div><p>ATF Tests fails to report test result when excessi","articleBody":"<h2>ATF Tests fails to report test result when excessive console logs are generated</h2><br/><div style=\"overflow-x:auto\"><h2>Description</h2><div><p>ATF Tests fails to report test result when excessive console logs are generated</p></div><h2>Steps to Reproduce</h2><div> <p></p><p>1) Navigate to the Automated Test Framework (ATF) > Properties module and enable the following checkboxes:<br />- Under "Test/Test Suite Properties": "Enable test/test suite execution"<br />- Under "Test Debugging Properties": "Enable additional debugging"<br />2) Click "Save".<br />3) Navigate to the "System UI > UI Pages" module and click "New".<br />4) Input any name. Copy the following in the "HTML" field:<br /><span style=\"font-family: 'courier new', courier;\"><?xml version="1.0" encoding="utf-8" ?></span><br /><span style=\"font-family: 'courier new', courier;\"><j:jelly trim="false" xmlns:j="jelly:core" xmlns:g="glide" xmlns:j2="null" xmlns:g2="null"></span><br /><span style=\"font-family: 'courier new', courier;\">Hello ATF!</span><br /><span style=\"font-family: 'courier new', courier;\"></j:jelly></span><br />5) Copy the following in the "Client script" field:<br /><span style=\"font-family: 'courier new', courier;\">// Wait mechanism ignores setTimeouts with matching stack traces so call this recursively to "trick" it into thinking they are different</span><br /><span style=\"font-family: 'courier new', courier;\">function foo(i) {</span><br /><span style=\"font-family: 'courier new', courier;\">if (i < 200) {</span><br /><span style=\"font-family: 'courier new', courier;\">setTimeout(function() {</span><br /><span style=\"font-family: 'courier new', courier;\">// Intentionally blank</span><br /><span style=\"font-family: 'courier new', courier;\">}, 1000);</span><br /><span style=\"font-family: 'courier new', courier;\">foo(i + 1);</span><br /><span style=\"font-family: 'courier new', courier;\">}</span><br /><span style=\"font-family: 'courier new', courier;\">}</span><br /><span style=\"font-family: 'courier new', courier;\">foo(0);</span><br />6) Click "Submit".<br />7) Navigate to the "Automated Test Framework (ATF) > Tests" module and click "New".<br />8) Input any value for "Name" and press "Save".<br />9) Click "Add Test Step", navigate to "Form" category and select "Open an Existing Record".<br />10) Set Table to "UI Page [sys_ui_page]" and Record to the UI page created in Steps 1-5 and click "Submit".<br />11) Click "Add Test Step", navigate to "Form" category and select "Click a UI Action".<br />12) Set Table to "UI Page [sys_ui_page]" and UI action to "Try It", then click "Submit".<br />13) Click "Add Test Step", navigate to "Custom UI" category and select "Assert Text on Page (Custom UI)".<br />14) Set "Text" to "Hello ATF!" and "Assert type" to "Text is on page".<br />15) Click "Run Test", select "Start a new test runner" then click "Run Test".<br /><br />Observe the test seems to run in the client test runner, but gets stuck with a full progress bar at "Finished test execution, reporting result".<br />In the browser console, the message "ReportUITestProgress.reportBatchResult: failed X time(s) to report batch result, http status 0, trying again" is reported repeatedly until the batch times out after 10 minutes.<br />The test eventually ends in failure with output "The test timed out because the max execution time (600 seconds) was reached while executing steps 1 - 3".</p></div><h2>Workaround</h2><div><p><strong>Terminology</strong></p>\r\n<p>- <strong>UI batch</strong>: A sequence of continuous <strong>UI</strong> Step types, or Step Configurations where its Step environment is "UI". Example <strong>UI</strong> steps: <strong>Form > Set Field Values</strong>, <strong>Service Catalog > Order catalog Item</strong>. To break up a batch, use a <a title=\"server steps\" href=\"https://docs.servicenow.com/csh?topicname=server-test-steps.html&version=latest\" rel=\"nofollow\">server step</a> in between two <strong>UI</strong> steps.</p>\r\n<p>- <strong>Batch order constraint</strong>: Step Configurations have a column that defines where in a UI or Server batch they can be run. Steps with batch order constraint "Start a batch execution" tend to navigate to a page without any dependency on the prior state of the browser, other than that a Client Test Runner is available to run that UI batch.</p>\r\n<p><br /></p>\r\n<p>The goal of this workaround is to decrease the amount of console logs and reporting data being sent from the Client Test Runner back to the server in a single AJAX call. The below solutions attempt to decrease the data being generated per test UI batch.</p>\r\n<p>There are two recommendations:</p>\r\n<p>1) Disable ATF Debugging</p>\r\n<p>Disable Debugging to decrease the number of console logs that the Client Test Runner reports back to the server:</p>\r\n<p>Directions</p>\r\n<ol style=\"list-style-type: upper-alpha; list-style-position: inside;\"><li>Navigate to the <strong>Automated Test Framework (ATF) > Properties</strong> module</li><li>Under "Test Debugging Properties" -> "Enables additional debugging" -> uncheck the checkbox, click "Save".</li><li>Try running problematic tests again. If they continuously report success without issue, you can skip other workaround recommendations.</li></ol>\r\n<p><br /></p>\r\n<p>2) Break up problematic tests that have long continuous UI batches so that they send less data back to the server in a single reported test progress AJAX call.</p>\r\n<p>This can be accomplished by adding no-operation <strong>Server</strong> environment steps in the middle of long sequences of <strong>UI</strong> environment steps.</p>\r\n<p>The <strong>UI</strong> steps that you can add a <strong>Server</strong> step before, without interrupting a dependent sequence of UI behavior, are any that have a <strong>Step configuration</strong> > <strong>Batch order constraint</strong> of "Start a batch execution". To see a list of these, navigate to <strong>Automated Test Framework (ATF) > Administration > Step Configurations</strong>, then filter by <strong>Batch order constraint</strong> is "Start a batch execution" and <strong>Step environment</strong> is "UI".</p>\r\n<p>Directions for tests that demonstrate this defect:</p>\r\n<ol style=\"list-style-type: upper-alpha; list-style-position: inside;\"><li>Find continuous UI steps in the test, for example: more than 10 steps in a row. Break them up where possible with a <strong>Server</strong> step <strong>Server > Run Server Side Script</strong> but with no embedded script. This <strong>Server</strong> step can be placed before any <strong>UI</strong> step with a <strong>Batch Order Constraint</strong> fo "Start a batch execution".</li><li>Run the test and confirm it still passes without introducing a breaking change due to a dependency between the two steps that were separated by the <strong>Server</strong> step. Confirm the test passes.</li><li>repeat for all problematic tests</li></ol>\r\n<p><br /></p></div><div><br /><strong>Related Problem: PRB1388207</strong></div></div>"}</script>