<script custom-tag="" type="application/ld+json">{"@context":"https://schema.org","@type":"TechArticle","headline":"ATF Tests fails to report test result when excessive console logs are generated","image":"http://support.servicenow.com/29f1d2661baf6010cafa53d8624bcb43.iix","author":{"name":"ServiceNow Support","url":"http://support.servicenow.com/now"},"keywords":"ServiceNow, NowSupport public Knowledge Base articles, Known Error,KB0819732,","wordcount":"871","publisher":"ServiceNow","url":"https://support.servicenow.com/kb?id=kb_article_view&sysparm_article=KB0819732","datePublished":"2020-07-13","dateCreated":"2020-03-30","dateModified":"2024-05-23","description":"<h2>ATF Tests fails to report test result when excessive console logs are generated</h2><br/><div style=\"overflow-x:auto\"><h2>Description</h2><div><p>ATF Tests fails to report test result when excessi","articleBody":"<h2>ATF Tests fails to report test result when excessive console logs are generated</h2><br/><div style=\"overflow-x:auto\"><h2>Description</h2><div><p>ATF Tests fails to report test result when excessive console logs are generated</p></div><h2>Steps to Reproduce</h2><div> <p></p><p>1) Navigate to the Automated Test Framework (ATF) &gt; Properties module and enable the following checkboxes:<br />- Under &#34;Test/Test Suite Properties&#34;: &#34;Enable test/test suite execution&#34;<br />- Under &#34;Test Debugging Properties&#34;: &#34;Enable additional debugging&#34;<br />2) Click &#34;Save&#34;.<br />3) Navigate to the &#34;System UI &gt; UI Pages&#34; module and click &#34;New&#34;.<br />4) Input any name. Copy the following in the &#34;HTML&#34; field:<br /><span style=\"font-family: &#39;courier new&#39;, courier;\">&lt;?xml version&#61;&#34;1.0&#34; encoding&#61;&#34;utf-8&#34; ?&gt;</span><br /><span style=\"font-family: &#39;courier new&#39;, courier;\">&lt;j:jelly trim&#61;&#34;false&#34; xmlns:j&#61;&#34;jelly:core&#34; xmlns:g&#61;&#34;glide&#34; xmlns:j2&#61;&#34;null&#34; xmlns:g2&#61;&#34;null&#34;&gt;</span><br /><span style=\"font-family: &#39;courier new&#39;, courier;\">Hello ATF!</span><br /><span style=\"font-family: &#39;courier new&#39;, courier;\">&lt;/j:jelly&gt;</span><br />5) Copy the following in the &#34;Client script&#34; field:<br /><span style=\"font-family: &#39;courier new&#39;, courier;\">// Wait mechanism ignores setTimeouts with matching stack traces so call this recursively to &#34;trick&#34; it into thinking they are different</span><br /><span style=\"font-family: &#39;courier new&#39;, courier;\">function foo(i) {</span><br /><span style=\"font-family: &#39;courier new&#39;, courier;\">if (i &lt; 200) {</span><br /><span style=\"font-family: &#39;courier new&#39;, courier;\">setTimeout(function() {</span><br /><span style=\"font-family: &#39;courier new&#39;, courier;\">// Intentionally blank</span><br /><span style=\"font-family: &#39;courier new&#39;, courier;\">}, 1000);</span><br /><span style=\"font-family: &#39;courier new&#39;, courier;\">foo(i &#43; 1);</span><br /><span style=\"font-family: &#39;courier new&#39;, courier;\">}</span><br /><span style=\"font-family: &#39;courier new&#39;, courier;\">}</span><br /><span style=\"font-family: &#39;courier new&#39;, courier;\">foo(0);</span><br />6) Click &#34;Submit&#34;.<br />7) Navigate to the &#34;Automated Test Framework (ATF) &gt; Tests&#34; module and click &#34;New&#34;.<br />8) Input any value for &#34;Name&#34; and press &#34;Save&#34;.<br />9) Click &#34;Add Test Step&#34;, navigate to &#34;Form&#34; category and select &#34;Open an Existing Record&#34;.<br />10) Set Table to &#34;UI Page [sys_ui_page]&#34; and Record to the UI page created in Steps 1-5 and click &#34;Submit&#34;.<br />11) Click &#34;Add Test Step&#34;, navigate to &#34;Form&#34; category and select &#34;Click a UI Action&#34;.<br />12) Set Table to &#34;UI Page [sys_ui_page]&#34; and UI action to &#34;Try It&#34;, then click &#34;Submit&#34;.<br />13) Click &#34;Add Test Step&#34;, navigate to &#34;Custom UI&#34; category and select &#34;Assert Text on Page (Custom UI)&#34;.<br />14) Set &#34;Text&#34; to &#34;Hello ATF!&#34; and &#34;Assert type&#34; to &#34;Text is on page&#34;.<br />15) Click &#34;Run Test&#34;, select &#34;Start a new test runner&#34; then click &#34;Run Test&#34;.<br /><br />Observe the test seems to run in the client test runner, but gets stuck with a full progress bar at &#34;Finished test execution, reporting result&#34;.<br />In the browser console, the message &#34;ReportUITestProgress.reportBatchResult: failed X time(s) to report batch result, http status 0, trying again&#34; is reported repeatedly until the batch times out after 10 minutes.<br />The test eventually ends in failure with output &#34;The test timed out because the max execution time (600 seconds) was reached while executing steps 1 - 3&#34;.</p></div><h2>Workaround</h2><div><p><strong>Terminology</strong></p>\r\n<p>- <strong>UI batch</strong>: A sequence of continuous <strong>UI</strong> Step types, or Step Configurations where its Step environment is &#34;UI&#34;. Example <strong>UI</strong> steps: <strong>Form &gt; Set Field Values</strong>, <strong>Service Catalog &gt; Order catalog Item</strong>. To break up a batch, use a <a title=\"server steps\" href=\"https://docs.servicenow.com/csh?topicname&#61;server-test-steps.html&amp;version&#61;latest\" rel=\"nofollow\">server step</a> in between two <strong>UI</strong> steps.</p>\r\n<p>- <strong>Batch order constraint</strong>: Step Configurations have a column that defines where in a UI or Server batch they can be run. Steps with batch order constraint &#34;Start a batch execution&#34; tend to navigate to a page without any dependency on the prior state of the browser, other than that a Client Test Runner is available to run that UI batch.</p>\r\n<p><br /></p>\r\n<p>The goal of this workaround is to decrease the amount of console logs and reporting data being sent from the Client Test Runner back to the server in a single AJAX call. The below solutions attempt to decrease the data being generated per test UI batch.</p>\r\n<p>There are two recommendations:</p>\r\n<p>1) Disable ATF Debugging</p>\r\n<p>Disable Debugging to decrease the number of console logs that the Client Test Runner reports back to the server:</p>\r\n<p>Directions</p>\r\n<ol style=\"list-style-type: upper-alpha; list-style-position: inside;\"><li>Navigate to the <strong>Automated Test Framework (ATF) &gt; Properties</strong> module</li><li>Under &#34;Test Debugging Properties&#34; -&gt; &#34;Enables additional debugging&#34; -&gt; uncheck the checkbox, click &#34;Save&#34;.</li><li>Try running problematic tests again. If they continuously report success without issue, you can skip other workaround recommendations.</li></ol>\r\n<p><br /></p>\r\n<p>2) Break up problematic tests that have long continuous UI batches so that they send less data back to the server in a single reported test progress AJAX call.</p>\r\n<p>This can be accomplished by adding no-operation <strong>Server</strong> environment steps in the middle of long sequences of <strong>UI</strong> environment steps.</p>\r\n<p>The <strong>UI</strong> steps that you can add a <strong>Server</strong> step before, without interrupting a dependent sequence of UI behavior, are any that have a <strong>Step configuration</strong> &gt; <strong>Batch order constraint</strong> of &#34;Start a batch execution&#34;. To see a list of these, navigate to <strong>Automated Test Framework (ATF) &gt; Administration &gt; Step Configurations</strong>, then filter by <strong>Batch order constraint</strong> is &#34;Start a batch execution&#34; and <strong>Step environment</strong> is &#34;UI&#34;.</p>\r\n<p>Directions for tests that demonstrate this defect:</p>\r\n<ol style=\"list-style-type: upper-alpha; list-style-position: inside;\"><li>Find continuous UI steps in the test, for example: more than 10 steps in a row. Break them up where possible with a <strong>Server</strong> step <strong>Server &gt; Run Server Side Script</strong> but with no embedded script. This <strong>Server</strong> step can be placed before any <strong>UI</strong> step with a <strong>Batch Order Constraint</strong> fo &#34;Start a batch execution&#34;.</li><li>Run the test and confirm it still passes without introducing a breaking change due to a dependency between the two steps that were separated by the <strong>Server</strong> step. Confirm the test passes.</li><li>repeat for all problematic tests</li></ol>\r\n<p><br /></p></div><div><br /><strong>Related Problem: PRB1388207</strong></div></div>"}</script>
  Loading...