Known issues for Batch Testing Feature - NLU Workbench - Advanced Features 1.0(Feb release)


Feature:
Batch Testing

Description:
With Batch Testing, customers can test their NLU models against a large set of utterances to evaluate the performance of the model.
Batch Testing can process several utterances without having to test each utterance individually.

Release:
NLU Workbench - Advanced Features 1.0  - Feb release

How to install?
Install the NLU Workbench - Advanced Features (com.snc.nlu.workbench.advanced) ServiceNow® Store plugin.

Known Issues: The following are the known issues in the February Release. 

1. Description: Batch Testing will not work when user annotates Vocabulary Sources (for both Table  and List) as Mapped entity and use the entity in their utterances.
Defect Number: DEF0194462


2. Description: Incorrect percentages are displayed in the batch test results header. 
Defect Number: DEF0190809


3. Description: Incorrect page navigation for batch test results page.
The user will either displayed a blank page when navigated between pages of results or incorrect list.
This issue occurs when user visits page 2 which is after the display of 100 utterances in page 1. 

Defect Number: DEF0185089

Work Around: In order to view more than 100 utterances without any problem, users will need to navigate nlu_batch_test_result table. 


The filter criteria to look at the results of a particular test is by Test Execution Mapping to right Sys ID of a particular test run. 



However, Sys ID is generated for every test run. In order to identify the right test run id, the tables are nlu_batch_test_run_definition and nlu_batch_test_run_execution needed. 


Example: To find the results of restaurant_batch_test, the following steps are needed. 

Go to nlu_batch_test_run_definition table and copy the sys_id by right clicking on Models. 


Go to nlu_batch_test_run_execution and filter out based on Test Run Definition and paste the sys_id copied from nlu_batch_test_run_definitiond table.


Copy sys_id from the resulted list (sorted by latest date)

Go to nlu_batch_test_result table and filter out based on the sys_id copied. 



Known Limitations:

1. Description: Only 20 models are displayed in the drop-down for selecting the models to run the batch test.
    Defect Number: DEF0191743
    Work Around: Use the search box to select the all the models available.