With Batch Testing, customers can test their NLU models against a large set of utterances to evaluate the performance of the model.
Batch Testing can process several utterances without having to test each utterance individually.
NLU Workbench - Advanced Features 1.0 - Feb release
How to install?
Install the NLU Workbench - Advanced Features (com.snc.nlu.workbench.advanced) ServiceNow® Store plugin.
Known Issues: The following are the known issues in the February Release.
1. Description: Batch Testing will not work when user annotates Vocabulary Sources (for both Table and List) as Mapped entity and use the entity in their utterances.
Defect Number: DEF0194462
2. Description: Incorrect percentages are displayed in the batch test results header.
Defect Number: DEF0190809
3. Description: Incorrect page navigation for batch test results page.
The user will either displayed a blank page when navigated between pages of results or incorrect list.
This issue occurs when user visits page 2 which is after the display of 100 utterances in page 1.
Defect Number: DEF0185089
Work Around: In order to view more than 100 utterances without any problem, users will need to navigate nlu_batch_test_result table.
The filter criteria to look at the results of a particular test is by Test Execution Mapping to right Sys ID of a particular test run.
However, Sys ID is generated for every test run. In order to identify the right test run id, the tables are nlu_batch_test_run_definition and nlu_batch_test_run_execution needed.
Example: To find the results of restaurant_batch_test, the following steps are needed.
Go to nlu_batch_test_run_definition table and copy the sys_id by right clicking on Models.
Go to nlu_batch_test_run_execution and filter out based on Test Run Definition and paste the sys_id copied from nlu_batch_test_run_definitiond table.
Copy sys_id from the resulted list (sorted by latest date)
Go to nlu_batch_test_result table and filter out based on the sys_id copied.