[NLU] How to determine the version of the NLU Platform used to train your NLU Model and debugging NLU predictions on a single NLU Model using the Test button in the NLU Workbench. - Known Error

[NLU] How to determine the version of the NLU Platform used to train your NLU Model and debugging NLU predictions on a single NLU Model using the Test button in the NLU Workbench. Description In Paris and earlier versions , there is no simple way to determine the NLU version used for training the NLU Model nor when testing predictions in Studio with the trained NLU Model. Hence you need to enable all debugging on the instance, open the Script Debugger and test some utterances on the NLU Model to determine the NLU Version used for both training and making predictions on the NLU Model.

In Quebec , we introduced the capability and capability version fields on table [ml_solution] , which makes it easier to determine the NLU Platform version, however the version numbering is different compared to the "internal" version that is used in the debug logs.

Steps to Reproduce

1. Submit your custom NLU Model for training 2. Once the NLU Model is trained, there is no way to easily determine which NLU Version was used to train the NLU Model.

Workaround The NLU version is shown in Paris and earlier versions , when you open the Script Debugger in a new browser window, click on the Session Log tab, then click on Settings and you only need to enable Log to see the NLU debug output showing the NLU Platform version when testing utterances, as follows -

IMPORTANT NOTE:

To determine the correct NLU Platform version , you must debug when testing utterances after the NLU Model training has completed. Do not debug the NLU Model training , as this will not contain the correct version number.

When testing utterances in your NLU Model using NLU Workbench or the NLU Model Builder in Studio , using the Test button, within the Script Debugger window, it has the following output under the transaction for / api/now/v1/nlu/test -

[DEBUG] NLU Model predict response : {"status":"success","response":{"utterance":"I need help with changing my password","intents":[{"intentName":"GetPasswordResetLink","nluModelName": "sn_itsm_nlu_ITSM_NLU_for_VA","score":0.8976299,"intents":[]}],"properties":{"all: sn_itsm_nlu_ITSM_NLU_for_VA":"0.65","entity:all":"0.01","inference.time":"225","intent: all":"0.01","nluPlatformLanguage":"en","nluPlatformVersion":"3.1.2-HYB"}}}

However, the NLU Model is not always available in Studio , usually because it was installed via a Scoped Application. When using the Virtual Agent Designer to test the NLU Model, the Script Debugger does not show the NLU Platform Version in the debug output. Therefore, the best way to find the ML/NLU Engine version is to use the System Logs > Utilities > Node Log File Browser module using the following steps -

Important Note: All log timestamps are in the PST timezone, so when searching the logs, please use PST timestamps.

Train and publish an NLU Model Open Collaboration > Virtual Agent > Designer and click on the Test Topics button You can test many utterances and make a note of the time of testing using the PST timezone Once you have completed your utterance testing, open System Logs > Utilities > Node Log File Browser Change the Start time and End time fields to match the time of your utterance testing using the PST timezone. Add "nluPlatformVersion" In the Message field and click on the Submit button to see the Prediction Server log output.

NLU Platform versions: The instance family version can use multiple NLU Platform versions. For example, an instance on Paris can use either Version 2.1.2-HYB [Paris] or Version 3.1.2-HYB [Quebec] of the NLU Platform. Please review KB0858445 to determine how to set the NLU Platform version to be used for your NLU Model.

When using the Script Debugging or the Node Log File Browser, the internal version numbers are as follows -

Version 1.0 [Orlando] Version 2.1.2-HYB [Paris] Version 3.1.2-HYB [Quebec] Version rome.0 [Rome] Version sandiego.0 [San Diego] Version tokyo.0 [Tokyo] Version utah.0 [Utah] Version vancouver.0 [Vancouver] Please note that version numbers are different between the debug logs and the new capability and capability version fields on table [ml_solution] introduced with Quebec . The capability version field shows the external version numbers, which is used in the Fixed In section on a PRB where we released a fix to the NLU Platform, as follows -

Version 6.0 [Paris] Version 7.0 [Quebec] Version 20.0 [Rome] Version 21.0 [San Diego] Version 22.0 [Tokyo] Version 23.0 [Utah] Version 24.0 [Vancouver] In Quebec and onwards , once the NLU Model has been trained successfully, you can open the List View for table [ml_solution] and add the capability and capability version fields to the List View to show the version of the NLU Platform, the NLU Models were trained on.