Predictive Intelligence common issues causing poor Classification model performance that gives incorrect or low-quality predictions: How to analyze and improve the Classification predictions by tuning the model - Support and Troubleshooting

Predictive Intelligence common issues causing poor Classification model performance that gives incorrect or low-quality predictions: How to analyze and improve the Classification predictions by tuning the model Summary Predictive Intelligence Classification Updates

Predictive Intelligence has gotten several welcome updates in Zurich . The core concepts of this article are still applicable.

Classification has new Advanced Solution Settings and are covered later in this article. We added a feature analysis capability called " model explainability " which provides visual impact of features on the predicted class. You can use that along with the Predictability Estimate method shown below. We added a new LightGBM algorithm for classification model training, in addition to the Feed Forward Neural Net and XGBoost algorithms for classification and also covered in this article. Access Predictive Intelligence from AI Agents or Now Assist skills using the Now Assist Skill Kit (NASK) Predictive Intelligence tool . This allows your AI Agents and NASK skills to access your Predictive Intelligence models, combining deterministic and generative AI into one pipeline. Word Corpus is no longer needed for vectorization. Google Universal Sentence Encoder (GUSE) is used as the default. You have the ability to exclude classes in the prediction.

Tuning Classification Predictive Intelligence Models

Classification auto-populates fields on records to help route work to the right group. Machine learning isn't perfect, but the aim is to outperform humans. In practice, models trained on real-world data with precision above 70% and coverage above 80% generally outperform agents who manually review thousands of incidents per month.

Figure 1 represents a good classification model. The blue circles represent our assignment groups, the larger the circle the greater the distribution. What's distribution? Let's look at the blue bubble "Deskside Support". This assignment group represents 22% of the incidents from the training set. What makes this a good model is that all the blue bubbles which represent assignment groups are in the upper far right quadrant, which means they have good precision and good coverage. Note: the x-axis changes is changed to "recall" instead of precision.

Figure 1: A good classification model

In Figure 2 we notice that there are a couple of outliers that are not in the far upper right quadrant. Those are assignment groups that could be excluded from the model as they have a very little distribution and are most likely rare cases where a human is required to determine when an incident goes to that assignment group. Our Predictive Intelligence documentation lists out the steps on how to exclude classes from the model.

Figure 2: Outlier classes outside the upper right quadrant

Below, Figure 3 represents a model that needs a lot of work. You'll notice many of the assignment group bubbles with large distributions are scattered all over the classification graph, with several in the low precision and low coverage sections.

Figure 3: A classification model that needs tuning

So how do you move all those assignment group bubbles to the upper right, like our ideal model in Figure 1?

You have the ability to set the target precision and coverage for a solution definition. For most customers, setting a target solution level precision should be the first step when they start using Predictive Intelligence. By setting the target level precision, Predictive Intelligence will automatically change recall and coverage values to get all classes close to the target precision of 90% (see Figure 4). Important Note: When the model is retrained, it will not retain this tuned setting, as the data will have changed, and so it needs to be applied again after every training.

Figure 4: Setting a target solution precision

What if target metrics aren't helping? I've listed a few approaches below that can help you move the model to a more ideal state.

Techniques to improve the classification model precision and coverage

A Predictive Intelligence classification solution (for example, predicting Category, Assignment Group, Urgency or Impact on Incident) returns predictions that are frequently wrong, empty, or reported as low accuracy. Poor results are almost never caused by a single defect. They are the combined effect of the training data, the solution definition, the way predictions are triggered, and the way results are measured. A solution is only as good as the labeled data it is trained on, and its real performance cannot be known until it has been trained and its solution statistics reviewed.

This article explains how to analyze the prediction results and the training data, how to read the solution statistics, how to remediate the most common problems, and how to use the advanced solution settings available for classification from Zurich.

Understand the quality of the data. If there are lots of assignment groups that mean the same thing with different spellings, or if there are empty/null assignment groups, this will impact the precision of the model. You'll want to either exclude those from the training set or clean up your data. "More is not always better". Sometimes training using 200k records vs a smaller selection of 80k records from the past 3 months may drive down precision and coverage. If your precision is low, try training with a smaller data set first (for example 30k records), then increase the number of records to see if that increases the precision. In some cases, 200k records may reduce precision because there is a chance for more bad data in that training set. Make sure there is good distribution of the assignment groups. If 90% of the incidents are skewed toward "Deskside Support" then you don't need a machine learning model to tell you where new incidents will land. Also, you don't want to use inputs with lots of unique values. For example, configuration item, which may have millions or more records, may make it harder for the machine learning algorithm to predict if you use CI as an input field. Analyze your data for better inputs. If you started with the OOTB classification solution definition and it's giving a low precision/coverage, then try analyzing the data to determine what other inputs may help the ML predict a more precise result. Running lists/reports with these different input combinations should shed light on whether additional inputs such as category, subcategory, or location might help improve the precision/coverage of the model. Simplify your model. Because Predictive Intelligence is easy to configure, many customers will add lots of inputs to the solution definition without understanding that "more is not better". If you have lots of inputs and your model has low precision/coverage, then strip out all the inputs and start again with just short description. Update and retrain, then add input #2 and so forth until you get the desired precision. Plan to use multiple models. You might not be able to predict assignment group with just one model. One model may not be able to predict all the assignment groups. You can call different solution definitions in multiple passes. In Figure 3, four classification models were used to predict the assignment group. Use Performance Analytics (PA) with Predictive Intelligence. PA has helpful OOTB indicators such as reassignment count that can help identify areas where you want to focus Predictive Intelligence. The issues reported with classification solutions fall into the following categories. Several of them often occur together on the same solution.

Wrong predictions The solution returns a value, but the value is incorrect for the record.

Predictions on records the model never learned from. The training filter includes only certain records (for example, records created by agents through the portal or phone), but predictions are triggered on every new record, including records created by integrations, email, monitoring tools or other automation. The text and patterns in those records are different from the training data, so predictions on them are mostly wrong. In Predictor Results ( ml_predictor_results ), the incorrect predictions cluster around one creation channel, caller or user. Labels the model learned are themselves wrong. Users close records without correcting the predicted or default output value, or automation sets the value. The model then trains on incorrect labels at the next retrain, which repeats and amplifies the errors over time. Predictions of values that are no longer valid. The model predicts output values that have since been made inactive, such as an Assignment Group that was retired or reorganized, because those values are still in the training data. ml_predictor_results shows predictions of inactive values. Weak inputs. Only a short text field is used, the text is too generic to separate the classes (for example, "please help" or "not working"), or a related input field (such as Service or Service offering) is still empty when the prediction runs. Precision and recall stay low even with enough records. Missing or empty predictions The field is left empty, or certain values are never predicted at all.

Classes not generated. A class needs at least 30 records (default) before it can be generated. Values with less data, or values excluded by a filter that is too restrictive, are never predicted. The solution statistics show few classes generated and many classes excluded. Classes generated but never used. A class can show precision 100 and coverage 0 in the solution statistics. The data for that class was too varied, so the class is trained but never produces a prediction. Score below threshold. The prediction score does not reach the class or solution threshold, so no value is returned. Classes removed by advanced settings. Settings such as top N labels, minimum records per label, or remove others label drop classes before training. Those classes cannot appear in predictions. Predictions dominated by a few values The model predicts only a small set of values, regardless of the record.

Skewed data. One value represents most of the training data (for example, 98% of records have Urgency "Low"), so the model almost always predicts that value and minority values are excluded. Long tail of classes. With hundreds of possible values (for example, assignment groups), only the most common ones (for example, 30 out of several hundred) have enough data to be predicted. Heavy de-duplication. Machine-generated records with identical or near-identical text collapse into a single training record, so the effective data volume is much lower than the record count suggests. Problems in the trained solution itself The solution statistics or class list show something unexpected after training.

Duplicate classes with the same name. The Class Confidence tab on the trained solution ( ml_solution ) lists the same class name more than once, and the duplicated classes show empty statistics. Classes are generated per distinct output value, so when the output field has two values with the same label but different sys_ids (common with choice lists), one label produces two classes. Statistics change unexpectedly after retraining. Manually tuned thresholds or precision settings are reset when the solution retrains, because the statistics are regenerated. New values never appear. The solution was trained once on a curated dataset, or is not retrained regularly, so output values added since then are never learned. Accuracy reported lower than it really is The model may be performing reasonably, but the reported accuracy is misleading. Dashboards or reports built on prediction results show low accuracy, even though the solution statistics looked acceptable.

Final values that are not trained classes. A value is applied automatically at closure (for example, a default group set by a flow), so every prediction on those records is counted as incorrect in ml_predictor_results , even though the model could never have predicted it. Open records included in reports. Rows for records that are not yet closed have no final value and cannot be scored, but are included in accuracy reports. Input changes after prediction. A prediction is marked incorrect when the input fields were edited after the prediction was made, even if the predicted output was correct. Changes that do not improve the results Adjustments to the solution make no difference, or make the results worse.

Unmeasured changes. Fields are added or removed, or the filter is changed, based on assumptions (for example, "removing the description will improve accuracy") without comparing the solution statistics before and after. More data is not better data. Extending the training period adds older records with outdated values or labels, which can lower quality instead of raising it. Data problems remain. Changing the algorithm or advanced settings does not fix skew, wrong labels, duplicate values or inactive values in the training data. Tuning Predictive Intelligence Models using PredictabilityEstimate In this part I'll cover two methods to tune your Predictive Intelligence classification model.

Method 1: Auto-tuning a classification model

The simplest way to "tune" a Predictive Intelligence classification model is to set the Target Metric Value. However, this activity must be done after every solution training as the change in data will require new auto-tuning. Navigate to your trained solution and click into the Solution Statistics tab. In the below example my Estimated Solution Precision is 74.09%.

If I wanted to improve the precision, I can set the Target Metric Value to a higher precision (in this case 80%) and press Apply Values. The target metric value will optimize the precision by adjusting the recall and coverage percentages for the model. The below picture shows that we have improved our model to a precision of 80.07%, but we did drop in recall and coverage to get the precision gain.

Method 2: Adding inputs to your model

A second way of tuning our classification model is to add additional inputs to our classification solution definition. The question is, which inputs will have the most impact in improving the precision of the model? To answer this question we can use the PredictabilityEstimate object.

We'll use the ML API to instantiate and use the PredictabilityEstimate object in three steps.

STEP 1: Create and submit the PredictabilityEstimate

The PredictabilityEstimate is a scriptable object that provides an estimation of which features can be useful for predicting the target field for classification. To read more about the ML API and the PredictabilityEstimate object please see our documentation . To learn more about the inputs see the DatasetDefinition doc .

Copy and paste the below code and run it as a background script, by going to System Definition > Scripts - Background. The below JavaScript uses the PredictabilityEstimate object to recommend input fields to improve the classification prediction for category. If you want to use a different table or a different predicted target field, just swap them out.

//Create Estimate Object //Submit Training Job for Predictability Estimate try { var myIncidentData = new sn_ml.DatasetDefinition({ 'tableName' : 'incident', 'encodedQuery' : 'activeANYTHING' }); var eligibleFields = JSON.parse(myIncidentData.getEligibleFields('classification')); var myEstimate = new sn_ml.PredictabilityEstimate({ 'label': "predictability estimate", 'dataset' : myIncidentData, 'inputFieldNames':eligibleFields['eligibleInputFieldNames'], 'predictedFieldName': 'category' }); var myEstimateName = sn_ml.PredictabilityEstimateStore.add(myEstimate); var myEstimateVersion = myEstimate.submitTrainingJob(); var trainingStatus = myEstimateVersion.getStatus(); gs.print(JSON.stringify(JSON.parse(trainingStatus), null, 2)); } catch(ex) { gs.print(ex.message); } Here is a quick synopsis of the key blocks of code 1-4.

Block 1: specifies the "incident" table as the table with our training data. Block 2: says we want to test for "classification" in our solution. Block 3: defines the target predict field as category. Block 4: adds our PredictabilityEstimate object, myEstimate, to the PredictabilityEstimateStore and submits the training job.

STEP 2: Get the unique name of the PredictabilityEstimate

Go to the ML Solutions table and get the unique name for the predictability estimate object that you just created. In my example it's ml_x_snc_global_global_predictability_estimate .

STEP 3: Query the PredictabilityEstimate for the recommended input fields

Run the below script to retrieve the Predictability Estimate score for predicting category using classification. Replace the estimateName value with the unique name from Step 2.

//Get suggested input fields after training is complete try { var estimateName = 'ml_x_snc_global_global_predictability_estimate_2'; var estimateSolution = sn_ml.PredictabilityEstimateStore.get(estimateName); var estimateResults = JSON.parse(estimateSolution.getActiveVersion().getResults()); gs.print(JSON.stringify(estimateResults, null, 2)); } catch(ex) { gs.print(ex.message); } Interpreting the results

The script outputs two valuable insights when predicting category. The first being the impact of "nominal field inputs" on improving the model. The second calculates the "density of text input fields". We use feature ranking that leverages a mix of weighted information gain techniques, AUPRC (area under precision recall curve), ReliefF Scoring, and a pre-trained model to select and rank relevant nominal input fields for the recommendation.

Nominal Field Inputs are discrete values that can be used as inputs into the classification model. The below modelImprovement numbers show the correlation percentage to our predicted target field, in this case "category". We can see that subcategory is highly correlated to category, by 47%.

Now here is where we have to apply some human logic to the recommended input field output. We're trying to predict category, however we're not going to know the subcategory at the time of prediction. Also, category typically drives subcategory, so these two fields are typically highly correlated. So we should skip subcategory as a potential input to improve the model and look at the other recommended input fields. For example, assignment_group and service_offering both look like potential input fields that we might be able to use to improve the precision of our classification model. You may need to experiment with different combinations of the recommended input fields to get the best precision. I would suggest you make a copy of the original solution definition, add one new input, and train that model to see if precision improves.

If you scroll to the bottom of the output, you will see textInputFields density.

Density of text input fields represents the fullness or the sparsity of a potential input text field. Your Predictive Intelligence models will perform better if the text input fields have a high density of data. In our example above, short_description is a good text input field because it has a density of 99%. We want to avoid any text input fields that have low percentages.

In summary, we just learned about two methods for tuning your classification model. There is a third method that analyzes the trained solution and the prediction results to see how well the model is performing, and to avoid common issues with the data in the training dataset, so that the classification model performance does improve over time.

Tuning a model is an iterative process, as you won't know how well it will perform until you train it, as that will generate the classes for prediction and produces the solution statistics, so that it can be compared with other versions of the solution that were trained with different parameters or filters on the data to determine which solution will perform the best.

For reference, see the documentation for Using the Machine Learning APIs .

Facts Product area: Predictive Intelligence, Classification solution definitions ( ml_solution_definition ), trained solutions ( ml_solution ), prediction audit ( ml_predictor_results ). Release: The analysis and tuning guidance applies to all supported releases. The advanced solution settings for classification are available from Zurich onwards. How training works: The records matching the solution definition filter are labeled by the selected output field. Duplicate records (identical input field values) are de-duplicated, so only one copy is trained. 80% of the data is used to train and 20% is held back to test the trained solution and generate the solution statistics (precision, recall, coverage per class and the aggregate in the Solution Statistics). Minimum data: A class (output value) needs at least 30 records before it can be generated. Classification solutions as a whole need at least 10,000 training records by default. This limit is controlled by a system property [glide.platform_ml.api.csv_min_line], as per our documentation on Predictive Intelligence properties . Prediction is not filtered by the training filter. The solution definition filter controls only what is trained. Predictions are triggered (by Recommended Actions, Flow Designer, Business Rule or Script) on every record the trigger logic covers, whether or not that record would have qualified for training. Results are scored at closure. A row in ml_predictor_results with an empty Final Value Date belongs to an open record and cannot be scored yet. When the record is closed, a business rule populates Final Input Value, Final Output Value and Final Value Date and sets Predicted Correctly. A prediction is marked incorrect if the final output differs from the predicted output or the input values changed after the prediction was made. Threshold behavior: A threshold of 0 means a prediction is returned for any score. A class with estimated precision of 100 and coverage of 0 can never be predicted. Manual threshold tuning is lost when the solution is retrained, because the statistics are regenerated. Release All releases.

Instructions Step 1. Analyze the prediction results

Navigate to Predictive Intelligence > Results > Predictor Results (list ml_predictor_results ). Filter Solution = the solution under review (optional, as you may want to review predictions for all solutions on a record, so filter by Source ID). Exclude open records: set Final Value Date is not empty. Open records cannot be scored. Group by Predicted Correctly to get the overall correct/incorrect ratio. Record this ratio so you can compare it after making changes. With the incorrect rows isolated, group by Final Output Value. Look for: A final value that is not a class in the training data. Every prediction against it is counted wrong. Either stop predicting on those records, remove the value so that it can't be selected by the user or include the value as a trained class. A single dominant final value, which indicates skewed data. Inactive output values, such as inactive Assignment Groups. Open sample incorrect rows and compare Predicted Input Values / Final Input Value and Predicted Output Value / Final Output Value. Determine whether the input text was edited after prediction, or the output was legitimately changed by a user or automation. Open the source records of the incorrect rows and check how they were created (channel, caller, integration user, automation). If the incorrect rows cluster around one creation method that the training filter excludes, records created that way should not be predicted on. Group correct rows by Predicted Output Value to see which classes the solution actually predicts well. Compare with the classes you expect.

Step 2. Analyze the training data

Open the solution definition and review the filter and input fields. Select Update Count after any filter change, then open the matching-records link "View results in new tab".

Group the list by the output field. This shows how many records each class would receive. Fewer than 30 records: the class cannot be generated. One class holding the large majority of records: the data is skewed towards that class. The same label in more than one group: the output field has duplicate values with different sys_ids. Check that records are in a final state (Active=false", so output values are settled. Review for incorrectly labeled data: values set by default, values never changed from the prediction, and values populated by automation. Check whether any output values in the training data are inactive (for example, inactive Assignment Groups). These are trained as classes and can still be predicted.

Step 3. Train and read the solution statistics

Performance cannot be determined in advance. The solution statistics are the only reliable quality indicator. After every change, train the solution and open the Solution Statistics on the trained solution. Review:

Statistic What to look for Classes generated The number of classes the model can actually predict. Compare with the number of distinct values you expect. Classes excluded Classes dropped because of insufficient or inconsistent data. These are never predicted. Precision = 100, Coverage = 0 The data for that class was too varied. The class is trained but will never make a prediction. Precision = 0, Coverage = 100 The class is problematic: it predicts, but the predictions are wrong. Investigate its training data. Overall precision, recall, coverage Use these to compare candidate solutions side by side. Also open the Class Confidence tab on the trained solution and check for class names that appear more than once with empty statistics.

Notes:

100/100/100 is not achievable in practice. Results in the mid-nineties have only been seen where data was manually curated with an even number of records per class, which requires substantial effort. As a guide, precision above 70% and coverage above 80% is generally considered effective for classification. Use Test Solution on the trained solution to try example text, and run a batch test (up to 50 existing records) to compare against the actual values. To compare variants, create separate solution definitions (for example, short description only, versus short description plus description plus related fields), train each, and compare statistics. Do not assume a change helps or hurts before measuring it.

Step 4. Remediate based on what the analysis found

Stop predicting on records that were not trained on

If the training filter excludes automated records, add the same condition to the prediction trigger (Flow Designer condition, business rule condition, or Recommended Actions trigger), so predictions are not made on them. If no field currently identifies how the record was created, add one (for example, a distinct Channel/Contact type value for automated records) and use it in both the training filter and the trigger. Where a value is applied automatically at closure, either skip prediction for those records or add that value to the training data as a real class or remove the value so that it cannot be applied. Consider two solutions: one for human-created records and one for machine-generated records, triggered by channel. Machine-generated text is highly repetitive and de-duplicates heavily, so it behaves differently from human text. Give every class enough data

Broaden the filter (for example, a longer date range) so the classes you care about reach at least 30 records. A very restrictive filter can leave most classes excluded. More data is not always better. Test a smaller recent set against a larger one and keep whichever produces the better statistics. Remove or ignore duplicates and empty values in input fields. Handle skewed output fields

When one value represents almost all records (for example, 98%), the model will favor it:

Train only on the non-dominant classes, and apply the dominant value deterministically (with a flow, business rule or script) when the prediction is empty or below threshold. Classes with too few records cannot be trained. They must be set manually by users. Training on fewer than the default minimum of 10,000 records is possible through the system property [[glide.platform_ml.api.csv_min_line], but the resulting quality must be validated through the statistics. Improve input fields

Start with Short description and add fields incrementally, validating each addition with statistics. Including Description often adds context and can improve results slightly, but this is data dependent and must be measured. Related fields (for example, Service or Service offering) can strongly correlate with the output. If they are themselves predicted by Predictive Intelligence, predict them first, otherwise they are empty when the dependent field is predicted. Avoid high-cardinality fields (such as a Configuration Item reference with a very large number of unique values) and fields that rarely contain a value. Where a related record (for example, a Service offering) already defines the target value (such as a support group), consider deterministic logic based on that relationship instead of predicting the field. Improve label quality

Require users or automation to confirm or correct the output field before a record is closed (for example, a mandatory-on-close field or a confirmation step). Records closed with unchanged predicted or defaulted values train the model on wrong labels. Retrain, do not hand-tune

Per-class or solution-level threshold and precision settings are reset when the solution is retrained. Use manual tuning only for a fixed, curated dataset that will not be retrained. Schedule a retraining frequency so new values (new assignment groups, categories) are picked up. Setting a target precision at the solution level (for example, starting at 90%) automatically calibrates recall and coverage. A higher precision target lowers coverage, and vice versa. Select Apply so the visualization updates. Merge duplicate output values

Classes are generated from the output field value. If the training data contains two output values with the same label but different sys_ids (common with choice lists), two classes are generated for the same label, and both show empty statistics in the Class Confidence tab.

Identify which of the duplicate values should be kept. Make the other value inactive. Update the existing records that use the inactive value so they use the active value. Retrain the solution, and confirm in the Class Confidence tab that only one class is generated for that label. Exclude inactive output values

When output values change frequently (new values added, old ones made inactive), make sure the training data only contains active values. If ml_predictor_results shows predictions of inactive values, such as inactive Assignment Groups:

Open the solution definition and add a filter condition that includes only records whose output value is active (for example, Assignment group.Active is true). Select Update Count to confirm the effect on the training data. Retrain the solution.

Step 5. Use the advanced solution settings for classification (Zurich and later)

From Zurich, classification solutions have five advanced solution settings of their own: include only top N labels, minimum records needed for a label, remove others label, LightGBM, and config parameters for model config. Other advanced settings (class recall, TF-IDF, XGBoost, and the clustering options) are shared with other capabilities and are not covered here.

Advanced settings are optional. Use one only when you understand the technique it enables and your use case benefits from it. Classes that are excluded or show precision 100 / coverage 0 in the solution statistics are candidates for the minimum records or top N settings, and a long tail of tiny classes is a candidate for remove others label.

All five settings are added the same way:

Role: ml_admin or admin . Where: All > Predictive Intelligence > Classification > Solution Definitions > open or create a solution > Advanced Solution Settings tab > New. This opens the Advanced Solution Setting form ( ml_advanced_solution_settings ). Search the setting name in Solution Parameters, fill User Inputs where needed, and select Submit. Result: The setting appears as a row on the Advanced Solution Settings tab. To change or remove it, select the setting name on that tab. Afterwards: Retrain the solution. No setting takes effect until the next training. Compare the new statistics with the previous version. The benefits and drawbacks for each setting come from what the setting does to the training data. Always check them against your own solution statistics before and after retraining.

Configure include only top N labels What it does: Trains the model only on the N most frequent classes. You set N in User Inputs (for example 100 ). Limiting the model to a fixed number of the most frequent labels improves focus and performance, especially on large datasets.

Search term in Solution Parameters: include only top n labels Script: config.setTopNLabels(100); JSON: {"include_top_n_labels": 100} Benefits Drawbacks Removes the long tail of rare classes that often have too little data to be learned well. Classes outside the top N can never be predicted. Records that belong to those classes will always get a wrong prediction or none, which lowers the Predicted Correctly ratio in ml_predictor_results . Faster training and a smaller model on datasets with many distinct values (for example, hundreds of assignment groups). N is a fixed count, not a share of the data. When classes are added or retired, the cut-off moves and a class can drop in or out at the next retrain without anyone noticing. Easy to understand and explain: "the model covers the N most common values." Ranking is by record count only, so a business-critical but low-volume class (for example, a security team) can be left out. Duplicate labels and inactive values still take up places in the top N, so clean those up first. Minimum records needed for label to include it What it does: Drops any class with fewer records than the minimum before training. You enter the minimum in User Inputs and this will remove labels that occur infrequently, which can reduce overfitting and unreliable predictions. Classes dropped before training cannot appear in predictions.

Before you begin: make sure the dataset contains label values suitable for classification. Search term in Solution Parameters: minimum records needed for label to include it Script: config.setMinLabelCount(50); JSON: {"min_label_count": 50} Benefits Drawbacks Removes classes with too few examples, which are the main source of unreliable, overfitted predictions. Dropped classes can never be predicted, which lowers the Predicted Correctly ratio for records that belong to them. Based on the volume of each class, so it adapts as data grows: a class becomes eligible once it reaches the minimum. It does not depend on how many other classes exist. Setting the minimum too high on a dataset with many mid-volume classes can remove most of the classes. Compare Classes generated in the statistics before and after. Makes the 30-record minimum for generating a class adjustable per solution. The effect of a value below the default 30-record minimum is not documented, but it is unlikely to generate a class with many precision coverage lookups Duplicate labels split one class's records across two values, so each half can fall below the minimum. Merge duplicates first. Remove others label What it does: When set to true, removes records whose class makes up less than 1% of the total data from training. The interface labels these records "others". The 1% threshold cannot be changed. This reduces noise in the model and should improves predictive accuracy.

Search term in Solution Parameters: Remove others label . No User Inputs are needed; select Submit. Script: config.removeOthersLabel(true); JSON: {"remove_others_label": true} Benefits Drawbacks Removes the noisy "others" group, so the model no longer learns a catch-all class made of unrelated rare values. The threshold is a fixed 1% share, not a record count. On a dataset with many classes (for example, 300 assignment groups), most classes can be under 1%, so this setting can remove most of the classes at once. One switch with no value to tune, so it is simple to apply and to roll back. Because it is a share, a class with plenty of records (for example 500 out of 100,000) is still removed. Use the minimum records setting instead when you need control based on record count. Works well for output fields with a handful of dominant values and a scattered tail (for example, Category or Urgency). Removed classes cannot be predicted. Check that no business-critical low-volume class falls below 1%. Use LightGBM algo for classification model training What it does: Trains the classification model with the LightGBM (Light Gradient-Boosting Machine) algorithm instead of the default algorithm where LightGBM offers performance and efficiency for large datasets.

Search term in Solution Parameters: Use LightGBM algo for classification model training . Leave User Inputs blank and select Submit. Script: new MLPredictorConfig(' ') , then config.useLightGBM(); and config.updateConfig(); JSON: {"use_lightgbm": true} Benefits Drawbacks Faster training and lower resource use on large datasets. Retraining large solutions finishes sooner, which makes frequent retraining more practical. Gains appear on large datasets. On small or medium datasets, gradient-boosting algorithms like LightGBM are known to overfit more easily, so statistics can get worse. Often handles many classes and many records more efficiently than the default. Does not fix data problems. Skew, wrong labels, duplicate labels and inactive values in the training data affect LightGBM in the same way. No values to enter, so it is easy to switch on and off and compare. Whether the config parameters setting applies to LightGBM as well as the default algorithm is not documented [VERIFY]. Train the solution with and without LightGBM and compare Solution Statistics before keeping it. Config parameters for model config in classification What it does: Passes a dictionary of machine learning hyperparameters to customize how the classification model is trained. This an advanced option that should be modified with care, and says you should understand the hyperparameters relevant to your model objective first.

Search term in Solution Parameters: Config parameters for model config in classification . User Inputs format: enter only the key-value pairs, without the enclosing label, braces or parentheses. Edit the values from the documented example: 'max_depth': 10,'learning_rate': 0.1,'objective': 'multi:softprob','num_class': 11,'random_state': 10,'eval_metric': 'mlogloss','booster': 'gbtree','n_estimators': 150,'verbosity': 1,'use_label_encoder': false,'tree_method': 'hist','num_parallel_tree': 1,'gamma': 0.25,'subsample': 0.8,'max_delta_step': 5,'reg_alpha': 0,'reg_lambda': 1,'colsample_bytree': 0.8,'colsample_bylevel': 1,'colsample_bynode': 1,'min_child_weight': 1.25,'n_jobs': 11,'validate_parameters': true Script format: include the braces and parentheses: (function(){ var config = new MLPredictorConfig(' '); config.setModelParams({ 'max_depth': 10, 'learning_rate': 0.1, 'objective': 'multi:softprob', 'num_class': 11, 'n_estimators': 150 }); config.updateConfig(); })(); JSON format: the key is classification_model_params , with the same dictionary as its value. Main parameters in the example:

Parameter Example Meaning max_depth 10 Maximum depth of each tree. Higher learns more detail but overfits more easily. learning_rate 0.1 Step size for each boosting round. Lower is more stable but needs more rounds. n_estimators 150 Number of boosting rounds (trees). objective / eval_metric multi:softprob / mlogloss Multi-class probability output, measured by multi-class log loss. num_class 11 Number of classes. subsample , colsample_bytree 0.8 Share of rows and columns sampled per tree, which reduces overfitting. gamma , reg_alpha , reg_lambda , min_child_weight 0.25, 0, 1, 1.25 Regularization: higher values make the model more conservative. tree_method , n_jobs hist , 11 Training method and parallelism. random_state 10 Fixed seed for repeatable training. The parameter names in the example ( booster , gbtree , use_label_encoder , multi:softprob ) are XGBoost hyperparameters.

Benefits Drawbacks Full control over model complexity and regularization, for example lowering max_depth or raising gamma to reduce overfitting on noisy data. Needs real machine learning expertise. Wrong values can lower quality noticeably. Saved as a setting, so it is applied again at every retrain (unlike manual threshold tuning, which is lost). num_class is a fixed number. If the number of classes changes at a later retrain (new values, or classes dropped by other advanced settings), the value can stop matching. Many parameters interact. Change one at a time and compare Solution Statistics after each retrain.

Step 6. Verify

Retrain the solution and open the new Solution Statistics. Confirm that the number of classes generated increased or is as expected, and that the classes you depend on are not in the excluded list. In the Class Confidence tab, confirm that no class name is duplicated and that no inactive output values are listed as classes. Run Test Solution and a batch test with recent records. After the changes have been live long enough for records to close, open Predictor Results again, exclude empty Final Value Date, and compare the Predicted Correctly ratio with the ratio recorded before the changes. Alternatives

Large language model (LLM) based classification is an alternative for fields where labeled training data is poor or classes have very few records. It does not use training data and instead finds similar existing records (retrieval and hybrid search) to predict the value. For further details, please review our documentation on the Record field value prediction AI agent .