[Similarity] - Infrequently used similarity solutions with high update frequency cause slow performance when invoked, such as opening [em_alert] record or can throw a Malformed URL error when predictingDescriptionWhen a Similarity Solution with a high update frequency has not provided a prediction on the solution for several hours, a backlog of updates are processed during the first prediction call. For solutions that are not used very often this produces a malformed URL error - A common performance issue that is reported is when Find similar alerts is enabled, and sporadically it can take 2-3 mins to open an [em_alert] record. When checking the logs, we can see Push Model calls to update the solution on the Prediction server, which can take many seconds to process for each update, and why it can take so long for the [em_alert] record to be opened - 2023-06-02 01:23:53 (468) Default-thread-11 E56D8CB347C3A114AFADB604836D4388 txid=611550fb7a42 DxC_ML: dxc_id=2595069575673261 Starting push model call for ml_x_global_alert_similarity2023-06-02 01:24:13 (856) Default-thread-11 E56D8CB347C3A114AFADB604836D4388 txid=611550fb7a42 DxC_ML: dxc_id=2595069575673261 End push model call for ml_x_global_alert_similarity took 20387 ms2023-06-02 01:24:13 (856) Default-thread-11 E56D8CB347C3A114AFADB604836D4388 txid=611550fb7a42 DxC_ML: dxc_id=2595069575673261 Starting push model call for ml_x_global_alert_similarity2023-06-02 01:24:20 (028) Default-thread-11 E56D8CB347C3A114AFADB604836D4388 txid=611550fb7a42 DxC_ML: dxc_id=2595069575673261 End push model call for ml_x_global_alert_similarity took 6172 ms2023-06-02 01:24:20 (028) Default-thread-11 E56D8CB347C3A114AFADB604836D4388 txid=611550fb7a42 DxC_ML: dxc_id=2595069575673261 Starting push model call for ml_x_global_alert_similarity2023-06-02 01:24:32 (148) Default-thread-11 E56D8CB347C3A114AFADB604836D4388 txid=611550fb7a42 DxC_ML: dxc_id=2595069575673261 End push model call for ml_x_global_alert_similarity took 12119 ms2023-06-02 01:24:32 (148) Default-thread-11 E56D8CB347C3A114AFADB604836D4388 txid=611550fb7a42 DxC_ML: dxc_id=2595069575673261 Starting push model call for ml_x_global_alert_similarity2023-06-02 01:24:49 (262) Default-thread-11 E56D8CB347C3A114AFADB604836D4388 txid=611550fb7a42 DxC_ML: dxc_id=2595069575673261 End push model call for ml_x_global_alert_similarity took 17114 ms2023-06-02 01:24:49 (262) Default-thread-11 E56D8CB347C3A114AFADB604836D4388 txid=611550fb7a42 DxC_ML: dxc_id=2595069575673261 Starting push model call for ml_x_global_alert_similarity2023-06-02 01:25:01 (759) Default-thread-11 E56D8CB347C3A114AFADB604836D4388 txid=611550fb7a42 DxC_ML: dxc_id=2595069575673261 End push model call for ml_x_global_alert_similarity took 12495 ms2023-06-02 01:25:01 (759) Default-thread-11 E56D8CB347C3A114AFADB604836D4388 txid=611550fb7a42 DxC_ML: dxc_id=2595069575673261 Starting push model call for ml_x_global_alert_similarity2023-06-02 01:25:18 (265) Default-thread-11 E56D8CB347C3A114AFADB604836D4388 txid=611550fb7a42 DxC_ML: dxc_id=2595069575673261 End push model call for ml_x_global_alert_similarity took 16506 msSteps to Reproduce 1. Create a Similarity solution with an Update Frequency set to "Every 15 mins".2. Train the Similarity solution and create 100s of new records that matches the condition on the Similarity solution definition, but ensure no predictions are made on the solution during this time.3. Once the Similarity solution has a big backlog of records to be processed, do a test prediction using the Test Solution tab on the trained Similarity solution.4. Prediction Output returns "Malformed URL". 1. Ensure Event Management [com.glideapp.itom.snac] and Predictive Intelligence [com.glide.platform_ml] applications are installed.2. Check that the system property [evt_mgmt.similarity_use_ml] is set to “true”.3. Check table [ml_solution] and see if the Similarity solution [ml_x_global_alert_similarity] has been trained and is active.4. If the [em_alert] table has > 300k records, on occasion when the solution has not been invoked for a while, when opening a record in table [em_alert], it can take considerably longer than normal.WorkaroundAfter carefully considering the severity and frequency of the issue, as well as the cost and risk of attempting a fix, a decision has been made not to address this case in any current or future releases. We do not make this decision lightly and we apologize for any inconvenience. However, the following workaround has been successfully implemented and fixed the performance issue when opening the Event Management Alert [em_alert] record. The Similarity solution [ml_x_global_alert_similarity] has the Update Frequency set to "Every 15 minutes", and you will need to create a scheduled job that runs every 15 minutes and ensure the System ID field is set to 'ALL NODES'. By selecting 'ALL NODES', it will create child jobs on all nodes, as each application node has affinity with a Prediction Server in the datacenter, hence the Similarity updates need to be processed on all Prediction Servers. By opening the [ml_update_set] table, you can check the [sys_updated_on] timestamp when the Similarity solution update was trained. The scheduled job to perform a dummy prediction to invoke the Similarity Updates processing should start 5 seconds after the [sys_updated_on] timestamp on the associated [ml_update_set] record. The dummy script should look as follows - var solution_name = '<SOLUTION_NAME>';var mlSolution = sn_ml.SimilaritySolutionStore.get(solution_name);var options = {};options.top_n = 1;options.apply_threshold = false;var dummy_input = [{'short_description' : 'test'}];var results = mlSolution.getActiveVersion().predict(dummy_input , options);// gs.info(JSON.stringify(JSON.parse(results), null, 2));Note: In this example script, please update the solution_name and dummy_input with the Input fields on the solution Related Problem: PRB1575241