Controlling the number of "Kubernetes Visibility Agent" (formerly CNO for Visibility) concurrent worker threads<!-- /*NS Branding Styles*/ --> .ns-kb-css-body-editor-container { p { font-size: 12pt; font-family: Lato; color: var(--now-color--text-primary, #000000); } span { font-size: 12pt; font-family: Lato; color: var(--now-color--text-primary, #000000); } h2 { font-size: 24pt; font-family: Lato; color: var(--now-color--text-primary, black); } h3 { font-size: 18pt; font-family: Lato; color: var(--now-color--text-primary, black); } h4 { font-size: 14pt; font-family: Lato; color: var(--now-color--text-primary, black); } a { font-size: 12pt; font-family: Lato; color: var(--now-color--link-primary, #00718F); } a:hover { font-size: 12pt; color: var(--now-color--link-primary, #024F69); } a:target { font-size: 12pt; color: var(--now-color--link-primary, #032D42); } a:visited { font-size: 12pt; color: var(--now-color--link-primary, #00718f); } ul { font-size: 12pt; font-family: Lato; } li { font-size: 12pt; font-family: Lato; } img { display: ; max-width: ; width: ; height: ; } } Controlling the Maximum Number of Concurrent Worker Threads Messages sent by the Kubernetes Visibility Agent clients (a.k.a Informers) running in the Kubernetes clusters are first placed in the ECC queue. Later on, they are routed to one of pre-configured queues and handled by the corresponding scheduled job. By default, the system configures 4 queues which are handled by 4 scheduled jobs. This ensures that "Kubernetes Visibility Agent" will never use more than 4 concurrent worker threads across all the instance's nodes. This number should be sufficient for handling up to around 300 Kubernetes clusters. However, if the number of clusters is higher or clusters are going through frequent changes, you can increase the maximum number of concurrent worker thread up to 14. To do so, navigate to System Properties/All Properties and edit the property: sn_acc_visibility.k8s_informer_max_worker_threads. The values allowed are 1 to 14. Routing ECC Messages to Queues The default routing algorithm decides on the target queue based on the last character of the informer internal name. The system converts this hexadecimal character to a number, divides it by the number in sn_acc_visibility.k8s_informer_max_worker_threads and takes the reminder. Example 1: Informer name: k8s_informer_4724f0d3-121a-4331-a9e0-7a91178d75a4 Last character: 4 Last character as a decimal number: 4 Queue number: 4 % 4 (sn_acc_visibility.k8s_informer_max_worker_threads) = 0 Message will be routed to event queue: sn_acc_visibility.k8s_informer.message_0 Example 2: Informer name: k8s_informer_8afc893b-d304-4e16-b345-d72871ad0d21 Last character: 1 Last character as a decimal number: 1 Queue number: 1 % 4 (sn_acc_visibility.k8s_informer_max_worker_threads) = 1 Message will be routed to event queue: sn_acc_visibility.k8s_informer.message_1 Automatic Rerouting The system is monitoring constantly the load on each queue. When a certain queue becomes overloaded, the system may reroute messages coming from a given informer to another queue. This decision is persistent and the informer's messages will continue to be routed to the new queue.