Event based Discovery for large k8s clustersFor the large Kubernetes clusters, sometimes CNO pattern-based discovery can impact the Kubernetes API server. The alternative is to use the Kubernetes Informer which is part of the Kubernetes client-go library. This library brings efficiently all required resources and stores them in memory. From that point on it ingests changes and updates the memory data structure. And once a day, the program will send the whole picture, in order to cover for changes missed for some reason. This will be sent in multiple chunks and throttled in a way the instance could handle it. Steps to enable CNO with Informer :- Install CNO v3.1.0+ & ACCV v3.1.0+ from ServiceNow Store.After installing above plugins, go to Agent Client Collector -> Policies and activate K8s Informer policy.After activating k8s informer policy all the CNO deployments will work with Kubernetes Informer. CMDB Objects which are covered in this new way of K8s discovery:- Kubernetes Workloads (cmdb_ci_kubernetes_workload): - Kubernetes Cronjob (cmdb_ci_kubernetes_cronjob) Kubernetes DaemonSet (cmdb_ci_kubernetes_daemonset) Kubernetes Deployment (cmdb_ci_kubernetes_deployment) Kubernetes Job (cmdb_ci_kubernetes_job) Kubernetes ReplicaSet (cmdb_ci_kubernetes_replicaset) Kubernetes StatefulSet (cmdb_ci_kubernetes_statefulset) Kubernetes Componentes: - Kubernetes Cluster (cmdb_ci_kubernetes_cluster) Kubernetes Pod (cmdb_ci_kubernetes_pod) Kubernetes Node (cmdb_ci_kubernetes_node) Kubernetes Service (cmdb_ci_kubernetes_service) Kubernetes Namespace (cmdb_ci_kubernetes_namespace) Docker Container (cmdb_ci_docker_container) Container Repository (cmdb_ci_container_repository) Docker Image (cmdb_ci_docker_image) Container Repository Entry (cmdb_ci_container_repository_entry) Troubleshooting/Supportability features:- ServiceNow also provides some troubleshooting/supportability features like checking k8s informer program logs (which are separated from mid logs for better visibility), triggering full discovery manually, restart informer program or reload some parameters of informer program through two check definitions:- k8s_informer_supportability_features check definition:- Using this check definition, we can do the following:- 1. Trigger full discovery -> to trigger full discovery enable full_discovery check param and trigger Test check. 2. Restart Informer -> to restart informer enable restart_informer check param and trigger Test check. 3. Reload Parameter -> to reload informer parameter enable reload_parameter check param and trigger Test check. We support only 4 params for this:- "refresh_time_min", "verbose_logging", "max_cis_in_msg" & "ignore_labels". (Format for this param should be reload_parameter:param_name=param_value) Possible values:- reload_parameter:refresh_time_min=1000, reload_parameter:verbose_logging=true, reload_parameter:max_cis_in_msg=50 & reload_parameter:ignore_labels=app Note:- Enable only one check param at a time. k8s_informer_log check definition:- To get k8s Informer specific logs. Other useful links:- Deploy CNO on your Kubernetes platformHow to run test check on any check definition for containerised agents?