Kubernetes Visibility Agent (formerly CNO for Visibility) extensibility and customization - Support and Troubleshooting

Kubernetes Visibility Agent (formerly CNO for Visibility) extensibility and customization .ns-kb-css-body-editor-container { p { font-size: 12pt; font-family: Lato; color: var(--now-color--text-primary, #000000); } span { font-size: 12pt; font-family: Lato; color: var(--now-color--text-primary, #000000); } h2 { font-size: 24pt; font-family: Lato; color: var(--now-color--text-primary, black); } h3 { font-size: 18pt; font-family: Lato; color: var(--now-color--text-primary, black); } h4 { font-size: 14pt; font-family: Lato; color: var(--now-color--text-primary, black); } a { font-size: 12pt; font-family: Lato; color: var(--now-color--link-primary, #00718F); } a:hover { font-size: 12pt; color: var(--now-color--link-primary, #024F69); } a:target { font-size: 12pt; color: var(--now-color--link-primary, #032D42); } a:visited { font-size: 12pt; color: var(--now-color--link-primary, #00718f); } ul { font-size: 12pt; font-family: Lato; } li { font-size: 12pt; font-family: Lato; } img { display: ; max-width: ; width: ; height: ; } } *, *::before, *::after { box-sizing: border-box; margin: 0; padding: 0; } body { font-family: "Segoe UI", Arial, sans-serif; font-size: 14px; line-height: 1.7; color: #1a1a1a; background: #fff; padding: 0; } .page { max-width: 100%; margin: 0; background: #fff; border-radius: 0; box-shadow: none; padding: 0; } /* ── Header ── */ .kb-header { border-bottom: 3px solid #293e40; padding-bottom: 18px; margin-bottom: 32px; } .kb-id { font-size: 12px; font-weight: 600; color: #5b7a7d; letter-spacing: .06em; text-transform: uppercase; margin-bottom: 6px; } h1 { font-size: 24px; font-weight: 700; color: #293e40; line-height: 1.3; } /* ── TOC ── */ .toc { background: #f0f4f5; border-left: 4px solid #293e40; border-radius: 4px; padding: 20px 24px; margin-bottom: 40px; } .toc-title { font-size: 13px; font-weight: 700; text-transform: uppercase; letter-spacing: .08em; color: #293e40; margin-bottom: 10px; } .toc ol { list-style: decimal; padding-left: 20px; } .toc li { margin: 3px 0; } .toc a { color: #1a6b8a; text-decoration: none; font-size: 13.5px; cursor: pointer; } .toc a:hover { text-decoration: underline; } /* ── Headings ── */ h2 { font-size: 19px; font-weight: 700; color: #293e40; margin-top: 40px; margin-bottom: 12px; padding-bottom: 6px; border-bottom: 2px solid #e0e6e8; } h3 { font-size: 15px; font-weight: 700; color: #1a6b8a; margin-top: 24px; margin-bottom: 8px; } /* ── Body text ── */ p { margin-bottom: 14px; } ul, ol { padding-left: 22px; margin-bottom: 14px; } li { margin-bottom: 4px; } /* ── Code / pre ── */ code { font-family: "Consolas", "Courier New", monospace; font-size: 12.5px; background: #edf2f4; border-radius: 3px; padding: 1px 5px; } pre { font-family: "Consolas", "Courier New", monospace; font-size: 12.5px; background: #1e2b2d; color: #d4e6e8; border-radius: 6px; padding: 18px 20px; overflow-x: auto; margin: 12px 0 18px; line-height: 1.55; tab-size: 2; } /* ── Callout / note ── */ .note { background: #fff8e1; border-left: 4px solid #f0a500; border-radius: 4px; padding: 12px 16px; margin: 16px 0; font-size: 13.5px; } .note strong { color: #7a5200; } /* ── Section anchor ── */ .section { margin-top: 48px; } .section:first-child { margin-top: 0; } a { color: #1a6b8a; } em { font-style: italic; } strong { font-weight: 700; }

KB1638668 Kubernetes Informer – Additional Resources & Custom Configuration Table of Contents Goals Pulling Additional Resources Default Information Pulled by the System Filtering Resources Custom Information Input Config Files Custom Relations Custom Related Entries Configuring Resource Extraction from the Instance Processing the Information on the Instance Filtering Out Resources on the Instance Scripted API for Retrieving Resource Information On-Demand Goals The goal of this feature is to enable customers to bring into their database additional resources and attributes from the Kubernetes environment and process them according to their needs. In addition, it provides capabilities for manipulating and filtering the resources reported by the informer. This document first explains the information the system can pull, followed by concrete examples of the configuration files.

Pulling Additional Resources Users may declare on any Kubernetes resource to pull into the CMDB. The resource definition will include the APIVERSION and NAME as seen in the output of the command kubectl api-resources .

Default Information Pulled by the System By default, the system will pull the following information:

Name Namespace (for namespaced resources) Kubernetes UID API Version Kind Labels and Annotations By default, the system will store the information in the cmdb_ci_kubernetes_component class.

The system will create a relation and a reference from the cmdb_ci_kubernetes_component to the cluster CI.

In case the resource is namespaced, the system will create a relation to the namespace CI.

If you would like to place the resource in a dedicated new table, it is possible to declare which is the target table. The target table should extend cmdb_ci_kubernetes_component and should have an identification rule that inherits the one from the parent class.

Filtering Resources Users may declare on a subset of the resources of a given kind to bring to the CMDB. For example, you may request to bring ConfigMap resources whose name is "version" . The filtering mechanism is applicable both to out-of-the-box resources or to additional resources defined by customers.

Basic Filtering The filtering is done by specifying a JSON path of the field in the resource JSON whose value should be inspected, and a regular expression applied to this value. It is possible to define multiple such conditions. If any of the conditions show a match, the resource will be sent to the CMDB. An example is shown later in this article.

Complex Filtering (from version 3.x.x) In case we need to filter resources using a complex condition, we provide a way to define the condition using the expression syntax described at expr-lang.org . See example later in this article.

Custom Information Users may define additional information to retrieve from a given resource whether this is a resource CNO brings out-of-the-box or an additional resource.

Per each resource, it is possible to define any number of custom fields. Per each such field, you should define the field name and JSON path to the information within the resource JSON and an optional regular expression to extract part of that value.

By default, the system will store the information in the attributes field of cmdb_ci_kubernetes_component as a key/value in a JSON string. You can define a custom target field name in the relevant table in the CMDB.

Note: If the aggregated size of the attributes field exceeds 65 K bytes (the field size in the database), the data will not be sent to the instance. Input Config Files The Helm chart will now include an additional value with two sub-keys:

additionalResources: resources: mappings: Both sub-key values will be loaded from YAML files with the following structures.

resources YAML The resources part is a YAML array, where each element contains arrays of apiGroups , apiVersions , and resources . Example:

- apiGroups: - '' apiVersions: - v1 resources: - configmaps - apiGroups: - config.openshift.io apiVersions: - v1 resources: - clusterversions In this example, we request the system to bring into the CMDB v1/configmaps and config.openshift.io/v1/clusterversions .

When using the ClusterRole provided in the Helm chart, the system will pull the resource list into the role definition and add the verbs list , get , and watch .

With the example above, the system will create a cmdb_ci_kubernetes_component CI per each ConfigMap and ClusterVersion resource in the cluster.

mappings YAML (with basic filtering) The mappings part is a YAML map where each key represents a resource, and sub-keys represent fields, optional target class, and filters. Example:

v1/configmaps: targetClass: "u_cmdb_kubernetes_configmap" filter: - jsonPath: ".metadata.name" regex: ^version$ fields: - fieldExtractor: jsonPath: ".data.clusterFqdn" name: cluster_fqdn targetColumn: "u_cluster_fqdn" config.openshift.io/v1/clusterversions: fields: - fieldExtractor: jsonPath: ".status.desired.version" name: version In this example:

The system will bring into a custom table u_cmdb_ci_kubernetes_configmap all ConfigMap resources in the cluster whose name is "version" . It will extract from the ConfigMap the key clusterFqdn and place the value in a field called u_cluster_fqdn . The system will also bring resources of kind ClusterVersion and place them in the cmdb_ci_kubernetes_component table. It will extract the value status.desired.version and place it in a JSON map in the attributes column under a key called "version" . Rules: Both resources and mappings parts are optional, but each resource listed in the mappings part must also be listed in the resources part . Within mappings , all parts are optional. Within fields , targetColumn is optional. Within each fieldExtractor , regex is optional. Within filter , regex is optional. Negative Filtering (from version 2.4.x) It is possible to declare negative filtering — resources matching the filter will not be brought to the CMDB. In the following example, the system will bring to the CMDB all pods that do not have a label environment with value production .

The field isNegativeCondition is optional and its default value is false .

v1/pods: filter: - jsonPath: ".metadata.labels.environment" regex: "production" isNegativeCondition: true Complex Filtering (from version 3.x.x) In this example, we will bring ConfigMaps whose name is paas-info and namespace is my_namespace , or whose name contains the string some_name . The first part is an array of fields with JSON paths; the second part is a conditional expression.

{ "v1/configmaps": { "targetClass": "u_cmdb_ci_kubernetes_configmap", "complexFilter": { "fields": { "name": { "fieldExtractor": { "jsonPath": ".metadata.name" } }, "namespace": { "fieldExtractor": { "jsonPath": ".metadata.namespace" } } }, "expression": "(name == 'paas-info' and namespace == 'my_namespace') or (name matches '(.*some_name.*)')" } } } Reducing Memory Footprint (excludeFromResource) By default, the system loads the full information from the Kubernetes API server into the informer's memory. For large resources, you can reduce the memory footprint by specifying JSON paths of parts of the resource that should be excluded from the saved object.

In the mappings definition, under a specific object definition, add the key excludeFromResource followed by an array of excluded JSON paths. Example:

excludeFromResource: - ".report.components" Setting Files via Helm To set resources.yaml while running the Helm command:

--set-file additionalResources.resources=/path/to/resources.yaml To set mappings.yaml while running the Helm command:

--set-file additionalResources.mappings=/path/to/mappings.yaml Note: If you are using k8s_informer.yaml , resources and mappings values have to be provided as a JSON string. An example has been provided in k8s_informer.yaml for reference. Custom Relations On top of the default relations created by the system, you can define additional relations between the additional resources and other CIs in the Kubernetes cluster. The system requires the user to define:

JSON path and optional regex to the target resource kind. If the target resource kind is constant, use the prefix const: (e.g. const:Pod ). JSON path and optional regex to the target resource UID, or JSON path and optional regex to the target resource namespace and name. The relation type (e.g. Contains::Contained by ). The relation direction (optional) — true / false , default is true . true means the parent of the relation is the additional resource and the child is the CI identified by the declared JSON paths. Lookup by UID Create a relation from controllerrevisions to its owners. The system will look up the owner using its UID:

apps/v1/controllerrevisions: relations: - targetResourceUid: jsonPath: ".metadata.ownerReferences[:].uid" targetKind: jsonPath: ".metadata.ownerReferences[:].kind" relationType: Controller for::Controlled by relationDirection: true Lookup by Namespace and Name Create a relation from ClusterRoleBinding to the corresponding ServiceAccount (namespaced, so provide name and namespace) and to the ClusterRole (not namespaced, so provide only name):

rbac.authorization.k8s.io/v1/clusterrolebindings: relations: - targetKind: jsonPath: ".subjects[:].kind" targetResourceName: jsonPath: ".subjects[:].name" targetResourceNamespace: jsonPath: ".subjects[:].namespace" relationType: Used by::Uses relationDirection: false - targetKind: jsonPath: ".roleRef.kind" targetResourceName: jsonPath: ".roleRef.name" relationType: Used by::Uses relationDirection: false Important: At this point, the system will not delete relations that were created in the past but are no longer discovered due to changes in the declaration or in the object data. Custom Related Entries Related entries are records in a non-CMDB table that holds a reference to a CMDB record — in this case a CI that extends cmdb_ci_kubernetes_component . Users can define a custom table to hold those records and populate it using data coming from the Kubernetes API server.

The definition is in mappings.yaml and includes:

targetTable: the name of the table of the related records. referenceField: the name of the field holding the reference to the parent cmdb_ci_kubernetes_component record. arrayJsonPath: JSON path of an array object within the payload returned by the Kubernetes API server. fields: array of fields to be populated in the related entry. Each field contains: name: a friendly name. targetColumn: the column in the target table. fieldExtractor / jsonPath: how to extract the field from each element in the array. regExp (optional): extract part of the value using a regular expression. filter (optional): a map of field names to regular expressions. If provided, the system will bring only related entries whose value is fully matched by the regular expression. Note: The custom related table should be added as a related entry to the parent class identification rules. Example (all rules) rbac.authorization.k8s.io/v1/clusterroles: targetClass: cmdb_ci_kubernetes_cluster_role relatedEntries: - targetTable: u_cluster_role_rule referenceField: u_cluster_role arrayJsonPath: ".rules.*" fields: - name: api_groups targetColumn: u_api_groups fieldExtractor: jsonPath: ".apiGroups" - name: resources targetColumn: u_resources fieldExtractor: jsonPath: ".resources" - name: verbs targetColumn: u_verbs fieldExtractor: jsonPath: ".verbs" Example (with filter – write verbs only) rbac.authorization.k8s.io/v1/clusterroles: targetClass: cmdb_ci_kubernetes_cluster_role relatedEntries: - targetTable: u_cluster_role_rule referenceField: u_cluster_role arrayJsonPath: ".rules.*" filter: verbs: '.*(create|delete|update|patch).*' fields: - name: api_groups targetColumn: u_api_groups fieldExtractor: jsonPath: ".apiGroups" - name: resources targetColumn: u_resources fieldExtractor: jsonPath: ".resources" - name: verbs targetColumn: u_verbs fieldExtractor: jsonPath: ".verbs" Configuring Resource Extraction from the Instance All the extraction and mapping options available via the additionalResources.mappings parameter can be configured from the instance as well. On the instance, the configuration is provided within the informer parameter Additional Resources ConfigMap .

The parameter should contain a JSON text with the following structure:

{ "mappings": } The same structure explained above when creating the mappings.yaml applies here as well, but instead of YAML, provide JSON text. For example:

{ "mappings": { "v1/configmaps": { "targetClass": "u_cmdb_kubernetes_configmap", "filter": [ { "jsonPath": ".metadata.name", "regex": "^version$" } ], "fields": [ { "fieldExtractor": { "jsonPath": ".data.clusterFqdn" }, "name": "cluster_fqdn", "targetColumn": "u_cluster_fqdn" } ] }, "config.openshift.io/v1/clusterversions": { "fields": [ { "fieldExtractor": { "jsonPath": ".status.desired.version" }, "name": "version" } ] } } } Processing the Information on the Instance In addition to traditional business rules, ServiceNow provides hooks for inspection, manipulation, and post-processing of the IRE (Identification and Reconciliation Engine), similar to the pattern pre/post script mechanism.

Once the informer sends payloads to the instance, it is possible to handle them in several ways:

Traditional business rule – hook into a specific change and act upon it. Note that when a CI is inserted, relations are still not there and references are not yet populated. Pre-sensor script – accepts the IRE payload as a string, can examine and manipulate it, and returns either the original or manipulated payload. This option should be used with care since the script will run on every payload and should be very efficient. Navigate to Pre Post Processing , press New , select the pattern "K8s Informer Place Holder Pattern for Pre Post Scripts" , select Pre sensor in "When to execute", and write your script. Post-sensor script – takes the IRE output string as input and can take additional actions. Same navigation path; select Post sensor in "When to execute". Pre-Sensor Script Example The script below picks the version information from the ClusterVersion CI and places it in the comments field of the cluster CI: