Performance results for CNO for VisibilityIntroduction This article lists out the performance metrics and results on CNO for Visibility. These results are collected by performing tests both on Informer side and Instance side. Purpose These performance results will help in estimating the load, memory, CPU etc on the Instance as well as on the k8s cluster on which the Informer is deployed. Performance Results Following are the performance results from multiple scenarios that are tested: Informer side test results This test is measuring the performance of K8S Informer which in deployed in a cluster with 50K pods. The below results are based on the tests performed using - Instance Details: Nodes - 2, Version: Utah K8S Cluster Details: OS: CentOS, CPU - 8 cores, Memory - 16GB, Nodes - 8 ParametersValuesDiscovery Time33 minNo. of Pods in Cluster 50040No of Pods Discovered50040Informer CPU6.25%Informer Memory (Max)1.21GBNo. of ECC Queue records created67ECC Queue to CMDB population timeMin : 44 secondsMax : 725 secondsAvg : 413 seconds The following table gives a view on required Informer memory per number of Pods No of PodsInformer Memory(Max)1K50 MB5K139 MB25K549 MB Instance side test results This test is measuring the performance of 300 clusters with both small and large payloads for a duration of 1hour. The below results are based on the tests performed using - Instance Details on which tests are performed: Nodes = 2Semaphores = 16 * 2 = 32Semaphores API_INT Available semaphores: 4Queue depth: 0Max queue depth: 47Queue depth limit: 50Maximum transaction concurrency: 4Maximum concurrency achieved: 4 DB Size : SmallDB Connection Pool Size: 32 Below are the results that are collected by using a simulator which acts as an Informer. Scenario 1: 300 simulators are used and each one sends a payload every 30seconds to the instance for 1 hour. The below table shows the results for this scenario. ParameterResult 1Result 2Payload Size3.4KB340KBTest Duration1 hr1 hrNo of Clusters30050CIs per Payload (per cluster)6600Send interval (seconds)3030Change Rate (per sec)0.220No of ECC Queue Created (per hour)351755988No of ECC Queue Processed351752884ECC Queue to CMDB Population Time (Max)19 seconds3605 secondsNo. of 429 Rejections00API_INT Queue Depth MaxNode 1 : 10Node 1 : 0Node 2 : 1Node 2 : 0Instance App Host CPU Increase during discovery1%8%Instance App Host Memory Increase during discoveryNo change512 MBDB Host CPU Increase during discovery11%14% Scenario 2: 300 simulators are used and each one sends a payload every 30seconds to the instance for 5minutes. This is a scenario where there is a sudden surge in the incoming requests. The below table shows the results for this scenario. ParametersValuesTest Duration5 minNo of Clusters300CIs per Payload (per cluster)200Send interval (seconds)30Change Rate (per sec)6.66No of ECC Queue Created (per hour)2997No of ECC Queue Processed2997ECC Queue to CMDB Population TimeMin : 9 secondsMax : 3122 secondsAvg : 1660 seconds Note: These performance tests are conducted on Instance and K8S cluster with no other load.