AWS Discovery Datacenter Filtering: mid.cloud.discovery.sonar.discover_all_aws_datacentersSummary<!-- /*NS Branding Styles*/ --> .ns-kb-css-body-editor-container { p { font-size: 12pt; font-family: Lato; color: #000000; } span { font-size: 12pt; font-family: Lato; color: #000000; } h2 { font-size: 24pt; font-family: Lato; color: black; } h3 { font-size: 18pt; font-family: Lato; color: black; } h4 { font-size: 14pt; font-family: Lato; color: black; } a { font-size: 12pt; font-family: Lato; color: #00718F; } a:hover { font-size: 12pt; color: #024F69; } a:target { font-size: 12pt; color: #032D42; } a:visited { font-size: 12pt; color: #00718f; } ul { font-size: 12pt; font-family: Lato; } li { font-size: 12pt; font-family: Lato; } img { display: ; max-width: ; width: ; height: ; } } Overview Starting with Discovery and Service Mapping Patterns store version 1.29.0, ServiceNow introduced an AWS datacenter discovery model intended to focus discovery on datacenters with resources and exclude datacenters that do not contain resources. To enable that behavior, set the MID Server property mid.cloud.discovery.sonar.discover_all_aws_datacenters to false.This property enables a filtering library inside the AWS datacenter discovery pattern. The library classifies each region and removes only the regions it classifies as Passive from the working region set. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Where the property is used (pattern and library) Pattern name: Amazon AWS Datacenter discoveryPattern link: https://<instance_name>.service-now.com/nav_to.do?uri=sa_pattern.do?sys_id=370a7716db765300b91e85184b9619f1 Library name: Filter empty aws datacentersLibrary link: https://<instance_name>.service-now.com/nav_to.do?uri=sa_pattern.do?sys_id=464ef9a82b3bea104987fa466e91bf89 Key behavior: the property is used as a gate to decide whether the library runs; if the library does not run, there is no Passive-based filtering. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Pattern flow that matters for this property (step names in order) Step 1: Fill cmdb_ci_logical_datacenter table→ Creates the working set of AWS datacenter records (regions) in cmdb_ci_aws_datacenter Step 2: Remove Duplicate Data Centers→ De-duplicates the working set Step 3: check if we need to discover all datacenters→ Reads mid.cloud.discovery.sonar.discover_all_aws_datacenters from MID configuration and stores it in discoverAllDatacenters→ The read is treated as a string and defaults to true if the property is missing Step 4: Reference to library "Add sonar state to aws datacenters"→ Invokes Filter empty aws datacenters only when all are true→ discoverAllDatacenters equals false (strict match to literal false)→ cmdb_ci_aws_datacenter is not empty→ flow_type is empty Step 5: Relation between cmdb_ci_aws_datacenter to cmdb_ci_cloud_service_account→ Relationships are created after filtering, so the post-filter working set is carried forward Important prerequisite gate→ If cmdb_ci_aws_datacenter is empty, Step 3 is not executed and Step 4 cannot run ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ What changes when the property is false When the property is false and Step 4 gates are met→ The library runs→ The library writes a classification into cmdb_ci_aws_datacenter.comments for each region→ The library then filters the working set and removes only regions whose comments contains PassiveThis is the documented optimization described as focusing discovery on datacenters with resources. Important detail that affects troubleshooting→ The gate is a strict string match to the literal value false, so the property value must match exactly to trigger Step 4 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Library flow (step names) Library step 1: Fetch active AWS regions→ Checks each region and produces a status per region in a map named region_status_map Library step 2: Add comments to cmdb_ci_aws_datacenter→ Copies the status into cmdb_ci_aws_datacenter.comments Library step 3: Filter active datacenters→ Filters the in-memory working set by removing only rows where comments contains Passive ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ AWS APIs used by the library Primary API: GetDiscoveredResourceCounts→ The library uses this to list what resource types AWS Config reports as discovered in that region Secondary API: DescribeConfigurationRecorderStatus→ The library uses this only when it sees zero discovered resources, to check whether the recorder is present and recording ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Region classification (Active, Passive, blank in comment field) The library repeats the same story for each region in the working datacenter list. Part 1: The excluded list it uses The library treats some resource families as “not enough to call the region active” and keeps them in an excluded array. Excluded resource family prefixes (exact array values, in one line)AWS::Config, AWS::S3, AWS::IAM, AWS::SNS, AWS::Lambda, AWS::CloudWatch, AWS::Logs, AWS::KMS, AWS::Events How it compares→ It takes a discovered resource type like AWS::EC2::Instance→ It keeps only the family prefix AWS::EC2→ It compares that family prefix against the excluded list Part 2: When a region becomes Active in comment A region becomes Active when the library finds at least one discovered resource family that is not in the excluded list.→ The moment it finds one non-excluded family, it marks the region Active and stops checking further types for that regionPractical meaning→ Active means this region stays in the working set and is not removed by the filter step Part 3: When a region becomes Passive in comment A region becomes Passive only when it never became Active and the library concludes it should be excluded.There are two paths where the library sets Passive. Passive path 1→ The region never became Active→ The library reaches the non-active fallback logic that marks the region Passive Passive path 2→ The region never became Active→ The library sees AWS Config reports total discovered resources as zero→ It calls DescribeConfigurationRecorderStatus→ If the recorder status exists and recording is true, it marks the region Passive Practical meaning→ Passive is the only status that causes the region to be removed by the filter step Part 4: When a region becomes blank in comment A region becomes blank when the library cannot confidently mark it Active or Passive.Blank happens in these situations. Blank situation 1→ The library did not get a usable response payload back for the region from GetDiscoveredResourceCounts Blank situation 2→ AWS Config reported zero discovered resources→ The follow-up recorder-status check did not confirm recording is true→ The library leaves the status empty Blank situation 3→ The library hit an exception while checking the region→ In the error path it stores an empty status Practical meaning→ Blank stays in the working set because the filter removes only Passive ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ How the filter is applied (the exact effect) After the library writes cmdb_ci_aws_datacenter.comments→ The filter step removes only rows where comments contains Passive hence→ Rows with comments = Active remain→ Rows with comments = blank remain ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Troubleshooting: property is false but datacenters still appear to be scanned This troubleshooting uses only facts from the pattern gates and the library outputs. Troubleshooting step 1: confirm the library step executed in the pattern run Open the Amazon AWS Datacenter discovery pattern execution and verify these step names executed.→ check if we need to discover all datacenters→ Reference to library "Add sonar state to aws datacenters" If the library step did not execute, filtering did not happen in that run. The code-level reasons are.→ The property value did not match the literal string false→ cmdb_ci_aws_datacenter was empty so the property read step did not execute→ flow_type was not empty so the library gate did not pass Troubleshooting step 2: confirm that the library produced any Passive regions The filter removes regions only when a region’s comments contains Passive.→ Check cmdb_ci_aws_datacenter.comments for the regions created or updated by the runIf no region has comments containing Passive, the working set cannot be reduced by this feature. Troubleshooting step 3: use comments values to identify what happened per region If comments = Active→ The library found at least one discovered resource family outside the excluded list If comments = Passive→ The region never became Active and the library set Passive by one of the two Passive paths If comments is blank→ One of these happened→ No usable response payload from GetDiscoveredResourceCounts→ Zero discovered resources but recorder-status did not confirm recording true→ Exception path stored empty status→ Blank is retained, so those regions remain eligible in the working set Troubleshooting step 4: confirm that filtering could have any visible impact The filtering step can reduce the region list only when there is at least one Passive region.→ If your run produces Active and blank only, the discovery will still fan out across regions even though the property is false and the library ran Troubleshooting step 5: prerequisites This behaviour is documented starting with Patterns store version 1.29.0 and requires Discovery and Service Mapping Patterns installed at that version or later. Related Links<!-- /*NS Branding Styles*/ --> .ns-kb-css-body-editor-container { p { font-size: 12pt; font-family: Lato; color: #000000; } span { font-size: 12pt; font-family: Lato; color: #000000; } h2 { font-size: 24pt; font-family: Lato; color: black; } h3 { font-size: 18pt; font-family: Lato; color: black; } h4 { font-size: 14pt; font-family: Lato; color: black; } a { font-size: 12pt; font-family: Lato; color: #00718F; } a:hover { font-size: 12pt; color: #024F69; } a:target { font-size: 12pt; color: #032D42; } a:visited { font-size: 12pt; color: #00718f; } ul { font-size: 12pt; font-family: Lato; } li { font-size: 12pt; font-family: Lato; } img { display: ; max-width: ; width: ; height: ; } } Limit AWS discovery to datacenters with resources