Servicenow Discovery - AWS Discovery - Discovery not finding the VMs in af-south-1 regionIssue Using the AWS Main Member account discovery system, Cloud Discovery was activated for accounts that have enabled regions other than the default.However, it was unsuccessful in retrieving Cloud resources and resulted in an error message stating that the access credentials provided could not be validated.This error was logged in the Cloud API Trail/Pattern Logs.CauseWhen checking out the section in "Regions and Endpoints in STS (Security Token Service)", the sections mention When attempting to call the DescribeInstances operation, an error occurred stating "AuthFailure," which was caused by AWS being unable to verify the access credentials providedThe reason for this is that the IAM service for the main/management account, where my IAM user is located, has the STS region compatibility set to only allow session tokens for legacy global regionsTherefore, the temporary credentials provided by the STS service cannot be used in the af-south-1 regionTo resolve this, I toggled the policy to allow "All AWS Regions" in the account where my IAM user is located (139769765351) and requested new temporary credentialsThis worked successfully for the af-south-1 region. It's important to note that the session token is longer when it's valid across all regions, as stated in the documentationResolutionWe conducted a thorough investigation and tested various solutions in a lab environment. As a result, we have developed some workarounds that may be helpful to you. Please refer to the "Managing global endpoint session tokens" section. By default, most AWS Regions are enabled for use with all AWS services and AWS STS. However, some regions like Asia Pacific (Hong Kong) need to be enabled manually.For more information on enabling and disabling AWS Regions, refer to the "Managing AWS Regions" section in the AWS General Reference. When these AWS Regions are enabled, they are automatically activated for use with AWS STS. Note that you cannot activate the STS endpoint for a disabled Region. Tokens that are valid across all AWS Regions are longer than tokens that are only valid in Regions enabled by default. Changing this setting may have an impact on existing systems where tokens are temporarily stored.The setting for Region compatibility of session tokens for the global endpoint can be modified using the AWS Management Console, AWS CLI, or AWS API. To change this setting through the console, refer to the attached screenshots for guidance (Note: please modify the setting for af-south-1 Region). Open the IAM console. In the navigation pane, choose Account settingsIf needed, locate the Security Token Service (STS) section and expand it. In the first table next to the Global endpoint, look for the column labeled "Region compatibility of session tokens." It should currently read "Valid only in AWS Regions enabled by default." Click on the "Change" option.In the Change region compatibility of session tokens for global endpoint dialog box, select Valid in all AWS Regions. Then choose Save changes. Refer to AWS Discovery Issue with non-default regions (e.g., af-south-1) for more details Another possible solution is to create credentials for the member account in question, allowing discovery over the af-south-1 Region to be conducted through credentials. This can be achieved by removing the parent_account reference and updating the discovery_credentials reference in the corresponding Cloud Service Account entry.Related Links To address the issue, it is recommended to use the region-specific STS endpoint instead of the global endpoint. This issue is now related to the Discovery Infrastructure rather than CMP or Discovery content. However, before transferring the issue, it is suggested to have a discussion with their team to avoid unnecessary back and forth and ensure an accurate resolution. "New AWS Regions not enabled by default (for example, Hong Kong and Bahrain) use the updated AWS STS format. The global AWS STS endpoint (sts.amazonaws.com) issues tokens in the previous format by default. This error can occur if a session token is trying to use the previous format with an AWS Region not enabled by default.""Version 1 tokens are valid only in AWS Regions that are available by default. These tokens do not work in manually enabled Regions, such as Asia Pacific (Hong Kong). Version 2 tokens are valid in all Regions. However, version 2 tokens include more characters and might affect systems where you temporarily store tokens.""By default, AWS Security Token Service (STS) is available as a global service, and all STS requests go to a single endpoint at https://sts.amazonaws.com. AWS recommends using Regional STS endpoints to reduce latency, build in redundancy, and increase session token availability.""If you make an STS call to the global endpoint, the resulting session tokens might be valid in some regions but not others. It depends on the version that is set in this operation. Version 1 tokens are valid only in AWS Regions that are available by default. These tokens do not work in manually enabled Regions, such as Asia Pacific (Hong Kong). Version 2 tokens are valid in all Regions. However, version 2 tokens are longer and might affect systems where you temporarily store tokens" URL(s):https://aws.amazon.com/premiumsupport/knowledge-center/iam-validate-access-credentials/https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp_enable-regions.html#sts-regions-manage-tokenshttps://docs.aws.amazon.com/IAM/latest/APIReference/API_SetSecurityTokenServicePreferences.htmlIt is now clear why session tokens were being generated in both accounts but not authenticated when logging into the member account. This is because the API call was being made to the Global STS endpoint to retrieve the session token, instead of the region-specific endpoint, which is different for each region. For example, the STS endpoint for the Africa (Cape Town) region is "sts.af-south-1.amazonaws.com".CloudServiceAccountCredentialUtil.java: https://code.devsnc.com/dev/mid/blob/master/mid/src/com/service_now/mid/util/CloudServiceAccountCredentialUtil.java