Migrating to a new Cloud Image modelDescription This KB article explains the process for migrating to the new cloud image model 'Discovery and Service Mapping Patterns' store app version 1.10.0 or later Index OverviewRelease VersionPrerequisitesSteps to Migrate new Cloud Image ModelSteps to clean the old Model OS image type recordsIssue with the Update set solution prior to 1.10.0Pattern Execution Flow Overview Machine Image (Example: AWS Machine Image - AMI, Azure Virtual Machine Image) provides the information required to launch a Compute/VM instance. We must specify an Image when you launch an instance. Old Image Model This model uses the class cmdb_ci_os_template (label: Image). This class is a dependent class on cmdb_ci_logical_datacenter or cmdb_ci_cloud_service_account.This modelling is that each public Image is modeled in the CMDB N times where N = number of service accounts.Customers with thousands of service accounts end up discovering millions of Image records in the CMDB. New Image Model The new model will reduce the millions of Image records to thousands of records in the CMDB.Images are populated in cmdb_ci_cloud_os_image (label: Cloud Image). This is an independent class. It will create one record for each public Image (object id) irrespective of the Service account.New Model support was added only for these cloud providers - AWS, Azure, GCP, IBM and OCI Release Version This feature is available starting from: Release Version: Discovery and Service Mapping Patterns store app version 1.10.0 (November 2023) or later. Note: We recommend upgrading the discovery and Service Mapping Patterns store app to version 1.10.0 or later to adopt the new solution to migrate to New Image Model.In versions 1.10.0 and earlier, an Update Set was provided to migrate to a image type model for AWS, Azure, GCP, IBM, and OCI. If you have already migrated using the Update Set, no further action is required. Prerequisites The New Cloud Image model offers support for exclusively Discovery customers and is not available for CPG customers. If a customer is currently utilizing CPG Provisioning flows or has intentions to do so, it is advisable not to proceed with migrating to the new Image Model. It's important to note that CPG has not yet integrated support for utilizing the New Cloud Image(cmdb_ci_cloud_os_image) table. Before migrating to the new model, the following criteria should be met: The Discovery and Service Mapping Patterns plugin should be active and at version 1.10.0 or later.AWS/ Azure Customers, This new model is relevant who have already migrated from CPG to Patterns( AWS, Azure) -Migrate CPG to Pattern. This prerequisite is not applicable to GCP, IBM, and OCI customers, as GCP, IBM, and OCI follows a direct Patterns flow without a migration from CPG.The Cloud Provisioning and Governance plugin should not be active. Steps to Migrate new Cloud Image Model This solution is applicable for AWS, Azure, GCP, IBM, and OCI customers. Migration Steps: 1. Set the System Property: To migrate to a new cloud image model, set the System Property "sn_cmdb_ci_class.use_single_cloud_os_image" to true. To do this, Navigate to Filter Navigator -> enter sys_properties.list -> Search for sn_cmdb_ci_class.use_single_cloud_os_image Note: Once this property is enabled, future discoveries will populate records in the new model table (cmdb_ci_cloud_os_image). Restrictions and Important Notes: Starting from 1.10.0 and later releases, It is advised not to apply the previously provided update set.Release version 1.10.0 onwards, following patterns, won't get future updates. Amazon AWS - Executable Template (LP)Amazon AWS - Owned Template (LP)Azure - Image (LP)Oracle OCI - Image (LP) Changing the System property "sn_cmdb_ci_class.use_single_cloud_os_image" from true to false by flipping it is not supported (It means New model to Old model). Steps to clean the old Model OS image type records Follow these steps only if you need to delete records from old table (cmdb_ci_os_template): If you haven't deleted old model hardware type records, migrated to a new model and run cloud discovery, you may observe both old and new model records and relations in CMDB. To Delete old model CI records in Image(cmdb_ci_os_template) table, import the update set in Global Scope and commit it. Once the update set has been committed, follow the steps below: Navigate from the Filter Navigator -> System Definition -> Scheduled Jobs -> Search for job Named "Old Image Model records Mark as Absent"Activate the Scheduled job by clicking on "Active" and then click on "Update"This Schedule Job will periodically update the old model(cmdb_ci_os_template) cis status as Absent. Subsequently, Table cleaner will clean the Absent records in cmdb_ci_os_template table. Notes: You can adjust the repeat interval and follow system property values to optimize the performance. However, please note that the following system properties are not listed in the 'sys_properties' table. Should you wish to update these properties, you can either add them to the 'sys_properties' table or directly modify values within the scheduled job script. delete_old_image_job_count - Defines the number of jobs executed simultaneously. The default value is 10delete_old_image_batch_size - Specifies the number of records processed by each job. The default value is 10000 Considerations for Running the 'Old Image Model records Mark as Absent' Scheduled Job: When running the scheduled job that marks old image model records as absent, consider the following to avoid potential performance issues: System Resource Management: For instances with large datasets, schedule this job during off-peak times to reduce the load on the system. Ensure that no other scheduled jobs are running concurrently, and verify that the instance has adequate resources available.Table Cleaner Operation: Once records are marked as absent, the Table Cleaner process will continuously scan and remove these records. Although it works in regular intervals, this constant background activity may lead to performance degradation, especially if a large number of records need to be processed.Performance Impact: Since the Table Cleaner may take time to process absent records, it can put additional load on your instance, potentially affecting overall performance. Best Practice: To reduce performance risks and prevent further impact on instance performance, manually schedule this job during low-activity periods, ideally for couple of hours when system load is minimal. Rather than marking all CIs as absent at once, break the job into multiple intervals, processing smaller batches of data during each run for the Table Cleaner. This approach helps avoid significant performance degradation. Issue with the Update set solution prior to 1.10.0 For versions prior to 1.10.0, migrating to the new Cloud Image model using the Update Set requires the execution of a fix script provided in the update set.The fix script will: Enable the system property "sn_cmdb_ci_class.use_single_cloud_os_image"Implement necessary orchestration changes to AWS, Azure and OCI Patterns. Important: Any changes made by the fix script will be treated as customizations and will not be included in future updates. Be aware that these customizations will need to be maintained separately. Pattern Execution Flow Note: GCP cloud discovery use the 'Google Cloud Platform (GCP) - Virtual Server' pattern for both old and new models to populate the cmdb_ci_os_template and cmdb_ci_cloud_os_image table based on System property value. In the 1.10.0 version, four new patterns were introduced: Amazon AWS - Executable Cloud OS Image (LP)Amazon AWS - Owned Cloud OS Image (LP)Azure - Cloud OS Image (LP)Oracle OCI - Cloud OS Image (LP) CloudNew PatternCorresponding Old PatternAWSAmazon AWS - Executable Cloud OS Image (LP)Amazon AWS - Executable Template (LP)Amazon AWS - Owned Cloud OS Image (LP)Amazon AWS - Owned Template (LP)AzureAzure - Cloud OS Image (LP)Azure - Image (LP)OCIOracle OCI - Cloud OS Image (LP)Oracle OCI - Image (LP) Before the 1.10.0 release, AWS, Azure, and OCI customers executed a fix script to implement the necessary orchestration changes. Depending on the migration scenario, customers may observe the following behaviors: ScenarioOld Pattern BehaviourNew Pattern BehaviourNot migrated to New hardware type modelProcess the pattern and populate cmdb_ci_os_template tableGracefully terminate the patternMigrated to New hardware type model before 1.10.0 version (via Fix Scripts)Process the pattern and populate cmdb_ci_cloud_os_image tableGracefully terminate the patternMigrated to New hardware type model version 1.10.0 or laterGracefully terminate the patternProcess the pattern and populate cmdb_ci_cloud_os_image table Note: GCP and IBM discoveries use the same pattern for both old and new flows, so these flow changes do not apply to GCP and IBM clouds. Flow Diagram: Note: This flow diagram does not apply to GCP and IBM customers.