Migrating to a new hardware type modelDescription Hardware types are configuration items (CI)s that represent pre-defined virtual machine types provided by cloud vendors. Cloud vendors provide several hundreds of out-of-box types. In most cases, customers are using the out-of-box types(Example- t2.nano, t2.micro, t2.large in AWS). Old Hardware Type Model This model uses the class cmdb_ci_compute_template (label: Hardware Type). This class is a dependent class on cmdb_ci_logical_datacenter.The result of this modeling is that each out-of-box hardware type is modeled in the CMDB N times where N = number of service accounts * number of logical datacenters.Customers with hundreds of service accounts end up discovering millions of hardware type records in the CMDB. This slows down discovery and other related flows that can cause performance issues.Hence, customers are puzzled by the fact that hundreds of hardware types present in the cloud (AWS, AZURE, GCP) result in millions of records being persisted in the CMDB. New Hardware Type Model This model reduces the number of hardware type records in the CMDB.Hardware types are populated in cmdb_ci_cloud_hardware_type (label: Cloud Hardware Type). This is an independent class. It will create one record for each hardware type irrespective of the Service account and data center.This will reduce the millions of hardware type records to hundreds of records. Prerequisites Before migrating to the new model, the following criteria should be met: For AWS/ Azure Customers, this new model is relevant who have already migrated from CAPI to Patterns( AWS, Azure) -Migrate CAPI to pattern. Customization should not be done in the below-mentioned patterns "Discovery and Service Mapping Patterns" store app version 1.0.74 or before it. If customization is done on or before 1.0.74 version, the customer needs to revert their changes before upgradation and upgrade it. After upgrading, the customer needs to add their customization changes. Below OOB patterns are modified in version 1.0.75 to support the new model. Amazon AWS - Hardware Type (LP)Amazon AWS - Virtual Server (LP)Amazon AWS Virtual Server EventsAzure - Hardware Type (LP)Azure - Virtual Machine (LP)Azure Virtual Machine EventGoogle VM - Get Compute templateGoogle GCP - Get instance templates Note: Release version June release 1.6.0 (i.e. Next version after 1.3.0) onwards of the Discovery and Service Mapping Patterns store app, this KB is no longer considered valid. Please refer to KB1285337 for information related to June 2023 or later. Instructions Ensure there are no active Discovery jobs running for Cloud services (AWS, AZURE, GCP).Import the update set (MigrateToNewHardwareTypeModel) and follow the below mentioned steps in "Steps to follow after committing UpdateSet".After following the steps mentioned in "Steps to follow after committing UpdateSet", Verify that the system property - sn_itom_pattern.use a single hardware type for cloud data centers is set to true.Sync the Patterns with mid servers( Pattern sync with Mid).Run Discovery jobs for cloud services to populate a new hardware type table - cmdb_ci_cloud_hardware_type. Steps to follow after committing UpdateSet 1. Set the System Property - delete_old_hardware_type_batch_size. Default vaule is 1000. From the Filter Navigator -> enter sys_properties.list -> Search for delete_old_hardware_type_batch_size 2. Run the Schedule Job- Delete Old Hardware Types Model records This Scheduled Job periodically deletes the old model(cmdb_ci_compute_template) records. Note: If you have not deleted old model records, migrated to a new model and run cloud discovery, you can observe both old and new model records and relations in CMDB. a. Navigate from the Filter Navigator -> System Definition -> Scheduled Jobs -> Search for Delete Old Hardware Types Model records b. On Delete Old Hardware Types Model records, Click on Active ->Click on Update c. Now Delete Old Hardware Types Model records is active. This scheduled Job repeat interval is 2 minutes and deletes delete_old_hardware_type_batch_size records. Note: You can modify the repeat interval and batch size values that work optimally for you. d. This schedule Job deletes all the records from the cmdb_ci_compute_template table, expect this step to take time(maybe days depending on Ci's in cmdb_ci_compute_template table). e. After cleaning the cmdb_ci_compute_template table, disable the "Delete Old Hardware Types Model records" schedule. Because VMware cloud discovery is still populating the Hardware types in the cmdb_ci_compute_template table. Note: If you are using VMware cloud discovery, this scheduled job deletes VMware cloud discovery populated cmdb_ci_compute_template CIs if the scheduled job is Active. 3. Run the Fix script Migrate to new Hardware Model a.Navigate from the Filter Navigator -> System Definition -> Fix Scripts then search for Migrate to new Hardware Model. b.Click on the Fix script titled, Migrate to new Hardware Model then verify the whether Unloadable check box is enabled or not. If not, Check whether the committed UpdateSet is the latest or not. In the attached UpdateSet, Unloadable check box is enabled. c. Click on Run Fix script d. When the Run Fix Script window shows up, click proceed. e. Validate the fix script has been executed whether successfully or not by viewing the logs. If it has been not completed successfully then check the logs to identify the cause. Then close the window. Note 1: Fix script-made changes are considered as customization. Hence, the below patterns won't receive future OOB updates. Amazon AWS - Hardware Type (LP)Azure - Hardware Type (LP) Note 2: If you wish to capture the customizations to the "Discovery and Service Mapping Patterns" plugin then publish the Fix script generated Customer Update[sys_update_xml] records. You can claim these changes(Mentioned in Step#:5) done by the Fix script before the next upgrade/repair to avoid an unexpected reversion. For more details, you can refer Manage customizations to store applications Note: This step is only applicable when you are using App customization to manage customizations. 4. Verify the System Property - sn_itom_pattern.use a single hardware type for cloud data centers is updated to true From the Filter Navigator -> enter sys_properties.list -> Search for sn_itom_pattern.use a single hardware type for cloud data centers 5. After Run the Fix script, the below records are generated in Customer Updates(sys_update_xml) table and those records are considered as customized. Hence, the below records won't receive future OOB updates. S.NOTypeTarget NameNameScope of the FileRemarks1System Propertysn_itom_pattern.use a single hardware type for cloud data centerssys_properties_8560e8b21b6a605036afc8851a4bcb56Discovery and Service Mapping PatternsValue changed from false to true2Discovery PatternsAmazon AWS - Hardware Type (LP)sa_pattern_f80c73351bda4010e1b863913d4bcb9dDiscovery and Service Mapping Patternsci_type is changed from cmdb_ci_compute_template to cmdb_ci_cloud_hardware_type3Discovery PatternsAzure - Hardware Type (LP)sa_pattern_c1d05d691b15101010a6dce7cc4bcb9cDiscovery and Service Mapping Patternsci_type is changed from cmdb_ci_compute_template to cmdb_ci_cloud_hardware_type4Pattern Outputs sn_pattern_outputs_9291fe9fdb54d4901f2d9207db9619b1Discovery and Service Mapping Patternsci_class_type is changed from cmdb_ci_compute_template to cmdb_ci_cloud_hardware_type5Pattern Outputs sn_pattern_outputs_d4ae4ee1db951010b2a1f962759619d1Discovery and Service Mapping Patternsci_class_type is changed from cmdb_ci_compute_template to cmdb_ci_cloud_hardware_type6Extended Pattern Input Parameterscompute_templatesdiscovery_ptrn_lnch_param_def_ext_5868be5bdb94d4901f2d9207db96199cDiscovery and Service Mapping Patternsci_class is changed from cmdb_ci_compute_template to cmdb_ci_cloud_hardware_type7Extended Pattern Input Parameterscmdb_ci_cloud_hardware_typediscovery_ptrn_lnch_param_def_ext_420f0625db951010b2a1f9627596195bDiscovery and Service Mapping Patterns1. ci_class is changed from cmdb_ci_compute_template to cmdb_ci_cloud_hardware_type2. "key" is changed from cmdb_ci_compute_template to cmdb_ci_cloud_hardware_type"8View Tablecmdb_ci_cloud_hardware_typesys_db_view_table_b69072665b755010c937839db881c777Discovery and Service Mapping PatternsChanged table from cmdb_ci_compute_template to cmdb_ci_cloud_hardware_type9View Tablecmdb_ci_cloud_hardware_typesys_db_view_table_b4c496be5b395010c937839db881c7f1Discovery and Service Mapping PatternsChanged table from cmdb_ci_compute_template to cmdb_ci_cloud_hardware_type10View Tablecmdb_ci_cloud_hardware_typesys_db_view_table_280458575bf19010c937839db881c758Discovery and Service Mapping PatternsChanged table from cmdb_ci_compute_template to cmdb_ci_cloud_hardware_type11Related CI Typesb9e53aec1be30010f6f2ea08bc4bcbebsa_ci_to_pattern_b9e53aec1be30010f6f2ea08bc4bcbebGlobalci_type changed from cmdb_ci_compute_template to cmdb_ci_cloud_hardware_type12Related CI Typescf6f339bdb3f8410a480265d13961980sa_ci_to_pattern_cf6f339bdb3f8410a480265d13961980Global 1. ci_type changed from cmdb_ci_compute_template to cmdb_ci_cloud_hardware_type 2. deletion_strategy changed from Mark as Retired to Keep 13Related CI Types4a538120db561300d504788dbf961913sa_ci_to_pattern_4a538120db561300d504788dbf961913Globalci_type changed from cmdb_ci_compute_template to cmdb_ci_cloud_hardware_type14Related CI Types12869e211b15101010a6dce7cc4bcbc3sa_ci_to_pattern_12869e211b15101010a6dce7cc4bcbc3Globalci_type changed from cmdb_ci_compute_template to cmdb_ci_cloud_hardware_type15Related CI Types1576105adbb91c10b2a1f962759619c5sa_ci_to_pattern_1576105adbb91c10b2a1f962759619c5Globalci_type changed from cmdb_ci_compute_template to cmdb_ci_cloud_hardware_type16Related CI Types43a05cf21bf91c1010a6dce7cc4bcbb7sa_ci_to_pattern_43a05cf21bf91c1010a6dce7cc4bcbb7Global1. ci_type changed from cmdb_ci_compute_template to cmdb_ci_cloud_hardware_type2. deletion_strategy changed from Mark as Retired to Keep17Related CI Typesdc52279b0b832300a47f8e4335673acbsa_ci_to_pattern_dc52279b0b832300a47f8e4335673acbGlobalci_type changed from cmdb_ci_compute_template to cmdb_ci_cloud_hardware_type Note: UpdateSet (CloudOsImageMigrationUpdateSet.xml) was created in Global Scope and it contains the below records. S.NOTypeTarget NameNameRemarks1Fix ScriptMigrate to new Hardware Modelsys_script_fix_69d7cb5e1b7e2810a2d92f092a4bcbe7This script will change the patterns orchestrator to support the new model 2System Propertydelete_old_hardware_type_batch_sizesys_properties_746d476e1bdf6c10e2c665b6624bcb63Number of cmdb_ci_compute_template records delete in each batch3Scheduled Script ExecutionDelete Old Hardware Types Model recordssysauto_script_fa804c0597e4e5108d3b350e6253af78 This Scheduled Job deletes the old model records (cmdb_ci_compute_template) records periodically. Note: If you have not deleted old model records and migrated to a new model and run cloud discovery, you can observe both old and new model records and relations in CMDB. 6. If you wish to capture the customizations to the "Discovery and Service Mapping Patterns" plugin then publish the Fix script generated Customer Update[sys_update_xml] records. You can claim these changes(Mentioned in Step#:5) done by the Fix script before the next upgrade/repair to avoid an unexpected reversion. For more detials, you can refer Manage customizations to store applications Note: This step is only applicable when you are using App customization to manage customizations. Frequently Asked Questions 1. If the customer is using CPG and family version prior to San Diego Release, Can the customer migrate to New Model? Ans. The customer can not migrate to the new model. If migrates, it causes issues in CPG flow. Currently, don't have WA. CPG supporting this new model from San Diego release onwards. 2. Is recommended to delete records directly from individual tables to avoid the delay caused by cascade delete? Ans. No, Not recommended. 3. Can we change the sequence of jobs? Ans. Not recommended to change the sequence of jobs/steps order. Run the migration fix script Before cleaning old CI's from 'cmdb_ci_compute_template' table may cause issues- CMDB contains both old and new CI records and relations. 4. Changes done by Fix Script(Migrate to new Hardware Model) are overriding after Store App upgradation. how to handle it? Most Probable cause - If the Fix script runs without enabling unloadable checkbox then the System property "sn_itom_pattern.use a single hardware type for cloud data centers" and some other records that are modified by the Fix script are not present in Customer Update[sys_update_xml] table. Hence, after the Store App upgradation, the System property value reverted to false (OOB, value is false) [Amazon AWS - Hardware Type (LP)/ Azure - Hardware Type (LP) Patterns CI type is cmdb_ci_cloud_hardware_type and System Property 'sn_itom_pattern.use a single hardware type for cloud data centers' is false]. So, it causes pattern failures with this message "Discovery using patterns failed due to lack of existence of the main CI type in the results. Check the discovery logs for more details". WA: Import the update set -> From the above Instructions section -> Step 2. Import the update set in this KB and follow the steps mentioned in "Run the Fix script - Migrate to new Hardware Model ". Note 1: Fix script in latest Updateset contains logic to create records in customer updates(sys_update_xml) table to avoid unexpected reversion of records while upgrade. Re-running the fix script won't cause any issues. Note 2: Re-run the fix script won't delete any CIs in CMDB. Note 3: Fix script runs with the Unloadable check box enabled, it creates Customer Update[sys_update_xml] records when the fix script runs. 5. Changes done by the Fix script are present (Migrate to new Hardware Model) and still facing issues- Discovery using patterns failed due to lack of existence of the main CI type in the results. Check the discovery logs for more details. how to resolve it? Cause: Amazon AWS - Hardware Type (LP)/ Azure - Hardware Type (LP) Patterns CI type is cmdb_ci_cloud_hardware_type and System Property 'sn_itom_pattern.use a single hardware type for cloud data centers' is true. Still, Patterns are failing with the error "Discovery using patterns failed due to lack of existence of the main CI type in the results. Check the discovery logs for more details". Then Might be Mid server Sync not happened properly or the Patterns did not Sync to the MID Server (NDL still shows old hardware type as the CI Type) WA: sync the Patterns with mid servers( Pattern sync with Mid). If you've already performed this step and still getting the same error then restarting the MID Server will often resolve this. if it doesn't then please open a Case with ServiceNow Support to investigate further for any MID Server issues.