Migrating to a new hardware type model - Support and Troubleshooting

Migrating to a new hardware type model Description Hardware types are configuration items (CI)s that represent pre-defined virtual machine types provided by cloud vendors. Cloud vendors provide several hundreds of out-of-box types. In most cases, customers are using the out-of-box types(Example- t2.nano, t2.micro, t2.large in AWS). Old Hardware Type Model This model uses the class cmdb_ci_compute_template (label: Hardware Type). This class is a dependent class on cmdb_ci_logical_datacenter. The result of this modeling is that each out-of-box hardware type is modeled in the CMDB N times where N = number of service accounts * number of logical datacenters. Customers with hundreds of service accounts end up discovering millions of hardware type records in the CMDB. This slows down discovery and other related flows that can cause performance issues. Hence, customers are puzzled by the fact that hundreds of hardware types present in the cloud (AWS, AZURE, GCP) result in millions of records being persisted in the CMDB. New Hardware Type Model This model reduces the number of hardware type records in the CMDB. Hardware types are populated in cmdb_ci_cloud_hardware_type (label: Cloud Hardware Type). This is an independent class. It will create one record for each hardware type irrespective of the Service account and data center. This will reduce the millions of hardware type records to hundreds of records. Prerequisites Before migrating to the new model, the following criteria should be met:

For AWS/ Azure Customers, this new model is relevant who have already migrated from CAPI to Patterns( AWS, Azure) - Migrate CAPI to pattern . Customization should not be done in the below-mentioned patterns " Discovery and Service Mapping Patterns" store app version 1.0.74 or before it. If customization is done on or before 1.0.74 version, the customer needs to revert their changes before upgradation and upgrade it. After upgrading, the customer needs to add their customization changes. Below OOB patterns are modified in version 1.0.75 to support the new model. Amazon AWS - Hardware Type (LP) Amazon AWS - Virtual Server (LP) Amazon AWS Virtual Server Events Azure - Hardware Type (LP) Azure - Virtual Machine (LP) Azure Virtual Machine Event Google VM - Get Compute template Google GCP - Get instance templates Note: Release version June release 1.6.0 (i.e. Next version after 1.3.0) onwards of the Discovery and Service Mapping Patterns store app, this KB is no longer considered valid. Please refer to KB1285337 for information related to June 2023 or later. Instructions Ensure there are no active Discovery jobs running for Cloud services (AWS, AZURE, GCP). Import the update set ( MigrateToNewHardwareTypeModel ) and follow the below mentioned steps in " Steps to follow after committing UpdateSet ". After following the steps mentioned in "Steps to follow after committing UpdateSet", Verify that the system property - sn_itom_pattern.use a single hardware type for cloud data centers is set to true . Sync the Patterns with mid servers( Pattern sync with Mid). Run Discovery jobs for cloud services to populate a new hardware type table - cmdb_ci_cloud_hardware_type.

Steps to follow after committing UpdateSet 1. Set the System Property - delete_old_hardware_type_batch_size. Default vaule is 1000.

From the Filter Navigator -> enter sys_properties.list -> Search for delete_old_hardware_type_batch_size

2. Run the Schedule Job- Delete Old Hardware Types Model records

This Scheduled Job periodically deletes the old model(cmdb_ci_compute_template) records.

Note: If you have not deleted old model records, migrated to a new model and run cloud discovery, you can observe both old and new model records and relations in CMDB.

a. Navigate from the Filter Navigator -> System Definition -> Scheduled Jobs -> Search for Delete Old Hardware Types Model records

b. On Delete Old Hardware Types Model records , Click on Active ->Click on Update

c. Now Delete Old Hardware Types Model records is active . This scheduled Job repeat interval is 2 minutes and deletes delete_old_hardware_type_batch_size records.

Note: You can modify the repeat interval and batch size values that work optimally for you.

d. This schedule Job deletes all the records from the cmdb_ci_compute_template table, expect this step to take time(maybe days depending on Ci's in cmdb_ci_compute_template table).

e. After cleaning the cmdb_ci_compute_template table, disable the " Delete Old Hardware Types Model records " schedule. Because VMware cloud discovery is still populating the Hardware types in the cmdb_ci_compute_template table.

Note: If you are using VMware cloud discovery, this scheduled job deletes VMware cloud discovery populated cmdb_ci_compute_template CIs if the scheduled job is Active.

3. Run the Fix script Migrate to new Hardware Model

a.Navigate from the Filter Navigator -> System Definition -> Fix Scripts then search for Migrate to new Hardware Model .

b.Click on the Fix script titled, Migrate to new Hardware Model then verify the whether Unloadable check box is enabled or not. If not, Check whether the committed UpdateSet is the latest or not. In the attached UpdateSet, Unloadable check box is enabled.

c. Click on Run Fix script

d. When the Run Fix Script window shows up, click proceed.

e. Validate the fix script has been executed whether successfully or not by viewing the logs. If it has been not completed successfully then check the logs to identify the cause. Then close the window.

Note 1: Fix script-made changes are considered as customization. Hence, the below patterns won't receive future OOB updates.

Amazon AWS - Hardware Type (LP) Azure - Hardware Type (LP) Note 2: If you wish to capture the customizations to the " Discovery and Service Mapping Patterns" plugin then publish the Fix script generated Customer Update[sys_update_xml] records. You can claim these changes(Mentioned in Step#:5) done by the Fix script before the next upgrade/repair to avoid an unexpected reversion. For more details, you can refer Manage customizations to store applications

Note: This step is only applicable when you are using App customization to manage customizations.

4. Verify the System Property - sn_itom_pattern.use a single hardware type for cloud data centers is updated to true

From the Filter Navigator -> enter sys_properties.list -> Search for sn_itom_pattern.use a single hardware type for cloud data centers

5. After Run the Fix script, the below records are generated in Customer Updates(sys_update_xml) table and those records are considered as customized . Hence, the below records won't receive future OOB updates.

S.NO Type Target Name Name Scope of the File Remarks 1 System Property sn_itom_pattern.use a single hardware type for cloud data centers sys_properties_8560e8b21b6a605036afc8851a4bcb56 Discovery and Service Mapping Patterns Value changed from false to true 2 Discovery Patterns Amazon AWS - Hardware Type (LP) sa_pattern_f80c73351bda4010e1b863913d4bcb9d Discovery and Service Mapping Patterns ci_type is changed from cmdb_ci_compute_template to cmdb_ci_cloud_hardware_type 3 Discovery Patterns Azure - Hardware Type (LP) sa_pattern_c1d05d691b15101010a6dce7cc4bcb9c Discovery and Service Mapping Patterns ci_type is changed from cmdb_ci_compute_template to cmdb_ci_cloud_hardware_type 4 Pattern Outputs sn_pattern_outputs_9291fe9fdb54d4901f2d9207db9619b1 Discovery and Service Mapping Patterns ci_class_type is changed from cmdb_ci_compute_template to cmdb_ci_cloud_hardware_type 5 Pattern Outputs sn_pattern_outputs_d4ae4ee1db951010b2a1f962759619d1 Discovery and Service Mapping Patterns ci_class_type is changed from cmdb_ci_compute_template to cmdb_ci_cloud_hardware_type 6 Extended Pattern Input Parameters compute_templates discovery_ptrn_lnch_param_def_ext_5868be5bdb94d4901f2d9207db96199c Discovery and Service Mapping Patterns ci_class is changed from cmdb_ci_compute_template to cmdb_ci_cloud_hardware_type 7 Extended Pattern Input Parameters cmdb_ci_cloud_hardware_type discovery_ptrn_lnch_param_def_ext_420f0625db951010b2a1f9627596195b Discovery and Service Mapping Patterns 1. ci_class is changed from cmdb_ci_compute_template to cmdb_ci_cloud_hardware_type 2. "key" is changed from cmdb_ci_compute_template to cmdb_ci_cloud_hardware_type" 8 View Table cmdb_ci_cloud_hardware_type sys_db_view_table_b69072665b755010c937839db881c777 Discovery and Service Mapping Patterns Changed table from cmdb_ci_compute_template to cmdb_ci_cloud_hardware_type 9 View Table cmdb_ci_cloud_hardware_type sys_db_view_table_b4c496be5b395010c937839db881c7f1 Discovery and Service Mapping Patterns Changed table from cmdb_ci_compute_template to cmdb_ci_cloud_hardware_type 10 View Table cmdb_ci_cloud_hardware_type sys_db_view_table_280458575bf19010c937839db881c758 Discovery and Service Mapping Patterns Changed table from cmdb_ci_compute_template to cmdb_ci_cloud_hardware_type 11 Related CI Types b9e53aec1be30010f6f2ea08bc4bcbeb sa_ci_to_pattern_b9e53aec1be30010f6f2ea08bc4bcbeb Global ci_type changed from cmdb_ci_compute_template to cmdb_ci_cloud_hardware_type 12 Related CI Types cf6f339bdb3f8410a480265d13961980 sa_ci_to_pattern_cf6f339bdb3f8410a480265d13961980 Global 1. ci_type changed from cmdb_ci_compute_template to cmdb_ci_cloud_hardware_type

2. deletion_strategy changed from Mark as Retired to Keep

13 Related CI Types 4a538120db561300d504788dbf961913 sa_ci_to_pattern_4a538120db561300d504788dbf961913 Global ci_type changed from cmdb_ci_compute_template to cmdb_ci_cloud_hardware_type 14 Related CI Types 12869e211b15101010a6dce7cc4bcbc3 sa_ci_to_pattern_12869e211b15101010a6dce7cc4bcbc3 Global ci_type changed from cmdb_ci_compute_template to cmdb_ci_cloud_hardware_type 15 Related CI Types 1576105adbb91c10b2a1f962759619c5 sa_ci_to_pattern_1576105adbb91c10b2a1f962759619c5 Global ci_type changed from cmdb_ci_compute_template to cmdb_ci_cloud_hardware_type 16 Related CI Types 43a05cf21bf91c1010a6dce7cc4bcbb7 sa_ci_to_pattern_43a05cf21bf91c1010a6dce7cc4bcbb7 Global 1. ci_type changed from cmdb_ci_compute_template to cmdb_ci_cloud_hardware_type 2. deletion_strategy changed from Mark as Retired to Keep 17 Related CI Types dc52279b0b832300a47f8e4335673acb sa_ci_to_pattern_dc52279b0b832300a47f8e4335673acb Global ci_type changed from cmdb_ci_compute_template to cmdb_ci_cloud_hardware_type

Note: UpdateSet (CloudOsImageMigrationUpdateSet.xml) was created in Global Scope and it contains the below records.

S.NO Type Target Name Name Remarks 1 Fix Script Migrate to new Hardware Model sys_script_fix_69d7cb5e1b7e2810a2d92f092a4bcbe7 This script will change the patterns orchestrator to support the new model 2 System Property delete_old_hardware_type_batch_size sys_properties_746d476e1bdf6c10e2c665b6624bcb63 Number of cmdb_ci_compute_template records delete in each batch 3 Scheduled Script Execution Delete Old Hardware Types Model records sysauto_script_fa804c0597e4e5108d3b350e6253af78 This Scheduled Job deletes the old model records (cmdb_ci_compute_template) records periodically.

Note: If you have not deleted old model records and migrated to a new model and run cloud discovery, you can observe both old and new model records and relations in CMDB.

6. If you wish to capture the customizations to the " Discovery and Service Mapping Patterns" plugin then publish the Fix script generated Customer Update[sys_update_xml] records. You can claim these changes(Mentioned in Step#:5) done by the Fix script before the next upgrade/repair to avoid an unexpected reversion. For more detials, you can refer Manage customizations to store applications

Note: This step is only applicable when you are using App customization to manage customizations.

Frequently Asked Questions 1. If the customer is using CPG and family version prior to San Diego Release, Can the customer migrate to New Model?

Ans . The customer can not migrate to the new model. If migrates, it causes issues in CPG flow. Currently, don't have WA. CPG supporting this new model from San Diego release onwards.

2. Is recommended to delete records directly from individual tables to avoid the delay caused by cascade delete?

Ans. No, Not recommended.

3. Can we change the sequence of jobs?

Ans. Not recommended to change the sequence of jobs/steps order. Run the migration fix script Before cleaning old CI's from 'cmdb_ci_compute_template' table may cause issues- CMDB contains both old and new CI records and relations.

4. Changes done by Fix Script(Migrate to new Hardware Model) are overriding after Store App upgradation. how to handle it?

Most Probable cause - If the Fix script runs without enabling unloadable checkbox then the System property "sn_itom_pattern.use a single hardware type for cloud data centers" and some other records that are modified by the Fix script are not present in Customer Update[sys_update_xml] table.

Hence, after the Store App upgradation, the System property value reverted to false (OOB, value is false) [Amazon AWS - Hardware Type (LP)/ Azure - Hardware Type (LP) Patterns CI type is cmdb_ci_cloud_hardware_type and System Property 'sn_itom_pattern.use a single hardware type for cloud data centers' is false ]. So, it causes pattern failures with this message " Discovery using patterns failed due to lack of existence of the main CI type in the results. Check the discovery logs for more details ".

WA: Import the update set -> From the above Instructions section -> Step 2. Import the update set in this KB and follow the steps mentioned in " Run the Fix script - Migrate to new Hardware Model ".

Note 1: Fix script in latest Updateset contains logic to create records in customer updates(sys_update_xml) table to avoid unexpected reversion of records while upgrade. Re-running the fix script won't cause any issues.

Note 2 : Re-run the fix script won't delete any CIs in CMDB.

Note 3: Fix script runs with the Unloadable check box enabled, it creates Customer Update[sys_update_xml] records when the fix script runs.

5. Changes done by the Fix script are present (Migrate to new Hardware Model) and still facing issues- Discovery using patterns failed due to lack of existence of the main CI type in the results. Check the discovery logs for more details. how to resolve it?

Cause: Amazon AWS - Hardware Type (LP)/ Azure - Hardware Type (LP) Patterns CI type is cmdb_ci_cloud_hardware_type and System Property 'sn_itom_pattern.use a single hardware type for cloud data centers' is true . Still, Patterns are failing with the error "Discovery using patterns failed due to lack of existence of the main CI type in the results. Check the discovery logs for more details". Then Might be Mid server Sync not happened properly or the Patterns did not Sync to the MID Server (NDL still shows old hardware type as the CI Type)

WA: sync the Patterns with mid servers( Pattern sync with Mid). If you've already performed this step and still getting the same error then restarting the MID Server will often resolve this. if it doesn't then please open a Case with ServiceNow Support to investigate further for any MID Server issues.