Dynamic Schema Zurich Migration Guide - Support and Troubleshooting

Dynamic Schema Zurich Migration Guide .ns-kb-css-body-editor-container { p { font-size: 12pt; font-family: Lato; color: #000000; } span { font-size: 12pt; font-family: Lato; color: #000000; } h2 { font-size: 24pt; font-family: Lato; color: black; } h3 { font-size: 18pt; font-family: Lato; color: black; } h4 { font-size: 14pt; font-family: Lato; color: black; } a { font-size: 12pt; font-family: Lato; color: #00718F; } a:hover { font-size: 12pt; color: #024F69; } a:target { font-size: 12pt; color: #032D42; } a:visited { font-size: 12pt; color: #00718f; } ul { font-size: 12pt; font-family: Lato; } li { font-size: 12pt; font-family: Lato; } img { display: ; max-width: ; width: ; height: ; } } Table of Contents Purpose and Audience Brief Overview of Architectural Changes Discontinuation of Dynamic Attribute Groups Introduction of Dynamic Namespaces Narrowing of Dynamic Schema definitions Automated Migration (During Upgrade) Migration of Dynamic Attribute Store fields Migration of Dynamic Attributes Migration of Dynamic Categories Post-Upgrade Recommendations Create new Dynamic Attribute Store fields Deprecate Legacy Dynamic Attributes Update Dynamic Schema References Update Dynamic Namespace References Appendix: Glide Properties Purpose and Audience The Dynamic Schema feature underwent significant architectural changes in the Zurich release. This document is for developers that have used the Dynamic Schema feature prior to the Zurich release and have existing data being migrated to Zurich . Although every effort has been made to maintain backward compatibilty with many items being automatically migrated during upgrade, there are a few additional steps we ask customers to perform post-Upgrade in order to complete the migration in a way most appropriate for them and their data. These steps are crucial to ensure a seamless transition and to take full advantage of the new features and improvements introduced in the Zurich release.

References Dynamic Schema - Yokohama Dynamic Schema - Zurich Brief Overview of Architectural Changes Discontinuation of Dynamic Attribute Groups Prior to the Zurich release, every Dynamic Attribute was defined within an associated Dynamic Attribute Group . In Zurich , all attributes have been flattened and exist without this concept of group.

For example, given a Dynamic Attribute Store element named store1 ,

Prior to Zurich , an attribute named attribute1 might be defined in a group named group1 and would be targeted within a store field using the syntax store1->group1->attribute1 and stored in json as { "_group1" : { "attribute1" : "value" }} . In Zurich , an attribute named attribute1 would be targeted within a store field using the syntax store1->attribute1 and stored in json as { "attribute1" : "value" } . The concept of group no longer exists. For customers migrating existing Dynamic Attribute definitions to Zurich , steps have been taken to allow for backward compatability. See the Automated Migration section below for details.

Introduction of Dynamic Namespaces In Zurich , the concept of a Dynamic Namespace has been introduced. A Dynamic Namespace is a scoped container of Dynamic Attribute , Dynamic Category , and other Dynamic Schema components.

Items created within one Dynamic Namespace cannot be shared or referenced from any other Dynamic Namespace . Every Dynamic Attribute Store field is configured with a single Dynamic Namespace as specified by the dynamic_namespace attribute configured on its sys_dictionary record. By default, every Dynamic Attribute Store field receives its own Dynamic Namespace , but they can be created independently of, or used by multiple, store fields.

Narrowing of Dynamic Schema definitions A Dynamic Attribute Store field could specifiy a dependent-on-field relationship to another field which provides schema support for the store field.

Prior to Zurich , the type of this schema field could be either reference or glide_list referencing the dynamic_category , dynamic_attribute_group , or dynamic_attribute tables. In Zurich , the type of this schema field must be a reference (not glide_list ) to dynamic_category record. Furthermore the Dynamic Attribute Store field and the dynamic_category record selected in each row must be configured with the same Dynamic Namespace .

Automated Migration (During Upgrade) If there exist columns of type dynamic_attribute_store on the instance, or there are any records in the dynamic_attribute or dynamic_category tables, then the GlideProperty glide.db.dynamic_schema.legacy_attribute_support.enabled will be set to true . This indicates that the instance should continue to allow the Legacy store->group->attribute syntax to be used which is needed for backward compatibility. Customers are urged to complete the migration process by moving their data away from Legacy attributes that rely on groups, to new attributes defined without groups, and then set this GlideProperty to false . See Post-Upgrade Recommendations section below for more details.

As the new Dynamic Schema architecture scopes all components within namespaces, if there exist any records in the dynamic_attribute or dynamic_category table that need to be migrated from a previous release, a dynamic_namespace record with name sn/Migrated will be created for all migrated items to be placed in. This namespace will only be created if there are items to migrate.

Migration of Dynamic Attribute Store fields The Dynamic Namespace used by a Dynamic Attribute Store field is specified by the value of the dynamic_namespace attribute configured on the field's sys_dictionary record.

If the instance has dynamic_attribute records, then each Dynamic Attribute Store field will be configured to use the sn/Migrated namespace. This is done in order to mimic previous scoping behavior where all attributes on the instance shared the same scope and were available in all Dynamic Attribute Store fields. If the instance has no dynamic_attribute records, then each Dynamic Attribute Store field receives it's own dynamic_namespace . This is done in order to more closely align the field to a format it would receive if it were created new in the Zurich release where each store field receives its own namespace. Note that the Dynamic Attribute Store field's dependent-on-field relationship, which specifies its schema, is not modified. This value may have previously pointed to a reference or list of groups, attributes, or categories, whereas in Zurich , it can only reference a single category. Correcting this is left up to the user as a recommended Post-Upgrade action, see below.

Note also that no data stored in these fields is modified. Migration of this data is left as a Post-Upgrade Recommendation , see below.

Migration of Dynamic Attributes The Zurich release removed the concept of Dynamic Attribute Group . Existing dynamic_attribute definitions need to modified in such as way as to maintain backward compatibilty but also function in the new architecture. For these reasons the following modifications will be made to all dynamic_attribute definitions.

All dynamic_attribute s will be associated with the sn/Migrated namespace. The following dynamic_attribute field values will also be modified for each attribute. As an example, consider a dynamic_attribute record with name attribute1 created within a dynamic_attribute_group with name group1 . The dynamic_attribute.name of the attribute will be prefixed with the group name, separated with a double underscore, e.g.: group1__attribute1 The dynamic_attribute.legacy_group_name field will be populated with the group name, e.g.: group1 Furthermore, a dynamic_attribute record is considered Legacy if it exhibits the above properties.

For each such record, if the Legacy Attribute Support glide property is enabled, the platform will continue to allow the attribute to be referenced using the legacy syntax store1->group1->attribute1 .

Note that this is not the same as the non-Legacy attribute store1->group1__attribute1 . Although one or both of these may exist as transient attributes in a single Dynamic Attribute Store field, only one may have a dynamic_attribute definition within the same namespace. As they would both have the same dynamic_attribute.name value, which is unique within a namespace, the definition could either be legacy or not, but not both.

Existing dynamic_attribute_group records are no longer used, but remain in the instance and are not modified during migration.

Migration of Dynamic Categories All dynamic_category s will be associated with the sn/Migrated namespace.

Post-Upgrade Recommendations The primary goal of these recommendations is to help customers migrate away from their use of Legacy Dynamic Attributes (which use Dynamic Attribute Groups) to Dynamic Attributes without groups. Future releases of the platform may no longer be backward compatible with attributes of this form.

Create new Dynamic Attribute Store fields Although not strictly required, it is recommended to create a new Dynamic Attribute Store field for each existing Dynamic Attribute Store field being migrated. This makes the migration cleaner and ultimately a bit less confusing. The remainder of this document assumes a new store field is created for this migration.

If it is not possible or desired to create new store fields, it's entirely possible to perform the migration in-place. In this case, the following facts may be particularly useful to keep in mind,

During upgrade, if there were any dynamic_attribute s migrated, they will exist in a single sn/Migrated namespace which is configured on all Dynamic Attribute Store fields. New store fields each receive their own namespace by default. You may manually create new dynamic_namespace records and update the namespace used by any Dynamic Attribute Store field by modifying the dynamic_namespace attribute on the store's sys_dictionary record. Modifying the dynamic_namespace attribute of a Dynamic Attribute Store field does not modify any of the data stored in that field. Attribute and category definitions may be entirely different between the two namespaces and any data in the store field will be re-interpretted using definitions present (or not present) in the new namespace. After modifying the dynamic_namespace of a Dynamic Attribute Store field, you must ensure that the categories referenced by the store's dependent-on-field are also modified to be ones present in the new namespace or they will become ineffective. Both Legacy and non-Legacy attributes are separately addressable and can exist within a single Dynamic Attribute Store field. However, a Legacy attribute group1->attribute1 and non-Legacy attribute group1__attribute1 would share the same dynamic_attribute.name value which is required to be unique within a namespace. Although one or both may exist as transients, only one may have a formal dynamic_attribute definition within the same namespace.

Deprecate Legacy Dynamic Attributes During upgrade, all migrated dynamic_attribute definitions have now become Legacy attributes. These can be identified by filtering the dynamic_attribute table for records with non-blank legacy_group_name values. Any application logic that depends on these attribute definitions should be migrated to new, non-Legacy attributes, and any existing data copied from the old attribute to the new. There are many ways in which this can be accomplished, but consider the following as one example.

Suppose a pre-Zurich attribute named color exists in a group named body . A table mytable also exists with a Dynamic Attribute Store field named store1 . The following would print the value of the attribute for all rows in the table

var gr = new GlideRecord('mytable'); gr.query(); while(gr.next()) { gs.info(gr.getDynamicAttributeValue('store1->body->color')); } For migrated Zurich instances, this script continues to work due to Legacy Attribute Support being enabled. To migrate this attribute,

(Recommended) Create a new Dynamic Attribute Store field called store2 . This will also implicitly provide you with a new dynamic_namespace of mytable/store2 Determine a new attribute name to migrate to: body_color for example. ( Optional ) Go to the mytable/store2 dynamic_namesapace and create a dynamic_attribute definition for this new attribute. Copy all data from the old attribute to the new one, see example script below. Delete the legacy dynamic_attribute definition var gr = new GlideRecord('mytable'); gr.query(); while(gr.next()) { // get the stored value of the original attribute var attribute_value = gr.getDynamicAttributeStorageValue('store1->body->color'); // set the new attribute's stored value gr.setDynamicAttributeStorageValue('store2->body_color', attribute_value); // update the record gr.update(); } In addition to Legacy dynamic_attribute definitions, an instance may also be using the double-arrow syntax ( store->group->attribute ) to save and retrieve transient attributes (attributes without formal dynamic_attribute definitions present). This code will also need to be modified to remove group information ( store->attribute ).

Finally, once all occurrences of the double-arrow syntax are removed from an instance, the GlideProperty glide.db.dynamic_schema.legacy_attribute_support.enabled should be deleted (or set to false ) indicating that all attributes in the instance have been fully migrated. Disabling this property will render the double-arrow syntax invalid and further usage of it will return no results. This is the default for new instances in Zurich .

Update Dynamic Schema References Dynamic Attribute Store fields can specify a dependent-on-field relationship to another field. The value of this field determines the store's schema, which may vary between rows.

Prior to Zurich , this schema field could be of type reference or glide_list , referencing the dynamic_category , dynamic_attribute_group , or dynamic_attribute tables.

In Zurich , this schema field must be of type reference to a single dynamic_category . Furthermore, for each row in the table, the category selected in that field must exist within the same namespace the store field is configured with.

By default, creating a new Dynamic Attribute Store field on a table will also create a companion reference field correctly configured to provide this schema support for the store. However, during migration, dependent-on-field relationships of any pre-existing Dynamic Attribute Store fields are not modified. This means that unless the relationship already points to a reference field to dynamic_category , it will need to be fixed manually or the schema feature will not function properly.

To fix these relationships, for each misconfigured Dynamic Attribute Store field on the instance:

Create or identify a field of type reference to dynamic_category that you wish to provide schema support for the store field. This field must be on the same table, or a dot-walk away from one of its fields. View the sys_dictionary record for the store field and select the Dependent Field tab Modify the Dependent on field value and select your chosen field from Step 1 Save the record Finally, for each row in the table, ensure that the dynamic_category record selected for this field exists within the same namespace that the store field is configured with.

Update Dynamic Namespace References In Zurich, every Dynamic Attribute Store field has an associated Dynamic Namespace . If there were dynamic_attribute or dynamic_category records migrated from previous releases, they have been placed in a dynamic_namespace named sn/Migrated . All Dynamic Attribute Store fields have been configured to use this namespace to preserve similar pre- Zurich behavior where all items existed within the same scope.

As mentioned above, it is recommended to create new Dynamic Attribute Store fields for migration purposes, and if this is done, each will receive it's own namespace. However, if migration is performed without creating new Dynamic Attribute Store fields, it is still possible to create one or more new dynamic_namespace records and re-configure any existing store fields to use them.

To do this

Create a new entry in the dynamic_namespace table Open the sys_dictionary record for the Dynamic Attribute Store field to configure. Modify the dynamic_namespace attribute on the sys_dictionary record to the name of the record created in Step 1. Note that updating this namespace will not affect any data stored in the table. However, it will cause that data to be interpretted using attribute, category and other configurations from this new namespace. This can greatly affect how usage of that data behaves.

Appendix: Glide Properties glide.db.dynamic_schema.legacy_attribute_support.enabled = true (Default: false )

If present and set to true , glide will allow referencing Legacy dynamic attributes which include a group with the double arrow syntax: store->group->field . If disabled, this syntax becomes invalid and any data that requires it to be referenced becomes unreachable.