Instance Data Replication (IDR) FAQ Contents: Is IDR part of the platform (free plugin)?Can I use IDR to copy data once from one instance to another?Is it required that Producer and Consumer be on the same build?There is a data synchronization lag. I make an update on producer and it takes a while for consumer record to update. Should I worry?How does IDR deal with parent-child table hierarchy?How does IDR deal with consumer-side records that cannot be updated due to non-IDR rules such as Business Rules or ACL?Are the original Created [sys_created*] and Updated [sys_updated*] values from the producer kept when data is replicated to consumer?Do target tables/columns need to exist on the consumer instance?When troubleshooting, I need to get access to the raw data that was received by the consumer instance. Is this possible?A field on the consumer instance is not being updated. There are no ERRORs or WARNINGs. What can cause this issue?Is there a limit in the number of records one seeding action can perform? 1. Is IDR part of the platform (free plugin)? IDR is a paid product that requires a license. 2. Can I use IDR to copy data once from one instance to another? Yes, this is the "Seeding" feature within IDR. Please review the "IDR limitations and when not to use IDR" section of this official document to find out if this method of copying data from one instance to another is right for you. Another option for copying data from one instance to another is Import Sets. Please review our Import Set official documentation for more information. 3. Is it required that Producer and Consumer be on the same build? It is not a requirement for Producer and Consumer to be on the same version, but it is strongly recommended. For example, in order for some features to work, producer and consumer must be running on the same build. For example: The consumer and producer must both be running the Tokyo release or later to support the custom coalesce feature. Official Documentation.The consumer and producer must both be running the Tokyo release or later to support concurrent job execution. Official Documentation. 4. There is a data synchronization lag. I make an update on producer and it takes a while for consumer record to update. Should I worry? First determine if your expectation is correct. Expect a consumer record to be updated 30 seconds to 1 minute after the producer record is updated. If the lag is around this time, then there is no issue and the lag is expected. If the lag is greater than one minute, then there may be a delay in lag. Please note that if there is seeding taking place, then this lag may also be expected. The lag should get back to normal as soon as seeding is complete. 5. How does IDR deal with parent-child table hierarchy? IDR is highly configurable, which means that a user must be very explicit in what you wish to replicate. If you wish to replicate a particular table, you should have a replication entry for that table as part of the replication set. Within Servicenow, data is organized into tables with a hierarchy, and the user needs to be explicit in which part of the hierarchy they want to replicate. For example, if all the data from the cmdb_ci_computer (child of cmdb_ci_hardware) is desired to be replicated, a replication entry should be created for the cmdb_ci_computer table. In general, a child table will inherit columns from its parent table, which means that there are columns that exist on the child table, but not the parent. In this example, if the user decides to replicate the cmdb_ci_hardware table, they will not replicate the full data for its child table, cmdb_ci_computer. This, however, could be a valid use case - a user may want to replicate all of the data from the cmdb_ci_hardware table, but not be interested in the child tables' data in particular. For this reason, it is important that the replicated tables are explicitly defined. Because of this specificity, choosing the table to replicate also explicitly tells IDR where the data should be replicated. The table indicated by the replication entry will define where IDR will attempt to replicate the data. If the user selects the cmdb_ci_hardware table for replication, it will replicate to the cmdb_ci_hardware table. Even if the user provides a filter, (e.g. replication entry: cmdb_ci_hardware, filter: Class=Computer), this will still replicate the data from the cmdb_ci_hardware table, not to the child table (cmdb_ci_computer). This makes a lot of sense when IDR is being used for its primary purpose, which is keeping a particular subset of data in-sync between two instances. The specificity needed is key to the usage This is understandably inconvenient when IDR is being used to migrate a huge selection of CMDB data because each of the child tables will need a replication entry to ensure that the data ends up where they want it to. But IDR was not designed for this purpose See the official documentation for more details. 6. How does IDR deal with consumer-side records that cannot be updated due to non-IDR rules such as Business Rules or ACL? Sometimes the consumer instance will not be able to perform an INSERT/UPDATE due to non-IDR enforcements such as Business Rules or ACL. If this occurs, a Log [syslog] error record will be created with the following message: Error while inserting payload into consumer {"table_name": ... <complete payload here> ... fLabelParameters":{}}: com.snc.db.data_replicate.replicator.DataReplicationLoader$DataReplicationLoaderException: Error loading UPDATE: com.snc.db.data_replicate.replicator.DataReplicationLoader.updateOrThrow(DataReplicationLoader.java:221) 7. Are the original Created [sys_created*] and Updated [sys_updated*] values from the producer kept when data is replicated to consumer? There is a checkbox in the replication entry where customers decide whether to keep the original or replace them with the producer values. 8. Do target tables/columns need to exist on the consumer instance? Technically, you don't need those tables on the consumer instance. If the consumer instance does not have those tables/columns then a transformation must be setup on the consumer instance to transform to existing tables/columns. 9. When troubleshooting, I need to get access to the raw data that was received by the consumer instance. Is this possible? Yes. Enable debug System Properties idr.debug=true and glide.data_replicate.debug=true to force the raw XML for each record to be printed in the localhost logs. If an instance has several nodes, you will have to get the localhost logs for the node that handle this request. Therefore, you will first have to go to the Transactions table to find the transaction that handled the request (for a seeding request, this job is 'JOB: IDRSeedingConsumerJob', for example). The 'system' field in the Transaction record will let you know which node handled the request. The 'DEBUG: ReplicationPayloadDeserializer' line contains the exact content received from producer. The 'DEBUG: KeyValueEntryTransformer' line contains the content pushed/transformed into the consumer's target tables. Keep in mind that you can also get this data via the MySQL binlogs INSERT/UPDATE actions correlating to the IDR updates. 10. A field on the consumer instance is not being updated. There are no ERRORs or WARNINGs. What can cause this issue? It is possible that the Replication Entries are out-of-sync. To fix this press the 'Synchronize Replication Entries' Related Link in the Consumer Instance's Consumer Replication Set. 11. Is there a limit in the number of records one seeding action can perform? Yes, the seeding limit is 3 million records per replication set. Break up the data in batches if you need to seed more than 3 million records. .