IDR — Applying Filters for Partial Seeding via Producer Outbound Entry - Support and Troubleshooting

IDR — Applying Filters for Partial Seeding via Producer Outbound Entry Issue .ns-kb-css-body-editor-container { p { font-size: 12pt; font-family: Lato; color: var(--now-color--text-primary, #000000); } span { font-size: 12pt; font-family: Lato; color: var(--now-color--text-primary, #000000); } h2 { font-size: 24pt; font-family: Lato; color: var(--now-color--text-primary, black); } h3 { font-size: 18pt; font-family: Lato; color: var(--now-color--text-primary, black); } h4 { font-size: 14pt; font-family: Lato; color: var(--now-color--text-primary, black); } a { font-size: 12pt; font-family: Lato; color: var(--now-color--link-primary, #00718F); } a:hover { font-size: 12pt; color: var(--now-color--link-primary, #024F69); } a:target { font-size: 12pt; color: var(--now-color--link-primary, #032D42); } a:visited { font-size: 12pt; color: var(--now-color--link-primary, #00718f); } ul { font-size: 12pt; font-family: Lato; } li { font-size: 12pt; font-family: Lato; } img { display: ; max-width: ; width: ; height: ; } } When using Instance Data Replication (IDR) to replicate data from a producer instance to a consumer instance, there are scenarios where the consumer needs to receive only a subset of records — a pattern referred to as partial seeding. This article explains why filters for partial seeding cannot always be applied on the consumer side, how to apply the filter correctly on the producer's outbound entry, and how to work around a UI limitation when filtering on the payload field.

Use Case A consumer replication set has inbound entries where the source table does not exist on the consumer instance. Instead, the consumer uses a transform map to route the incoming data to a target table. In this scenario, if you need to apply a filter to seed only a subset of records (for example, only records where the payload contains data for the incident table), the filter cannot be applied on the consumer inbound entry.

The filter must instead be applied on the producer's outbound entry.

Release .ns-kb-css-body-editor-container { p { font-size: 12pt; font-family: Lato; color: var(--now-color--text-primary, #000000); } span { font-size: 12pt; font-family: Lato; color: var(--now-color--text-primary, #000000); } h2 { font-size: 24pt; font-family: Lato; color: var(--now-color--text-primary, black); } h3 { font-size: 18pt; font-family: Lato; color: var(--now-color--text-primary, black); } h4 { font-size: 14pt; font-family: Lato; color: var(--now-color--text-primary, black); } a { font-size: 12pt; font-family: Lato; color: var(--now-color--link-primary, #00718F); } a:hover { font-size: 12pt; color: var(--now-color--link-primary, #024F69); } a:target { font-size: 12pt; color: var(--now-color--link-primary, #032D42); } a:visited { font-size: 12pt; color: var(--now-color--link-primary, #00718f); } ul { font-size: 12pt; font-family: Lato; } li { font-size: 12pt; font-family: Lato; } img { display: ; max-width: ; width: ; height: ; } } All versions of com.glide.idr

Cause .ns-kb-css-body-editor-container { p { font-size: 12pt; font-family: Lato; color: var(--now-color--text-primary, #000000); } span { font-size: 12pt; font-family: Lato; color: var(--now-color--text-primary, #000000); } h2 { font-size: 24pt; font-family: Lato; color: var(--now-color--text-primary, black); } h3 { font-size: 18pt; font-family: Lato; color: var(--now-color--text-primary, black); } h4 { font-size: 14pt; font-family: Lato; color: var(--now-color--text-primary, black); } a { font-size: 12pt; font-family: Lato; color: var(--now-color--link-primary, #00718F); } a:hover { font-size: 12pt; color: var(--now-color--link-primary, #024F69); } a:target { font-size: 12pt; color: var(--now-color--link-primary, #032D42); } a:visited { font-size: 12pt; color: var(--now-color--link-primary, #00718f); } ul { font-size: 12pt; font-family: Lato; } li { font-size: 12pt; font-family: Lato; } img { display: ; max-width: ; width: ; height: ; } } 1. The Payload field on the outbound staging table is a compressed/large text field type, which restricts the available operators in the filter builder UI to only is, is not, is anything. 2. The source table mentioned in the inbound entry is not available in the consumer instance, preventing the custom filter from showing fields to add filter conditions.

Resolution .ns-kb-css-body-editor-container { p { font-size: 12pt; font-family: Lato; color: var(--now-color--text-primary, #000000); } span { font-size: 12pt; font-family: Lato; color: var(--now-color--text-primary, #000000); } h2 { font-size: 24pt; font-family: Lato; color: var(--now-color--text-primary, black); } h3 { font-size: 18pt; font-family: Lato; color: var(--now-color--text-primary, black); } h4 { font-size: 14pt; font-family: Lato; color: var(--now-color--text-primary, black); } a { font-size: 12pt; font-family: Lato; color: var(--now-color--link-primary, #00718F); } a:hover { font-size: 12pt; color: var(--now-color--link-primary, #024F69); } a:target { font-size: 12pt; color: var(--now-color--link-primary, #032D42); } a:visited { font-size: 12pt; color: var(--now-color--link-primary, #00718f); } ul { font-size: 12pt; font-family: Lato; } li { font-size: 12pt; font-family: Lato; } img { display: ; max-width: ; width: ; height: ; } } Why the Filter Cannot Be Applied on the Consumer The consumer inbound entry processes records that have already been replicated from the producer. At the consumer side:

The source table referenced in the inbound entry does not exist on the consumer instance. ServiceNow cannot evaluate filter conditions against a table that is absent from the instance schema. Any filter defined on the consumer inbound entry would either fail silently or throw a table-not-found error, resulting in all records being processed without filtering. Since the filter evaluation requires the source table to be queryable, the filtering logic must be applied at the point of origin — the producer — before records are sent across.

Solution — Apply the Filter on the Producer Outbound Entry The correct approach is to configure the filter on the outbound entry within the producer replication set. This controls which records are included in the replication payload before they leave the producer instance.

Steps On the producer instance, navigate to the relevant replication set. Open the outbound entry for the source table. In the Filter section, add the appropriate condition to limit which staging records are replicated. Save and test by triggering a replication run. Only records matching the filter condition on the outbound entry will be included in the replication payload sent to the consumer.

Filtering on the Payload Field — UI Limitation Problem When the filter criterion involves the payload field on the outbound staging table — for example, to replicate only records where the payload contains data for a specific table such as incident — you may find that the "contains" operator is not available in the filter builder UI.

The filter builder only presents the following operators for the payload field:

is is not is anything The contains operator is absent.

Reason The payload field on the outbound staging table is stored as a compressed large text field. ServiceNow's filter builder UI restricts available operators based on field type. Compressed and large text fields do not expose string-based operators such as contains (which maps to the LIKE SQL operator) in the condition builder, because:

These fields are stored in a compressed binary format in the database. The UI operator list is driven by field type metadata ( sys_glide_object ), and compressed fields are not registered with the string operator set. Allowing LIKE queries on compressed fields at scale can cause significant performance issues, so they are intentionally excluded from the UI to prevent accidental use. Workaround — Update the Filter via Background Script Since the UI does not expose the contains / LIKE operator for compressed fields, you can manually set the encoded query on the replication entry record using a background script.

ServiceNow's encoded query syntax uses LIKE as the equivalent of "contains" for text fields.

Step 1 — Find the replication entry sys_id and review the current filter javascript var gr = new GlideRecord('idr_replication_entry'); gr.addQuery('source_table_name', ' '); gr.query(); while (gr.next()) { gs.info('sys_id: ' + gr.getUniqueValue()); gs.info('Current filter: ' + gr.getValue('filter')); } Step 2 — Apply the payload contains filter Replace YOUR_SYS_ID_HERE with the sys_id from Step 1. Adjust the search value to match the expected content in the payload (e.g., a JSON key-value pair identifying the source table).

javascript var gr = new GlideRecord('idr_replication_entry'); if (gr.get('YOUR_SYS_ID_HERE')) { // Use a specific JSON key-value pattern to avoid false positives // e.g., payloadLIKE"tableName":"incident" rather than just payloadLIKEincident var filterValue = 'payloadLIKE"tableName":"incident"'; gr.setValue('filter', filterValue); gr.setWorkflow(false); gr.update(); gs.info('Filter updated to: ' + gr.getValue('filter')); } Important: Before applying the filter, verify the exact JSON structure of the payload field by inspecting a sample record from the outbound staging table. Use the most specific pattern possible to avoid unintended matches (e.g., "tableName":"incident" instead of incident , which would also match incident_task , sn_incident , etc.).

Step 3 — Verify a sample payload value javascript var staging = new GlideRecord('x_snc_impact_csp_outbound_staging'); staging.setLimit(3); staging.query(); while (staging.next()) { gs.info('Payload sample: ' + staging.getValue('payload')); } Use the output to confirm the correct JSON key name and construct the LIKE pattern accordingly.

Summary Scenario Recommended Action Consumer inbound entry source table does not exist on consumer Apply filter on producer outbound entry Filter required on payload field Use background script to set payloadLIKE encoded query on idr_replication_entry "contains" operator not available in filter builder UI Expected behaviour — compressed field type restricts available operators in UI; use encoded query workaround Notes Modifying idr_replication_entry via background script should be done in a non-production environment first and validated before applying to production. Performance impact: LIKE queries on compressed fields can be expensive at high record volumes. Consider complementary indexing or data volume controls if applying this filter to large datasets. If the payload structure changes on the producer side, the encoded query filter must be updated accordingly.