Troubleshooting Methods for Import Sets - Support and Troubleshooting

Troubleshooting Methods for Import Sets .ns-kb-css-body-editor-container { p { font-size: 12pt; font-family: Lato; color: var(--now-color--text-primary, #000000); } span { font-size: 12pt; font-family: Lato; color: var(--now-color--text-primary, #000000); } h2 { font-size: 24pt; font-family: Lato; color: var(--now-color--text-primary, black); } h3 { font-size: 18pt; font-family: Lato; color: var(--now-color--text-primary, black); } h4 { font-size: 14pt; font-family: Lato; color: var(--now-color--text-primary, black); } a { font-size: 12pt; font-family: Lato; color: var(--now-color--link-primary, #00718F); } a:hover { font-size: 12pt; color: var(--now-color--link-primary, #024F69); } a:target { font-size: 12pt; color: var(--now-color--link-primary, #032D42); } a:visited { font-size: 12pt; color: var(--now-color--link-primary, #00718f); } ul { font-size: 12pt; font-family: Lato; } li { font-size: 12pt; font-family: Lato; } img { display: ; max-width: ; width: ; height: ; } }

This article provides step-by-step troubleshooting guidance for Import Set issues. Step 1: Use the Master Triage Table below to identify your issue type. Step 2: Follow the link to the relevant section. Step 3: Work through the troubleshooting steps for that issue. If you exhaust all steps without resolution, contact ServiceNow Support with the information gathered during your investigation.

Table of Contents Master Triage Table Section 1: Core Import and Transform 1.1 — Import Set stuck in Loading state 1.2 — Import Set stuck in Loaded state (transform never starts) 1.3 — Transform producing incorrect results 1.4 — Duplicate records in target table 1.5 — Business rule behavior during transform Section 2: LDAP Import Sets 2.1 — LDAP connection failing or test connection unsuccessful 2.2 — LDAP running but data not loading or missing records 2.3 — LDAP importing wrong data / updating records incorrectly Section 3: Easy Import 3.1 — Cannot access Easy Import or download the template 3.2 — Easy Import not creating or updating records correctly Section 4: Import Set API (Web Service Import Sets) 4.1 — API call succeeds but transform does not run 4.2 — API creating new columns instead of mapping to existing staging fields Section 5: Scheduled Imports 5.1 — Scheduled import not triggering at expected time 5.2 — Scheduled import fetching fewer records than expected Section 6: Import Set Deleter and Table Cleanup 6.1 — Import Set Deleter not cleaning up staging data Section 7: Concurrent Import 7.1 — Concurrent import not creating multiple import sets Section 8: Data Sources 8.1 — JDBC data source: connection failing, authentication error, or returning no data 8.2 — File or attachment data source failing to load data Section 9: Performance 9.1 — Import or transform is slow or taking excessively long Section 10: Data Quality 10.1 — Fields not populated or missing data in target table after transform Section 11: Character Encoding and Date Format Issues 11.1 — Characters displaying incorrectly after import 11.2 — Date values not populating or populating with wrong values Reference: Import Set Row States Reference: Key Diagnostic Tools and Navigation Paths Master Triage Table Identify your primary symptom in the left column. Follow the condition in the middle column to the section link on the right. If a symptom matches multiple rows, start with the first match from the top.

Symptom Condition / What to check first Go to Staging tables growing large, or Import Set Deleter job not running Always Section 6.1 Import Set Deleter Import or transform running slow or timing out Always Section 9.1 Performance LDAP import — connection / test connection failing LDAP data source in use Section 2.1 LDAP connection LDAP import — records missing or not loading LDAP data source in use Section 2.2 LDAP missing records LDAP import — wrong data or incorrect updates LDAP data source in use Section 2.3 LDAP wrong data Easy Import — cannot access or download template Easy Import in use Section 3.1 Easy Import access Easy Import — records not created or updated correctly Easy Import in use Section 3.2 Easy Import records Import Set REST API — transform not running after POST Data sent via Import Set REST API Section 4.1 API no transform Import Set REST API — new staging columns created instead of mapping Data sent via Import Set REST API Section 4.2 API new columns Scheduled import not triggering at expected time Scheduled import in use Section 5.1 Scheduled trigger Scheduled import fetching fewer records than expected Scheduled import in use Section 5.2 Scheduled fewer records Concurrent import enabled but not splitting into multiple import sets Concurrent Import checkbox checked Section 7.1 Concurrent Import JDBC data source connection failing, returning no data, or authentication error after password change JDBC data source in use Section 8.1 JDBC data source File or attachment data source failing to load data File / FTP / SFTP / HTTP data source in use Section 8.2 File data source Import Set stuck in "Loading" state Import Set state = Loading Section 1.1 Stuck Loading Import Set loaded but transform never starts Import Set state = Loaded Section 1.2 Stuck Loaded Duplicate records in target table Duplicate records appearing after transform Section 1.4 Duplicates Characters garbled or non-ASCII characters corrupted Text contains accented / CJK / special characters Section 11.1 Encoding Date fields null or populated with wrong value Date or datetime fields affected Section 11.2 Date format Fields empty or missing in target table after transform Specific fields null / not populated Section 10.1 Missing data Business rule firing unexpectedly or not firing during transform Business rule behavior in question Section 1.5 Business rules Transform running but results incorrect (wrong values, wrong inserts/updates) Transform completes without error but target data is wrong Section 1.3 Wrong results Section 1: Core Import and Transform Issue 1.1 — Import Set stuck in Loading state The import set record remains in "Loading" state and never advances. Data is either not reaching the staging table, or the import process was interrupted mid-run.

Troubleshooting Steps Step What to do What you find Next action 1 Check sys_diagnostic_event.list filtered to name CONTAINS "Node Start" near the import start time. Node restart event present Node restart interrupted import. Manually re-process the Import Set. Note the restart timestamp. 1 Check sys_diagnostic_event.list filtered to name CONTAINS "Node Start" near the import start time. No node restart event Step 2 2 If you have admin access: run the Import Transform Summary Tool (KB0870045) in Scripts-Background for the ISET number. Otherwise, note the ISET number and contact ServiceNow Support. Import rate is 0 and no active transaction Transform stalled — check System Log around the last-processed row timestamp for errors. 2 If you have admin access: run the Import Transform Summary Tool (KB0870045) in Scripts-Background for the ISET number. Otherwise, note the ISET number and contact ServiceNow Support. Summary Tool shows active transaction and low import rate Go to step 3 to check the transform map and Transformer job. 3 Confirm a transform map exists for this staging table. Navigate to System Import Sets > Administration > Transform Maps and filter by source table. No transform map found Import Set will remain in Loading indefinitely without a transform map. Create or fix the transform map, then re-run. 3 Confirm a transform map exists for this staging table. Transform map exists Step 4 4 Verify the Import Set Transformer job is active: System Scheduler > Scheduled Jobs, search "Import Set Transformer." Job is inactive or shows an error Activate the job or investigate the error in sys_trigger. 4 Verify the Import Set Transformer job is active. Job is active and running Provide Summary Tool output and Issue 1.2 — Import Set stuck in Loaded state (transform never starts) The data loaded successfully into the staging table (state is "Loaded"), but the transform has not started. The target table has not been updated.

Troubleshooting Steps Step What to do What you find Next action 1 Check the data source / import set Mode field on the data source record. Mode = Asynchronous Transform is queued and depends on the Import Set Transformer job. Check job status (see step 2). The transform may just be delayed. Section 1.2 Step 2 . 1 Check the data source / import set Mode field on the data source record. Mode = Synchronous Transform should have run immediately. Step 2 2 Check the Import Set Transformer job: System Scheduler > Scheduled Jobs, search "Import Set Transformer." Check Active status and last run time. Job is inactive or not running Activate the job or investigate sys_trigger. For concurrent imports, check mutex properties are not set to unreasonably long values. 2 Check the Import Set Transformer job. Job is active and running normally Step 3 3 Check whether the "Transform Synchronously" business rule on sys_import_set has been customized. Business rule is customized Custom BR is preventing the auto-transform trigger. Review and restore OOB logic or correct the customization. Provide BR content and ISET details. 3 Check whether the "Transform Synchronously" business rule is customized. Business rule is OOB / uncustomized Manually trigger the transform from the ISET record ("Transform" related link) to verify the transform map itself works. Issue 1.3 — Transform map running but producing incorrect results The transform completes without errors, but the data in the target table is wrong — wrong values, wrong fields populated, records inserted when they should have been updated, or records skipped unexpectedly.

Troubleshooting Steps

Step What to do What you find Next action 1 Check the Import Set Run record (Transform History). Review the insert, update, error, ignored, and skipped counts on sys_import_set_run . High error or skipped count Step 2 1 Check the Import Set Run record counts. Counts look expected but values in target are wrong Step 3 2 Open a sample error row from the staging table. Check the error_label field on the sys_import_set_row record. Error mentions choice or reference A Choice action of "Reject" is rejecting rows. Check the field map Choice action setting. 2 Open a sample error row and check error_label . Script error in error_label A transform script is throwing. Check onBefore / onAfter scripts for exceptions. 3 Pick one affected record. Compare the staging table row value to the field map configuration for that field. Check whether a field script is active on this field map. Field script is present The script may be computing a wrong value. Add gs.info() to log the computed value and check System Log. 3 Pick one affected record and check field mapping. No field script Confirm the source field name, target field name, and whether a field map exists for the affected field. Verify the transform map target table is correct. If mapping looks correct, gather a sample staging row and target record and contact ServiceNow Support with the information gathered. Issue 1.4 — Duplicate records created in target table The transform is creating new records instead of updating existing ones, resulting in duplicates in the target table.

Troubleshooting Steps

Step What to do What you find Next action 1 Open the transform map. Check the field maps for any field with Coalesce = true. No coalesce configured Root cause found. Configure coalesce on a unique identifier field and re-run. 1 Check field maps for Coalesce = true. Coalesce is configured Step 2 2 Pick one duplicate pair. Compare the coalesce field value in the staging row to the value on the existing target record exactly (case, whitespace, format). Values differ (case / format / whitespace mismatch) Coalesce is not finding a match. Normalize the source value using a field map script before the coalesce lookup runs. 2 Compare coalesce field values between staging row and target record. Values are identical Step 3 3 Check if Concurrent Import is enabled on the Scheduled Data Import record. If yes, check whether "Synchronize Inserts" is enabled. Concurrent Import ON, Synchronize Inserts OFF Two transform threads are racing to insert the same record. Enable Synchronize Inserts on the scheduled import record. 3 Check Concurrent Import and Synchronize Inserts settings. Not concurrent, or Synchronize Inserts already enabled Step 4 4 Check for an active onBefore transform script. Look for any code that sets action = 'insert' or overrides the transform action. Script forces action = 'insert' The script overrides coalesce. Fix the script logic to respect the action variable set by coalesce. 4 Check onBefore script for action override. No script override Gather: the transform map name, the coalesce field name and its data type, a sample duplicate pair (staging row sys_id and both target record sys_ids), and whether the coalesce field is indexed on the target table. Contact ServiceNow Support with the information gathered. Issue 1.5 — Business rule behavior during transform Business rules on the target table are not firing when expected, or are firing unexpectedly and causing unintended side effects (notifications, approvals, field normalization).

Troubleshooting Steps

Step What to do What you find Next action 1 Identify which table the business rule is defined on: the staging table (u_ table extending sys_import_set_row) or the target table. BR is on the staging table BRs on staging tables never fire during data load. This is platform behavior by design and cannot be overridden. See KB0825734. 1 Identify which table the BR is on. BR is on the target table Step 2 2 Open the transform map record. Check whether "Run Business Rules" is checked. Run Business Rules is unchecked BRs on the target table will not fire. Enable it if BRs are required. Note: this also enables workflows, approvals, and field normalization — review the performance impact before enabling on large imports. 2 Check "Run Business Rules" checkbox on the transform map. Run Business Rules is checked Step 3 3 Run the Summary Tool (KB0870045). In the Definition section, find "Number of non-async business rules to run on target: N." High count (N > 5) AND import is slow Business rules are a performance bottleneck. Review Section 9.1 . Consider using KB0538161 to identify the slowest BRs. 3 Run the Summary Tool and check non-async BR count. Unexpected automation triggering (notifications, approvals) Step 4 4 Disable "Run Business Rules" temporarily and re-run a small test set (10–20 records). Observe whether the unexpected behavior stops. Behavior stops with BRs off A specific target BR is causing the side effect. Disable target table BRs one by one to isolate the responsible rule. 4 Disable "Run Business Rules" temporarily and retest. Behavior continues with BRs off The source is not a BR. Check transform event scripts or Flow Designer triggers active on the target table. Section 2: LDAP Import Sets LDAP imports use an LDAP Server record, an OU Definition, and a Data Source to pull directory data into the staging table. An LDAP Transform Map then moves users or groups to the target table. Debugging always starts with the LDAP Browse feature — it shows exactly what the platform sees before any transform runs.

Issue 2.1 — LDAP connection failing or test connection unsuccessful The "Test Connection" button on the LDAP Server record returns a failure, or imports are not starting because connectivity cannot be established.

Troubleshooting Steps

Step What to do What you find Next action 1 Click "Test Connection" on the LDAP Server record. Capture the exact error message. Navigate: System LDAP > LDAP Servers. "Authentication failed" or "Invalid credentials" Bind DN or password is wrong. Verify and re-enter credentials. 1 Click "Test Connection" and capture the error. Network or timeout error Step 2 2 Determine if this LDAP import uses a MID Server. Check the LDAP Server record's "Use MID Server" field. Check MID Server status in ecc_agent.list . MID Server is down or in error state Fix the MID Server first. Check MID logs for errors. LDAP imports via MID cannot proceed until MID is healthy. 2 Check MID Server status. No MID Server used, or MID Server is Up Step 3 (if MID is in use) or Step 4 (if no MID) 3 Navigate to ecc_queue.list . Filter by agent = MID Server name and topic CONTAINS "LDAP". Check whether output records are being created after the test connection. Check whether they are being picked up (state = Processed) and whether the payload contains any error or warning messages. No ECC output records created after test connection Confirm the LDAP Server record references the correct MID Server. Check that the MID Server is registered and that its capabilities include LDAP. Restart the MID Server if the ECC output records remain absent after a retest. 3 Check ECC queue for LDAP records. ECC output records exist but state = Ready or Error (not being picked up) Restart the MID Server. Check MID Server logs for errors that may be preventing it from processing the queue. Verify the MID Server has no active errors in ecc_agent.list . 3 Check ECC queue for LDAP records. ECC records are being processed but payload shows an error or warning Read the error payload from the ECC output record. Use the error text to identify the specific failure (credential, network, certificate) and proceed to Step 4 with this context. 4 Enable LDAP debug logging. Set system property glide.ldap.debug = true . If using MID, set it as a MID Server property and restart MID. Re-run test connection and check System Log > All. Log shows connection refused or timeout Firewall or network is blocking the connection. Confirm instance IPs are whitelisted at the customer's LDAP server. 4 Enable LDAP debug and check logs. Log shows successful bind but data import fails Connectivity is confirmed. Section 2.2 for missing records investigation. Issue 2.2 — LDAP running but data not loading or records missing The LDAP import completes without a connection error, but the staging table is empty or contains fewer records than expected. Some users or groups are missing.

Troubleshooting Steps

Step What to do What you find Next action 1 Use the LDAP Browse feature on the LDAP Server record. Navigate to the OU containing the missing records. Note exactly what is visible. Record not visible in LDAP Browse The record does not exist in the LDAP directory as expected. This is a directory issue, not a ServiceNow issue. 1 Use LDAP Browse to locate the missing record. Record is visible in LDAP Browse Step 2 2 Check the OU Definition record for the affected import. Confirm the DN (Distinguished Name) is correct and the scope covers the missing records. DN is wrong or scope is too narrow Fix the OU Definition DN or change scope to subtree. 2 Check OU Definition DN and scope. DN looks correct and scope is subtree Step 3 3 Check the LDAP server paging configuration. Verify glide.ldap.max_results does not exceed the LDAP server's paging limit (default ServiceNow value: 1000). Compare total expected record count to the limit. Total records exceed paging limit Import is truncated at the page limit. Reduce glide.ldap.max_results to match the LDAP server limit, or ask customer to adjust LDAP server paging settings. 3 Check paging configuration. Paging is not the cause Issue 2.3 — LDAP importing wrong data or updating records incorrectly Records are imported and transformed, but the data in the target table does not match the LDAP source — wrong field values, wrong group memberships, or fields being overwritten incorrectly.

Troubleshooting Steps

Step What to do What you find Next action 1 Use LDAP Browse to view the raw value of the affected field for a specific record. LDAP Browse shows the wrong value at the source The problem is in the LDAP directory, not in ServiceNow. 1 Check the raw LDAP attribute value in LDAP Browse. LDAP Browse shows the correct value Step 2 2 Check the staging table row for the affected record after the import. Compare the staging value to what the LDAP Browse showed. Staging value is wrong (differs from Browse) Issue is in the import / data load phase. Check the OU Definition attribute mappings. 2 Compare staging row value to LDAP Browse value. Staging value matches Browse but target is wrong Step 3 3 Open the LDAP transform map. Find the field map for the affected field. Check for any field-level script or transform event script modifying the value. Script is modifying the value incorrectly Review and fix the script logic. 3 Check field map and transform scripts for the affected field. No script, mapping looks correct Check whether a business rule on sys_user or sys_user_group is overwriting the value post-transform. Section 3: Easy Import Easy Import is a simplified import process that allows users to download an Excel template from the list view of any table, populate it, and upload it. Under the hood it creates a standard Import Set and Transform Map. Errors at access and upload can stem from ACLs, UI context menu conditions, or the underlying import set infrastructure.

Issue 3.1 — Cannot access Easy Import or download the template Troubleshooting Steps

Step What to do What you find Next action 1 Confirm the user can see the "Import" option in the list context menu (right-click or gear icon). Check the UI Action conditions for the Easy Import action. Option not visible Check the UI Action conditions — they may restrict by role or table. Use the Access Analyzer for the affected user. 1 Check whether the context menu option is visible. Option visible but template is wrong or empty Step 2 2 Check whether the user has access to the fields that should appear in the template. Use the Access Analyzer (elevated privilege menu) for the affected user and table. Fields restricted by ACL The ACL or field-level security is preventing those fields from appearing in the template. This is expected security behavior — resolve with the customer's security team. 2 Check field access using the Access Analyzer. Access looks correct Step 3 3 Enable Business Rule debug for the affected user session. Attempt to download the template and check whether any BR is preventing the template attachment from being created or accessed. Check if the user has read access to sys_attachment. BR or ACL blocking sys_attachment Grant access to sys_attachment or fix the BR condition. 3 Check for BR or ACL on sys_attachment. No obstruction found Issue 3.2 — Easy Import not creating or updating records correctly Troubleshooting Steps

Step What to do What you find Next action 1 Find the Import Set created by Easy Import (filter by source="Easy Import" or by timestamp). Check the Import Set Run record counts. Error rows exist Open a staging row with state=error and check the error_label . Proceed through the Section 1.3 error path. 1 Check the Import Set Run counts. All rows show Success / Updated Step 2 2 Open the transform map created by Easy Import. Check whether all expected fields are mapped. Check for field length limits or field type mismatches between staging and target. Field type mismatch or length truncation The staging table auto-generates fields as String. If the target is a different type (integer, date, reference), the mapping may silently fail. Fix the field map or normalize the source data. 2 Check field mappings and types. Mappings look correct Step 3 3 Check the coalesce configuration on the Easy Import transform map. For updates to work, coalesce must be set on a field that uniquely identifies the target record. No coalesce or wrong coalesce field Records will always insert (not update). Set coalesce on the correct unique field. 3 Check coalesce configuration. Coalesce configured correctly but behavior still wrong Check whether the property glide.import_set_row.dynamically_add_fields is true. If false, new columns in the Excel may be silently ignored. Section 4: Import Set API (Web Service Import Sets) The Import Set REST API ( /api/now/import/{staging_table} ) allows external systems to push data directly into a staging table. By default the transform runs synchronously after the POST. If the data source mode is set to Asynchronous, the transform is queued for background processing. Required roles: import_set_loader , import_transformer .

Issue 4.1 — API call succeeds but transform does not run The POST returns HTTP 200 with status "inserted," but the target table records are not created or updated. The staging table has the data, but no transform has executed.

Troubleshooting Steps

Step What to do What you find Next action 1 Check the import set mode for this staging table's data source. Find the "Mode" field on the sys_data_source record. Mode = Asynchronous Transform is queued. Check the Import Set Transformer job status. Section 1.2 . 1 Check the Mode field on the data source record. Mode = Synchronous Transform should have run immediately. Step 2 2 Check whether an active transform map exists for this staging table. Navigate to System Import Sets > Administration > Transform Maps, filter by source table. No active transform map The API inserts into staging but has no map to run. Create or activate the transform map. 2 Check for an active transform map. Transform map exists and is active Step 3 3 Check the API user's roles. The user making the API call needs both import_set_loader and import_transformer roles. Missing import_transformer role The user can insert into staging but cannot trigger the transform. Add the import_transformer role. 3 Check the API user's roles. Roles are correct Check whether the "Transform Synchronously" BR is customized ( Section 1.2 Step 3 ). Note: For high-volume REST API integrations, Asynchronous mode is strongly recommended (KB0793315) — the transform happens in bulk rather than per-call, which is significantly more performant. To force asynchronous behavior for specific web service imports, see KB0781666.

Issue 4.2 — API creating new columns instead of mapping to existing staging fields After a POST, new columns appear in the staging table with prefixed names instead of mapping to existing fields. Records insert but the field mapping is broken.

Troubleshooting Steps

Step What to do What you find Next action 1 Compare the JSON body field names in the API request to the actual column names in the staging table. Staging table columns are auto-prefixed with u_ . POST body uses "fieldname" but staging has "u_fieldname" The mismatch causes a new column to be created. Use the staging column names (without the u_ prefix) in the POST body — ServiceNow adds u_ automatically. 1 Compare POST body field names to staging column names. Names match Step 2 2 Check the property glide.import_set_row.dynamically_add_fields . If true, any unrecognized field in the POST body will create a new column. Property is true and new columns are unexpected Ensure the POST body uses the exact staging table column names. Also check whether the POST body or source data contains duplicate field names — if two fields share the same name, ServiceNow cannot map either to an existing column and will create a new one instead. Deduplicate all field names in the POST body before sending. 2 Check glide.import_set_row.dynamically_add_fields . Property is true and new columns are expected (schema expansion) This is expected behavior for auto-schema expansion. No action needed if intentional. Section 5: Scheduled Imports Scheduled Data Imports ( scheduled_import_set ) run on a timer and automatically load data from a data source. They create an execution context and plan in sys_execution_context and sys_execution_plan . Hierarchical imports (parent → child) complete all child imports before the parent completes.

Issue 5.1 — Scheduled import not triggering at expected time Troubleshooting Steps

Step What to do What you find Next action 1 Open the Scheduled Data Import record. Check: is it Active? What is the Run field (Daily, Weekly, Periodically)? What is the next scheduled run time? Navigate: System Import Sets > Administration > Scheduled Imports. Record is not Active Activate it. Imports only run if the record is active. 1 Check the Scheduled Data Import record Active status and schedule. Record is Active Step 2 2 Check the underlying sys_trigger entry for this scheduled import. Navigate: System Scheduler > Scheduled Jobs, search by name. Look for error state or blocked jobs. sys_trigger is in Error state Check System Log for the error at the last run time. Fix the error and re-activate. 2 Check sys_trigger state and last/next run times. sys_trigger looks correct Step 3 3 Check whether this import is a child of a hierarchical import (has a "Run After" parent import configured). If yes, confirm the parent import is running. Parent import is not running The child will not run until the parent completes. Fix the parent import first. 3 Check the "Run After" field for a parent import dependency. No parent, or parent is running Check whether property glide.scheduled_import.stop_on_error is true and a prior run errored out. Clear the error and re-run. Issue 5.2 — Scheduled import fetching fewer records than expected The scheduled import runs and completes, but the staging table contains fewer records than the data source should have provided.

Troubleshooting Steps

Step What to do What you find Next action 1 Run "Load 20 Records" from the data source record. Count how many records arrive in the staging table vs. the expected 20. Also run a full import and compare staging count to source count. 20-record test returns fewer than expected The data source connectivity or query is filtering results. Check the SQL query (for JDBC) or file content (for file sources). 1 Run "Load 20 Records" and compare counts. Test load is fine but full load is short Step 2 2 For JDBC sources: review the SQL query on the data source record. Does it include any WHERE clause that would filter records by date, status, or other field? Query has a restrictive filter Adjust the WHERE clause to include the missing records. 2 Review the SQL query for restrictive filters. No restrictive filter Step 3 3 For JDBC via MID Server: check MID Server logs for any row-count limits, connection timeouts, or memory errors during the data retrieval. Review KB0538460 for known JDBC/MID missing data patterns. MID Server log shows timeout or memory error The MID ran out of resources retrieving large datasets. See KB0538460 for mitigations. 3 Check MID Server logs for resource errors. No MID errors Section 6: Import Set Deleter and Table Cleanup Issue 6.1 — Import Set Deleter not cleaning up staging table data The staging tables ( sys_import_set and its children extending sys_import_set_row ) are growing unboundedly. The OOB "Import Set Deleter" scheduled job should remove records older than 7 days, but the tables are not being cleaned. See also KB0755790.

Troubleshooting Steps

Step What to do What you find Next action 1 Verify the Import Set Deleter job is active. Navigate: System Import Sets > Import Set Tables > Scheduled Cleanup. Check the Active checkbox and last run time. Job is not Active Activate it. The job runs nightly at midnight. This is the most common cause of staging table growth. See KB0755790. 1 Check the Import Set Deleter job status. Job is Active and ran recently Step 2 2 Check the System Log for the time the Deleter last ran. Look for errors, timeouts, or transaction cancellations during its execution. Log shows transaction cancelled or timeout The table has grown so large the cleanup itself is timing out. Run manual cleanup first (System Import Sets > Import Set Tables > Cleanup) to clear the backlog, then re-enable scheduled cleanup. 2 Check System Log around the last Deleter run time. Job ran cleanly but tables are still large Step 3 3 Check the volume of records being added per day. If records are added faster than the 7-day window can clear them, the table grows regardless. Volume exceeds 7-day window capacity Reduce the retention period to 3 days or less by modifying the Deleter job script: ic.setDaysAgo(3); (script path: /sysauto_script.do?sys_id=a060c407c0a8006f0067fef2befb357d). 3 Check daily import volume against the retention window. Volume is reasonable but tables are still large Run Manual Cleanup (System Import Sets > Import Set Tables > Cleanup) to clear the existing backlog, then let the scheduled job maintain going forward. For tables exceeding 5 million rows, use the Manual Cleanup module to delete records in batches. Note: For imports where debugging is not needed and rows can be discarded immediately after transform, use an onComplete transform script with the ImportSetCleaner script include to drop rows right after each run. This prevents accumulation without waiting for the nightly job.

Section 7: Concurrent Import Issue 7.1 — Concurrent import not creating multiple import sets The "Concurrent Import" checkbox is enabled on the Scheduled Data Import record, but the system creates only one import set (running serially) or creates no import sets at all despite the option being checked. See KB0720801 for an overview of concurrent import.

Troubleshooting Steps

Step What to do What you find Next action 1 Confirm the Concurrent Import field is checked on the Scheduled Data Import record. Also confirm this is a scheduled job (not a manual "Load All Records" run). See KB0832611. Running manually (Load All Records) Concurrent import only works via scheduled jobs. Manual runs are always serial by design. 1 Confirm import is triggered via a scheduled job with Concurrent Import checked. Running as scheduled job with option checked Step 2 2 Check the property glide.scheduled_import.max.concurrent.import_sets (default = 10) and the number of active nodes in the cluster (System Diagnostics > Stats). Property is set to 1 This forces serial execution even with concurrent enabled. Increase the property to at least 2. 2 Check max.concurrent.import_sets property and cluster node count. Property is correct (2 or higher) Step 3 3 Check the sys_concurrent_import_set_job table for entries from the failing run. Check their state (Pending, Running, Completed, Error). See KB0753011. Jobs are stuck in Pending The Import Set Transformer job is not polling the queue. Check that the Transformer job is running ( Section 1.2 ). 3 Check sys_concurrent_import_set_job entries. No entries created at all The execution plan failed to create jobs. Check sys_execution_context and sys_execution_plan for errors. Note: Concurrent import only affects the transform phase — the import phase is always serial. For duplicate records with concurrent imports, ensure "Synchronize Inserts" is enabled on the scheduled import record ( Section 1.4 ).

Section 8: Data Sources Issue 8.1 — JDBC data source: connection failing, authentication error, or returning no data The "Test Connection" or "Load 20 Records" operation on a JDBC data source fails, returns an error, or returns no records despite valid data in the external database. This section covers three common JDBC failure patterns: general connectivity issues, password cache errors after a password change (KB1641538), and Windows Integrated Authentication failures with SQL Server (KB0719341).

Triage: Identify the failure pattern first Pattern Key indicator Go to General connectivity failure (driver, network, credentials) Connection fails immediately on first setup or consistently; restarting MID does not change the behavior Section 8.1 -A General JDBC troubleshooting below Password cache issue after DB password change Connection worked before a DB password change; restarting the MID Server fixes it temporarily; error includes "null" or key set errors; fails about 1 hour after password change Section 8.1 -B JDBC password cache (KB1641538) SQL Server Windows Integrated Authentication failure Error includes "Login failed for user 'DOMAIN\username'" or "login is from an untrusted domain"; SQL Server database uses Windows Authentication Section 8.1 -C SQL Server integrated authentication (KB0719341) Section 8.1 -A General JDBC Troubleshooting Step What to do What you find Next action 1 Verify the MID Server associated with this JDBC data source is Up and not reporting errors. Navigate to ecc_agent.list. MID Server is Down or in Error Resolve MID Server health before testing the data source. Check MID logs for the root cause. 1 Check MID Server status. MID Server is Up Step 2 2 Run "Test Connection" on the data source record. Check the ECC queue for the output record generated by this test: ecc_queue.list, filter queue=output, topic like "%JDBC%". Also check if the MID has the correct JDBC driver in its ext/ directory. ECC shows authentication error or missing driver JDBC credentials are wrong or the driver is missing. Verify credentials and confirm the JDBC jar is in the MID's ext/ directory. 2 Check ECC queue output for the test connection. Connection succeeds but no data Step 3 3 Review the SQL query on the data source record. Test the exact query directly against the database (outside ServiceNow) to confirm it returns data. Query returns no records outside ServiceNow The query itself is filtering all records. Fix the WHERE clause or check permissions on the database user. 3 Test the SQL query directly against the database. Query returns records outside ServiceNow but not via JDBC Likely an IP whitelist or firewall issue between the MID Server and the database. Confirm the database allows connections from the MID Server IP. See KB0538460. Section 8.1 -B JDBC Password Cache Issue (KB1641538) Before applying this fix, confirm the pattern: If the customer restarts the MID Server and the problem goes away temporarily (until the next password change), this is the correct issue. If restarting the MID Server does not fix it, this is a different problem — do not apply this workaround.

Root cause: The MID Server caches JDBC connection credentials in a connection pool. When the database user's password is changed, the MID Server continues using the old cached password. The failure appears approximately one hour after the password change (when the cached connections expire and new ones are attempted with the old password). The error is generic: Cannot invoke "java.util.Map.keySet()" because "this.fImportSourceFields" is null .

Workaround: Add a unique timestamp parameter to the JDBC Connection URL in the data source record. The JDBC driver ignores the unknown parameter, but the different URL forces the MID Server to create a new connection pool with the new credentials.

Database type URL format example MySQL / MariaDB jdbc:mysql://localhost/my_database?mid_pool=2024_04_20__12_00 MySQL / MariaDB (with existing properties) jdbc:mysql://localhost/my_database?abc=1&xyz=2&mid_pool=2024_04_20__12_00 SQL Server jdbc:sqlserver://localhost/my_database;mid_pool=2024_04_20__12_00 Oracle jdbc:oracle:thin:@localhost:1521:ORCL?mid_pool=2024_04_20__12_00 Important notes:

Each database type has its own URL syntax. MySQL and Oracle use ? to start properties and & to delimit them. SQL Server uses ; to start and delimit properties. Teradata and Databricks JDBC drivers do not ignore unknown properties. This workaround does not work for those databases — restart the MID Server instead. Use the same timestamp every time the same password is used. This allows DB connections to be shared across data sources using the same password. Never use the same timestamp for different passwords. Do not use the "flush credentials" operation as the workaround — it has a separate defect that leaks JDBC connections. Related defect: PRB1757699. Section 8.1 -C SQL Server Windows Integrated Authentication Failure (KB0719341) Symptom: Connection fails with error: Login failed for user 'DOMAIN\username' or Login is from an untrusted domain and cannot be used with Windows authentication , even when the correct credentials are in the data source record.

Root cause: The SQL Server database is configured to require Windows Integrated Authentication. In this mode, the database uses the Windows account that the MID Server service is running under — not the username and password in the data source record. If the MID Server service account does not have access to the SQL Server, the connection fails regardless of the data source credentials.

Resolution:

Configure the MID Server service account: On the MID Server host, open Windows Services. Locate the ServiceNow MID Server service. Right-click > Properties > Log On tab > select "This account" > enter the domain account credentials (for example, DOMAIN\svc-account) > OK > Restart the service. Configure the data source: In ServiceNow, navigate to System Import Sets > Administration > Data Sources. Open the data source record. Set "Use integrated authentication" to true. Save. Update the JDBC connection string if still failing: Add the integratedSecurity parameter: jdbc:sqlserver://database-server.example.com:1433;selectMethod=cursor;databaseName=exampledb;integratedSecurity=true Additional considerations:

Multiple integrations requiring different service accounts: The MID Server Windows service can only run as a single user account. If you have multiple SQL Server connections requiring Integrated Authentication with different user accounts, you must use a separate MID Server for each account. "Login is from an untrusted domain" error: This is an Active Directory trust issue (not a ServiceNow issue) and typically occurs when a new MID Server is deployed in a different network location from the SQL Server. Work with the customer's Active Directory and SQL Server administrators to resolve the trust relationship. Issue 8.2 — File or attachment data source failing to load data A file-type data source (CSV, Excel, XML, JSON via FTP, HTTP, or attachment) is failing to load data into the staging table, is empty after a run, or (for XML) is missing some expected columns in the staging table despite successful file retrieval.

Troubleshooting Steps

Step What to do What you find Next action 1 Identify the file retrieval method: Attachment, FTP, HTTP, or SFTP. For Attachment type, confirm a file is attached to the data source record. Attachment type with no file attached The import has nothing to read. Attach the source file to the data source record. 1 Identify the file retrieval method. FTP / HTTP / SFTP Step 2 2 Click "Load 20 Records" on the data source record. Check the System Log for file retrieval errors (FTP login failed, file not found, connection refused). File retrieval error (auth / connection) Fix server connectivity, credentials, or IP whitelist. For SFTP: check if the server requires a specific key authentication method. 2 Run "Load 20 Records" and check System Log. File is retrieved but staging table is empty Step 3 3 Validate the file format and contents. Check for: password protection, encoding issues, header row format, empty pages (Excel), XPath expression (XML). For CSV/Excel: check for duplicate column headers — ServiceNow cannot map two columns with the same name. Duplicate column headers in CSV/Excel ServiceNow cannot resolve duplicate column names. Fix the source file by removing or renaming duplicate column headers. 3 Validate file format and contents. File looks correct but staging is empty Enable import debug: set glide.import.debug=true and re-run to capture detailed logs. Check whether the header row format matches the expected format. 3 For XML data sources only: check whether all expected XML fields are represented as columns in the staging table. Open the staging table and verify the column count matches the unique elements in the raw XML. Some XML elements appear in the file but are missing as staging table columns Step 4 4 Check the system property glide.db.impex.XMLLoader.max.scan_nodes . Navigate to System Properties and search for this property. Property is unset or set to a value lower than the position where missing fields first appear in the XML Increase glide.db.impex.XMLLoader.max.scan_nodes to a value higher than the record number where the latest-appearing field appears (e.g., 80 or higher). Re-run the import. 4 Check the system property glide.db.impex.XMLLoader.max.scan_nodes value. Property exists and is set to a value sufficiently high Enable import debug: set glide.import.debug=true and re-run. Check System Log for schema inference warnings. Section 9: Performance Issue 9.1 — Import or transform is slow or taking excessively long The import or transform is taking significantly longer than expected. The import phase and the transform phase have different root causes for slowness — use the Summary Tool to identify which phase is the bottleneck before investigating further. See KB0793356 and KB0814735 for extended investigation steps.

Phase identification triage What the Summary Tool shows Likely phase Go to Import rate is in single digits (recs/sec); transform rate is normal or not reached Import phase is the bottleneck Section 9.1 -A Import phase troubleshooting Import rate is normal; transform rate is in single digits Transform phase is the bottleneck Section 9.1 -B Transform phase troubleshooting Both rates are in single digits Both phases are affected — likely DB contention or system-wide load Section 9.1 -A then Section 9.1 -B; also check KB0793356 Section 9.1 -A Import phase troubleshooting Check What to look for Fix Staging table size Very large staging table (millions of undeleted rows from previous imports) Clean the staging table first ( Section 6.1 ). Large staging tables slow every import operation. Schema alteration during import New columns in the incoming file that do not exist in the staging table (logs show table-alter operations during the import) Pre-define all staging table columns before running large imports. Schema changes lock the table. Importing unchanged data Source system exports all records every run regardless of changes Implement delta / change detection at the source, or add a WHERE clause filter in the JDBC SQL query. Simultaneous imports Multiple large imports running concurrently Schedule large imports at different times to prevent database resource contention. Slow queries in localhost log Transaction ID from Summary Tool shows slow SQL queries in the localhost log (see KB0814735 step 2) Engage the Performance team with the slow query output from the localhost log. Slow Business Rules in localhost log Transaction ID shows Business Rules consuming significant time per row (see KB0814735 step 3) Identify the specific BRs in the slow BR log output. Optimize or conditionally disable them for bulk import runs. DB contention or CPU load Other long-running processes or high CPU on the instance during the import window Review instance activity during the import window. Reschedule imports during off-peak hours. Section 9.1 -B Transform phase troubleshooting Check What to look for Fix Business rules on target table Summary Tool Definition section: "Non-async BRs on target: N" where N > 5. Use KB0538161 to identify the slowest BRs. See also KB0814735 step 3. Disable "Run Business Rules" on the transform map for bulk imports, or optimize the slowest BRs. Every non-async BR fires on every insert or update. Coalesce on non-indexed field Coalesce is configured on a field that does not have a database index on the target table. See KB0818304. Add a database index to the coalesce field on the target table. This is one of the highest-impact performance fixes available — a full table scan runs on every row without an index. Slow transform scripts Active onBefore / onAfter scripts with GlideRecord queries or expensive operations running per row Cache repeated GlideRecord lookups in onStart variables. Move logic out of onBefore/onAfter into onStart / onComplete where possible. Large staging table (during coalesce) Staging table has millions of rows; coalesce queries against it are slow Clean staging tables ( Section 6.1 ) before running large transforms. No coalesce configured Transform map has no coalesce field — every row is an insert; import is slower than expected for the record count Configure coalesce on an indexed field if update-or-insert behavior is needed. If insert-only is intentional, ensure the table is not growing unboundedly. Note: Concurrent Import should only be considered after all other performance optimizations are exhausted. It only improves transform throughput on very large datasets (millions of rows) and adds overhead for smaller imports. Section 7.1 and KB0720801.

Section 10: Data Quality Issue 10.1 — Fields not populated or missing data in target table after transform The transform completes but specific fields in the target table are empty, null, or contain incorrect values. The source data exists in the staging table but is not arriving correctly in the target.

Troubleshooting Steps

Step What to do What you find Next action 1 Check the staging table row for the affected field. Navigate to {staging_table}.list, find the affected row, check the column value for the missing field. Field is empty in staging too The issue is in the import phase, not the transform. The source data is not reaching staging. Check the data source or file format for that column (KB0538154). 1 Check staging row value for the affected field. Field has data in staging Step 2 2 Open the transform map. Find the field map for the affected field. Confirm it exists and that the source and target fields are correctly mapped. No field map exists for this field The field is not mapped. Add a field map for it. 2 Find the field map for the affected field. Field map exists Step 3 3 Check the "Copy empty fields" checkbox on the transform map. If unchecked and the source value is empty for some rows, the target field will not be overwritten. Copy Empty Fields is unchecked and source has empty values The transform is intentionally skipping empty source fields. Enable Copy Empty Fields if overwriting with empty is required. 3 Check the Copy Empty Fields setting. Copy Empty Fields is on, field still missing Step 4 4 Check if a field-level script is active on this field map. Also check whether a transform event script (onBefore/onAfter) is overwriting the value back to null. Add gs.info() logging and re-run a test. Script overwriting to null or empty Fix the script logic. 4 Check field-level and event scripts. No script issue Section 11: Character Encoding and Date Format Issues Issue 11.1 — Characters displaying incorrectly after import After import, text fields in the staging table or target table show garbled characters — typically non-ASCII characters such as accented letters, Chinese/Japanese/Korean characters, or special symbols appear as question marks or corrupted strings. See also: Video: Troubleshooting Truncated Import Data .

Troubleshooting Steps

Step What to do What you find Next action 1 Check whether the corruption is in the staging table or only in the target table. Check the staging row for the affected record after the import phase completes. Corruption already in staging The encoding issue is in the import phase — the file or source is not being read in the correct character set. Step 2 1 Compare staging row value to the expected value. Staging is correct, target is wrong A transform script or field map script is corrupting the value. Review script logic. See youtu.be/ffmQg8aOXEo for truncation troubleshooting. 2 Identify the encoding of the source file. Open the file in a text editor that shows encoding (UTF-8, UTF-8 with BOM, Latin-1, Shift-JIS, etc.). File is non-UTF-8 encoding Convert the source file to UTF-8 before importing, or configure the data source to specify the correct encoding. 2 Check source file encoding. File is UTF-8 but still garbled Step 3 3 Check the property com.glide.loader.verify_target_field_size . If enabled and the field is too short, the value may be truncated. Check field length limits on the staging table column. Field length too short Increase field length or enable dynamic field resizing. See youtu.be/ffmQg8aOXEo for truncation troubleshooting. 3 Check field length limits. Not a length issue Issue 11.2 — Date values not populating or populating with wrong values After transform, date or datetime fields in the target table are null, show the wrong date, or the import sets all dates to the same incorrect value. See also: Video: Troubleshooting Missing Data in Date Fields .

Troubleshooting Steps

Step What to do What you find Next action 1 Check the staging row value for the date field after import. Is the date string present and readable in staging? Staging field is empty Date is not being captured at the import phase. Check the data source column name mapping and whether the column exists in the source file. 1 Check staging row for the date string. Staging has the date string Step 2 2 Open the field map for the date field on the transform map. Check the "Date format" field. It must exactly match the format of the date string in the staging table (e.g., yyyy-MM-dd , dd/MM/yyyy HH:mm:ss ). Format mismatch The platform cannot parse the date string. Update the Date format field on the field map to exactly match the source format. Format mismatches produce silent failures with no visible error in transform results — look in the System Log for "Unparseable date" messages. 2 Check the Date format field on the field map. Format looks correct Step 3 3 Check the timezone context. Dates may parse correctly but be stored in the wrong timezone if the instance timezone differs from the source data's implied timezone. Check: System Properties > Basic Configuration for the instance timezone. Timezone offset matches the difference observed Use a field map script to parse the date and apply the correct timezone offset before setting the value. 3 Check timezone context. No timezone issue Reference: Import Set Row States Each row in the staging table ( sys_import_set_row ) has a state field that reflects its processing status. Understanding these states is essential for diagnosing partial failures.

State Meaning What to do Pending Row has been loaded into staging but not yet processed by the transform. Normal if the transform has not run yet. If stuck, check the Import Set Transformer job. Inserted A new record was created in the target table for this row. Expected outcome. Verify the inserted record in the target table. Updated An existing target record was updated using the coalesced field match. Expected outcome. Verify the update was applied correctly. Ignored Row was intentionally skipped by a transform script ( ignore = true ), or all values were identical to the existing record (coalesce found a match but nothing changed). Normal if expected. Review onBefore scripts if unexpected. A high Ignored count is not an error unless duplicates are also appearing. Skipped Row was skipped due to a Choice action of "Reject" — a field value was not recognized in the reference or choice list. Check the error_label field for the specific field that caused the skip. Fix the Choice action or normalize the source data. Error The transform failed for this row. The error_label field contains the reason. Read the error_label field. Common causes: script exception, mandatory field missing, field type mismatch. Reference: Key Diagnostic Tools and Navigation Paths Import Transform Summary Tool (KB0870045) This tool provides a detailed snapshot of an Import Set — its configuration, runtime status, and transaction context. Requires admin access (Scripts-Background). It produces three sections: Definition (configuration warnings, field maps), Runtime (import and transform rates, current state, timestamps), and Transaction (transaction ID, node). If you do not have Scripts-Background access, share the ISET number with your ServiceNow administrator or contact ServiceNow Support.

Requires admin access. Navigate to System Import Sets > Scripts-Background. Paste the script from KB0870045 and replace the ISET number with your ISET#.

Key Navigation Paths Task Navigation View import sets and their state System Import Sets > Advanced > Import Sets View transform history / import set runs System Import Sets > Advanced > Import Set Runs View / edit transform maps System Import Sets > Administration > Transform Maps View / edit scheduled imports System Import Sets > Administration > Scheduled Imports View staging table rows Navigate to {staging_table_name}.list in the navigation bar Load data manually System Import Sets > Load Data Check IS Transformer job System Scheduler > Scheduled Jobs — search "Import Set Transformer" Manual table cleanup System Import Sets > Import Set Tables > Cleanup Scheduled cleanup settings System Import Sets > Import Set Tables > Scheduled Cleanup Node restart check sys_diagnostic_event.list — filter name CONTAINS "Node Start" ECC queue (for JDBC/MID issues) ecc_queue.list — filter queue=output, agent=MID server name Concurrent import aggregated view System Import Sets > Advanced > Concurrent Import Sets LDAP Browser System LDAP > LDAP Servers > [record] > LDAP Browser Key Properties Note: The following system properties require System Administrator access to modify. If you do not have admin access, share these property names with your ServiceNow administrator.

Property Purpose Default glide.import.debug Enable detailed debug logging for all import processes false glide.xlsx.import.debug Enable debug logging for Excel (XLSX) imports false com.glide.import_set.importlog_level Control import log verbosity normal glide.ldap.debug Enable LDAP debug logging false glide.ldap.listen.verbose Enable verbose LDAP listener logging false glide.ldap.max_results Max LDAP records per page (must not exceed LDAP server limit) 1000 glide.import_set_row.dynamically_add_fields Whether to auto-create staging columns for unrecognized source fields true glide.scheduled_import.max.concurrent.import_sets Max concurrent import sets per run 10 glide.scheduled_import.stop_on_error Stop the import chain if a scheduled import errors false Additional Resources KB0867752 — Import/Transform Execution Flow and Table Reference (foundational reference) KB0870045 — Import Transform Summary Tool KB0814735 — Troubleshooting Slow Import Sets (tips and tricks) KB0538460 — JDBC/MID Missing Data Patterns KB1641538 — JDBC Password Cache Issue After DB Password Change KB0719341 — SQL Server JDBC Connection Fails with "Login failed for user" Error KB0541631 - Import and export resources page Import/Transform Learning Path (Community Article)