<h2>ServiceNow Discovery — Troubleshooting for VMware ESX / vCenter Discovery</h2><br/><div style="overflow-x:auto"><article><div ><h3 >Summary</h3><section><style type="text/css"><!-- /*NS Branding Styles*/ --> .ns-kb-css-body-editor-container { p { font-size: 12pt; font-family: Lato; color: var(--now-color--text-primary, #000000); } span { font-size: 12pt; font-family: Lato; color: var(--now-color--text-primary, #000000); } h2 { font-size: 24pt; font-family: Lato; color: var(--now-color--text-primary, black); } h3 { font-size: 18pt; font-family: Lato; color: var(--now-color--text-primary, black); } h4 { font-size: 14pt; font-family: Lato; color: var(--now-color--text-primary, black); } a { font-size: 12pt; font-family: Lato; color: var(--now-color--link-primary, #00718F); } a:hover { font-size: 12pt; color: var(--now-color--link-primary, #024F69); } a:target { font-size: 12pt; color: var(--now-color--link-primary, #032D42); } a:visited { font-size: 12pt; color: var(--now-color--link-primary, #00718f); } ul { font-size: 12pt; font-family: Lato; } li { font-size: 12pt; font-family: Lato; } img { display: ; max-width: ; width: ; height: ; } } </style> <div class="ns-kb-css-body-editor-container"> <div style="max-width: 920px; margin: 0 auto;"> <div style="background: #005baa; color: #fff; border-radius: 10px; padding: 28px 32px; margin-bottom: 24px;"> <div style="font-size: 11px; text-transform: uppercase; letter-spacing: 1px; opacity: 0.75; margin-bottom: 6px;">ServiceNow Discovery — Troubleshooting Guide</div> <div style="font-size: 26px; font-weight: 800; margin-bottom: 6px;">VMware ESX / vCenter Discovery</div> <div style="font-size: 14px; opacity: 0.88;">Complete reference: credential setup, probe chain, CI hierarchy, properties, error codes & known issues</div> <div style="margin-top: 14px; display: flex; gap: 10px; flex-wrap: wrap;"><span style="background: rgba(255,255,255,0.18); padding: 3px 12px; border-radius: 12px; font-size: 12px;">Protocol: HTTP + SOAP</span> <span style="background: rgba(255,255,255,0.18); padding: 3px 12px; border-radius: 12px; font-size: 12px;">CI Type: cmdb_ci_esx_server</span> <span style="background: rgba(255,255,255,0.18); padding: 3px 12px; border-radius: 12px; font-size: 12px;">Target: vCenter IP (not ESX directly)</span> <span style="background: rgba(255,255,255,0.18); padding: 3px 12px; border-radius: 12px; font-size: 12px;">Versions: Zurich / Xanadu</span></div> </div> <div style="background: #fff; border: 1px solid #d1d5db; border-radius: 8px; padding: 18px 24px; margin-bottom: 24px;"> <div style="font-size: 13px; font-weight: bold; text-transform: uppercase; letter-spacing: 0.5px; color: #374151; margin-bottom: 10px;">Contents</div> <ol style="margin: 0; padding-left: 20px; font-size: 13px; line-height: 2.1; color: #005baa;"><li><a style="color: #005baa; text-decoration: none;" href="#section-overview">Overview & Architecture</a></li><li><a style="color: #005baa; text-decoration: none;" href="#section-checklist">Pre-Discovery Checklist</a></li><li><a style="color: #005baa; text-decoration: none;" href="#section-symptoms">Troubleshoot by Symptom</a></li><li><a style="color: #005baa; text-decoration: none;" href="#section-errors">Error Message Reference</a></li><li><a style="color: #005baa; text-decoration: none;" href="#section-properties">System Properties Reference</a></li><li><a style="color: #005baa; text-decoration: none;" href="#section-probe">Probe/Sensor Chain Deep Dive</a></li><li><a style="color: #005baa; text-decoration: none;" href="#section-known">Known Issues & Limitations</a></li></ol> </div> <div id="section-overview" style="margin-bottom: 28px;"> <div style="background: #005baa; color: #fff; padding: 12px 20px; border-radius: 8px 8px 0 0; font-size: 16px; font-weight: bold;">1. Overview & Architecture</div> <div style="background: #fff; border: 1px solid #d1d5db; border-top: none; border-radius: 0 0 8px 8px; padding: 20px 22px;"> <div style="border-left: 5px solid #dc2626; background: #fef2f2; padding: 14px 18px; border-radius: 0 8px 8px 0; margin-bottom: 20px;"> <div style="font-weight: 800; font-size: 14px; color: #991b1b; margin-bottom: 6px;">Critical: Target vCenter — not ESX hosts</div> <div style="font-size: 13px; color: #7f1d1d; line-height: 1.7;">Unlike Network Switch or Router Discovery which targets device IPs directly, VMware Discovery <strong>must target the vCenter Server IP</strong>. Discovery authenticates to vCenter via its SOAP/REST API (port 443) and retrieves the complete inventory tree — datacenters, clusters, ESX hosts, and all VMs — in a single orchestrated probe chain. Pointing Discovery at individual ESX host IPs discovers only basic IP/host info with no relationships.</div> </div> <p style="margin: 0 0 14px; font-size: 14px;">ServiceNow Discovery uses the standard 4-phase engine for VMware, but the technology in Phase 4 is different from SNMP-based patterns. Instead of a Pattern (.sa_pattern), it uses a legacy <strong>Probe/Sensor chain</strong> that calls the VMware vSphere Web Services (SOAP) API.</p> <table style="width: 100%; border-collapse: collapse; font-size: 13px; margin-bottom: 20px;"><thead><tr><th style="background: #005baa; color: #fff; padding: 9px 12px; text-align: left; width: 18%;">Phase</th><th style="background: #005baa; color: #fff; padding: 9px 12px; text-align: left; width: 24%;">Mechanism</th><th style="background: #005baa; color: #fff; padding: 9px 12px; text-align: left;">What happens for VMware</th></tr></thead><tbody><tr style="background: #f9fafb;"><td style="padding: 9px 12px; border-bottom: 1px solid #e5e7eb; font-weight: bold;">1 — Scanning</td><td style="padding: 9px 12px; border-bottom: 1px solid #e5e7eb;">Shazzam + port probe</td><td style="padding: 9px 12px; border-bottom: 1px solid #e5e7eb;">Port probe hits vCenter IP on TCP 443 / 5480 / 9443. Shazzam confirms host is up.</td></tr><tr><td style="padding: 9px 12px; border-bottom: 1px solid #e5e7eb; font-weight: bold;">2 — Classification</td><td style="padding: 9px 12px; border-bottom: 1px solid #e5e7eb;">Process/port classifier</td><td style="padding: 9px 12px; border-bottom: 1px solid #e5e7eb;">Port 5480 (VAMI) or process <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">vmware-vpxd</code> identifies the host as <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">cmdb_ci_vcenter</code>.</td></tr><tr style="background: #f9fafb;"><td style="padding: 9px 12px; border-bottom: 1px solid #e5e7eb; font-weight: bold;">3 — Identification</td><td style="padding: 9px 12px; border-bottom: 1px solid #e5e7eb;">IRE (for vCenter CI only)</td><td style="padding: 9px 12px; border-bottom: 1px solid #e5e7eb;">IRE creates/updates the <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">cmdb_ci_vcenter</code> record. ESX hosts are identified separately by <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">VCenterESXHostsSensor</code> — not IRE.</td></tr><tr><td style="padding: 9px 12px; font-weight: bold;">4 — Exploration</td><td style="padding: 9px 12px;">Probe/Sensor chain (SOAP)</td><td style="padding: 9px 12px;">Full vCenter inventory walk via SOAP API: datacenters, clusters, ESX hosts, VMs, networks, datastores, folders.</td></tr></tbody></table> <p style="margin: 0 0 10px; font-size: 14px; font-weight: bold; color: #1e3a5f;">How VMware Discovery differs from Switch/Router Discovery</p> <table style="width: 100%; border-collapse: collapse; font-size: 13px; margin-bottom: 20px;"><thead><tr><th style="background: #374151; color: #fff; padding: 8px 12px; text-align: left; width: 22%;">Dimension</th><th style="background: #374151; color: #fff; padding: 8px 12px; text-align: left;">Network Switch/Router</th><th style="background: #374151; color: #fff; padding: 8px 12px; text-align: left;">VMware ESX/vCenter</th></tr></thead><tbody><tr style="background: #f9fafb;"><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb; font-weight: 600;">Target IP</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb;">Device IP directly</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb; font-weight: bold; color: #dc2626;">vCenter IP only — never ESX IPs</td></tr><tr><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb; font-weight: 600;">Protocol</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb;">SNMP (UDP 161)</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb;">HTTPS + SOAP (TCP 443)</td></tr><tr style="background: #f9fafb;"><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb; font-weight: 600;">Credential type</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb;">SNMP (v2c/v3)</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb; font-weight: bold; color: #dc2626;">VMware (not SSH/SNMP/WMI)</td></tr><tr><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb; font-weight: 600;">Exploration engine</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb;">Pattern (sa_pattern)</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb;">Probe/Sensor chain</td></tr><tr style="background: #f9fafb;"><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb; font-weight: 600;">ESX identification</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb;">IRE (CIIdentification)</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb; font-weight: bold; color: #005baa;">VCenterESXHostsSensor (bypasses IRE)</td></tr><tr><td style="padding: 8px 12px; font-weight: 600;">CIs created</td><td style="padding: 8px 12px;">1 CI + network adapters</td><td style="padding: 8px 12px;">10+ CI types across full inventory</td></tr></tbody></table> <p style="margin: 0 0 10px; font-size: 14px; font-weight: bold; color: #1e3a5f;">All CI types created by vCenter Discovery</p> <table style="width: 100%; border-collapse: collapse; font-size: 13px; margin-bottom: 20px;"><thead><tr><th style="background: #005baa; color: #fff; padding: 8px 12px; text-align: left; width: 36%;">CI Table</th><th style="background: #005baa; color: #fff; padding: 8px 12px; text-align: left; width: 28%;">Represents</th><th style="background: #005baa; color: #fff; padding: 8px 12px; text-align: left;">Created by</th></tr></thead><tbody><tr style="background: #e8f1fb;"><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb; font-family: monospace; font-size: 12px; font-weight: bold;">cmdb_ci_vcenter</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb;">vCenter Server</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb;">IRE (Phase 3)</td></tr><tr style="background: #fff3cd;"><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb; font-family: monospace; font-size: 12px; font-weight: bold;">cmdb_ci_esx_server</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb; font-weight: bold;">ESX/ESXi Host</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb; font-weight: bold;">VCenterESXHostsSensor</td></tr><tr style="background: #f9fafb;"><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb; font-family: monospace; font-size: 12px; font-weight: bold;">cmdb_ci_vmware_instance</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb; font-weight: bold;">VM (vCenter metadata)</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb; font-weight: bold;">VCenterVMsSensor</td></tr><tr><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb; font-family: monospace; font-size: 12px;">cmdb_ci_linux_server<br />cmdb_ci_win_server</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb;">VM Guest OS</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb;">Separate IP-based discovery of VM's IP</td></tr><tr style="background: #f9fafb;"><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb; font-family: monospace; font-size: 12px;">cmdb_ci_vmware_datacenter</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb;">vSphere Datacenter</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb;">VCenterDatacenterSensor</td></tr><tr><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb; font-family: monospace; font-size: 12px;">cmdb_ci_vcenter_cluster</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb;">vSphere Cluster</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb;">VCenterClustersSensor</td></tr><tr style="background: #f9fafb;"><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb; font-family: monospace; font-size: 12px;">cmdb_ci_vcenter_network</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb;">vSphere Port Group / Network</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb;">VCenterNetworkSensor</td></tr><tr><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb; font-family: monospace; font-size: 12px;">cmdb_ci_vcenter_datastore</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb;">vSphere Datastore (storage)</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb;">VCenterDatastoresSensor</td></tr><tr style="background: #f9fafb;"><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb; font-family: monospace; font-size: 12px;">cmdb_ci_vcenter_folder</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb;">vSphere Folder</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb;">VCenterFoldersSensor</td></tr><tr><td style="padding: 8px 12px; font-family: monospace; font-size: 12px;">cmdb_ci_vmware_template</td><td style="padding: 8px 12px;">VM Template</td><td style="padding: 8px 12px;">VCenterVMsSensor (template=true rows)</td></tr></tbody></table> <p style="margin: 0 0 10px; font-size: 14px; font-weight: bold; color: #1e3a5f;">Key fields on cmdb_ci_esx_server (VMware-specific)</p> <table style="width: 100%; border-collapse: collapse; font-size: 13px; margin-bottom: 20px;"><thead><tr><th style="background: #374151; color: #fff; padding: 8px 12px; text-align: left; width: 22%;">Field</th><th style="background: #374151; color: #fff; padding: 8px 12px; text-align: left; width: 28%;">VMware API Source</th><th style="background: #374151; color: #fff; padding: 8px 12px; text-align: left;">Purpose / Notes</th></tr></thead><tbody><tr style="background: #f9fafb;"><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb; font-family: monospace; font-size: 12px; font-weight: bold;">morid</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb; font-family: monospace; font-size: 11px;">_moref.value</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb;">Managed Object Reference ID. vCenter-internal identifier. Changes if ESXi is removed and re-added to vCenter.</td></tr><tr><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb; font-family: monospace; font-size: 12px; font-weight: bold;">vcenter_ref</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb; font-family: monospace; font-size: 11px;">vCenter sys_id</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb;">Reference to the parent <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">cmdb_ci_vcenter</code> CI. Use to see which vCenter manages this host.</td></tr><tr style="background: #f9fafb;"><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb; font-family: monospace; font-size: 12px; font-weight: bold;">vcenter_uuid</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb; font-family: monospace; font-size: 11px;">hardware.systemInfo.uuid</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb; font-weight: bold; color: #1e3a5f;">Primary identifier key. This is the hardware UUID of the ESX host. Critical for deduplication. Also stored as <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">correlation_id</code> in some versions.</td></tr><tr><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb; font-family: monospace; font-size: 12px; font-weight: bold;">cluster_id</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb; font-family: monospace; font-size: 11px;">ClusterComputeResource MoRef</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb;">Identifies which vSphere cluster this ESX host belongs to.</td></tr><tr style="background: #f9fafb;"><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb; font-family: monospace; font-size: 12px; font-weight: bold;">connection_state</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb; font-family: monospace; font-size: 11px;">summary.runtime.connectionState</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb;">Values: connected / disconnected / notResponding. Disconnected hosts are still reported by vCenter.</td></tr><tr><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb; font-family: monospace; font-size: 12px; font-weight: bold;">power_state</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb; font-family: monospace; font-size: 11px;">summary.runtime.powerState</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb;">poweredOn / poweredOff / standBy / unknown.</td></tr><tr style="background: #f9fafb;"><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb; font-family: monospace; font-size: 12px; font-weight: bold;">hyper_threading</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb; font-family: monospace; font-size: 11px;">config.hyperThread.active</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb;">Boolean. True if HyperThreading is enabled on this host.</td></tr><tr><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb; font-family: monospace; font-size: 12px; font-weight: bold;">logical_processor</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb; font-family: monospace; font-size: 11px;">hardware.cpuInfo.numCpuThreads</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb;">Total logical processors (cores × threads per core when HT enabled).</td></tr><tr style="background: #f9fafb;"><td style="padding: 8px 12px; font-family: monospace; font-size: 12px; font-weight: bold;">object_id</td><td style="padding: 8px 12px; font-family: monospace; font-size: 11px;">hardware.systemInfo.uuid</td><td style="padding: 8px 12px;">Used as a secondary identifier. Populated from the ESX hardware UUID. Differs from morid.</td></tr></tbody></table> <p style="margin: 0 0 10px; font-size: 14px; font-weight: bold; color: #1e3a5f;">Relationship types created by vCenter Discovery</p> <table style="width: 100%; border-collapse: collapse; font-size: 13px;"><thead><tr><th style="background: #005baa; color: #fff; padding: 8px 12px; text-align: left; width: 34%;">Relationship type</th><th style="background: #005baa; color: #fff; padding: 8px 12px; text-align: left;">Parent CI</th><th style="background: #005baa; color: #fff; padding: 8px 12px; text-align: left;">Child CI</th><th style="background: #005baa; color: #fff; padding: 8px 12px; text-align: left;">Created by</th></tr></thead><tbody><tr style="background: #f9fafb;"><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb; font-family: monospace; font-size: 11px;">Virtualized by::Virtualizes</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb; font-family: monospace; font-size: 11px;">cmdb_ci_esx_server</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb; font-family: monospace; font-size: 11px;">cmdb_ci_linux/win_server</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb;">VCenterESXHostsSensor (fixVirtualizes)</td></tr><tr><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb; font-family: monospace; font-size: 11px;">Instantiates::Instantiated by</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb; font-family: monospace; font-size: 11px;">cmdb_ci_vmware_instance</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb; font-family: monospace; font-size: 11px;">cmdb_ci_linux/win_server</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb;">VmwareVmCorrelator</td></tr><tr style="background: #f9fafb;"><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb; font-family: monospace; font-size: 11px;">Virtualized by::Virtualizes</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb; font-family: monospace; font-size: 11px;">cmdb_ci_esx_server</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb; font-family: monospace; font-size: 11px;">cmdb_ci_vmware_instance</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb;">vCenter probe chain</td></tr><tr><td style="padding: 8px 12px; font-family: monospace; font-size: 11px;">Runs on::Runs</td><td style="padding: 8px 12px; font-family: monospace; font-size: 11px;">cmdb_ci_esx_server</td><td style="padding: 8px 12px; font-family: monospace; font-size: 11px;">cmdb_ci_vcenter_cluster</td><td style="padding: 8px 12px;">VCenterClustersSensor</td></tr></tbody></table> </div> </div> <div id="section-checklist" style="margin-bottom: 28px;"> <div style="background: #005baa; color: #fff; padding: 12px 20px; border-radius: 8px 8px 0 0; font-size: 16px; font-weight: bold;">2. Pre-Discovery Checklist</div> <div style="background: #fff; border: 1px solid #d1d5db; border-top: none; border-radius: 0 0 8px 8px; padding: 20px 22px;"> <details style="margin-bottom: 12px; border: 1px solid #bfdbfe; border-radius: 6px;" open=""> <summary style="background: #eff6ff; padding: 12px 16px; border-radius: 6px; cursor: pointer; font-weight: bold; font-size: 14px; color: #1e3a5f; list-style: none;">▶ Step 1 — Create VMware Credential (most critical step)</summary> <div style="padding: 16px; font-size: 13px;"> <div style="border-left: 4px solid #dc2626; background: #fef2f2; padding: 10px 14px; border-radius: 0 6px 6px 0; margin-bottom: 14px;"><strong>Wrong credential type is the #1 cause of vCenter discovery failure.</strong> Using SSH, SNMP, or WMI credential types against a vCenter IP will never work. The credential MUST be type <strong>"VMware"</strong>.</div> <p style="margin: 0 0 10px;"><strong>Navigate to:</strong> Discovery → Credentials → New → Type: <strong>VMware</strong></p> <table style="width: 100%; border-collapse: collapse; font-size: 13px; margin-bottom: 12px;"><tbody><tr style="background: #f3f4f6;"><th style="padding: 7px 10px; text-align: left; font-size: 12px;">Field</th><th style="padding: 7px 10px; text-align: left; font-size: 12px;">Required value</th><th style="padding: 7px 10px; text-align: left; font-size: 12px;">Notes</th></tr><tr><td style="padding: 7px 10px; border-bottom: 1px solid #e5e7eb; font-weight: 600;">Type</td><td style="padding: 7px 10px; border-bottom: 1px solid #e5e7eb; font-weight: bold; color: #dc2626;">VMware</td><td style="padding: 7px 10px; border-bottom: 1px solid #e5e7eb;">Not SSH, not SNMP, not WMI</td></tr><tr style="background: #f9fafb;"><td style="padding: 7px 10px; border-bottom: 1px solid #e5e7eb; font-weight: 600;">Username</td><td style="padding: 7px 10px; border-bottom: 1px solid #e5e7eb; font-family: monospace; font-size: 12px;">user@domain.com</td><td style="padding: 7px 10px; border-bottom: 1px solid #e5e7eb;">Format: UPN or DOMAIN\User. Must be a vCenter local or AD account.</td></tr><tr><td style="padding: 7px 10px; border-bottom: 1px solid #e5e7eb; font-weight: 600;">Password</td><td style="padding: 7px 10px; border-bottom: 1px solid #e5e7eb;"><vCenter password></td><td style="padding: 7px 10px; border-bottom: 1px solid #e5e7eb;">Avoid special XML chars: &, <, >, " in passwords (see Known Issues)</td></tr><tr style="background: #f9fafb;"><td style="padding: 7px 10px; font-weight: 600;">IP ranges</td><td style="padding: 7px 10px;">vCenter IPs only</td><td style="padding: 7px 10px;">Assign this credential only to the vCenter management IP range, not to all IPs</td></tr></tbody></table> <p style="margin: 0 0 8px;"><strong>Required vCenter permissions (assign at root vCenter object):</strong></p> <ul style="margin: 0; padding-left: 20px; line-height: 1.9; font-size: 13px;"><li><strong>Role:</strong> Read-Only (or custom role with read permissions)</li><li><strong>Global permission:</strong> Global → Licenses (required for vCenter license data)</li><li><strong>"Propagate to children" checkbox: MUST be checked.</strong> Without this, the vCenter API cannot traverse the inventory tree. ESX hosts and VMs will not be discovered.</li></ul> <p style="margin: 12px 0 4px;"><strong>Test the credential before running Discovery:</strong></p> <p style="margin: 0; font-size: 13px;">On the credential record → click <strong>Test Credential</strong> → enter the vCenter IP, leave port as 443, select MID Server → click OK. Expect "Credential validated." Any other result means the credential is wrong.</p> </div> </details> <details style="margin-bottom: 12px; border: 1px solid #bfdbfe; border-radius: 6px;"> <summary style="background: #eff6ff; padding: 12px 16px; border-radius: 6px; cursor: pointer; font-weight: bold; font-size: 14px; color: #1e3a5f; list-style: none;">▶ Step 2 — Verify Firewall Ports (MID Server to vCenter)</summary> <div style="padding: 16px; font-size: 13px;"> <table style="width: 100%; border-collapse: collapse; font-size: 13px; margin-bottom: 12px;"><thead><tr><th style="background: #374151; color: #fff; padding: 8px 12px; text-align: left;">Port</th><th style="background: #374151; color: #fff; padding: 8px 12px; text-align: left;">Protocol</th><th style="background: #374151; color: #fff; padding: 8px 12px; text-align: left;">Purpose</th><th style="background: #374151; color: #fff; padding: 8px 12px; text-align: left;">Required?</th></tr></thead><tbody><tr style="background: #f9fafb;"><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb; font-weight: bold; font-family: monospace;">443</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb;">HTTPS</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb;">vCenter SOAP API (primary). All probe chain calls use this port.</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb; font-weight: bold; color: #16a34a;">Required</td></tr><tr><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb; font-weight: bold; font-family: monospace;">5480</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb;">HTTPS</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb;">VAMI (vCenter Appliance Management Interface). Required for the port-probe classification method.</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb; font-weight: bold; color: #b45309;">If using port probe</td></tr><tr style="background: #f9fafb;"><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb; font-weight: bold; font-family: monospace;">9443</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb;">HTTPS</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb;">vSphere Web Client API. Required for port-probe method.</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb; font-weight: bold; color: #b45309;">If using port probe</td></tr><tr><td style="padding: 8px 12px; font-weight: bold; font-family: monospace;">135, 49152+</td><td style="padding: 8px 12px;">TCP</td><td style="padding: 8px 12px;">WMI/Windows. For <strong>Windows Guest OS discovery only</strong> — not for vCenter connectivity.</td><td style="padding: 8px 12px;">For Windows VM OS discovery</td></tr></tbody></table> <p style="margin: 0;"><strong>Test from MID Server:</strong> <code style="background: #f3f4f6; padding: 2px 6px; border-radius: 3px;">Test-NetConnection -ComputerName <vCenter_IP> -Port 443</code> (PowerShell) or <code style="background: #f3f4f6; padding: 2px 6px; border-radius: 3px;">nc -zv <vCenter_IP> 443</code> (Linux)</p> </div> </details> <details style="margin-bottom: 12px; border: 1px solid #bfdbfe; border-radius: 6px;"> <summary style="background: #eff6ff; padding: 12px 16px; border-radius: 6px; cursor: pointer; font-weight: bold; font-size: 14px; color: #1e3a5f; list-style: none;">▶ Step 3 — Configure Discovery Schedule (target vCenter IP, not ESX IPs)</summary> <div style="padding: 16px; font-size: 13px;"> <p style="margin: 0 0 10px;">Navigate to: <strong>Discovery → Discovery Schedules → New</strong></p> <ul style="margin: 0 0 12px; padding-left: 20px; line-height: 1.9;"><li><strong>IP Range:</strong> Enter the vCenter Server's management IP address. Do not enter ESX host IPs.</li><li><strong>Credential:</strong> Select the VMware credential created in Step 1.</li><li><strong>Include IPs / Networks:</strong> Use the specific vCenter management IP; avoid broad /16 ranges that include ESX host IPs — ESX IPs discovered directly return only basic IP/hostname data with no inventory or relationship data.</li><li><strong>MID Server:</strong> Must have network connectivity to vCenter on port 443. Verify via the Test Credential step.</li></ul> <div style="border-left: 4px solid #b45309; background: #fffbeb; padding: 10px 14px; border-radius: 0 6px 6px 0; font-size: 12px;"><strong>Scope tip:</strong> One vCenter discovery run discovers ALL ESX hosts and VMs managed by that vCenter. You do not need one schedule entry per ESX host. Adding all ESX IPs individually to the Discovery range wastes resources and produces duplicate or partial CIs.</div> </div> </details> <details style="border: 1px solid #bfdbfe; border-radius: 6px;"> <summary style="background: #eff6ff; padding: 12px 16px; border-radius: 6px; cursor: pointer; font-weight: bold; font-size: 14px; color: #1e3a5f; list-style: none;">▶ Step 4 — MID Server Requirements</summary> <div style="padding: 16px; font-size: 13px;"> <ul style="margin: 0; padding-left: 20px; line-height: 1.9;"><li>MID Server must have the <strong>VMware</strong> capability enabled. Navigate to: MID Server record → Capabilities tab → confirm <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">VMware</code> capability is present and active.</li><li>If multiple MID Servers exist, Discovery MID selection API chooses the best MID based on capability matching. Ensure at least one MID with VMware capability is in the same network segment as vCenter.</li><li>MID Server JVM must have sufficient heap space for large vCenter inventories. VMware probe payloads can exceed the default 5 MB limit for very large environments (10,000+ VMs). See Section 5 for <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">mid.discovery.max_payload_size</code>.</li><li>Verify: MID Server log (<code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">agent0.log</code>) should show no proxy errors or SSL handshake failures when connecting to vCenter.</li></ul> </div> </details> </div> </div> <div id="section-symptoms" style="margin-bottom: 28px;"> <div style="background: #005baa; color: #fff; padding: 12px 20px; border-radius: 8px 8px 0 0; font-size: 16px; font-weight: bold;">3. Troubleshoot by Symptom</div> <div style="background: #fff; border: 1px solid #d1d5db; border-top: none; border-radius: 0 0 8px 8px; padding: 20px 22px;"> <details style="margin-bottom: 10px; border: 1px solid #fca5a5; border-radius: 6px;"> <summary style="background: #fef2f2; padding: 11px 16px; border-radius: 6px; cursor: pointer; font-weight: bold; font-size: 13px; color: #991b1b; list-style: none;">▶ vCenter runs, only the vCenter CI is created — no ESX hosts, no VMs</summary> <div style="padding: 14px; font-size: 13px;"> <p style="margin: 0 0 8px;">This is the most common symptom. Work through in order:</p> <ol style="margin: 0; padding-left: 20px; line-height: 2.0;"><li><strong>Credential type wrong:</strong> Open the credential used → confirm Type = VMware. Change if needed, then re-test.</li><li><strong>"Propagate to children" unchecked:</strong> In vCenter, open the permission assigned to the Discovery service account → confirm the Propagate to children checkbox is checked. This is the most commonly missed step.</li><li><strong>Insufficient permissions:</strong> Verify the account has Read-Only role at the root vCenter object AND Global → Licenses privilege.</li><li><strong>VCenterESXHostsSensor disabled:</strong> Navigate to System Definition → Script Includes → search "VCenterESXHostsSensor" → confirm Active = true.</li><li><strong>VCenterVMsSensor disabled:</strong> Same check for "VCenterVMsSensor" and "VmwareVmCorrelator".</li><li><strong>ECC Queue check:</strong> Search ECC Queue for the vCenter IP → confirm probe entries for "VMWarevCenterVMsProbe" are present. If no probe entries: the vCenter Datacenters probe did not fire (credential or port issue). If probe entries exist but errored: see Section 4 Error Reference.</li></ol> </div> </details> <details style="margin-bottom: 10px; border: 1px solid #fca5a5; border-radius: 6px;"> <summary style="background: #fef2f2; padding: 11px 16px; border-radius: 6px; cursor: pointer; font-weight: bold; font-size: 13px; color: #991b1b; list-style: none;">▶ vCenter shows as a "Credentialless" CI in cmdb_ci_computer (not cmdb_ci_vcenter)</summary> <div style="padding: 14px; font-size: 13px;"> <p style="margin: 0 0 8px;"><strong>Symptom:</strong> The vCenter appliance (VCSA) appears in <code style="background: #f3f4f6; padding: 2px 5px; border-radius: 3px;">cmdb_ci_computer</code> as a generic CI with no relationships to ESXi hosts, clusters, VMs, or datastores. Discovery_source is empty or shows a non-VMware source.</p> <p style="margin: 0 0 8px;"><strong>Cause:</strong> Discovery hit the vCenter IP but the VMware credential was not selected (credential ordering issue) or authentication failed silently at the VMware API layer.</p> <ol style="margin: 0; padding-left: 20px; line-height: 1.8;"><li>Verify the VMware credential has a lower Order value than SSH/SNMP credentials in the same IP range. Discovery tries credentials in Order ascending — SSH credentials with lower Order may fire first and classify vCenter as a generic Linux host.</li><li>Create an IP Service Affinity record: Discovery → Administration → IP Service Affinities → map the vCenter IP to VMware credential type explicitly.</li><li>Delete the <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">cmdb_ci_computer</code> CI created by the wrong credential, then re-run Discovery.</li><li>Re-test the VMware credential against the vCenter IP using the Test Credential button.</li></ol> </div> </details> <details style="margin-bottom: 10px; border: 1px solid #fca5a5; border-radius: 6px;"> <summary style="background: #fef2f2; padding: 11px 16px; border-radius: 6px; cursor: pointer; font-weight: bold; font-size: 13px; color: #991b1b; list-style: none;">▶ ESX hosts discovered but "Virtualized by" relationship to Server CIs is missing</summary> <div style="padding: 14px; font-size: 13px;"> <p style="margin: 0 0 8px;"><strong>Root cause:</strong> The "Virtualized by::Virtualizes" relationship between ESX server and Linux/Windows Server CIs is created by <code style="background: #f3f4f6; padding: 2px 5px; border-radius: 3px;">VCenterESXHostsSensor</code>'s <code style="background: #f3f4f6; padding: 2px 5px; border-radius: 3px;">fixVirtualizes</code> function. This function relies on the "Instantiates::Instantiated by" relationship between the VMware Instance CI and the Server CI already existing (created by <code style="background: #f3f4f6; padding: 2px 5px; border-radius: 3px;">VmwareVmCorrelator</code>). If the VMs' OS was not separately discovered, the Server CIs do not exist yet and no relationship can be formed.</p> <ol style="margin: 0; padding-left: 20px; line-height: 1.8;"><li>Ensure both vCenter Discovery AND IP-based OS Discovery are running for the VMs' IP addresses. vCenter Discovery creates the VMware Instance CI; OS Discovery creates the Server CI. The relationship needs both.</li><li>If Server CIs exist but no relationship: check the <code style="background: #f3f4f6; padding: 2px 5px; border-radius: 3px;">Guest Reconciled</code> flag on the VMware Instance CI and <code style="background: #f3f4f6; padding: 2px 5px; border-radius: 3px;">Is Virtual</code> on the Server CI. If both are True but no relationship exists, reset both fields to False and re-run vCenter Discovery.</li><li>Confirm <code style="background: #f3f4f6; padding: 2px 5px; border-radius: 3px;">VmwareVmCorrelator</code> Script Include is Active.</li></ol> </div> </details> <details style="margin-bottom: 10px; border: 1px solid #fca5a5; border-radius: 6px;"> <summary style="background: #fef2f2; padding: 11px 16px; border-radius: 6px; cursor: pointer; font-weight: bold; font-size: 13px; color: #991b1b; list-style: none;">▶ Duplicate ESX server CIs in cmdb_ci_esx_server</summary> <div style="padding: 14px; font-size: 13px;"> <p style="margin: 0 0 8px;"><strong>Cause:</strong> ESX identification bypasses IRE and uses three identification indexes in <code style="background: #f3f4f6; padding: 2px 5px; border-radius: 3px;">VCenterESXHostsSensor</code>. If the ESX host's <code style="background: #f3f4f6; padding: 2px 5px; border-radius: 3px;">correlation_id</code> or hardware UUID changes (ESXi reinstall, vCenter re-registration, vCenter migration), none of the indexes match the existing CI and a new one is created.</p> <p style="margin: 0 0 8px;"><strong>Property note:</strong> Setting <code style="background: #f3f4f6; padding: 2px 5px; border-radius: 3px;">glide.discovery.alter_esx_server_index=true</code> changes identification to use serial_number only (removes correlation_id from the index). Use this when correlation_id is unstable across your infrastructure.</p> <ol style="margin: 0; padding-left: 20px; line-height: 1.8;"><li>Identify which CI is the "stale" one by comparing <code style="background: #f3f4f6; padding: 2px 5px; border-radius: 3px;">last_discovered</code> timestamps.</li><li>Merge or retire the stale CI. Verify CMDB Deduplication is configured for <code style="background: #f3f4f6; padding: 2px 5px; border-radius: 3px;">cmdb_ci_esx_server</code>.</li><li>If duplicates keep appearing: enable <code style="background: #f3f4f6; padding: 2px 5px; border-radius: 3px;">glide.discovery.alter_esx_server_index=true</code> to use stable serial_number identification.</li></ol> </div> </details> <details style="margin-bottom: 10px; border: 1px solid #fcd34d; border-radius: 6px;"> <summary style="background: #fffbeb; padding: 11px 16px; border-radius: 6px; cursor: pointer; font-weight: bold; font-size: 13px; color: #92400e; list-style: none;">▶ ESX server CI has blank IP address</summary> <div style="padding: 14px; font-size: 13px;"> <p style="margin: 0 0 8px;">vCenter Discovery retrieves ESX host data from the vCenter API. Some vCenter configurations do not return IP address data in the same payload as other ESX attributes.</p> <p style="margin: 0 0 8px;"><strong>Workaround options:</strong></p> <ol style="margin: 0; padding-left: 20px; line-height: 1.8;"><li>Run a separate SNMP Discovery targeting the ESX host IP directly. This populates the IP address field via a lightweight SNMP walk without re-creating the CI.</li><li>Manually populate the IP Address field on the ESX server CI and mark it as not overwritable if needed.</li><li>Verify the VMware API returns IP data: <code style="background: #f3f4f6; padding: 2px 5px; border-radius: 3px;">summary.managementServerIp</code> or <code style="background: #f3f4f6; padding: 2px 5px; border-radius: 3px;">config.network.vnic[0].spec.ip.ipAddress</code> should be populated in the vCenter inventory for this host.</li></ol> </div> </details> <details style="margin-bottom: 10px; border: 1px solid #fcd34d; border-radius: 6px;"> <summary style="background: #fffbeb; padding: 11px 16px; border-radius: 6px; cursor: pointer; font-weight: bold; font-size: 13px; color: #92400e; list-style: none;">▶ VM templates or powered-off VMs not appearing in cmdb_ci_vmware_instance</summary> <div style="padding: 14px; font-size: 13px;"> <p style="margin: 0 0 8px;">By default, vCenter Discovery discovers all VM objects regardless of power state, including templates (<code style="background: #f3f4f6; padding: 2px 5px; border-radius: 3px;">template=true</code>). If templates are missing, the credential account likely lacks permission on the vSphere template folders specifically.</p> <p style="margin: 0 0 8px;">Powered-off VMs are included in the vCenter API payload. If they are missing: check <code style="background: #f3f4f6; padding: 2px 5px; border-radius: 3px;">cmdb_ci_vmware_instance</code> for records with <code style="background: #f3f4f6; padding: 2px 5px; border-radius: 3px;">state=poweredOff</code> — they may exist but not be visible in a default list view that filters on state.</p> <p style="margin: 0;">For templates specifically: verify the Discovery account has Read-Only permission on the <strong>vm</strong> folder in the vSphere hierarchy where templates reside, with Propagate to children enabled.</p> </div> </details> <details style="margin-bottom: 10px; border: 1px solid #fcd34d; border-radius: 6px;"> <summary style="background: #fffbeb; padding: 11px 16px; border-radius: 6px; cursor: pointer; font-weight: bold; font-size: 13px; color: #92400e; list-style: none;">▶ ESX server unexpectedly decommissioned / Operational Status changed to Retired</summary> <div style="padding: 14px; font-size: 13px;"> <p style="margin: 0 0 8px;"><strong>Cause:</strong> The built-in business rule <strong>"Cascade deprovisioned status to server"</strong> on <code style="background: #f3f4f6; padding: 2px 5px; border-radius: 3px;">cmdb_ci_vmware_instance</code> runs when a VM is retired or deprovisioned. In some cases it incorrectly cascades the retired status to the parent ESX server CI. This is tracked in DEF0617331 (Can't Confirm state).</p> <p style="margin: 0 0 8px;"><strong>Workaround:</strong></p> <ol style="margin: 0; padding-left: 20px; line-height: 1.8;"><li>Review the Business Rule "Cascade deprovisioned status to server" under System Definition → Business Rules. Confirm the table is <code style="background: #f3f4f6; padding: 2px 5px; border-radius: 3px;">cmdb_ci_vmware_instance</code> and the condition correctly filters for VM termination events only.</li><li>Alternatively, set <code style="background: #f3f4f6; padding: 2px 5px; border-radius: 3px;">glide.discovery.mark_esx_servers_as_absent=true</code> to change the behavior from Retired to Absent when ESX hosts are not found in the vCenter payload, which is less destructive than Retired.</li></ol> </div> </details> <details style="border: 1px solid #fcd34d; border-radius: 6px;"> <summary style="background: #fffbeb; padding: 11px 16px; border-radius: 6px; cursor: pointer; font-weight: bold; font-size: 13px; color: #92400e; list-style: none;">▶ Payload size error — discovery stops partway through large vCenter</summary> <div style="padding: 14px; font-size: 13px;"> <p style="margin: 0 0 8px;"><strong>Error in ECC Queue:</strong> <code style="background: #f3f4f6; padding: 2px 5px; border-radius: 3px;">Payload length 5671972 exceeded limit of 5000000</code></p> <p style="margin: 0 0 8px;">Occurs in environments with thousands of VMs. The VMware probe returns more data than the default payload size limit allows.</p> <ol style="margin: 0; padding-left: 20px; line-height: 1.8;"><li>Increase <code style="background: #f3f4f6; padding: 2px 5px; border-radius: 3px;">mid.discovery.max_payload_size</code> — set to 10000000 (10 MB) as a starting point.</li><li>Also increase <code style="background: #f3f4f6; padding: 2px 5px; border-radius: 3px;">com.glide.attachment.max_get_size</code> to match or exceed the same value.</li><li>Restart the MID Server after changing these properties.</li></ol> </div> </details> </div> </div> <div id="section-errors" style="margin-bottom: 28px;"> <div style="background: #005baa; color: #fff; padding: 12px 20px; border-radius: 8px 8px 0 0; font-size: 16px; font-weight: bold;">4. Error Message Reference</div> <div style="background: #fff; border: 1px solid #d1d5db; border-top: none; border-radius: 0 0 8px 8px; padding: 20px 22px;"> <p style="margin: 0 0 14px; font-size: 14px;">Check these error locations: <strong>Discovery → Discovery Log → [run] → ECC Queue</strong> and <strong>MID Server agent0.log</strong>.</p> <details style="margin-bottom: 10px; border: 1px solid #fca5a5; border-radius: 6px;"> <summary style="background: #fef2f2; padding: 11px 16px; border-radius: 6px; cursor: pointer; font-weight: bold; font-size: 13px; color: #991b1b; list-style: none; font-family: monospace;">▶ Authentication failure / Error connecting to host</summary> <div style="padding: 14px; font-size: 13px;"> <p style="margin: 0 0 8px;"><strong>Phase:</strong> Scanning / Classification. The MID Server connected to vCenter port 443 but authentication was rejected.</p> <p style="margin: 0 0 4px;"><strong>Causes and fixes (in order):</strong></p> <ol style="margin: 0; padding-left: 20px; line-height: 1.8;"><li>Wrong credential type — must be VMware, not SSH/SNMP.</li><li>Wrong password. Reset in vCenter and update the ServiceNow credential.</li><li>Username format incorrect. Try both <code style="background: #f3f4f6; padding: 2px 5px; border-radius: 3px;">user@domain.com</code> and <code style="background: #f3f4f6; padding: 2px 5px; border-radius: 3px;">DOMAIN\user</code> formats.</li><li>Special characters in password (&, <, >, ") cause silent XML parsing failure in the SOAP call body. Use Test Credential to confirm. Change the password to exclude these characters.</li><li>Account locked in Active Directory or vCenter SSO after repeated failed attempts.</li></ol> </div> </details> <details style="margin-bottom: 10px; border: 1px solid #fca5a5; border-radius: 6px;"> <summary style="background: #fef2f2; padding: 11px 16px; border-radius: 6px; cursor: pointer; font-weight: bold; font-size: 13px; color: #991b1b; list-style: none; font-family: monospace;">▶ Payload length N exceeded limit of 5000000</summary> <div style="padding: 14px; font-size: 13px;"> <p style="margin: 0 0 8px;"><strong>Phase:</strong> Exploration. The VMware probe response for a large vCenter inventory exceeds the MID Server payload size limit.</p> <p style="margin: 0 0 8px;">Increase both <code style="background: #f3f4f6; padding: 2px 5px; border-radius: 3px;">mid.discovery.max_payload_size</code> and <code style="background: #f3f4f6; padding: 2px 5px; border-radius: 3px;">com.glide.attachment.max_get_size</code> to 10485760 (10 MB) or higher. Restart the MID Server. Discovery has been confirmed stable up to 8.5 MB payload size.</p> </div> </details> <details style="margin-bottom: 10px; border: 1px solid #fca5a5; border-radius: 6px;"> <summary style="background: #fef2f2; padding: 11px 16px; border-radius: 6px; cursor: pointer; font-weight: bold; font-size: 13px; color: #991b1b; list-style: none; font-family: monospace;">▶ InstanceSoapClient doesn't recover if GetMIDInfo encounters an exception</summary> <div style="padding: 14px; font-size: 13px;"> <p style="margin: 0 0 8px;"><strong>Phase:</strong> Exploration. The SOAP client used by the VMware probe enters an unrecoverable error state when the MID Server info call raises an exception. Subsequent probe calls on the same MID Server session fail silently. Tracked in DEF0311630 (Fixed).</p> <p style="margin: 0;"><strong>Fix:</strong> Apply the fix version referenced in DEF0311630. As an interim workaround: restart the MID Server, which clears the stuck SOAP client state. If persistent, check <code style="background: #f3f4f6; padding: 2px 5px; border-radius: 3px;">agent0.log</code> for SSL handshake failures that may be triggering the exception.</p> </div> </details> <details style="margin-bottom: 10px; border: 1px solid #fca5a5; border-radius: 6px;"> <summary style="background: #fef2f2; padding: 11px 16px; border-radius: 6px; cursor: pointer; font-weight: bold; font-size: 13px; color: #991b1b; list-style: none; font-family: monospace;">▶ VMware vCenter VM Tags probe fails SSL certificate validation</summary> <div style="padding: 14px; font-size: 13px;"> <p style="margin: 0 0 8px;"><strong>Phase:</strong> Exploration (VM Tags step). When Discovery is run against a vCenter host identified by IP (not FQDN), the VM Tags probe enforces Subject Alternative Name (SAN) validation using the IP address. If the vCenter certificate's SAN does not include the IP (only the FQDN), validation fails. Tracked in DEF0752263 (Work in Progress).</p> <p style="margin: 0 0 8px;"><strong>Impact:</strong> VM Tags are not populated on VMware Instance CIs. All other ESX/VM discovery continues normally.</p> <p style="margin: 0;"><strong>Workaround:</strong> Use the vCenter FQDN instead of the IP address in the Discovery schedule IP range. Ensure the vCenter certificate's SAN includes the FQDN. Alternatively, configure the vCenter certificate to include both IP SAN and FQDN SAN entries.</p> </div> </details> <details style="border: 1px solid #fca5a5; border-radius: 6px;"> <summary style="background: #fef2f2; padding: 11px 16px; border-radius: 6px; cursor: pointer; font-weight: bold; font-size: 13px; color: #991b1b; list-style: none; font-family: monospace;">▶ Discovery Jobs on Mutual Auth MID Servers seeing a big increase in duration</summary> <div style="padding: 14px; font-size: 13px;"> <p style="margin: 0 0 8px;"><strong>Phase:</strong> All phases. MID Servers configured with Mutual Authentication (client certificates) experience significantly longer discovery run times for VMware probes. Tracked in DEF0639562 (Fixed).</p> <p style="margin: 0;"><strong>Fix:</strong> Apply the fix version referenced in DEF0639562. The issue is in the SOAP client certificate negotiation overhead on each probe call. After patching, run times should return to pre-mutual-auth levels.</p> </div> </details> </div> </div> <div id="section-properties" style="margin-bottom: 28px;"> <div style="background: #005baa; color: #fff; padding: 12px 20px; border-radius: 8px 8px 0 0; font-size: 16px; font-weight: bold;">5. System Properties Reference</div> <div style="background: #fff; border: 1px solid #d1d5db; border-top: none; border-radius: 0 0 8px 8px; padding: 20px 22px;"> <p style="margin: 0 0 14px; font-size: 14px;">Navigate to: <strong>Discovery → Administration → Properties</strong>. Changes take effect on the next Discovery run (no MID Server restart needed unless noted).</p> <table style="width: 100%; border-collapse: collapse; font-size: 13px; margin-bottom: 14px;"><thead><tr><th style="background: #005baa; color: #fff; padding: 9px 12px; text-align: left; width: 40%;">Property</th><th style="background: #005baa; color: #fff; padding: 9px 12px; text-align: left; width: 12%;">Default</th><th style="background: #005baa; color: #fff; padding: 9px 12px; text-align: left;">Effect & When to Change</th></tr></thead><tbody><tr style="background: #f9fafb;"><td style="padding: 9px 12px; border-bottom: 1px solid #e5e7eb; font-family: monospace; font-size: 12px; font-weight: bold; color: #1e3a5f;">glide.discovery.alter_esx_server_index</td><td style="padding: 9px 12px; border-bottom: 1px solid #e5e7eb; font-family: monospace; font-size: 12px;">false</td><td style="padding: 9px 12px; border-bottom: 1px solid #e5e7eb; font-size: 13px;">When <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">true</code>: removes <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">correlation_id</code> from the ESX identification index; uses <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">serial_number</code> only, and adds an FQDN identifier on <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">cmdb_ci_server</code>. <strong>Set true when you see duplicate ESX CIs caused by correlation_id changes</strong> (ESXi reinstalls, vCenter migrations).</td></tr><tr><td style="padding: 9px 12px; border-bottom: 1px solid #e5e7eb; font-family: monospace; font-size: 12px; font-weight: bold; color: #1e3a5f;">glide.discovery.mark_esx_servers_as_absent</td><td style="padding: 9px 12px; border-bottom: 1px solid #e5e7eb; font-family: monospace; font-size: 12px;">false</td><td style="padding: 9px 12px; border-bottom: 1px solid #e5e7eb; font-size: 13px;">When <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">false</code> (default): ESX hosts not found in the vCenter payload are set to <em>Retired</em>. When <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">true</code>: they are set to <em>Absent</em> instead. <strong>Set true</strong> if you prefer Absent over Retired for temporarily disconnected ESX hosts — Absent is less destructive and easier to re-activate.</td></tr><tr style="background: #f9fafb;"><td style="padding: 9px 12px; border-bottom: 1px solid #e5e7eb; font-family: monospace; font-size: 12px; font-weight: bold; color: #1e3a5f;">glide.discovery.esx_server_fields</td><td style="padding: 9px 12px; border-bottom: 1px solid #e5e7eb; font-family: monospace; font-size: 12px;">(empty)</td><td style="padding: 9px 12px; border-bottom: 1px solid #e5e7eb; font-size: 13px;">Comma-separated list of ESX server CI field names that Discovery will <strong>skip overwriting</strong>. Use when another discovery source (SCCM, SAM, manual entry) populates certain fields and you want Discovery to leave them alone. Example: <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">serial_number,model_number</code></td></tr><tr><td style="padding: 9px 12px; border-bottom: 1px solid #e5e7eb; font-family: monospace; font-size: 12px; font-weight: bold; color: #1e3a5f;">mid.discovery.max_payload_size</td><td style="padding: 9px 12px; border-bottom: 1px solid #e5e7eb; font-family: monospace; font-size: 12px;">5000000</td><td style="padding: 9px 12px; border-bottom: 1px solid #e5e7eb; font-size: 13px;">Maximum size in bytes of a probe response payload the MID Server will process. VMware environments with thousands of VMs frequently exceed 5 MB. <strong>Increase to 10485760 (10 MB)</strong> for large vCenter environments. Requires MID Server restart.</td></tr><tr style="background: #f9fafb;"><td style="padding: 9px 12px; font-family: monospace; font-size: 12px; font-weight: bold; color: #1e3a5f;">com.glide.attachment.max_get_size</td><td style="padding: 9px 12px; font-family: monospace; font-size: 12px;">5242880</td><td style="padding: 9px 12px; font-size: 13px;">Maximum attachment size (in bytes) the ServiceNow instance will accept from the MID Server. Must be set equal to or higher than <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">mid.discovery.max_payload_size</code>. <strong>Set to 10485760 together with the property above</strong> when dealing with payload size errors.</td></tr></tbody></table> </div> </div> <div id="section-probe" style="margin-bottom: 28px;"> <div style="background: #005baa; color: #fff; padding: 12px 20px; border-radius: 8px 8px 0 0; font-size: 16px; font-weight: bold;">6. Probe/Sensor Chain Deep Dive & Step-by-Step Debugging</div> <div style="background: #fff; border: 1px solid #d1d5db; border-top: none; border-radius: 0 0 8px 8px; padding: 20px 22px;"> <div style="border-left: 4px solid #005baa; background: #e8f1fb; padding: 10px 14px; border-radius: 0 6px 6px 0; margin-bottom: 16px; font-size: 13px;"><strong style="display: block; font-size: 12px; text-transform: uppercase; letter-spacing: 0.5px; color: #005baa; margin-bottom: 4px;">Architecture note: Probes vs Patterns</strong> Unlike Switch/Router Discovery which uses the newer Pattern engine (<code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">sa_pattern</code>), VMware Discovery uses the legacy <strong>Probe/Sensor architecture</strong>. Probes run on the MID Server; Sensors run on the ServiceNow instance. Probe output is queued in the ECC Queue; the instance processes it via sensor Script Includes. To debug: check <strong>Discovery → Discovery Log → [run] → ECC Queue</strong> for probe output and error messages.</div> <div style="background: #f8fafc; border: 1px solid #e2e8f0; border-radius: 6px; padding: 4px 0; margin-bottom: 16px;"> <div style="padding: 8px 16px; font-size: 12px; font-weight: bold; text-transform: uppercase; letter-spacing: 0.5px; color: #64748b; border-bottom: 1px solid #e2e8f0;">Group A: Scanning & Classification (Phases 1–2)</div> <details style="border-bottom: 1px solid #e2e8f0;"> <summary style="padding: 12px 16px; cursor: pointer; font-weight: bold; font-size: 13px; color: #1e3a5f; list-style: none;">▶ <span style="background: #005baa; color: #fff; padding: 1px 8px; border-radius: 10px; font-size: 11px; font-family: monospace; margin-right: 6px;">Step A1</span> Shazzam probe + port probe scan vCenter IP</summary> <div style="padding: 14px 16px; font-size: 13px; background: #fff;"> <p style="margin: 0 0 8px;"><strong>What it does:</strong> Shazzam confirms the host is up (ICMP). The port probe then connects to TCP ports 443, 5480, and 9443. Successful connections on port 5480 (<code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">vmapp_https</code>) or 9443 (<code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">vmapp6_https</code>) trigger the vCenter classification path.</p> <p style="margin: 0 0 8px;"><strong>Requirements:</strong> MID Server must reach the vCenter IP on TCP 443. Ports 5480/9443 are required for the port-probe classification method; port 443 alone is sufficient if using process classifier.</p> <p style="margin: 0;"><strong>Failure signs:</strong> No ECC Queue entries for this vCenter IP at all. Test connectivity: <code style="background: #f3f4f6; padding: 2px 5px; border-radius: 3px;">Test-NetConnection -ComputerName <vCenter_IP> -Port 443</code> from the MID Server host.</p> </div> </details> <details> <summary style="padding: 12px 16px; cursor: pointer; font-weight: bold; font-size: 13px; color: #1e3a5f; list-style: none;">▶ <span style="background: #005baa; color: #fff; padding: 1px 8px; border-radius: 10px; font-size: 11px; font-family: monospace; margin-right: 6px;">Step A2</span> vCenter Classification → cmdb_ci_vcenter CI type assigned</summary> <div style="padding: 14px 16px; font-size: 13px; background: #fff;"> <p style="margin: 0 0 8px;"><strong>What it does:</strong> The SNMP Classification is not used for VMware. Instead, a port classifier rule matches the <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">vmapp_https</code> or <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">vmapp6_https</code> port response to the CI type <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">cmdb_ci_vcenter</code>. This triggers IRE (Phase 3) to create the vCenter CI and then launches the exploration probe chain.</p> <p style="margin: 0 0 8px;"><strong>Failure signs:</strong> vCenter classified as <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">cmdb_ci_linux_server</code> or <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">cmdb_ci_computer</code> instead of <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">cmdb_ci_vcenter</code>. Cause: SSH credential has a lower Order value and fires first, classifying the VCSA as a Linux server before the VMware credential fires.</p> <p style="margin: 0;"><strong>Fix:</strong> Use IP Service Affinity to explicitly map the vCenter IP to VMware credential type. Delete the incorrectly classified CI and re-run Discovery.</p> </div> </details> </div> <div style="background: #f8fafc; border: 1px solid #e2e8f0; border-radius: 6px; padding: 4px 0; margin-bottom: 16px;"> <div style="padding: 8px 16px; font-size: 12px; font-weight: bold; text-transform: uppercase; letter-spacing: 0.5px; color: #64748b; border-bottom: 1px solid #e2e8f0;">Group B: vCenter Inventory Retrieval (Phase 4 Entry)</div> <details style="border-bottom: 1px solid #e2e8f0;"> <summary style="padding: 12px 16px; cursor: pointer; font-weight: bold; font-size: 13px; color: #1e3a5f; list-style: none;">▶ <span style="background: #0369a1; color: #fff; padding: 1px 8px; border-radius: 10px; font-size: 11px; font-family: monospace; margin-right: 6px;">Step B1</span> VMware — vCenter Datacenters probe</summary> <div style="padding: 14px 16px; font-size: 13px; background: #fff;"> <p style="margin: 0 0 8px;"><strong>What it does:</strong> The entry-point probe for vCenter exploration. Authenticates to the vCenter SOAP API on port 443 using the VMware credential. Calls <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">RetrievePropertiesEx</code> on the vCenter's ManagedObjectReference (type: Datacenter) to retrieve the full list of datacenters. Each datacenter becomes a child probe invocation for hosts and VMs.</p> <p style="margin: 0 0 8px;"><strong>VMware API call:</strong> <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">ServiceInstance → PropertyCollector.RetrievePropertiesEx</code> on Datacenter objects</p> <p style="margin: 0 0 8px;"><strong>CI created:</strong> <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">cmdb_ci_vmware_datacenter</code> for each vSphere datacenter</p> <p style="margin: 0;"><strong>Failure signs:</strong> No ESX or VM CIs created; ECC Queue has no VMWarevCenterVMsProbe entries. Check authentication — the probe fires but SOAP auth failed silently. Review ECC Queue error field for the Datacenters probe entry.</p> </div> </details> <details> <summary style="padding: 12px 16px; cursor: pointer; font-weight: bold; font-size: 13px; color: #1e3a5f; list-style: none;">▶ <span style="background: #0369a1; color: #fff; padding: 1px 8px; border-radius: 10px; font-size: 11px; font-family: monospace; margin-right: 6px;">Step B2</span> VMWare — vCenter Clusters probe + VCenterClustersSensor</summary> <div style="padding: 14px 16px; font-size: 13px; background: #fff;"> <p style="margin: 0 0 8px;"><strong>What it does:</strong> Retrieves all ClusterComputeResource objects from each datacenter. For each cluster: name, DRS/HA configuration, resource pool structure, and the list of host MoRefs assigned to the cluster. <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">VCenterClustersSensor</code> creates <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">cmdb_ci_vcenter_cluster</code> CIs and establishes "Runs on::Runs" relationships to ESX server CIs (after ESX CIs are created).</p> <p style="margin: 0 0 8px;"><strong>VMware API objects:</strong> <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">ClusterComputeResource</code></p> <p style="margin: 0;"><strong>Non-blocking:</strong> Clusters are optional in some vSphere configurations. Standalone ESX hosts not in a cluster will not appear in this step but are still discovered in Step C1.</p> </div> </details> </div> <div style="background: #f8fafc; border: 1px solid #e2e8f0; border-radius: 6px; padding: 4px 0; margin-bottom: 16px;"> <div style="padding: 8px 16px; font-size: 12px; font-weight: bold; text-transform: uppercase; letter-spacing: 0.5px; color: #64748b; border-bottom: 1px solid #e2e8f0;">Group C: ESX Host Discovery & Identification</div> <details style="border-bottom: 1px solid #e2e8f0;"> <summary style="padding: 12px 16px; cursor: pointer; font-weight: bold; font-size: 13px; color: #1e3a5f; list-style: none;">▶ <span style="background: #0e7490; color: #fff; padding: 1px 8px; border-radius: 10px; font-size: 11px; font-family: monospace; margin-right: 6px;">Step C1</span> VMWare — vCenter Hosts probe</summary> <div style="padding: 14px 16px; font-size: 13px; background: #fff;"> <p style="margin: 0 0 8px;"><strong>What it does:</strong> For each datacenter/cluster, calls <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">RetrievePropertiesEx</code> on HostSystem managed objects. Retrieves the full set of ESX host attributes: hardware info, configuration, runtime state, and network config. This is the data that feeds into ESX CI creation.</p> <p style="margin: 0 0 8px;"><strong>VMware API properties retrieved per HostSystem:</strong></p> <ul style="margin: 0 0 8px; padding-left: 20px; font-family: monospace; font-size: 11px; line-height: 1.8;"><li>hardware.systemInfo.uuid (maps to vcenter_uuid / correlation_id)</li><li>hardware.systemInfo.serialNumber (maps to serial_number)</li><li>hardware.systemInfo.model (maps to model_number)</li><li>hardware.systemInfo.vendor (maps to manufacturer)</li><li>hardware.cpuInfo.numCpuCores (maps to cpu_core_count)</li><li>hardware.cpuInfo.numCpuThreads (maps to logical_processor)</li><li>hardware.memorySize (maps to ram)</li><li>summary.config.name (maps to host_name)</li><li>summary.runtime.connectionState (maps to connection_state)</li><li>summary.runtime.powerState (maps to power_state)</li><li>_moref.value (maps to morid)</li></ul> <p style="margin: 0;"><strong>Failure signs:</strong> No ESX host data in ECC Queue output. Verify the VMware credential account has Read-Only on the Hosts & Clusters container with Propagate to Children.</p> </div> </details> <details> <summary style="padding: 12px 16px; cursor: pointer; font-weight: bold; font-size: 13px; color: #1e3a5f; list-style: none;">▶ <span style="background: #0e7490; color: #fff; padding: 1px 8px; border-radius: 10px; font-size: 11px; font-family: monospace; margin-right: 6px;">Step C2</span> VCenterESXHostsSensor — create/update cmdb_ci_esx_server CIs</summary> <div style="padding: 14px 16px; font-size: 13px; background: #fff;"> <div style="border-left: 4px solid #b45309; background: #fffbeb; padding: 8px 12px; border-radius: 0 6px 6px 0; margin-bottom: 10px; font-size: 12px;"><strong>Critical:</strong> ESX identification does NOT go through IRE. This sensor implements its own 3-key identification: (1) hardware UUID / correlation_id, (2) serial_number, (3) FQDN. If all three fail to match an existing CI, a new CI is created — the source of duplicate ESX records.</div> <p style="margin: 0 0 8px;"><strong>What it does:</strong> Processes the probe output from Step C1. For each ESX host: runs the 3-index identification, creates or updates the <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">cmdb_ci_esx_server</code> CI, populates all fields from the VMware API payload, then calls <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">fixVirtualizes</code> to create the "Virtualized by::Virtualizes" relationship to Server CIs (if they already exist from OS discovery).</p> <p style="margin: 0 0 8px;"><strong>CI fields set:</strong> <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">name, serial_number, model_number, manufacturer, cpu_core_count, logical_processor, ram, host_name, connection_state, power_state, morid, vcenter_ref, vcenter_uuid, cluster_id, hyper_threading, object_id, last_discovered, discovery_source</code></p> <p style="margin: 0;"><strong>Failure signs:</strong> ESX CI missing expected fields. Confirm Script Include <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">VCenterESXHostsSensor</code> is Active in System Definition → Script Includes.</p> </div> </details> </div> <div style="background: #f8fafc; border: 1px solid #e2e8f0; border-radius: 6px; padding: 4px 0; margin-bottom: 16px;"> <div style="padding: 8px 16px; font-size: 12px; font-weight: bold; text-transform: uppercase; letter-spacing: 0.5px; color: #64748b; border-bottom: 1px solid #e2e8f0;">Group D: VM Instance Discovery & Relationship Creation</div> <details style="border-bottom: 1px solid #e2e8f0;"> <summary style="padding: 12px 16px; cursor: pointer; font-weight: bold; font-size: 13px; color: #1e3a5f; list-style: none;">▶ <span style="background: #047857; color: #fff; padding: 1px 8px; border-radius: 10px; font-size: 11px; font-family: monospace; margin-right: 6px;">Step D1</span> VMWarevCenterVMsProbe — retrieve all VM objects</summary> <div style="padding: 14px 16px; font-size: 13px; background: #fff;"> <p style="margin: 0 0 8px;"><strong>What it does:</strong> Retrieves all VirtualMachine managed objects from the vCenter inventory. This probe typically appears multiple times in the ECC Queue (one entry per datacenter or batch). Each probe result contains the full property set for every VM in that datacenter scope.</p> <p style="margin: 0 0 8px;"><strong>VMware API properties retrieved per VirtualMachine:</strong></p> <ul style="margin: 0 0 8px; padding-left: 20px; font-family: monospace; font-size: 11px; line-height: 1.8;"><li>config.instanceUuid (maps to vm_instance_uuid — primary identifier)</li><li>config.uuid (maps to bios_uuid)</li><li>config.guestFullName (maps to guest_os_fullname)</li><li>config.guestId (maps to guest_id)</li><li>config.template (maps to template boolean)</li><li>config.name (maps to name)</li><li>config.files.vmPathName (maps to image_path)</li><li>summary.runtime.powerState (maps to state)</li><li>summary.config.memorySizeMB (maps to memory)</li><li>summary.config.numCpu (maps to cpus)</li><li>runtime.host (MoRef of ESX host — used for parent relationship)</li></ul> <p style="margin: 0;"><strong>Failure signs:</strong> Multiple ECC Queue entries with error state. Check the error field — most common: payload too large (increase max_payload_size) or SOAP authentication expired mid-run.</p> </div> </details> <details style="border-bottom: 1px solid #e2e8f0;"> <summary style="padding: 12px 16px; cursor: pointer; font-weight: bold; font-size: 13px; color: #1e3a5f; list-style: none;">▶ <span style="background: #047857; color: #fff; padding: 1px 8px; border-radius: 10px; font-size: 11px; font-family: monospace; margin-right: 6px;">Step D2</span> VCenterVMsSensor → VmwareVmCorrelator — create VM Instance CIs</summary> <div style="padding: 14px 16px; font-size: 13px; background: #fff;"> <p style="margin: 0 0 8px;"><strong>What it does:</strong> <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">VCenterVMsSensor</code> processes each VirtualMachine payload. For each VM: calls <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">VmwareVmCorrelator.processVmInstanceOrTemplate</code> which creates or updates the <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">cmdb_ci_vmware_instance</code> CI (or <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">cmdb_ci_vmware_template</code> if <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">template=true</code>).</p> <p style="margin: 0 0 8px;"><strong>CI fields set on cmdb_ci_vmware_instance:</strong> <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">name, vm_instance_uuid, bios_uuid, vcenter_uuid, vcenter_ref, state, guest_id, guest_os_fullname, template, memory, cpus, image_path, host_id, cluster_id, last_discovered</code></p> <p style="margin: 0;"><strong>Failure signs:</strong> <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">cmdb_ci_vmware_instance</code> records not created. Confirm <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">VmwareVmCorrelator</code> Script Include is Active.</p> </div> </details> <details> <summary style="padding: 12px 16px; cursor: pointer; font-weight: bold; font-size: 13px; color: #1e3a5f; list-style: none;">▶ <span style="background: #047857; color: #fff; padding: 1px 8px; border-radius: 10px; font-size: 11px; font-family: monospace; margin-right: 6px;">Step D3</span> VmwareVmCorrelator — create "Instantiates" relationship + set Guest Reconciled</summary> <div style="padding: 14px 16px; font-size: 13px; background: #fff;"> <div style="border-left: 4px solid #dc2626; background: #fef2f2; padding: 8px 12px; border-radius: 0 6px 6px 0; margin-bottom: 10px; font-size: 12px;"><strong>Prerequisite for Step C2 fixVirtualizes.</strong> If this step doesn't complete successfully, no "Virtualized by" relationship can be created between ESX and Server CIs.</div> <p style="margin: 0 0 8px;"><strong>What it does:</strong> After creating the VMware Instance CI, <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">VmwareVmCorrelator</code> searches for an existing Server CI (<code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">cmdb_ci_server</code>) whose IP address matches the VM's guest IP (from VMware Tools). If found: creates the "Instantiates::Instantiated by" relationship between the VMware Instance and the Server CI, and sets <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">Guest Reconciled = true</code> on the VMware Instance and <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">Is Virtual = true</code> on the Server CI.</p> <p style="margin: 0 0 8px;"><strong>Requirement:</strong> VMware Tools must be installed and running on the VM guest for the VM's IP to be available in the VMware API. Without VMware Tools, IP is null and no Server CI match is attempted.</p> <p style="margin: 0;"><strong>Failure signs:</strong> <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">Guest Reconciled = false</code> on VMware Instance CIs. No "Virtualized by" relationships. Fix: ensure VMware Tools is installed on VMs and OS-level Discovery has run on the VM IPs.</p> </div> </details> </div> <div style="background: #f8fafc; border: 1px solid #e2e8f0; border-radius: 6px; padding: 4px 0; margin-bottom: 16px;"> <div style="padding: 8px 16px; font-size: 12px; font-weight: bold; text-transform: uppercase; letter-spacing: 0.5px; color: #64748b; border-bottom: 1px solid #e2e8f0;">Group E: Infrastructure CIs (Networks, Datastores, Folders)</div> <details style="border-bottom: 1px solid #e2e8f0;"> <summary style="padding: 12px 16px; cursor: pointer; font-weight: bold; font-size: 13px; color: #1e3a5f; list-style: none;">▶ <span style="background: #7c3aed; color: #fff; padding: 1px 8px; border-radius: 10px; font-size: 11px; font-family: monospace; margin-right: 6px;">Step E1</span> VCenterNetworkSensor — vSphere networks and port groups</summary> <div style="padding: 14px 16px; font-size: 13px; background: #fff;"> <p style="margin: 0 0 8px;"><strong>What it does:</strong> Retrieves all Network managed objects (standard vSwitches and distributed port groups). Creates <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">cmdb_ci_vcenter_network</code> CIs. These represent the logical network constructs that VMs connect to. Links each network CI to the parent datacenter and to each VM that uses it.</p> <p style="margin: 0;"><strong>Non-blocking:</strong> If this sensor has errors, ESX and VM CIs are still created. Network relationship data may be incomplete but core discovery continues.</p> </div> </details> <details style="border-bottom: 1px solid #e2e8f0;"> <summary style="padding: 12px 16px; cursor: pointer; font-weight: bold; font-size: 13px; color: #1e3a5f; list-style: none;">▶ <span style="background: #7c3aed; color: #fff; padding: 1px 8px; border-radius: 10px; font-size: 11px; font-family: monospace; margin-right: 6px;">Step E2</span> VCenterDatastoresSensor — vSphere datastores</summary> <div style="padding: 14px 16px; font-size: 13px; background: #fff;"> <p style="margin: 0 0 8px;"><strong>What it does:</strong> Retrieves all Datastore managed objects. Creates <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">cmdb_ci_vcenter_datastore</code> CIs representing storage volumes (VMFS, NFS). Populates capacity, free space, and type. Links datastores to the ESX hosts that mount them.</p> <p style="margin: 0;"><strong>Non-blocking:</strong> Datastore data may be missing for restricted accounts that lack permission on the storage layer.</p> </div> </details> <details> <summary style="padding: 12px 16px; cursor: pointer; font-weight: bold; font-size: 13px; color: #1e3a5f; list-style: none;">▶ <span style="background: #7c3aed; color: #fff; padding: 1px 8px; border-radius: 10px; font-size: 11px; font-family: monospace; margin-right: 6px;">Step E3</span> VCenterFoldersSensor — vSphere folder hierarchy</summary> <div style="padding: 14px 16px; font-size: 13px; background: #fff;"> <p style="margin: 0 0 8px;"><strong>What it does:</strong> Retrieves the vSphere folder tree structure. Creates <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">cmdb_ci_vcenter_folder</code> CIs. Primarily used for Service Mapping topology and for organizing VM CIs in a visual representation of the vSphere inventory. Has no impact on ESX host or VM CI creation.</p> <p style="margin: 0;"><strong>Non-blocking:</strong> Missing folder CIs do not affect ESX or VM discovery completeness.</p> </div> </details> </div> <div style="border: 1px solid #d1d5db; border-radius: 6px; padding: 16px; background: #f8fafc; margin-top: 8px;"> <p style="margin: 0 0 10px; font-weight: bold; font-size: 13px; color: #1e3a5f;">Expected: "Dual CI" architecture for VMs (not a duplicate)</p> <p style="margin: 0 0 10px; font-size: 13px;">When both vCenter Discovery AND IP-based OS Discovery run for the same VM, two CIs are created intentionally:</p> <table style="width: 100%; border-collapse: collapse; font-size: 13px;"><tbody><tr style="background: #e8f1fb;"><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb; font-weight: bold; width: 40%; font-family: monospace; font-size: 12px;">cmdb_ci_vmware_instance</td><td style="padding: 8px 12px; border-bottom: 1px solid #e5e7eb;">Created by vCenter Discovery. Contains vCenter metadata: CPU count, memory, image_path, guest_os_fullname, template flag, vcenter_ref, vcenter_uuid.</td></tr><tr><td style="padding: 8px 12px; font-weight: bold; font-family: monospace; font-size: 12px;">cmdb_ci_linux/win_server</td><td style="padding: 8px 12px;">Created by OS Discovery. Contains OS-level data: processes, services, users, software installs, disk layout.</td></tr></tbody></table> <p style="margin: 10px 0 0; font-size: 13px;">These two CIs are linked via the <strong>"Instantiates::Instantiated by"</strong> relationship. This is the intended CMDB model, not a duplicate. Do not merge or delete either CI without understanding which data each carries.</p> </div> </div> </div> <div id="section-known" style="margin-bottom: 28px;"> <div style="background: #005baa; color: #fff; padding: 12px 20px; border-radius: 8px 8px 0 0; font-size: 16px; font-weight: bold;">7. Known Issues & Limitations</div> <div style="background: #fff; border: 1px solid #d1d5db; border-top: none; border-radius: 0 0 8px 8px; padding: 20px 22px;"> <details style="margin-bottom: 12px; border: 1px solid #fca5a5; border-radius: 6px;"> <summary style="background: #fef2f2; border-left: 4px solid #dc2626; padding: 12px 16px; border-radius: 0 6px 6px 0; cursor: pointer; font-weight: bold; font-size: 14px; color: #991b1b; list-style: none;">▶ KB0830586 — Duplicate ESX server records when correlation_id changes</summary> <div style="padding: 16px; font-size: 13px;"> <table style="width: 100%; border-collapse: collapse; font-size: 13px;"><tbody><tr><td style="padding: 6px 10px; border-bottom: 1px solid #f3f4f6; font-weight: 600; width: 140px;">Root cause</td><td style="padding: 6px 10px; border-bottom: 1px solid #f3f4f6;">ESX identification uses VCenterESXHostsSensor (not IRE). When the <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">correlation_id</code> changes (ESXi reinstall, vCenter re-registration, vCenter migration), none of the 3 identification indexes match the existing CI and a new CI is created.</td></tr><tr style="background: #f9fafb;"><td style="padding: 6px 10px; border-bottom: 1px solid #f3f4f6; font-weight: 600;">Symptom</td><td style="padding: 6px 10px; border-bottom: 1px solid #f3f4f6;">Multiple <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">cmdb_ci_esx_server</code> records for the same physical host with different correlation_id values.</td></tr><tr><td style="padding: 6px 10px; border-bottom: 1px solid #f3f4f6; font-weight: 600;">Workaround</td><td style="padding: 6px 10px; border-bottom: 1px solid #f3f4f6;">Set <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">glide.discovery.alter_esx_server_index=true</code> to switch to serial_number-based identification. Delete the stale duplicate CI. Also see KB1123141 for identification behavior details.</td></tr></tbody></table> </div> </details> <details style="margin-bottom: 12px; border: 1px solid #fca5a5; border-radius: 6px;"> <summary style="background: #fef2f2; border-left: 4px solid #dc2626; padding: 12px 16px; border-radius: 0 6px 6px 0; cursor: pointer; font-weight: bold; font-size: 14px; color: #991b1b; list-style: none;">▶ KB0750033 — IP address not populated on ESX server CIs after vCenter Discovery</summary> <div style="padding: 16px; font-size: 13px;"> <table style="width: 100%; border-collapse: collapse; font-size: 13px;"><tbody><tr><td style="padding: 6px 10px; border-bottom: 1px solid #f3f4f6; font-weight: 600; width: 140px;">Symptom</td><td style="padding: 6px 10px; border-bottom: 1px solid #f3f4f6;">vCenter Discovery runs successfully and creates all ESX CIs, but the <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">ip_address</code> field is blank on <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">cmdb_ci_esx_server</code> records.</td></tr><tr style="background: #f9fafb;"><td style="padding: 6px 10px; border-bottom: 1px solid #f3f4f6; font-weight: 600;">Impact</td><td style="padding: 6px 10px; border-bottom: 1px solid #f3f4f6;">No IP address on ESX CIs prevents OS-level Discovery from correlating a scan of the ESX management IP back to the correct <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">cmdb_ci_esx_server</code> record.</td></tr><tr><td style="padding: 6px 10px; font-weight: 600;">Workaround</td><td style="padding: 6px 10px;">Run a lightweight SNMP scan targeting ESX host IPs to populate <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">ip_address</code> without replacing other fields. Or populate manually and set the field to not be overwritten via <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">glide.discovery.esx_server_fields</code>.</td></tr></tbody></table> </div> </details> <details style="margin-bottom: 12px; border: 1px solid #fca5a5; border-radius: 6px;"> <summary style="background: #fef2f2; border-left: 4px solid #dc2626; padding: 12px 16px; border-radius: 0 6px 6px 0; cursor: pointer; font-weight: bold; font-size: 14px; color: #991b1b; list-style: none;">▶ DEF0752263 — VM Tags probe fails SSL certificate SAN validation</summary> <div style="padding: 16px; font-size: 13px;"> <table style="width: 100%; border-collapse: collapse; font-size: 13px;"><tbody><tr><td style="padding: 6px 10px; border-bottom: 1px solid #f3f4f6; font-weight: 600; width: 140px;">State</td><td style="padding: 6px 10px; border-bottom: 1px solid #f3f4f6;">Work in Progress (active defect)</td></tr><tr style="background: #f9fafb;"><td style="padding: 6px 10px; border-bottom: 1px solid #f3f4f6; font-weight: 600;">Symptom</td><td style="padding: 6px 10px; border-bottom: 1px solid #f3f4f6;">VM Tags not discovered. SSL certificate validation error in ECC Queue for the VM Tags probe when Discovery targets vCenter by IP (not FQDN).</td></tr><tr><td style="padding: 6px 10px; border-bottom: 1px solid #f3f4f6; font-weight: 600;">Root cause</td><td style="padding: 6px 10px; border-bottom: 1px solid #f3f4f6;">The VM Tags probe enforces SAN validation using the IP address. If the vCenter certificate's SAN does not list the IP (only the FQDN), the TLS handshake is rejected.</td></tr><tr style="background: #f9fafb;"><td style="padding: 6px 10px; font-weight: 600;">Workaround</td><td style="padding: 6px 10px;">Use vCenter FQDN in the Discovery schedule IP range instead of the IP. Alternatively, reissue the vCenter certificate to include an IP SAN entry.</td></tr></tbody></table> </div> </details> <details style="margin-bottom: 12px; border: 1px solid #fcd34d; border-radius: 6px;"> <summary style="background: #fffbeb; border-left: 4px solid #b45309; padding: 12px 16px; border-radius: 0 6px 6px 0; cursor: pointer; font-weight: bold; font-size: 14px; color: #92400e; list-style: none;">▶ DEF0601391 / DEF0601714 — Missing relations from ESXi discovery</summary> <div style="padding: 16px; font-size: 13px;"> <p style="margin: 0 0 8px;"><strong>State:</strong> Working as Expected (behavior confirmed, no code change planned)</p> <p style="margin: 0 0 8px;">DEF0601391: ESXi server CIs are created but missing certain parent/child relationships. DEF0601714: Some relations between vCenter CI and ESX server CI not populated.</p> <p style="margin: 0 0 8px;"><strong>Root cause:</strong> These relationships depend on the timing of the probe chain. If the datacenter/cluster CIs have not been fully committed when the ESX sensor runs, the foreign key references resolve to null and the relationships are not created on that run. They are typically present after the second Discovery run.</p> <p style="margin: 0;"><strong>Workaround:</strong> Run vCenter Discovery twice in sequence. The second run has all parent CIs already committed and creates all missing relationships. Configure a recurring Discovery schedule to ensure the relationship data stays current.</p> </div> </details> <details style="margin-bottom: 12px; border: 1px solid #fcd34d; border-radius: 6px;"> <summary style="background: #fffbeb; border-left: 4px solid #b45309; padding: 12px 16px; border-radius: 0 6px 6px 0; cursor: pointer; font-weight: bold; font-size: 14px; color: #92400e; list-style: none;">▶ DEF0821588 — ESX OS Software Install records: display_name hardcoded, stale versions not deactivated</summary> <div style="padding: 16px; font-size: 13px;"> <p style="margin: 0 0 8px;"><strong>State:</strong> Fixed</p> <p style="margin: 0 0 8px;">Software Installed records on ESX server CIs have a hardcoded <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">display_name</code>, the <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">last_scanned</code> field is not updated on Discovery runs, and stale version records are not marked inactive after an ESX upgrade.</p> <p style="margin: 0;"><strong>Fix:</strong> Apply the fix version referenced in DEF0821588. Until patched, stale software install records may persist after ESX host upgrades — manually deactivate outdated records if SAM or ITAM workflows depend on accurate ESX software inventory.</p> </div> </details> <details style="border: 1px solid #d1d5db; border-radius: 6px;"> <summary style="background: #f0f4f9; border-left: 4px solid #005baa; padding: 12px 16px; border-radius: 0 6px 6px 0; cursor: pointer; font-weight: bold; font-size: 14px; color: #003d75; list-style: none;">▶ VM Tag discovery requires vCenter 6.0+ and specific permissions</summary> <div style="padding: 16px; font-size: 13px;"> <p style="margin: 0 0 8px;">VM Tags are populated via the vSphere REST API (not the SOAP API). This API was introduced in vSphere 6.0. VMs managed by vCenter 5.x will have no tag data populated.</p> <p style="margin: 0 0 8px;"><strong>Permission required:</strong> The Discovery account needs READ permission at the vCenter level (not just at resource level) for the tagging namespace. The Read-Only role at root with Propagate to Children covers this in most vCenter configurations, but custom roles that exclude the <code style="background: #f3f4f6; padding: 1px 4px; border-radius: 3px;">com.vmware.content.library.vm.tag.read</code> privilege will silently omit tags.</p> <p style="margin: 0;">Also see DEF0752263 above for the SSL certificate issue that can prevent tag discovery regardless of permissions.</p> </div> </details> </div> </div> <div style="background: #f8fafc; border: 1px solid #e2e8f0; border-radius: 8px; padding: 16px 20px; margin-top: 4px; font-size: 12px; color: #64748b;"><strong style="color: #374151;">ServiceNow Discovery — VMware ESX / vCenter Pattern</strong> | Zurich / Xanadu | cmdb_ci_esx_server | June 2026<br /><span style="margin-top: 4px; display: block;">For questions or corrections, use the feedback button on this article. Related guides: <em>Network Switch Discovery</em> — <em>Network Router Discovery</em>.</span></div> </div> <span id="ns-kb-css-end-div-identifier" style="display: none; pointer-events: none;"></span></div></section></div></article></div>