RPM/DEB in-product upgrade permanently wedges agent after first upgrade — DestinationFileExistsError on cert extract due to missing $upgradeDir cleanup - Known Error

RPM/DEB in-product upgrade permanently wedges agent after first upgrade — DestinationFileExistsError on cert extract due to missing $upgradeDir cleanup Description Any RPM (or DEB) agent that has previously been upgraded in-product will fail every subsequent in-product upgrade attempt with:

Zip::DestinationFileExistsError: Destination '/var/cache/servicenow/agent-client-collector/upgrade/ServiceNow_Digicert_DGST.pem' already exists

Root cause is three compounding bugs in the upgrade pipeline:

1. unzip_file_linux "replace" branch does not replace (common_utils.rb ~line 759). When replaceExisting=true (the default used by runUpgrade.rb), zip_file.extract(entry, dest_path) is called without an overwrite block. rubyzip's on_exists_proc defaults to false, so entry.rb raises DestinationFileExistsError instead of overwriting the existing file.

2. $upgradeDir is never cleaned on the success path (runUpgrade.rb ~line 188-196). After a successful upgrade, the script spawns a detached systemd unit and calls exit(0) with no cleanup. The .pem and .bin verification files remain on disk in $upgradeDir.

3. The DestinationFileExistsError exception is unhandled at the call site (runUpgrade.rb ~line 101). It bypasses cannotUpgrade (the only place that calls FileUtils.rm_rf($upgradeDir)), so the directory is never self-healed and the agent is permanently wedged on every retry until manual intervention.

Net effect: the first successful in-product upgrade leaves $upgradeDir populated. Every subsequent in-product upgrade dies deterministically at the signature-extract step. The same pattern exists in the DEB upgrade path.

Steps to Reproduce

1. Install ACC RPM agent version 6.1.0 on a Linux host. 2. Trigger an in-product upgrade to 6.2.x or 6.3.0 via the ServiceNow instance UI. Upgrade succeeds. 3. Inspect /var/cache/servicenow/agent-client-collector/upgrade/ — note .pem and .bin files are left behind. 4. Trigger another in-product upgrade (e.g. 6.2.x → 6.3.0). 5. Upgrade fails immediately with: Zip::DestinationFileExistsError: Destination '.../upgrade/ServiceNow_Digicert_DGST.pem' already exists 6. $upgradeDir is not cleaned because the exception is unhandled and does not route through cannotUpgrade. 7. All further in-product upgrade retries fail identically until the operator manually removes the upgrade directory.

Reproduced on: Linux RPM agents (x86_64). Same pattern present in DEB path.

Workaround Primary (recommended) — Direct RPM/DEB upgrade, bypasses the failing unzip step:

sudo systemctl stop acc sudo rm -rf /var/cache/servicenow/agent-client-collector/upgrade sudo rpm -Uvh /path/to/agent-client-collector- -x86_64.rpm sudo systemctl start acc (For DEB: replace rpm -Uvh with dpkg -i agent-client-collector- _amd64.deb)

Alternative — In-product retry after manual cleanup:

sudo systemctl stop acc sudo rm -rf /var/cache/servicenow/agent-client-collector/upgrade sudo systemctl start acc

Then trigger "Upgrade Agent" from the instance. Note: the rm -rf step must be repeated before each future in-product upgrade attempt until the fix is released, as the success path re-populates the directory.

For large fleets, recommend batching the rollout in groups of ~50 agents to avoid simultaneous reconnection storms on the instance.