ACC: how to debug "is not recognized as an internal or external command, operable program or batch file." errorsSummaryWhen a Agent Client Collector runs a Check, it is actually executing a Ruby script from the cache folder. To get that script into the cache folder first, many steps are involved, and various things can go wrong. This KB hopes to explain known causes, and explain how to investigate these to confirm those or figure out others. Instance/MID Server side: A Check is an execution by an ACC of a Check Definitione.g. Enhanced DiscoveryThe Check Definition record defines the command that is to be run, which is usually a Ruby script (.rb)e.g. endpoint_discovery.rb --compact --select=basic_inventory,installed_software,file_systems,serial_numbers,network_adapters,tcp_connections,storage_devices,running_processesIt also lists the Plugins needed to be able to run that command. These Plugins, also known as Sensu Assets, contain the scripts and dependencies.e.g. acc-f-commonsThe "Plugin Name" record has a related list of 1 or more OS-Specific "Plugin" recordsEach of those Plugin records will have an attachment, all with the same <Plugin Name>.tar.gz name, but with contents specific to the OS. If signed, the tar.gz is included with a .sig file and wrapped within another tar.gz file of the same name. The contents must meet the Sensu asset format specificatione.g. windows/all/all/all/acc-f-commons.tar.gzThe Plugin table sn_agent_asset extends ecc_agent_sync_file, so the attachments will be automatically copied to MID Servers into the agent\static\acc_plugin folder, if the record is active, using the existing MID Server platform file sync feature. The folder tree is defined by the parent and directory fields of the records in the normal way too.e.g. C:\MID_Server\agent\static\acc_plugin\windows\all\all\all\acc-f-commons.tar.gzOnce on the MID Server disk, because they are within the agent\static folder, the files are part of the file system of the MID Server Web Server and can be downloaded from the MID Server with a normal URL.e.g. http://10.19.2.16:8095/static/windows/all/all/all/acc-f-commons.tar.gz All the above should have happened before you even install an ACC, and run a check. When you do that, here's the flow: The Check is run from the instance, and an ECC Queue record with topic=MonitoringProbe is created on the output queue with relevant Agent+Check information.The MID Server picks up this job from the ECC Queue and executes the probe in a thread, in the normal way, which then tell the ACC to run the check by sending a message to ACC via WebSocket over TLS.The ACC will then check its cache database to see if it has the required assets (asset.db), or if it needs to update/download them.The tar.gz files are downloaded from the MID Server's Web server extension, and extracted to a temp folder. see KB1640697Signed tarballs will have the signature checked, and if verify-plugin-signature property is set to True it has to pass that check.The files are then copied into the cache folder.e.g. C:\ProgramData\ServiceNow\agent-client-collector\cache\acc-f-commons\bin\*.rb, including endpoint_discovery.rbThe command of the check is then executed.e.g. C:\ProgramData\ServiceNow\agent-client-collector\cache\acc-f-commons\bin\endpoint_discovery.rb --compact --select=basic_inventory,installed_software,file_systems,serial_numbers,network_adapters,tcp_connections,storage_devices,running_processes InstructionsGeneral Debugging Tips MID Server Debug Logging Add MID Server parameter mid.log.level=trace to the MID Server that the ACC is connecting through The docs don't mention the "trace" value, only having "debug", but this will give even more output than debug, and will allow you to see what the MonitoringProbe and Web Server are doing in detail. ACC Debug Logging In the C:\ProgramData\ServiceNow\agent-client-collector\config\acc.yml file, change the log-level parameter to debug, and then restart the ACC to cause the initial discovery check to re-run: log-level: "debug" Known causes/problems Wrongly constructed tar.gz file The {PATH_TO_ASSET}/bin is injected into the PATH environment variable by Sensu, so if the asset is built properly, and is extracted to the cache, then the command should be found in the path. If it is not built correctly, then the asset may be in the cache folder, but the script that is trying to run may not be in the right directory. Perhaps when creating the custom asset, you archived the folder containing the bin folder, and not the bin folder itself. The system expects the script here:C:\ProgramData\ServiceNow\agent-client-collector\cache\<plugin name>\bin\<command.rb> But perhaps yours is actually here, when you look at the disk:C:\ProgramData\ServiceNow\agent-client-collector\cache\<plugin name>\<plugin name>\bin\command.rborC:\ProgramData\ServiceNow\agent-client-collector\cache\<plugin name>\command.rb Solution: The asset will need re archiving, based on the Sensu asset format specification. Missing Attachment on Plugin record If the instance is sub-production, and has recently been overwritten by a clone, then the clone may have excluded the attachment files on the plugin records. https://<instance>.service-now.com/sn_agent_asset_list.do?sysparm_query=directory=false The symptoms of this are seen in the mid server agent log when the MonitoringProbe runs, and it can't find the files on the MID Server disk: Worker-Standard:MonitoringProbe-e5bf4b3b1bc8c110c0e4ea0ee54bcbd3 Worker starting: MonitoringProbe source: on_demand_requestWorker-Standard:MonitoringProbe-e5bf4b3b1bc8c110c0e4ea0ee54bcbd3 Running system command: on_demand_requestWorker-Standard:MonitoringProbe-e5bf4b3b1bc8c110c0e4ea0ee54bcbd3 WARNING *** WARNING *** No asset found for asset name: acc-visibility-mainos: windows platform: microsoft_windows_server_2016_datacenter platformVersion: 10.0.14393 Build 14393 arch: amd64 Agent log for file sync confirms no files were synched. No files are listed between these 2 log entries: FileSync:sn_agent_asset Starting file synchronization: sn_agent_assetFileSync:sn_agent_asset Finishing file synchronization: sn_agent_asset Solution: Upgrading or Repairing the Agent Client Collector apps should put the attachments back. These could be split between several ACC apps.or, export as XML, the records in the source instance, and import them to the target instance. Attachments are included when you export records as XML. The Check Definition hasn't linked all the required Plugins If a custom plugin is used for a check, that depends on scripts in other plugins, make sure all those plugins are also listed on the check definition. Solution: Work out which plugin the command in the error is from, and make sure that plugin is entered on the check definition.