Command Validation Tool - Support and Troubleshooting

Command Validation Tool Command Validation Tool

What is the problem?

There is no tool available, which allows user to run a command on range of IP Address(es). In the absence of such tool, the validation of command on IP address(es) against the credentials present in the instance do not happen. In case of discovery failure, where certain command did not execute as expected, majority of the times, we ask the customer to perform such validation by running the command from the virtual machine where mid server is installed.

What are we doing?

We have implemented an application, which will allow the user to run a command on a range of IP addresses belonging a certain OS type. The user will also get to choose the mid server behaviour for this operation. The output of this operation will tell whether the execution was successful or not.

How it is being resolved?

With the introduction of the command validation tool, now user do not have to go anywhere as most popular operation types/protocols will be supported in this tool. The

Command validation tool supports operation types as:

Shell Command WMI Query SNMP Query (Tabular and Scalar) SNMP Walk HTTP GET (authenticated via Basic Authentication credentials)

Following MID selection behaviour is also provided:

Specific MID Server Auto Select MID Server Try All Eligible MID Servers Note: 2 nd and 3 rd behaviour work when you have mid configured with proper IP ranges or application configured to include ‘Discovery’.

The user can provide IPv4 addresses as input in the following manner:

Comma separated IP Addresses IP Range IP Network Comma separated IP Addresses, IP Range, and IP Network. Note: The IP range and IP network will only work, when you have ParseIPRanges script include imported to your instance. Script include ( sys_script_include_3389256b778201104cb984e35b5a99ad.xml ) is attached with the KB.

The output of the Command Validation Task will be Command Validation Result which will give us the following:

Status: Success/Failure State: pending/processing/done Result: Response from device/Reason for failure Steps to troubleshoot in case of failure (depending upon if we have error suggestion attached to the error code present in the response) Credentials used for command validation (Works for Shell, SNMP query and SNMP Walk in San Diego and Tokyo release)

Requirement and Use Cases:

Troubleshooting Discovery Failures: In case of Discovery failure, if you see that there are failures in command execution either in pattern log or agent logs and the if reason for failure is not clean enough the tool can be used re-run the command on the failing IP and by choosing an appropriate mid. The Command Validation Tool can be used to detect the following: Authentication issue Authorisation/Permission Issue No Valid credentials present IP not reachable from mid Command with wrong syntax

Bulk Execution of a Command: The tool can also be used to run a command of your choice on the range of IPs to check for a certain configuration. To also be able to export the command validation task result records for a certain execution.

Feature Flow:

The Command Validation Tool is going to be released via a new Scoped Application named ‘Pattern Designer Enhancements’. To avail this feature, we must install it from the list of plugins.

After the Plugin is installed, once you refresh and if you search for ‘Command Validation Tool’ in the navigation menu, you will notice the relevant menu

Under Command Validation Tool, we notice 3 sub menus:

Create New: To create a new Command Validation Task Record Command Validation Tasks: To see the list of tasks Command Validation Task Results: To see the list of Task Results

Creating a new Command Validation Task:

Once we click on ‘Create New’ the below UI gets loaded:

Note: There are relevant field messages for ‘OS Class’, ‘IP Addresses’ and ‘Command’ field to guide the user, on what input is expected. The field message for ‘Command’ gives information about the syntax of different commands and a working example. The field message for IP Addresses and Command fields will change depending upon the Command Type field.

Following fields needs Input:

Mid Selection Method: Choose either of the 3 options which are: Specific mid server Auto Select mid server Try all eligible mid servers (will try executing command via all eligible MIDs) Note: 2 nd and 3 rd behaviour work when you have mid configured with proper IP ranges or application configured to include Discovery.

Mid Server: When you choose ‘Specific Mid Server’ as Mid Selection Method.

OS Class: Concerned OS class to which the IP addresses belongs. Example: Linux Server when all IPs you entered as input are Linux servers.

Command Type: The type of command, you want to execute on the IP addresses. It can be Shell, WMI query, SNMP query, SNMP Walk or HTTP GET.

IP Addresses: The user can provide IP4 addresses as input in the following manner: Comma separated IP Addresses IP Range (1.1.1.1-1.1.1.100) IP Network (1.1.1.1/28) Comma separated IP Addresses, IP Range, and IP Network. Note: The IP range and IP network will only work, when you have ‘ParseIPRanges’ script includes imported to your instance. Script include ( sys_script_include_3389256b778201104cb984e35b5a99ad.xml ) is attached with the KB.

Command: command belonging to the chosen command type, which can run on the chosen OS Server on the IP Addresses provided.

Command Type

Command Syntax

IP Syntax

Shell

command

No change

WMI Query

["Namespace"][space]"WMI Query"

No change

SNMP Scalar Query

oid

No change

SNMP Tabular Query

oid[space][comma separated column number],

No change

SNMP Walk

oid

No change

HTTP GET

httpEndpoint,

ip:port

Note: The field message for ‘Command’ on the UI also gives information about the syntax of different commands and a working example for each.

Once done, you can save the record or click on ‘Run Command Validation’ button to begin the task.

The task records have:

Status field: which shows the status if it is in progress or completed Progress field: which will show the progress as percentage complete.

Batching:

Once the command validation task is active, the IP addresses will be picked up by the mid in batches. By default, the batch count is 10.

The following system properties is introduced, where batch count can be customised:

“command.validation.probe.batch.count”

State of the Command Validation Task Result Records:

Pending: Yet to be picked by scheduler and dispatched to mid server Processing: Scheduler has picked the record and dispatched to mid for execution Done: The execution of command is completed, and result is created.

Troubleshooting Steps when the Command validation Task Result is a failure:

Once we open the record of the any failed Command validation Task Result, you will see an UI action or a link by the name ‘Open Suggestions to troubleshoot failure’. Once you click on this link, you will see a dialog box, with suggestions on how to troubleshoot this failure. This suggestion is coming from Error Suggestions table. If the error code present in the command validation task result record is linked with any error suggestion OOB, then we will see a proper suggestion. Check the screenshot below:

ACL and Role:

Both the new tables ‘Command Validation Task’ and ‘Command Validation Task Results’ have been configured with ACL, where create, update, read and delete operation privilege is given to role ‘discovery_admin’.

Scheduled Job:

A scheduled job is also introduced, whose responsibility is to pick Command Validation Task Results record in batches (which is by default 10) in a regular interval (by default the schedular is configured to run every 15 seconds) and send their content to ECC queue via probe. User can customise the batch count and frequency as per their environment and instance and mid configuration.

Name of scheduled Job: ‘Schedule Command Validation Probe launch’

Working example, where we run ‘sudo dmidecode’ on a list of IP addresses belong to Linux servers

Things to Note:

IP ranges, IP Network as input will only work when you have imported ‘ParseIPRanges’ script include present in KB attachments. ‘Credential Used’ attribute in Command Validation Task Results on works for command type: Shell, SNMP query and SNMP walk. This works in San Diego and Tokyo. We might observe the feature working for SNMP query and SNMP walk in Rome. Mid Selection Method: Auto Select Mid and Try All Eligible Mid servers, will only work when: You mid servers are configured with desirable IP ranges or You mid servers have ‘application’ configured to include ‘Discovery’. Choose Command Type in such a way that it can be run on the chosen OS Server. The chosen IP Addresses should belong to the chosen OS Type. For example: WMI query cannot be run Linux OS. Choose MID such that the chosen command type can be run via it. For example: WMI query cannot be run using Linux mid. HTTP Get command type, will only work, when the credentials of API call is present as Basic Auth credentials. Token based credentials are not supported for this operation. This is also conveyed via field messages. Kindly use the Command Validation Tool carefully, as it enables us to run any kind of command on the range of IP Addresses. The application is supported on Domain Separated Environment. There are no APIs exposed to trigger Command Validation Task.

Release Information

Release: August 2022 Store

Application version : 2.0.0

Previous version: None

Family version Supported: Quebec P-10, Rome P-8, San Diego P-5, Tokyo. (Onwards Supported)

Note: This is the first release of this scoped application.