Health Log Analytics - Support and Troubleshooting

Health Log Analytics Table of Contents Health Log Analytics Overview Overall Setup RoadMap Request Health Log Analytics Installation Setup MID Server Setup Data Input Log Processing Log Viewer Alerts Troubleshooting Additional Information Health Log Analytics Overview What is Health Log Analytics (HLA)? An application which receives logs, processes logs, and makes such logs available for review on your ServiceNow instance. HLA collects metrics from the logs and detects anomalies. Alerts can be generated when deviations from normal behaviors are discovered. Once processed, the logs are available for review in the instance.

Health Log Analytics collects logs streaming into your ServiceNow instance from endpoints or data lakes, such as Splunk and Elasticsearch. The instance receives the logs via the MID Server connector instance. Health Log Analytics identifies and triages anomalies in your log data using unsupervised machine-learning (ML) models. It then groups the anomalies together and applies further algorithms to help identify the root cause of the issue.

Architecture

Log Flow The diagram shows the Health Log Analytics workflow from collecting the data through sending an event or alert to Event Management.

Components Overview Streaming Sources

Streaming sources are the application/servers which send the logs to the MID server.

MID Server

The MID Server is the "middle-man" which receives the logs from the streaming sources and forwards them to your ServiceNow instance.

Note: The MID server can use data input pre processors to modify raw log data, and drop or break up log messages, before they are processed by the log processing nodes. This happens before Health Log Analytics maps and structures the raw logs. See Edit raw log data before processing.

Log Analytics Core Server

This server/application receives the logs sent by the MID server and processes them. It will:

Send the structured logs to the log Elasticsearch server Detect anomalies and create alerts, em_alert Log metrics to the Metrics server Elasticsearch The Elasticsearch server keeps the structured logs and makes them available for searches.

Time Series DB The time series DB keeps all the metrics extracted from the logs.

Overall Setup RoadMap Request Health Log Analytics installation Request Health Log Analytics installation Configure the MID Server Configure the data inputs Request Health Log Analytics Installation Request Health Log Analytics installation Request Health Log Analytics installation Setup MID Server Open the MID Server which will be used to stream logs to Select the related list "Capabilities" Click "Edit" Add the "Log Ingestion" capability and save Note: If the data inputs need to connect to the MID server via a public IP address, configure mid server property mid.public_ip Setup Data Input Now that we have a MID server setup we can configure the data inputs.

Note: While configuring a data input we also configure what port the MID server will receive the data on

You can set up the data input process for Health Log Analytics in either of two ways:

Guided setup Manually Guided setup: Provides a sequence of tasks that help you configure data inputs on your ServiceNow instance. Using the guided setup ensures that you have the minimum required setup for the data input process. For more information, see Set up data inputs using guided setup .

Manually: For more information, see Set up data inputs manually .

Note: Regardless of how you choose to implement Health Log Analytics, you must first configure a MID Server and make sure its log ingestion capability is enabled.

The optional guided setup helps you create data input connectors for the following common data sources:

Rsyslog Beats Splunk Elasticsearch MID Server TCP Log Processing Structuring This layer deals with structuring log data and auto-mapping it to logical silos, called Components. Data structuring can be done automatically or manually.

The system auto-structures log data by extracting the following properties from incoming log messages: Message, Timestamp, Host, Severity, and External-IDs. It extracts explicit values, like "property-name" and "value is IP." and semantic ones such as length, number of English words, and variance.

Auto-mapping assigns log samples and metadata to the appropriate tags automatically. The system tries to map log lines by analyzing the source that streams the data. The mapping is based on agent hints and common transport header fields.

Enrichment This layer handles identifying the variable parts of a log message.

Health Log Analytics workflow: Enrichment

It also identifies keywords and contextual properties. In the image above, "WARN" and "Failed" are the keywords to track. "User," "source IP," and "port" are the contextual properties.

Analysis In this layer, each log line is indexed. Health Log Analytics extracts properties from the inner log message that contribute to models of behavior that the system learns to expect. Anomalous behavior departs from this expected behavior. You can search for an event and its most significant properties for manual triaging.

Machine Learning (ML) and Artificial Intelligence (AI)

Health Log Analytics uses advanced unsupervised machine-learning algorithms to discover patterns within logs and learn their unique data behavior. It then sets dynamic thresholds based on the data signature in real time to detect issues when they first occur. When the system detects a deviation from the typical pattern, it sends an event to Event Management.

Log Viewer The Log viewer tab enables you to browse the logs by timestamp or time range, to search for particular log text, and to visualize the frequency of anomaly occurrences in a particular time period. If you discover an important metric in the log data, you can use it to define a Log Analytics alert rule.

To open up the log viewer, navigate to:

Health Log Analytics > Log Viewer Example screenshot from viewing logs:

To learn more about the Query String Syntax, click in the info link above the search box.

For more information on the log viewer, please see:

Define, save, and share a search of log data Filter search results on the Log viewer Alerts Health Log Analytics sends events to Event Management. In Event Management, Health Log Analytics alerts appear in the All alerts list. This list enables operators to see alerts from the event and the Health Log Analytics alert type in a single location.

Troubleshooting For troubleshooting please see Health Log Analytics Troubleshooting

Additional Information Health Log Analytics