Use Logstash to ship logs to Loom - Loom Predictive Intelligence

Use Logstash to ship logs to Loom Summary Logstash is a popular log shipping tool, which can be used to forward data from almost every source to Loom.

Release Sophie standalone versions 3.4.x - 3.7.x

Instructions Step 1 - Installing Logstash Locate a server where LogStash will run. If forwarding files, this should be the server where the files are located. If forwarding data from a database, It is recommended to install Logstash on a dedicated server / VM.

On the designated server, install Java 11, then download and install Logstash.

Step 2 - Preparing for encryption To encrypt the traffic, download the Loom CA certificate and place under the Logstash folder in a sub-folder named cert . Next, generate a client-side certificate:

download openssl run the following commands: openssl genrsa -out selfsigned.key 2048 openssl req -new -key selfsigned.key -batch -out selfsigned.csr openssl x509 -req -days 3650 -in selfsigned.csr -signkey selfsigned.key -out selfsigned.crt rm selfsigned.csr Copy the key and crt files to the cert folder as well.

Step 3 - Create the configuration Under the Logstash folder, create a directory called conf and inside create a file named logstash.conf . Paste the following in the file:

input { # debug input # stdin {} }

output { # tcp { # host => "{{customer}}-data.loomsystems.com" # port => 5044 # codec => json_lines # ssl_enable => true # ssl_cert => "cert/selfsigned.crt" # ssl_key => "cert/selfsigned.key" # ssl_cacert => "cert/loom.cer" # }

# syslog { # host => "{{customer}}-data.loomsystems.com" # port => 6514 # appname => "{{application-name}}" # protocol => "ssl-tcp" # rfc => "rfc5424" # ssl_cert => "cert/selfsigned.crt" # ssl_key => "cert/selfsigned.key" # ssl_cacert => "cert/loom.cer" # }

# debug output # stdout { codec => rubydebug } } The configuration template includes two outputs.

To use the TCP output (recommended) First, make sure that your firewall is not blocking outbound traffic over port 5044. To test this, follow this article .

To use the TCP output with encryption you must have the TCP output plugin version 3.2 or later. To check your version, run:

bin/logstash-plugin list --verbose | grep tcp To update the plugin, run:

bin/logstash-plugin update logstash-output-tcp Uncomment the tcp clause in the configuration. Then, replace the {{customer}} handle with your customer name, which can be found in the domain you use to browse to Loom.

To use the Syslog output First, make sure that your Firewall isn't blocking traffic over port 6514. To test this, follow this article . Next, install the required output plugin by running from the Logstash directory:

bin/logstash-plugin install logstash-output-syslog Finally, uncomment the syslog clause in the configuration file, and replace the {{customer}} and {{application-name}} handles. If shipping more than one input, it's best to populate the appname field based on some property (or tag ) of the input - contact us if you need help with doing that!

Step 4 - Adding inputs Add inputs as appropriate. If you intend to read data from a database and forward to Loom, use this guide .

Step 5 - Test the configuration and run LogStash By now, your folder structure should look as follows:

logstash/ ├── bin/ ├── certs/ │ ├── loom.cer │ ├── selfsigned.crt │ └── selfsigned.key └── config/ └── logstash.conf Test your configuration file by running:

bin/logstash --configtest -f config/logstash.conf Start LogStash by running the following command:

bin/logstash -f config/logstash.conf Step 6 - (optional) Run Logstash as a service If you installed Logstash using a package manager, you should already be able to simply start the service . If running on Windows, we recommend using the life-saving NSSM .