Installing Logstash

General information

The Logstash installer must be run as the root user, specifying the path to it:

/opt/smart-monitor-installer/logstash/install.sh

If Logstash is used as a metrics collection point before sending to OpenSearch, we recommend using a separate server for Logstash installation. For Logstash, the default owner of directories and folders is logstash, and the group is logstash.

Our installer already includes the logstash-output-opensearch plugin, which is needed for connecting to OpenSearch.

Installing main directories

The installer checks for the presence of a running Logstash service and, if found, asks about continuing the installation:

This image also shows the found installation archive and the Logstash version to be installed. By default, the installation will be canceled, to continue you need to enter "y". When continuing the installation, the running service is turned off.

At the first step, the main installation directories are set. We recommend storing all files related to Smart Monitor in one directory /app, this will simplify maintenance in the future.

If you are doing a test installation and plan to install all Smart Monitor components on one server, then you must specify separate directories for each component. You need to enter the following directories:

• Home directory — Logstash installation directory
• Conf directory — location of configuration files
• Data directory — directory for storing various files used during Logstash operation
• Logs directory — directory for storing logs

After completing the input, confirm the correctness of the entered data. The installer will analyze the existing directories and if they are not empty, it will issue a warning again:

danger

If you continue, then all content of the specified directories will be deleted without possibility of recovery!

To continue the installation, you need to enter y again and press the Enter key.

Installing Logstash parameters

At this stage, we set basic Logstash parameters, such as:

• Node name — name of the current Logstash server
• JVM Heap Size — amount of memory in GB allocated for Logstash JVM, this parameter should not exceed the amount of free memory on the server

After this, confirm the entered data and continue the installation.

Installing certificate parameters

note

When creating a private key, the RSA cryptographic algorithm with a length of 2048 bits is used, and the SHA-256 algorithm is used to create a digital fingerprint. Admin and current server certificates are created for 1095 days.

At this step, the key and certificate parameters of the current Logstash server are configured. You must specify the existing CA certificate and private key that was used when installing OpenSearch.

• Do you want to use your existing certificates and keys? — Do you want to use existing certificate and key files? You can specify your own files or use the built-in algorithm. If you use the built-in algorithm, answer no (n) only for the first installation. For subsequent nodes, you should use files from the current configuration directory (by default: /app/opensearch/config/ca-cert.pem for certificate and /app/opensearch/config/ca-key.pem for private key)
• Do you want to use an existing CA certificate and private key to generate certificates and keys for this node? — If you choose no, then you will need to manually provide all necessary certificates for the node to work. Otherwise, by choosing yes, the remaining certificates will be automatically generated based on the certification center certificate and key.
• Path to CA certificate — path to the root certificate. The certificate must be the same on all Smart Monitor components;
• Path to CA key — path to the private key. The key must be the same on all Smart Monitor components;
• CN for node certificate — common name for the current server
• Country for node certificate — country name for the current server
• State for node certificate — state name for the current server
• Locality for node certificate — city name for the current server
• Organization for node certificate — organization name for the current server
• External IP — external IP address of the current server, which will be used for interaction with other components.

Below is an example with specified existing CA private key and certificate:

Confirm the correctness of the input and continue the installation.

Forming configuration files

At this stage, no actions are required from the administrator. The stage is conditionally divided into two:

• Preparing temporary directories, performing checks, preparing templates
• Forming configuration files from the specified parameters

Creating certificates

At this stage, no actions are required from the administrator, the private key and certificate of the current server are created from the specified parameters.

Unpacking the archive and changing configuration files

At this stage, no actions are required from the administrator, the archive with OpenSearch-Dashboards is extracted, configuration files extracted from the archive are changed, preparatory processes before system changes.

Beginning changes in OS

All operations until this moment are performed strictly within the installer directory and do not affect the OS, you can view the ready-to-install set of files in the staging directory.

For this, the administrator will be asked for permission to continue the installation:

Confirm the changes to continue the installation by pressing the «y» key and then «Enter».

Installation

At this stage, no actions are required from the administrator. Here several actions occur:

• creating groups and users
• creating directories
• installing Logstash
• installing plugins
• changing owners of files and directories, as well as setting SELinux rules

Preparatory work for starting Logstash

At this stage, no actions are required from the administrator, the auto-start of the logstash.service service is enabled.

Completing installation

At the end of installation, a message about firewall configuration should appear as an example with firewalld, as well as a message about successful installation as in the image below. With any other form of installation completion, an error occurred and additional corrective actions are required.

When the installer completes successfully, the node is deployed, but requires configuration files for data collection (pipelines). The logstash.service service will start automatically after rebooting the OS.

Note that you need to configure the firewall for port **9600/tcp** – port for accessing Logstash.

To check operability after adding a pipeline and enabling the logstash.service service, you can enter in the command line:

curl -X GET http://localhost:9600?pretty

Example output: