Version: 3.2

Updating Smart Monitor

Attention!

This instruction describes the process of updating Smart Monitor from version 3.1.* to 3.2.*.

Information

Conditional designations:

SMOS_32 - directory where the Smart Monitor version 3.2 installation package is unpacked
USER - system user with administrator rights, usually admin
OPENSEARCH_HOME - OpenSearch home directory, usually /app/opensearch/
OPENSEARCH_DATA - directory where indexed data is stored, usually /app/data/
OSD_HOME - OpenSearch Dashboards home directory, usually /app/opensearch-dashboards/
PATH_SSL - location of certificate, private admin key, and also ca-cert, usually coincides with /app/opensearch/config/

Attention!

Updating to version 3.2 from version 3.1 can be done using the update script or manually, but preliminary data migration actions must be performed in both cases.

First of all, you need to check which version of Smart Monitor is installed, view module versions on the main page or in the command line:

curl https://127.0.0.1:9200/_cat/plugins -k -u $USER

After entering this command, you will need to enter the password for the $USER account.

In version 3.2, OpenSearch, OpenSearchDashboards and Logstash components were updated. The document storage formats in indexes have also changed, so data migration should be performed. Let's consider the procedure for updating each component. The 3.2 installer needs to be unpacked into a directory, for example, /app/distr/.

Attention!

Before starting work, it is strictly recommended to make a backup of the main configuration files and Security settings.

Recommended actions

Create a directory, for example, /app/backup, where you should copy the following:

config directory, usually $OPENSEARCH_HOME/config or $OSD_HOME/config
systemd.service file, usually /etc/systemd/system/opensearch.service and /etc/systemd/system/opensearch-dashboards.service

copy of Security settings, this needs to be done once, for which you will need the certificate and private key of admin user

JAVA_HOME=$OPENSEARCH_HOME/jdk/ $OPENSEARCH_HOME/plugins/opensearch-security/tools/securityadmin.sh -backup my-backup-directory \
  -icl \
  -nhnv \
  -cacert $OPENSEARCH_HOME/config/ca-cert.pem \
  -cert $OPENSEARCH_HOME/config/admin-cert.pem \
  -key $OPENSEARCH_HOME/config/admin-key.pem

Preliminary actions

In Smart Monitor 3.2, the format for storing some system fields in documents has changed, so data migration is required.

Please note

Preliminary actions are mandatory and must be performed before the main update.

Open the web interface (OpenSearch-Dashboards), and go to the menu Menu -> System Settings -> Developer Console. Enter the following command to update settings and execute it:

POST .sm_settings/_update_by_query
{
  "script": {
    "source": """
    void updateDoc(Map source, Map oldToNewFieldNames, String id) {
      def meta = new HashMap();
      meta.put("id", id);

      for (Map.Entry field: oldToNewFieldNames.entrySet()) {
         if (source.containsKey(field.key)) {
          meta.put(field.value, source.get(field.key));
          source.remove(field.key)
        }
      }
      source.put("_meta", meta)
    }

    def oldToNewFieldNames = new HashMap();
    oldToNewFieldNames.put("_created", "created");
    oldToNewFieldNames.put("_updated", "updated");
    oldToNewFieldNames.put("_tag_ids", "tag_ids");
    oldToNewFieldNames.put("_type", "type");
    oldToNewFieldNames.put("_module", "module");
    oldToNewFieldNames.put("_object_version", "version");
    updateDoc(ctx._source, oldToNewFieldNames, ctx._id);
    """,
    "lang": "painless"
  }
}

The result of execution is visible in the screenshot below.

Then you need to add mapping for Knowledge Center with the command:

PUT .sm_kwc_*/_mapping
{
  "properties": {
    "_meta": {
      "properties": {
        "tag_ids": {
          "type": "keyword"
        },
        "id": {
          "type": "keyword",
          "fields": {
            "keyword": {
              "type": "keyword"
            }
          }
        }
      }
    }
  }
}

Insert and execute this in the developer console. If there were no Wikilogs articles created on your installation, you will see the following error:

This is normal when there are no articles, then this item can be skipped. Next, make a backup of system indexes by executing the command in the developer console:

POST _reindex
{
  "source": {
    "index": ".sm_sme_db_queries,.sm_sme_db,.sm_dashboards*,.sm_sme_scripts,.sm_sme_lookups,.sm_sme_tags*,.sm_search_favorite,.sm_search_history,.sm_sme_search_anywhere_sources,.sm_persons,.sm_incident_aggs,.sm_sme_limits_user,.sm_jsc_settings,.sm_settings,.sm_mitre*,.sm_rsm_models,.sm_rsm_metrics,.sm_rsm_indicators,.sm_jsc_jobs*,.sm_kwc*"
  },
  "dest": {
    "index": "backup"
  },
  "script": {
    "lang": "painless",
    "inline": "ctx._index = 'backup_.sm_' + (ctx._index.substring('.sm_'.length(), ctx._index.length()))"
  }
}

The result of execution is presented in the screenshot below:

Next, you need to migrate document IDs, execute the script below in the developer console:

POST .sm_sme_db_queries,.sm_sme_db,.sm_dashboards*,.sm_sme_scripts,.sm_sme_lookups,.sm_sme_tags*,.sm_search_favorite,.sm_search_history,.sm_sme_search_anywhere_sources,.sm_persons,.sm_incident_aggs,.sm_sme_limits_user,.sm_jsc_settings,.sm_settings,.sm_mitre*,.sm_rsm_models,.sm_rsm_metrics,.sm_rsm_indicators,.sm_jsc_jobs*,.sm_kwc*/_update_by_query
{
  "script": {
    "source": """
    void updateDoc(Map source, Map oldToNewFieldNames, String id) {
      def meta = new HashMap();
      meta.put("id", id);

      for (Map.Entry field: oldToNewFieldNames.entrySet()) {
         if (source.containsKey(field.key)) {
          meta.put(field.value, source.get(field.key));
          source.remove(field.key)
        }
      }
      source.put("_meta", meta)
    }

    def oldToNewFieldNames = new HashMap();
    oldToNewFieldNames.put("_created", "created");
    oldToNewFieldNames.put("_updated", "updated");
    oldToNewFieldNames.put("_tag_ids", "tag_ids");
    oldToNewFieldNames.put("_type", "type");
    oldToNewFieldNames.put("_module", "module");
    oldToNewFieldNames.put("_object_version", "version");
    updateDoc(ctx._source, oldToNewFieldNames, ctx._id);
    """,
    "lang": "painless"
  }
}

The last action will be RSM migration, execute the following command in the developer console:

POST .sm_rsm_metrics/_update_by_query
{
  "script": {
    "source": """
    ctx._source.put("internal_job_id", ctx._source.get("job_id"));
    """,
    "lang": "painless"
  }
}

At this step, the mandatory preliminary actions are completed.

Updating OpenSearch

In version 3.2, the OpenSearch version was updated to 2.11.1, OpenSearch and OpenSearch Dashboards plugins were updated. System document fields have changed. The 3.2 installer needs to be unpacked into a directory, for example, /app/distr/. Where you unpack the archive contents, mark as $SMOS_32. Change the user for the entire directory.

SMOS_32=/app/distr/smos_3.2

Automatic mode

For the script to work, the following pre-installed packages are required:

curl
zip
unzip

Please note!

If at the end you don't see the inscription that Smart Monitor has been updated, don't run the update script again, take a screenshot of where the script stopped and contact technical support.

The automatic update script allows automating actions during updating and is located at $SMOS_32/opensearch/update.sh. When calling the script, you can specify a configuration file. The YAML file format is similar to the configuration file during installation.

Please note!

Start updating with nodes without the master role, data nodes can connect to older versions of master nodes, but not vice versa.

To start the update, run the script:

$SMOS_32/opensearch/update.sh

You will need to enter correct paths to the following directories:

OpenSearch Home Directory - OpenSearch installation directory, usually /app/opensearch
OpenSearch Conf Directory - OpenSearch configuration files directory, usually /app/opensearch/config/
OpenSearch Data Directory - data directory, usually /app/data/
OpenSearch Logs Directory - logs directory, usually /app/logs/

The update script does not perform any actions with the data and logs directory, configuration directory and systemd files will be saved to the temporary directory $SMOS_32/opensearch/staging/. If you run the script again, the staging directory will be cleared.

After entering the directories, you need to confirm the entered data by pressing y, or re-enter by pressing n.

At the second step, you will need to enter the password from admin user (or another user with full rights). The password will not be displayed when entering.

Then preparatory actions will be performed before updating, before applying the update a question about continuing will be asked, until this moment no actions are performed in the system. Also, some information about the current node and the cluster as a whole will be displayed.

If you just press Enter - the update will be interrupted, to continue you need to press y.

Upon successful completion of the update, you should see the inscription SMART MONITOR SUCCESSFULLY UPDATED!, preliminary information about the cluster and current node will be displayed.

It should be noted that the update script takes into account the current list of installed plugins on OpenSearch nodes, if you need to install some plugin additionally, then the action should be performed manually additionally.

Updating OpenSearch Dashboards

For the script to work, the following pre-installed packages are required:

curl
zip
unzip

The automatic update script allows automating actions during updating and is located at $SMOS_32/opensearch-dashboards/update.sh. When calling the script, you can specify a configuration file. The YAML file format is similar to the configuration file during installation.

When working, the script will make a backup copy of the systedm service file, opensearch-dashboards.yml and the configuration directory to the temporary directory $SMOS_32/opensearch-dashboards/staging/.

To update, run the script:

$SMOS_32/opensearch-dashboards/update.sh

You will need to enter correct paths to the following directories:

OpenSearch Dashboards Home Directory - OpenSearch Dashboards installation directory, usually /app/opensearch-dashboards
OpenSearch Dashboards Conf Directory - OpenSearch Dashboards configuration files directory, usually /app/opensearch-dashboards/config/
OpenSearch Dashboards Data Directory - data directory, usually /app/data/
OpenSearch Dashboards Logs Directory - logs directory, usually /app/logs/

After entering the directories, you need to confirm the entered data by pressing y, or re-enter by pressing n.

Then preparatory actions will be performed before updating, before applying the update a question about continuing will be asked, until this moment no actions affecting system operability are performed. Also, some information about the current node and the cluster as a whole will be displayed.

Upon successful completion of the update script, the corresponding text SMART MONITOR DASHBOARDS SUCCESSFULLY UPDATED will be displayed.

Information​

Recommended actions​

Preliminary actions​

Updating OpenSearch​

Automatic mode​

Updating OpenSearch Dashboards​

Information

Recommended actions

Preliminary actions

Updating OpenSearch

Automatic mode

Updating OpenSearch Dashboards