SME-RE Configuration

SME-RE Startup Parameters

SME-RE startup parameters are specified in the application.properties configuration file, located in the same directory as the SME-RE utility.

Table of startup parameters:

Parameter	Description	Default Value
server.port	The port on which SME-RE will accept incoming connections.	18080
spring.servlet.multipart.max-request-size	Maximum file upload size in bytes.	100000000
server.tomcat.max-http-form-post-size	Maximum size for multipart/form-data requests in bytes.	104857600
sme.FileRotatePeriod	Request data file rotation time in seconds.	10000
sme.baseData	Directory for storing request data files.	./data
sme.directoryPollerInterval	Interval for checking the request data file storage directory in seconds.	50
sme.scriptConfigPath	Path to the scripts.yaml configuration file.	./scripts.yaml
sme.LogStringSize	Log string size.	300
sme.RapidDelete	Whether to delete request data files.	True
sme.redis.url	Address for connecting to Redis.	"127.0.0.1"
sme.redis.port	Port for connecting to Redis.	6379
sme.redis.pass	Password for connecting to Redis.	""
logging.file.path	Path for storing logs.	logs
server.ssl.enabled	Whether to use SSL/TLS for incoming connections.	true
server.ssl.certificate	Path to the public key (certificate) of the current SME-RE server in PKCS8 format.	/app/opensearch/config/node-cert.pem
server.ssl.certificate-private-key	Path to the private key of the current SME-RE server in PKCS8 format.	/app/opensearch/config/node-key.pem
server.ssl.trust-certificate	Path to the Certificate Authority (CA) chain in PKCS8 format.	/app/opensearch/config/ca-cert.pem
server.ssl.client-auth	Certificate validation mode, can be need, none, want.	need

General SME-RE Connection Settings

Table describing the general settings for connecting to SME-RE:

Setting	Description	Default Value	Setting Type
sme.core.remote_script.user	Username used to make requests to SME-RE.	sme_re_user	Cluster Setting
sme.core.remote_script.password	Password of the user used to make requests to SME-RE.		Keystore Setting
sme.core.remote_script.enable_ssl	Flag indicating whether to use SSL for connecting to SME-RE from the script command in search and in ScriptAction in the job scheduler.	true	Cluster Setting

Information for Running Scripts in Search

Connecting to SME-RE from the script Command in Search

Table describing cluster settings for connecting to SME-RE from the script command:

Setting	Description	Default Value
sme.core.remote_script.base_path	Path to the folder where scripts will be uploaded.	"/app/opensearch/utils/scripts/"
sme.core.remote_script.port	Port on which SME-RE is running.	18080
sme.core.remote_script.url	URL where SME-RE is running.	"https://127.0.0.1"
sme.core.remote_script.base_interpreter_name	The default interpreter name to use the script command without the intr parameter	"bash"
sme.core.remote_script.interpreters	List of available interpreters, as an array of strings. String format: "interpreter name"::"path to interpreter".	["python3::/app/opensearch/utils/python/bin/python3","bash::bash"]

Creating a Script

To use a script in search, you must place it in the folder specified in the sme.core.remote_script.base_path setting on all servers in the cluster, ensure the file is executable, and has opensearch:opensearch permissions.

To return data from the script for further processing in the search, output the result in JSON format.

Example for Python:

print(scriptResultJson)

If other commands are executed before the script command, their result is saved to a temporary file in the folder specified in sme.baseData. The name of this file will be passed to the script as a command-line argument.

Example for Python:

import sys

if __name__ == "__main__":
    # open the file with data from the request
    inputDataFile = open(sys.argv[1], "r")

    # read the file with data from the request
    inputData = inputDataFile.read()

Information for Running Scripts in the Job Scheduler

Connecting to SME-RE from ScriptAction in the Job Scheduler

Settings can be retrieved via the following path:

GET _core/settings/job-scheduler

Table describing the settings for connecting to SME-RE from ScriptAction:

Setting	Description	Default Value
sme-re_port	Port on which SM Engine RE is running.	18080
sme-re_host	Host on which SM Engine RE is running.	localhost

Creating a Script

You can run any script in the job scheduler.

If you need to pass sensitive data to the script, you can use the environment variables feature. This feature allows you to store sensitive data in the keystore so that SME-RE adds them to environment variables when the script is run.

Algorithm for configuring environment variables:

1. Store the necessary variables to be passed to the script in the keystore on all cluster servers, using the variable jobscheduler.script.pass.<variable_name>
2. Specify in the job scheduler settings which task can use which variables from the keystore To do this, execute the following command:

PUT _core/settings/job-scheduler
{
  "script_environments": """{
    "<task_id>": ["<variable_name>"]
  }"""
}

Now the scheduler will instruct SME-RE to run the script with the variables in the environment.

3. In the script, include code to retrieve the variable from the environment

Example for Python:

import os

if __name__ == "__main__":
    # get the variable from the environment
    try:
        envParamData = os.getenv("<variable_name>")
    except Exception as e:
        envParamData = e