peval

Description

Performs various operations on the data. It is based on internal storage mechanisms.

danger

Using peval in a query is allowed if it is preceded only by commands that also work with internal storage mechanisms. These include source, inputlookup and search. This condition must also be met for all subqueries in the query.

Syntax

 | peval <field>=<expression>["," <field>=<expression>]...

Parameter	Syntax	Description
field	<field>	Name of the field with the result of the operation.
expression	<expression>	The operation (computation) or a set of operations to be performed to create the new field.

---

Supported expression Operations

Operation Type	Syntax	Description
Arithmetic Operations	(+|-|*|/)	Arithmetic operations are applied to numeric fields of numeric type.
Logical Operations	(==|!=|>|>=|<|<=) (AND|OR|NOT)	Logical operations include comparisons and logical operators that return boolean values.
Conditional Operations	cidrmatch coalesce like nullif case if validate	Conditional operations provide additional capabilities for data processing.
Mathematical Operations	abs ceil exact exp floor ln log pi pow round sigfig sqrt	Mathematical operations with numerical values for various computations.
Time Operations	now strftime strptime	Time operations related to processing, managing, or analyzing data within a time interval.
Data Type Identification Operations	isbool isint isnotnull isnull isnum isstr typeof	Operations where the system or program evaluates the data type of a variable, object, or expression.
Data Conversion Operations	printf tobool tonumber tostring	These are processes of converting data into different types.
Statistical Operations	max min	These are operations that are defined for an object or type of objects but are not executed for an instance of an object.
Text Operations	len lower ltrim replace rtrim substr trim upper	These are sets of actions performed to analyze, transform, format, store, or process textual data.
Multivalue Operations	mvappend mvcount mvdedup mvjoin mvrange mvsort mvzip split	Operations designed for working with multivalue fields or creating and processing such arrays.
Regular Expressions	regex	Regular expression search operations.
Painless Scripts	_script	Operations described in the Painless scripting language.

Data Types

Peval operates with the following basic data types and attempts to retain the original or assigned data type in the response:

• string
• numeric
• boolean
• null
• multivalue

info

Multivalues represent arrays of values, where each element is a basic data type.

OpenSearch Specifics

• Scripts in OpenSearch handle numeric field types or keyword types for text fields. For text fields, you need to append <field name>.keyword
• Regular expressions must be wrapped in the regex function to ensure proper OpenSearch script handling:
  • Syntax: regex("<regular expression>")
• Allows executing custom painless scripts during command execution using the _script function:
  • Syntax: _script("<script>", (<arg_name>, <arg_value>)...)

---

Examples

Example 1

Concatenation of fields with addition of a string separator.

source nix_events-*
| search host="SPB-*"
| peval agent= agent + ":" + port
| search agent="*log*77"
| aggs values(per_number) by agent

In this example, the system filters events host with prefix SPB-*. Then a new field agent is created by concatenating string values agent and port with separator :. The query then selects events where the field agent contains substring log and ends with 77. After that, aggregation is performed, summing the values of field per_number for each unique agent.

Example input data:

host	agent	port	per_number
SPB-WS-01	nginx-log	77	ivanov
SPB-WS-02	nginx-log	77	petrov
MSK-DB-02	syslog	53363	ivanov
SPB-DB-03	syslog	57177	andreev
SPB-WS-03	nginx-error	77	andreev

The result of executing the query may be the following table:

agent	values (per_number)
nginx-log:77	ivanov petrov
syslog:57177	andreev

---

Example 2

Calculation of the difference between numeric fields.

... | peval temp_diff=ups_adv_battery_temperature - env_temperature
| where temp_diff > 10

In this example, the command calculates the difference between numeric fields ups_adv_battery_temperature and env_temperature, writing the result to temp_diff. The where filter keeps events where the difference exceeds 10 degrees.

Example input data:

ups_adv_battery_temperature	env_temperature
35	20
30	22
28	25

The result of executing the query may be the following table:

ups_adv_battery_temperature	env_temperature	temp_diff
35	20	15
30	22	8

---

Example 3

Classification by numeric field value using Painless scripts.

... | peval res=_script("if (doc['total_price'].value < 1000 ) { return 'low'; } else if (doc['total_price'].value < 2000) { return 'medium'; } else { return 'high'; }", (total_price, total_price))

In this example, the command classifies orders by the numeric field value total_price through doc['total_price'].value as low (less than 1000), medium (from 1000 to 1999) or high (2000 and above) using a Painless script with the _script function.

The result of executing the query may be the following table:

user_id	items_quantity	operation_status	total_price	res
1	3	success	1815	medium
2	7	success	4073	high
3	10	success	5520	high
4	1	success	599	low

---

Example 4

Determining the status level based on error count and threshold value for each document using the calcStatusByErrors script.

... | peval level=calcStatusByErrors(errorCount, 3)

The result of executing the query may be the following table:

errorCount	level	service
3	warning	auth
0	ok	cache
10	critical	search
5	critical	billing

---

Example 5

Creating a status array using multivalue operations:

| peval status_list=mvappend(case(http.response.status_code == 200, "OK", http.response.status_code == 404, "Not Found", "Error"), tostring(http.response.status_code))

In this example, the system processes numeric HTTP statuses http.response.status_code into an array status_list containing textual description and the original code as a string.

The result of executing the query may be the following table:

source.address	destination.address	http.request.method	http.response.status_code	status_list
172.24.76.116	192.168.12.17	GET	200	OK 200
172.23.10.193	192.168.12.98	GET	404	Not Found 404
172.27.99.179	192.168.12.1	GET	403	Error 403

---

Example 6

Extracting a text value from the host.name field and converting its first part to lowercase.

... | peval hn=lower(mvindex(split(host.name, "."), 0))

In this example, the split function separates the string host.name into a multivalue field hn by delimiter ., and the mvindex function extracts the element with index 0 from this field. As a result, the lower function converts this element to lowercase.

The result of executing the query may be the following table:

user	action	host	hn	hn (without mvindex function)
ivanov	added-user-account	WS-01.work.local	ws-01	ws-01 local work
popov	changed-password	WS-02.work.local	ws-02	ws-02 local work
andreev	removed-member-from-group	WS-03.work.local	ws-03	ws-03 local work

---

Example 7

Creating a field for event classification based on HTTP response status:

source web_indexes
| peval name=if(isnull(user.name), "None", user.name)

In this example, the if function checks the condition isnull (user.name), returning true if the field contains an empty value. If the condition is true, the string None is written to the field name; if false, the original value is written to user.name.

The result of executing the query may be the following table:

source	destination	path	status	name	user.name (before if expression)
172.26.0.240	192.168.12.100	/veil.jpg	200	Ivanov	Ivanov
172.26.0.241	192.168.12.101	/number/seashore	404	None	null
172.26.0.242	192.168.12.102	/advice	403	None	null
172.26.0.243	192.168.12.103	/walk/stove	200	Petrov	Petrov