Throttling in Job Scheduler

Important

This throttling mechanism is available starting with version 6.2.0.

Throttling prevents repeated processing of identical search query results.

The mechanism includes three types of rules: automatic suppression, manual suppression, and exclusions. Rules apply only when throttling is enabled in the job settings. Rules are stored as records: one automatic record is created for each new combination of throttling field values, and one manual record for each manual suppression or exclusion.

Rule Types

Rule type	Created by	Purpose	Duration
Automatic suppression	Job Scheduler	Suppresses repeated results based on fields configured in the job.	Set in the job settings.
Manual suppression	User	Suppresses results with specified field values.	Until the specified time or forever.
Exclusion	User	Allows a result to be processed despite matching a suppression.	Until the specified time or forever.

How Rules Work Together

For each search result, the rules are applied in this order:

1. Job Scheduler checks exclusions. If the result matches an exclusion, it is passed to active actions, and suppression rules are not checked for that result
2. If no exclusion matches, Job Scheduler checks automatic and manual suppressions. If the result matches either type, it is suppressed and is not passed to active actions
3. Results that do not match exclusions and are not suppressed are passed to active actions. If throttling fields are selected in the job, automatic records are created for these results

Processing Exclusions

A result matching an exclusion is passed to active actions on every run and does not create an automatic record.

Enabling Throttling

The throttling setting controls the entire mechanism:

• when throttling is enabled, automatic and manual suppressions and exclusions are applied
• when throttling is disabled, rules are not checked, no new automatic records are created, and all results are passed to active actions

Throttling is disabled by default for new jobs.

Disabling throttling does not delete existing records. When throttling is enabled again, rules that have not expired apply again.

For automatic suppression, configure these job settings:

Setting	Description
Suppression duration	Lifetime of an automatic record.
Throttling fields	Search result fields. Results with identical values for all these fields repeat.

For example, if the suppression duration is five minutes and the throttling fields are network.zone and status, results with identical values for these fields are considered repeats for five minutes.

If no throttling fields are selected, automatic records are not created, but existing manual suppressions and exclusions continue to apply.

SP Job Behavior

Manual suppressions and exclusions for an SP job apply to all its child jobs. Automatic records are created separately for each child job, so each has its own suppression window.

Important

Create manual suppressions and exclusions at the SP job level. A rule created for an individual child job does not apply when it runs as part of the SP job.

Rule Duration

Automatic Suppressions

The first result with a new combination of throttling field values is passed to active actions and creates an automatic record. Subsequent results with the same combination are suppressed while the record is active.

After the record expires, the next result with that combination is passed to active actions and creates a new automatic record.

Manual Suppressions and Exclusions

When creating a manual suppression or exclusion, you can specify an expiration time. After that time, the rule no longer applies. A rule without an expiration time remains active until a user deletes it.

Fields in Manual Suppressions and Exclusions

A manual suppression or exclusion can contain fields that are not selected for automatic suppression in the job.

Record Cleanup

Background cleanup removes expired records. Records without an expiration time are not removed automatically. Before deleting a job, delete all its manual suppressions and exclusions: otherwise, the remaining records without an expiration time stay in the index but become inaccessible through the Throttling API.

Resetting Throttling

A reset deletes only automatic records: for a regular job, its own records; for an SP job, the records of all its child jobs. Manual suppressions and exclusions remain, and the job settings do not change.

After a reset, the next result with each combination of throttling field values is passed to active actions again and creates an automatic record.

Example

In a job, automatic suppression is configured for five minutes using the network.zone and status fields. The following rules are also created:

• manual suppression for host.name = host-02
• exclusion for host.name = host-03

The search query returns results in this order:

Result	host.name	network.zone	status
event-1	host-01	office	unavailable
event-2	host-04	office	unavailable
event-3	host-02	dmz	unavailable
event-4	host-03	office	unavailable

The results are processed as follows:

Result	Processing
event-1	Passed to active actions; creates an automatic record for network.zone = office and status = unavailable.
event-2	Suppressed because an automatic record already exists for network.zone = office and status = unavailable.
event-3	Suppressed by the manual rule host.name = host-02.
event-4	Passed to active actions because the exclusion takes precedence over automatic suppression.

If the next search within five minutes returns the same results, event-1 and event-2 are suppressed by the automatic record, event-3 by the manual suppression, and event-4 is passed to active actions again because of the exclusion.

The Throttling API page describes how to manage manual suppressions and exclusions and reset automatic records.