IoC Lifecycle
The remove_expired_ioc.py script manages indicator expiration and restores IoCs from the recycle bin dashboard.
Automated Database Maintenance
The SCRIPT - Threat Intelligence - LifeTime IoC task runs every 10 minutes. Each time it runs, the script:
- Moves expired IoCs from the active database to the recycle bin
- Removes the moved IoCs from the active database
- Permanently deletes indicators that have been in the recycle bin for more than 30 days
To enable automated maintenance, open Main Menu - Job Scheduler - Jobs List, find SCRIPT - Threat Intelligence - LifeTime IoC, and enable the task. Run results are available in Job Scheduler; database status is shown on the module dashboards.
Restoring an Indicator
Find the indicator on the recycle bin dashboard and select Restore. The indicator is returned to the active database with a new 90-day expiration period.
To verify the result, open the Restored IoCs section on the active database dashboard.
Installation and Verification
Install the script at the following path on the SME task execution nodes:
/app/opensearch/utils/threat_intelligence/remove_expired_ioc.py
Run the commands as root. The example uses the SME account opensearch:opensearch; specify the account used by your installation.
mkdir -p /app/opensearch/utils/threat_intelligence/logs
cp /tmp/remove_expired_ioc.py /app/opensearch/utils/threat_intelligence/remove_expired_ioc.py
chown root:opensearch /app/opensearch/utils/threat_intelligence
chmod 0750 /app/opensearch/utils/threat_intelligence
chown root:opensearch /app/opensearch/utils/threat_intelligence/remove_expired_ioc.py
chmod 0640 /app/opensearch/utils/threat_intelligence/remove_expired_ioc.py
chown opensearch:opensearch /app/opensearch/utils/threat_intelligence/logs
chmod 0750 /app/opensearch/utils/threat_intelligence/logs
Configure the OpenSearch connection:
| Parameter | Value |
|---|---|
| OpenSearch address | https://localhost:9200 on the node where the script runs. |
| Credentials file | /app/opensearch/config/environment. |
| Credential variables | OPENSEARCH_USER, OPENSEARCH_PASSWORD. |
| CA certificate | /app/opensearch/config/ca-cert.pem. |
| Active database | threat_intelligence_database_ioc. |
| Recycle bin | threat_intelligence_recycle_bin. |
Grant the execution account read access to the environment file and CA certificate, and write access to the log directory. Grant the OpenSearch user permission to read, move, write, and delete documents in the module indexes. When editing the shared environment file, preserve the settings for other components.
Set the following query in the task:
| script intr=python3 "/app/opensearch/utils/threat_intelligence/remove_expired_ioc.py"
Set the task schedule to */10 * * * *.
After installation, enable the task and check the result of its next run in Job Scheduler. To verify restoration, use the Restore action on the recycle bin dashboard and find the restored indicator in the active database.
Activity Log
The log is located at:
/app/opensearch/utils/threat_intelligence/logs/threat_intelligence_recycle_bin.log
To view the latest messages, run:
tail -n 100 /app/opensearch/utils/threat_intelligence/logs/threat_intelligence_recycle_bin.log
Messages use UTC. The log is rotated daily, and seven archived files are retained.