Data Loading into the System

Introduction

Data analysis is crucial for ensuring the security of enterprise information systems. By analyzing information, it's possible to track user activity, identify potential security threats, and prevent incidents.

The use of monitoring tools, such as Smart Monitor, facilitates this process by providing automated data analysis and visualization capabilities.

The functionality of the Smart Monitor is an important tool in this process, providing comprehensive analysis and monitoring capabilities, as well as the ability to create customized reports.

About Data Import

The Smart Monitor platform supports various methods of information collection.

One of the most common methods is where data is ingested into the system directly from log sources.

The simplest method, recommended for familiarizing yourself with the capabilities of Smart Monitor, is direct data loading into the system through a specialized interface.

As an example for familiarization, we recommend using prepared data (jollymeal_wineventlog.csv).

What's Included in the Data

The data provided for familiarization contains information from the security audit log, which includes details about login attempts, changes in system settings, file access, and other actions that may pose a security risk to the system.

The example below represents a typical event presented in the prepared data sample.

JSON Example

{
  "agent": {
    "name": "jollymeal-demo",
    "id": "e13410f4-896d-4140-a4ba-4ed54ce58149",
    "type": "winlogbeat",
    "ephemeral_id": "02e29f56-c819-4371-ab81-ce9eb68c8b15",
    "version": "8.0.0"
  },
  "winlog": {
    "computer_name": "JM-MAN-014",
    "process": {
      "pid": 88463,
      "thread": {
        "id": 5651
      }
    },
    "keywords": [
      "Audit Failure"
    ],
    "level": "information",
    "channel": "Security",
    "event_data": {
      "TargetLogonId": "0x12345678",
      "WorkstationName": "JM-MAN-014",
      "TargetUserName": "SanchezThomas",
      "TargetDomainName": "JMCORP"
    },
    "opcode": 0,
    "record_id": "123456789",
    "task": "Logon",
    "event_id": 4625,
    "provider_guid": "{54849625-5478-4994-a5ba-3e3b0328c30d}",
    "time_created": "2024-03-06T07:15:09Z",
    "provider_name": "Microsoft-Windows-Security-Auditing",
    "outcome": "failure"
  },
  "log": {
    "file": {
      "path": "/app/auth-events/output/auth_events-2024-03-06.json"
    }
  },
  "destination": {
    "address": "TERM-SERV-JMCORP",
    "domain": "JMCORP",
    "ip": "192.168.16.220"
  },
  "source": {
    "address": "JM-MAN-014",
    "ip": "192.168.16.17",
    "domain": "JMCORP"
  },
  "@timestamp": "2024-03-06T07:15:09Z",
  "related": {
    "ip": [
      "192.168.16.17",
      "192.168.16.220"
    ],
    "user": [
      "SanchezThomas"
    ]
  },
  "ecs": {
    "version": "8.9.0"
  },
  "host": {
    "name": "JM-MAN-014"
  },
  "@version": "1",
  "event": {
    "original": "{\"@timestamp\": \"2024-03-06T07:15:09Z\", \"event\": {\"kind\": \"event\", \"category\": [\"authentication\"], \"type\": [\"start\"], \"outcome\": \"failure\", \"action\": \"logon-failed\", \"code\": 4625, \"provider\": \"Microsoft-Windows-Security-Auditing\", \"module\": \"security\"}, \"agent\": {\"name\": \"jollymeal-demo\", \"id\": \"e13410f4-896d-4140-a4ba-4ed54ce58149\", \"type\": \"winlogbeat\", \"ephemeral_id\": \"02e29f56-c819-4371-ab81-ce9eb68c8b15\", \"version\": \"8.0.0\"}, \"winlog\": {\"computer_name\": \"JM-MAN-014\", \"process\": {\"pid\": 88463, \"thread\": {\"id\": 5651}}, \"keywords\": [\"Audit Failure\"], \"level\": \"information\", \"channel\": \"Security\", \"event_data\": {\"WorkstationName\": \"JM-MAN-014\", \"TargetUserName\": \"SanchezThomas\", \"TargetDomainName\": \"JMCORP\", \"TargetLogonId\": \"0x12345678\"}, \"opcode\": 0, \"record_id\": \"123456789\", \"task\": \"Logon\", \"event_id\": 4625, \"provider_guid\": \"{54849625-5478-4994-a5ba-3e3b0328c30d}\", \"time_created\": \"2024-03-06T07:15:09Z\", \"provider_name\": \"Microsoft-Windows-Security-Auditing\", \"outcome\": \"failure\"}, \"source\": {\"address\": \"JM-MAN-014\", \"ip\": \"192.168.16.17\", \"domain\": \"JMCORP\"}, \"destination\": {\"address\": \"TERM-SERV-JMCORP\", \"ip\": \"192.168.16.220\", \"domain\": \"JMCORP\"}, \"related\": {\"ip\": [\"192.168.16.17\", \"192.168.16.220\"], \"user\": [\"SanchezThomas\"]}, \"user\": {\"domain\": \"JMCORP\", \"name\": \"SanchezThomas\", \"id\": \"0005\"}, \"host\": {\"name\": \"JM-MAN-014\"}, \"ecs\": {\"version\": \"8.9.0\"}, \"outcome\": \"failure\"}",
    "code": 4625,
    "provider": "Microsoft-Windows-Security-Auditing",
    "kind": "event",
    "module": "security",
    "action": "logon-failed",
    "category": [
      "authentication"
    ],
    "type": [
      "start"
    ],
    "outcome": "failure"
  },
  "user": {
    "domain": "JMCORP",
    "name": "SanchezThomas",
    "id": "0005"
  },
  "outcome": "failure"
}

Data Upload

Smart Monitor provides a simple data import interface designed so that users do not need specialized technical knowledge.

To upload data into Smart Monitor, follow these steps:

1. Go to the Navigation Menu
2. In the Main section, select Upload Data

3. The following interface will be presented, allowing you to import data into the system

warning

Only files in .xlsx, .csv, .json formats are available for import. Also, the file size should not exceed 100 MB.

4. Select the file jollymeal_wineventlog.csv for import

5. Click the Next button to proceed to the next import step

In the dropdown menu Select options for the index, choose New index, and specify its name in the corresponding field. It is recommended to name the index jollymeal_wineventlog

7. Configure the index data schema

This interface allows user-friendly customization of data types for imported fields without requiring special technical knowledge. Users can easily select the data type for each imported field, such as text field, numeric field, date and time, etc. This ensures correct interpretation and analysis of data according to its actual content, providing more accurate and useful results when analyzing data in Smart Monitor.

It is necessary to change the type of the following fields:

• event.code: integer
• winlog.event_id: integer
• winlog.opcode: integer
• winlog.process.pid: integer
• winlog.process.thread.id: integer
• @timestamp: date

Then click the Next button.

8. A message about successful import will be displayed

The following options are then available:

• Create a template
• Open in search
• Upload more

Event information search is already available, but for further work, an index template needs to be created, so select this option.

9. In the opened window, click the Create Index Template button

10. In the Index Template Name field, enter jollymeal_wineventlog

warning

The template name must match the index name. The * symbol at the end of the template name must be removed.

11. In the Time Field row, select @timestamp, then finish creating the template by clicking the Create Index Template button

12. Data upload is complete. The data is now available for search and analysis. To verify this, go to the section Main Menu - Main - Search

In the field, enter the query:

source jollymeal_wineventlog