Home > SOC Analysis

How SIEM Works

A SIEM is not a single tool that magically detects attacks. It is a pipeline with multiple stages, each performing a critical task that transforms raw logs into meaningful detections. To understand SIEM deeply, you must understand every layer: how data enters the system, how it is processed, how it is enriched, how detection rules work, how correlation chains are built, how alerts are generated, and how analysts use the SIEM for investigations.

This chapter explains the complete internal workflow of a SIEM end-to-end, with practical examples at every stage, real event structures, and real detection logic used in modern SOCs.

1. Log Collection and Data Ingestion

The SIEM begins by collecting logs from every asset. Without logs, nothing else works.

A mature SIEM collects logs from:

  • Windows servers and endpoints

  • Linux servers

  • Firewalls, routers, switches

  • IDS/IPS systems

  • EDR tools

  • Cloud platforms

  • Authentication servers

  • Web applications

  • Databases

  • Virtualization platforms

  • SaaS and third-party services

How devices send data to the SIEM

There are four primary ingestion methods:

  1. Agents
    Installed on endpoints. Example:

    Winlogbeat → Elastic SIEM
Filebeat → SIEM

  2. Syslog
    Common for firewalls, routers, switches.

    Firewall → Syslog → SIEM

  3. API Pull / API Streaming
    Used for cloud logs (AWS, Azure, GCP).

    CloudTrail → API → SIEM

  4. Direct connectors
    Built into the SIEM or vendor-specific.

Practical ingestion example

Windows:

Winlogbeat → Logstash → SIEM index → SIEM correlation engine

Linux:

/var/log/auth.log → Syslog → SIEM

Firewall:

Cisco ASA → UDP 514 → SIEM

Cloud:

AWS CloudTrail → S3 → SIEM bucket collector

The ingestion pipeline ensures constant, real-time delivery of logs from thousands of devices.

2. Parsing and Field Extraction

Every log arrives in different formats. A SIEM must extract usable fields from each log before analysis.

Raw Windows log example

4624 - Successful Logon
Account Name: tom
Process Name: C:\Windows\System32\winlogon.exe
Source IP: 185.12.33.77
Logon Type: 10

Parsed and normalized form

event_code: 4624
event_type: logon_success
username: tom
src_ip: 185.12.33.77
logon_type: remote_interactive
host: WIN-SERVER01

Normalization allows detection rules to work across different platforms.
A SIEM cannot correlate events unless fields are normalized.

3. Log Enrichment

The SIEM enriches every event with additional context that analysts need.

Examples of enrichment:

  • Threat intel (Is the IP malicious?)

  • User details (Is this user an administrator?)

  • Asset groups (Is the host critical?)

  • Geo-location (Is the login from a suspicious country?)

  • Vulnerability scanner data (Is this host unpatched?)

  • Past behavior patterns (Is the activity abnormal?)

Raw log without enrichment

src_ip: 185.44.12.9

Enriched log

src_ip: 185.44.12.9
TI_score: 98 (malicious)
geo: Russia
related_campaign: Cobalt Strike Infrastructure
user_role: domain_admin
asset_value: critical

Enrichment turns meaningless logs into investigation-ready events.

4. Threat Detection Logic

SIEM detection is built on rules, analytics, and patterns.
Detections usually fall into several categories:

  • Threshold rules

  • Correlation rules

  • Sequence rules

  • Machine learning anomalies

  • Behavior analytics

  • Reputation-based alerts

  • Signature-based rules

  • Custom detections based on MITRE ATT&CK

Practical Detection Rule Example 1: Brute Force

If event_type = failed_login from same IP
Count > 15 in 3 minutes
→ Alert: Possible Brute Force Attack

Example 2: Suspicious PowerShell Execution

If process_name = powershell.exe
AND command_line contains "-enc"
AND parent_process = winword.exe
→ Alert: Malicious PowerShell execution from macro

Example 3: Credential Theft

4624 (logon_success) AND
4672 (special privileges) AND
Sysmon 10 (process Access: LSASS)
→ Alert: Possible Mimikatz Credential Dumping

Real SIEMs run thousands of these rules every second.

5. Correlation Engine

Correlation is where the SIEM detects multi-stage attacks.

Example real-world kill chain correlation:

Stage 1: Phishing

Email logs:

Attachment: invoice.docm
From: unknown@risky-domain.com

Stage 2: Execution

Sysmon:

Event ID 1: powershell.exe launched by winword.exe

Stage 3: Lateral Movement

Windows Log:

4624: Successful login from HOST-A to HOST-B

Stage 4: Exfiltration

Firewall:

Outbound 2GB transfer to 91.22.113.44

The SIEM correlates these into:

ALERT: Multi-stage attack detected (Phishing → Execution → Lateral Movement → Exfiltration)

Correlation is what turns logs into a complete attack story.

6. Alert Generation and Prioritization

When a detection rule or correlation hits, the SIEM creates a structured alert.

A complete SIEM alert contains:

  • Summary

  • Host involved

  • User involved

  • Indicators (IP, hashes, domains)

  • Related logs

  • Timeline of events

  • Severity level

  • MITRE ATT&CK technique mapping

Example alert (realistic)

ALERT: Suspicious Lateral Movement via SMB
User: administrator
Source Host: WIN-LAPTOP-09
Target Host: FILE-SERVER-01
Events:
  - 4624 Logon Success
  - Sysmon 3 Network Connection
  - Sysmon 1 Remote Execution
Severity: High
ATT&CK Technique: T1021.002 - Remote Services (SMB)

The alert is then forwarded to the ticketing system.

7. Querying & Investigation in the SIEM

Analysts use the SIEM to investigate alerts using search queries, timelines, and visualizers.

Common investigation queries

Find malicious PowerShell:

process_name: powershell.exe AND command_line:*enc*

Check for lateral movement:

event_code:4624 AND logon_type:3

Find connections to suspected C2:

network.dest_ip:185.77.103.91

Trace attacker movement:

host:WIN-SERVER01 AND username:john AND event_code:*

The SIEM builds a minute-by-minute timeline of the attack.

8. Dashboards & Monitoring

SIEM dashboards help the SOC visualize:

  • Top failed logins

  • Malware distribution

  • High-risk users

  • Network anomalies

  • Cloud misconfiguration alerts

  • Authentication trends

  • Firewall block trends

  • Live attack attempts

Example dashboard widgets:

  • “Logon failures over time”

  • “Outbound traffic to high-risk countries”

  • “New admin accounts created”

Dashboards give real-time situational awareness.

9. Reporting & Compliance

SIEMs generate:

  • Audit trails

  • Regulatory reports

  • Incident compliance logs

  • Data access reports

  • Privileged user monitoring

Common compliance requirements supported:

  • PCI-DSS

  • HIPAA

  • GDPR

  • ISO 27001

Reports prove that the organization monitors and protects critical systems.

10. SOAR & Automation Integration

Modern SIEMs integrate with SOAR platforms to automate actions such as:

  • Isolating endpoints

  • Blocking IPs

  • Resetting passwords

  • Sending alerts to Slack/Teams

  • Running threat enrichment

  • Submitting files to sandboxes

Example automated workflow

SIEM Alert: Malware Detected
SOAR:
  1. Fetch hash reputation
  2. Block hash in EDR
  3. Quarantine endpoint
  4. Notify SOC channel
  5. Create ticket in TheHive

This reduces analyst workload and speeds up response.

Intel Dump

  • SIEM collects logs from every system via agents, syslog, and cloud connectors.

  • Parsing and normalization convert raw logs into structured fields.

  • Enrichment adds threat intel, asset data, and context.

  • Detection rules find suspicious activity using signatures, thresholds, and analytics.

  • Correlation links multiple events to reveal full attack chains.

  • Alerts contain structured details and are sent to ticketing systems.

  • Analysts query SIEM for deeper investigation and timeline analysis.

  • Dashboards provide real-time visibility into security trends.

  • SIEM supports compliance and reporting requirements.

  • SOAR integration automates response and reduces SOC workload.

HOME LEARN COMMUNITY DASHBOARD