Network Telemetry — NetFlow, Zeek, Firewall Logs By CyberDudeBivash

Executive Summary

Network telemetry is one of the most critical data sources in security monitoring and threat detection. It captures who is talking to whom, when, and how, enabling defenders to spot anomalies, investigate intrusions, and build network-aware threat intelligence.
This article provides a complete technical breakdown of the three major forms of network telemetry — NetFlow, Zeek, and Firewall Logs — along with their strengths, weaknesses, integration into SOC/SIEM/SOAR environments, and ATT&CK-aligned use cases.

1. What is Network Telemetry?

Network telemetry refers to metadata and sometimes content extracted from network traffic, used to:

Detect malicious activity without deep packet inspection (DPI) on all traffic.
Enrich security alerts from other sources (EDR, cloud logs, etc.).
Perform forensics after an incident.

2. NetFlow

Definition

NetFlow (Cisco’s standard, now generalized as IPFIX) is a flow-level telemetry format summarizing communication sessions between two endpoints.

Core Fields:

Source IP / Destination IP
Source Port / Destination Port
Protocol (TCP/UDP/ICMP, etc.)
Bytes sent / received
Packets count
Start and end timestamps

Use Cases:

Anomaly detection — Large outbound data volumes (potential exfiltration).
Lateral movement detection — Internal hosts talking to unusual subnets.
Beaconing detection — Regular, periodic flows to the same external IP.

Limitations:

No payload visibility.
Limited application-layer metadata.
Dependent on flow export interval for accuracy.

3. Zeek (formerly Bro)

Definition

Zeek is a network security monitoring framework that processes live traffic and generates rich, structured logs for multiple protocols.

Key Log Types:

conn.log — TCP/UDP connection summaries.
http.log — HTTP request/response metadata.
ssl.log — TLS handshake info, certificates.
dns.log — Queries, responses, record types.
files.log — File transfers over protocols (hashes, MIME type).
notice.log — Scripted detections and alerts.

Advantages:

Protocol-aware, deep parsing without full packet capture.
Flexible scripting for custom detections.
Ability to hash files, extract certificates, detect protocol anomalies.

Example Zeek HTTP Log:

iniCopyEditts=2025-08-10T09:35:12Z
uid=CYxX73fLlK9
id.orig_h=192.168.1.25
id.resp_h=185.199.110.153
method=GET
host=github.com
uri=/malicious/path
status_code=200

Use Cases:

Detect suspicious TLS certificate issuers.
Identify DNS tunneling attempts.
Correlate file hashes with threat intelligence.

4. Firewall Logs

Definition

Firewall logs record allow/deny actions based on security policies, including source/destination, ports, and actions taken.

Common Fields:

Action (ALLOW, DENY, DROP)
Source/Destination IP & Port
Protocol
Rule ID or Policy Name
Bytes transferred
Application name (Next-Gen Firewalls)

Advantages:

Direct view into what was blocked or allowed at perimeter or internal segments.
Rich application-layer metadata in NGFWs (App-ID, URL filtering logs).
Integration with threat prevention features (IPS, anti-malware).

Limitations:

Only logs traffic traversing that firewall.
Can miss east-west traffic if not inline.

5. Integrating Network Telemetry into SOC/SIEM

Ingestion Pipeline:

NetFlow/IPFIX: Export from routers/switches → Flow collectors → SIEM.
Zeek: Deploy on network tap/span ports → Output logs in JSON/TSV → SIEM.
Firewall Logs: Syslog/API → SIEM/Log management.

Normalization:

Map all to a common schema for correlation:

jsonCopyEdit{
  "src_ip": "192.168.1.25",
  "dest_ip": "185.199.110.153",
  "src_port": 54321,
  "dest_port": 443,
  "protocol": "TCP",
  "bytes_out": 10240,
  "bytes_in": 5120,
  "action": "ALLOW"
}

6. ATT&CK Mapping Examples

Data Source	Event Example	ATT&CK Technique ID	Technique Name
NetFlow	Periodic outbound traffic to rare IP	T1071.001	Application Layer Protocol: Web Protocols
Zeek DNS Logs	Long TXT records to unknown domain	T1071.004	Application Layer Protocol: DNS
Firewall Logs	Multiple denied connections to internal DB	T1046	Network Service Scanning

7. AI & Anomaly Detection on Network Telemetry

Beaconing detection with statistical models (e.g., inter-arrival time analysis).
Auto-clustering of flows to detect new C2 patterns.
Supervised models trained on labeled malicious/benign flow features.
UEBA integration — correlating network anomalies with user behavior.

Example: IsolationForest on NetFlow Data

pythonCopyEditfrom sklearn.ensemble import IsolationForest
import pandas as pd

# Example flow data
df = pd.DataFrame({
    "bytes_out": [200, 250, 50000, 210],
    "duration": [2, 3, 120, 2],
    "packet_count": [10, 12, 4000, 11]
})

model = IsolationForest(contamination=0.05)
model.fit(df)
df["anomaly"] = model.predict(df)  # -1 = anomaly
print(df)

8. Best Practices

Deploy Zeek at key choke points for protocol-rich visibility.
Enable full NetFlow (not sampled) on sensitive segments.
Integrate firewall logs with enrichment (geo-IP, threat feeds).
Retain telemetry long enough for threat hunting (90+ days recommended).
Correlate across data types — a Zeek alert + firewall allow log = high fidelity.
Automate IOC matching against threat intel feeds.

Conclusion

NetFlow, Zeek, and firewall logs are complementary pillars of network visibility.
By combining flow summaries, protocol-level metadata, and enforcement logs — and aligning them to MITRE ATT&CK techniques — SOC teams can detect, investigate, and respond to advanced threats more effectively.
When enriched, normalized, and automated, network telemetry becomes a force multiplier for both human analysts and AI-powered detection engines.

Cyberdudebivash