IR 3: Effective Log Analysis

Introduction

A well architected SIEM is one of the most valuable tools in a SOC, central to both detecting and responding to security incidents. This post will cover the essentials of setting up and configuring a SIEM, then move into effective log analysis techniques. For new analysts, this overview provides the necessary foundation to optimize a SIEM for incident detection, reduce false positives, and use logs to support a streamlined, effective response process.

Basics of SIEM Configuration

A SIEM is only as good as its configuration. Proper setup ensures accurate data collection and efficient alerting. Here’s how to approach initial configuration:

• Identify Data Sources: Select data sources critical to monitoring the network. Standard sources include firewalls, IDS/IPS, antivirus logs, VPN logs, and application logs. Prioritize sources based on your organization’s security needs.
• Define Log Collection Points: Configure log collection to include all necessary endpoints, servers, and network devices. This setup should align with the organization’s incident response priorities and threat landscape.
• Set Up Correlation Rules: Correlation rules help the SIEM detect potential incidents by connecting seemingly unrelated events. For example, multiple failed login attempts followed by a successful login from the same IP could trigger an alert. The goal is to define correlation rules that align with common attack patterns without overwhelming the SOC with false alerts.
• Establish Alert Thresholds: Alert thresholds define the frequency and sensitivity of alerts. Avoid setting thresholds too low, as this can create excessive noise. Start with moderate thresholds and adjust them based on observed activity patterns.
• Configure Data Retention Policies: Logs should be retained for a set period based on compliance requirements, storage capabilities, and investigation needs. Typical retention periods range from 90 days to a year, though highly regulated industries may require longer retention.

Optimizing SIEM for Log Analysis

Log analysis is a central part of incident detection, and a well tuned SIEM enhances this process. Here are steps for making log analysis more efficient and effective:

• Prioritize High Vallue Logs: Not all logs carry the same value in identifying threats. Prioritize logs that give insight into user activity, system integrity, and potential attack vectors. These may include authentication logs, VPN access logs, and critical server event logs.
• Apply Enrichment to Logs: Log enrichment adds context to raw data, making analysis faster and more meaningful. For example, adding geolocation data to IP addresses can help analysts quickly identify suspicious access attempts from unexpected regions. Similarly, adding user identity information to logs helps to highlight when unauthorized actions occur.
• Use Parsing and Field Extraction: Properly parsing logs allows the SIEM to structure unformatted log data. Field extraction (e.g., extracting IP addresses, usernames, event types) makes it easier to correlate and search logs. Most SIEMs support automated parsing, though some custom configurations may be needed.
• Regularly Update Detection Rules: As threats evolve, so should the SIEM’s detection capabilities. Schedule regular rule updates to keep up with emerging attack patterns. This can involve adding new rules based on the latest threats and adjusting existing ones based on recent incident analysis.
• Run Health Checks and Tuning: Periodically review the SIEM’s health and performance. This involves checking for log gaps, verifying data ingestion, and tuning existing rules based on recent performance. Regular tuning ensures that alerts remain relevant and accurate, reducing both false positives and false negatives.

Reducing False Positives

False positives can overwhelm analysts and lead to missed incidents. To reduce false positives, follow these steps:

• Fine Tune Correlation Rules: Correlation rules that are too broad or poorly configured can generate unnecessary alerts. For instance, a rule that triggers an alert for any failed login may produce excess noise. Adjust the rule to trigger only after a certain number of failed attempts within a set time frame.
• Implement Whitelisting: For known and trusted sources (e.g., internal IPs or frequently used service accounts), whitelist them where appropriate to minimize unnecessary alerts. Whitelisting should be done carefully to avoid creating blind spots.
• Adjust Severity Levels: Not every alert should be flagged as critical. Assign severity levels to alerts based on potential impact and confidence. For example, an alert on multiple failed login attempts could be set to a lower severity than one involving detected malware.
• Analyze Historical Data: Reviewing historical data helps identify recurring patterns of benign alerts. Using this information, refine rules and adjust thresholds to decrease the volume of known false positives.
• Test Rule Changes in Staging: Before deploying changes to rules in production, test them in a staging environment to validate effectiveness and adjust parameters. This helps avoid unexpected alert floods and keeps production alerts relevant.

Effective Log Analysis Techniques

With a well architected SIEM, analysts can perform effective log analysis to detect and investigate incidents. Here are some core techniques:

• Anomaly Detection: Identify deviations from normal behavior by comparing current data to historical patterns. For example, a spike in network traffic outside typical business hours could signal suspicious activity.
• Pattern Matching: Use pattern matching to find common indicators of compromise. Many SIEMs support regular expressions, which allow analysts to search logs for specific patterns (e.g., known malicious IP addresses or common exploit commands).
• Threat Intelligence Integration: Incorporate threat intelligence feeds that provide up to date information on IP addresses, domains, and file hashes associated with known threats. Matching logs against these feeds helps identify potential attacks early.
• Investigate Suspicious Activities in Context: When investigating an alert, view it in the context of related events. For instance, reviewing all activities from a suspicious IP can provide insight into its purpose and potential risk.
• Timeline Analysis: Creating timelines of events helps visualize the sequence of actions leading up to an incident. This technique is particularly useful when investigating complex incidents with multiple events, such as a phishing campaign followed by lateral movement within a network.

Wrap Up

Effective SIEM configuration and log analysis are fundamental to incident detection and response. By configuring data sources, tuning correlation rules, and focusing on high value logs, SOC analysts can create a more efficient and focused response environment. Regularly updating rules and reducing false positives further refines your SOC’s capabilities, allowing you to respond more quickly and accurately to real threats. As you gain experience, refining these configurations and analysis techniques will further enhance the SOC’s detection and response capabilities.