IMPROVE SITUATIONAL AWARENESS BY INTEGRATING SIEM AND NETFLOWHP Enterprise Security Business Whitepaper

IntroductionA network administrator sees a sudden traffic spike to several international domains outside their network. After some time investigating, she discovers that a new machine was added to the network as a guest email and Web browsing station.

A sudden increase of network traffic hits the organization’s Web servers. There is also a report of a new botnet being used to initiate attacks across the globe. The Web and network administrators spend all morning analyzing the traffic to discover that a new press release is causing users to access Web pages and documents from the company website.

A data breach is discovered by the network and database administrators. After a lengthy investigation, they find a steady but low volume of network traffic to the main customer database from an internal IP address. However, this is a dynamic IP address and they don’t know which machines or users were actually using it during the time of the breach.

Historically, network monitoring using NetFlow analysis and security monitoring with security information and event management (SIEM) technology have been the responsibility of different domains. However, organizations are increasing their reliance on IT systems, and are opening more internal systems to the Internet. At the same time, malicious threats are increasing in frequency and sophistication. To understand and counter them, IT administrators must rely on both network and security information.

This whitepaper describes the various technologies available for network and security monitoring, and benefits administrators can reap by combining security and network monitoring information into a single SIEM solution.

What is NetFlow?NetFlow is a network protocol for collecting Internet Protocol (IP) traffic information from networks. Routers that have the NetFlow feature activated generate NetFlow records, which are exported from the router as UDP or SCTP packets. Each flow record contains information about the source and destination address/port of the traffic, the amount of traffic, the protocol used for communication, as well as timestamps and routing information. These NetFlow records can be aggregated by NetFlow collectors. By analyzing the aggregated flow data, network administrators can build a picture of traffic flow and volume on a network.

NetFlow monitoring and analysis allows network administrators to understand how network bandwidth is being consumed, to identify and fix network problems and to optimize low-bandwidth but critical traffic links (e.g., expensive WAN connections to remote sites). However, while NetFlow monitoring can tell administrators about the performance health of their environment, it does not provide information about the security stance of those expensive and critical resources.

NetFlow and Security InformationArcSight Dramatically Enhances NetFlow AnalysisNetwork monitoring information alone is not enough to fully understand the activities occurring across your organization. For a complete picture, network activity must be correlated with server, access and security information. By correlating security event and network information, analysts are able to reduce false positives, prioritize their workload more efficiently, fix performance and security incidents faster and easily demonstrate compliance to auditors.

ArcSight Express combines the ArcSight award-winning, real-time, multi-vector correlation capability with market-leading NetFlow collection, normalization and categorization. ArcSight Express is able to collect NetFlow information directly from network devices via its own NetFlow connector or through integration with third party NetFlow analysis technologies. It then processes this data by normalizing the fields and adding important contextual data necessary for effective correlation, without any time lag. In the ArcSight correlation engine, hundreds of in-memory rules are used to correlate Netflow data with real-time security, server and user information. The result is drastically reduced false positives and false negatives, and an extremely accurate prioritized incident list.

ArcSight offers the industry’s leading collection and correlation technology and couples it with the highest performing NetFlow collection and analysis capability. By combining network data with security events, analysts are able to piece together a more accurate picture that goes beyond the infrastructure. Security analysts are able to understand the impact of network activity on the organization’s servers, and network analysts gain valuable root cause information that helps them eliminate network issues more effectively.

The combination of NetFlow and SIEM data also allows administrators to understand application activity, user behavior and business transactions. ArcSight Express employs multiple techniques to allow administrators to track application sessions. Its ability to correlate flow data with application logs, vulnerability information, compliance context, historical attack data and detailed asset information brings visibility to the critical incidents hidden in the millions of network events generated every day.

Example ScenariosCombining SIEM log events and NetFlow data improves the accuracy of prioritized events to better cut through the noise of low priority network “chatter.” The event also has better context and more detail, improving the effectiveness and response of IT administrators. Following are several scenarios that illustrate how integrating network and security information can improve overall security.

2

3

NetFlow Correlation – Interesting TrafficA machine on the internal network suddenly starts communicating with the outside world. A network monitoring utility that reads only NetFlow data may identify this activity as a threat, and would require a network administrator to perform additional analysis to determine the intent behind the traffic spike.

By tying in security information, ArcSight Express can determine that this activity is not a threat at all, but simply an internal user performing benign Web searches and then using normal online resources. This is an example of a false positive. Looking only at the variation and frequency of this activity can cause it to be incorrectly categorized as a hot item, thus wasting the limited time and resources of IT administrators. However, referencing it against a malicious website list, or one of several others provided by ArcSight Express, would have quickly identified it as harmless activity.

Cross-Device Correlation – Malicious Inbound TrafficNetwork traffic from various external sources to the organization’s Web servers suddenly spikes to four times the normal amount. This will trigger alarms in a network monitoring utility relying solely on NetFlow information. There is no way to evaluate the intent of this traffic.

However, if SIEM data is correlated with the traffic spike, determining the intent becomes easy. SIEM data can include botnets and malicious machines, lists of IP addresses that may have scanned or attacked the organization in the past and geographic information about the source of the network traffic. Using this information in conjunction with the network spike, it becomes easy to determine if the incident is malicious (e.g., a DDOS attack) or harmless (e.g., interested users responding to a new company announcement).

Multi-stage Correlation – Advanced Persistent ThreatAn advanced persistent threat (APT) is a set of techniques that sophisticated, determined and coordinated attackers have been using to systematically compromise U.S. government and commercial computer networks. The techniques are identified by a series of several steps, starting with reconnaissance and initial network intrusion; establishing backdoors, user credentials and utilities; extracting the critical data or causing the intended damage; and finally, maintaining persistence in the compromised network. APT cannot be discovered by NetFlow monitoring alone because of the extended period of time involved and the use of multiple different technologies.

SIEM becomes crucial in detecting APT because these attacks hide in regular network traffic such as HTTP and HTTPS protocols. APT traffic also avoids detection by not relying on inbound network connections, which are easily detected by network monitoring utilities. The key to detecting APT is to observe several different types of network, security, host and process indicators over an extended period of time, and then alert on the combination of these seemingly unrelated events. Relying on network monitoring data alone leads to these threats persisting in the target network and causing irreparable damage over time.

Figure 1: NetFlow Details Display screen, monitoring flow data in ArcSight Express

By combining network data with security events, analysts are able to piece together a more accurate picture that goes beyond the infrastructure.

Figure 2: Interesting traffic to multiple hosts and ports from internal machine

Session Correlation – User Activity TrackingNetwork data contains traffic flow information from source to destination IP addresses. However, because of the dynamic nature of IP addresses and assignment, it is difficult to understand who did what. For example, how does an analyst determine the difference between a flow showing traffic from a particular IP address at 9:00 a.m. vs. traffic from the same IP address 10 minutes later? The traffic could originate from different users, or even completely different machines.

Network and IP address information alone cannot determine the activities of any particular user or machine. However, by using session tracking, an administrator can analyze dynamically changing information. An efficient SIEM system can track DHCP lease allocations and logon/logoff data to understand the information beyond network data alone. SIEM dynamic lists coupled with source/destination in NetFlow records are used to track sessions automatically. As a result, analysts no longer need to piece the story together manually.

Historical Correlation – Low and Slow AttacksInformation security threats are becoming increasingly sophisticated. Most network and security incidents now involve several source machines (typically botnets) and extended planning, testing, and execution on the part of the attackers. This extended activity is performed over weeks or months, and can be picked up by an effective SIEM solution, but not by a network monitoring tool.

Visibility into both the short term NetFlow data and longer term historical behavioral analysis on the network is crucial to anticipating an attack before it happens. SIEM correlation is able to bring together the various unrelated events from networking and security infrastructure, and flag the combination as a potential incident. When observing this activity at the network level, typing it to the “low and slow detection” correlation rules can quickly surface it as a malicious event.

Vulnerability Correlation – Asset AwarenessAn internal server is the target of a virus attack. This incident is a minor blip on the network monitoring radar because the virus produces only a small amount of network traffic on the server. However, the impact of a virus that has access to the file system, privileged user accounts or other resources on a server can be disastrous.

SIEM systems can tie any source and destination network activity with asset information to provide the true impact. By correlating the vulnerability assessment data for this machine, an analyst can quickly figure out if the server needs immediate attention, or if the virus attack will fail because the server has the necessary protection applied to it. SIEM data also utilizes other information about the assets, including its importance, patch levels, zone/location in the network and any compliance associations – bringing true asset correlation to the analysis.

ArcSight Express NetFlow ArchitectureArcSight Express combines the industry-leading collection, correlation and log management capability with robust NetFlow collection and correlation to bring security and network monitoring together. With ArcSight Express, organizations can easily detect problems by consolidating security and network monitoring information to one central location. ArcSight Express comes with the necessary components to collect and correlate NetFlow and security information. It includes built-in, standards-based security and compliance best practices in the form of rules, alerts, reports and dashboards that are focused on the common areas of risk.

•ArcSightConnectors:ArcSight Connectors provide universal data collection from over 300 unique devices without the need to deploy agents across the enterprise. The ArcSight NetFlow connector collects flow data from routers and switches and transports it securely to prevent data loss. The ArcSight NetFlow connector also provides up to 60% improvement in collection efficiency through aggregation of flow data, while retaining litigation quality of the data. The data is normalized and categorized into the ArcSight Common Event Format (CEF) for easy correlation and analysis. The ArcSight Connector architecture enables “future proof” monitoring, as the system will continue to work even when network technologies are swapped out and replaced with those from new vendors.

•EfficientandSelf-ManagingStorage,andComplianceReporting:ArcSight Express includes the functionality of ArcSight Logger, a complete and efficient, market-leading log management solution for long-term data retention. Compliance mandates require companies to retain data for long periods of time, making storage efficiency and compression format very important. NetFlow monitoring generates high volumes of log data and can easily overwhelm an organization’s storage infrastructure. Demonstrating compliance also means the data must be readily available for fast searching, reporting and analysis. With ArcSight Logger, enterprises are able to efficiently collect, centrally store and provide interactive search and analysis capabilities across all log data. ArcSight Logger also allows easy compliance reporting, historical data analysis and fast forensic search. ArcSight Logger delivers search rates in the range of millions of events per second, without any significant trade-off on collection rates or storage efficiency.

•EnterpriseEventCorrelation:ArcSight uses the NetFlow and security event information from every system and device and provides an extremely accurate prioritized list of issues and alerts. It uses a number of different correlation techniques to reduce both false positives that reduce analyst efficiency, and false negatives that if unnoticed may become a major security incident. With the correlation capability of ArcSight Express, analysts are able to focus on the events that matter most. It allows security, compliance and anti-fraud personnel to quickly identify and prevent network and security attacks from a single, unified correlation and analysis console.

4

5

•Regulation-SpecificApplications:ArcSight Express enables faster compliance through the use of pre-built regulation specific compliance applications. The reports necessary for audits for a variety of standards and mandates (SOX, HIPAA, PCI, NIST, FISMA) are built in to the product in a simple, easy-to-read fashion. Administrators don’t have to spend days and weeks merging data from several different sources so auditors can make use of it. Furthermore, ArcSight is well positioned for any mandates that may be passed in the future. This means organizations can satisfy audit requirements faster and cheaper than before.

•Rules-BasedResponse:ArcSight Express allows organizations to respond faster to incidents, reducing the impact an event can have on operations. Through the response module, ArcSight Express can create the best threat mitigation plan. Once the plan is approved, administrators can execute on it and document the steps taken. This shrinks the response time to seconds, and provides a documented record for roll-back or IT audits.

Figure 3: NetFlow and event collection architecture for ArcSight Express

SummarySIEM solutions provide centralized security and activity monitoring for organizational activity through broad data collection, correlation and analysis. This crucial information has not been leveraged by network monitoring teams that rely solely on NetFlow information. By doing so, they are unable to understand the complete context of any suspicious network activity and often spend unnecessary time investigating false positives. At the same time malicious network activity that occurs over long periods of time or uses sophisticated attack vectors remains undetected by NetFlow analyzers.

ArcSight has been providing NetFlow collection and analysis capabilities since 2005. ArcSight Express combines network and security information out of the box, and provides the best solution to collect, store, analyze, report and respond to all activity occurring in the enterprise. By combining NetFlow data and security information using ArcSight Express, IT administrators can work more efficiently, reduce the effort spent chasing irrelevant incidents and improve the overall security posture of the organization. Bringing network data and security information together also provides for automated compliance reporting and faster time to resolution through workflow-driven response. In short, ArcSight SIEM with NetFlow increases security, improves compliance and reduces the resources required to do so.

© Copyright 2011 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein.

All other product and company names may be trademarks or registered trademarks of their respective owners.

ESP-BWP033-032810-04, Created August 2011