Bro: A System for Detecting Network

Intruders in Real-Time

Presented by Abdulmajid , Ameen, Yaseen ,mohammed

Lecturer :DR.Nur Izura

Bro Overview

Bro is an open-source, Unix-based Network Intrusion Detection System (NIDS) that passively monitors network traffic and looks for suspicious activity.

Bro was developed by Vern Paxson in the Network Research Group at Lawrence Berkley National Lab, and by the International Computer Science Institute.

Bro == Big Brother (“watching and taking care of you” and all your actions).

Original Goals

High-speed, large volume monitoring:

No packet filter drops (Low packet loss rate)

Would allow intruder to Avoid the NIDS.

Real-time notification

The monitor will be attacked

Mechanism separate from policy:

Bro Features

Rich Application-Layer Analysis

Custom Scripting Language

Pre-written Policy Scripts

Powerful Signature Matching Facility

Network Traffic Analysis

Detection Followed by Action

Snort Compatibility Support (snort2bro)

Target EnvironmentsBro is intended for use by sites requiring flexible, highly customizable intrusion detection.

Development of bro is primarily driven by research for intrusion detection and traffic analysis

Bro is specifically well-suited for scientific environments

• Extremely useful in networks with liberal (“default allow”) policies

• Supports intrusion prevention schemes

• High-performance on commodity hardware

• Runs on Unix-based systems (e.g., Linux, FreeBSD, MacOS)

• Open-source (BSD license)

Target Environments(con`t)

Bro is designed for use by Unix experts who place a premium on the ability to extend an intrusion detection system with new functionality as needed, which can greatly aid with tracking evolving attacker techniques as well as inevitable changes to a site's environment and security policy requirements.

Some sites may wish to run a commercial IDS as their front-line of defense, and then also run Bro as a way to:

• Verify the results of the commercial IDS / defense-in-depth

• Attain richer forensics capabilities

• Provide policy-checking capabilities not facilitated by the commercial IDS

• Experiment with new approaches and incorporate leading-edge research

Bro Deployment

Structure of the system

• Bro is layered into three major components . It uses the now-standard libpcap packet-capture library developed by LBNL's Network Research Group to filter the packet stream in the kernel for high performance.

• Bro is conceptually divided into an “event engine” that reduces a stream of (filtered) packets to a stream of higher-level network events.

• An interpreter for a specialized language that is used to express a site's security policy.

• The lower-most layers process the greatest volume of data, and hence must limit the work performed to a minimum.

• As we go higher up through the layers, the data stream diminishes, allowing for more processing per data item.

• This basic design reflects the need to conserve processing as much as possible, in order to meet the goals of monitoring high-speed, large volume traffic flows without dropping packets

Bro Architecture

Network

Policy Interpreter

Event Engine

libpcapKernel filters down high-

volume stream via packet capture library.

Distills filtered stream into high-level events.

“Policy Scripts” in Bro Policy language processes event stream ...

Tap network link passively, send up a copy of all traffic.

... and takes action!

10

Architecture

• Taps network link passively, sends up a copy of all network traffic.Network

11

Architecture

• Kernel filters down high-volume stream via standard libpcap packet capture library.

Network

libpcap

Packet Stream

Filtered PacketStream

TcpdumpFilter

12

Architecture

• “Event engine” distills filtered stream into high-level, policy-neutral events reflecting underlying network activity

• E.g., connection_attempt, http_reply, user_logged_in

• These span a range of semantic levels

• Currently about 300 different types

Network

libpcap

Event Engine

Packet Stream

Filtered PacketStream

TcpdumpFilter

EventStream

EventControl

13

Architecture

• “Policy script” processes event stream, incorporates:

• Context from past events

• Site’s particular policies

Network

libpcap

Event Engine

Policy Script Interpreter

Packet Stream

Filtered PacketStream

TcpdumpFilter

EventStream

EventControl

Real-time NotificationRecord To Disk

PolicyScript

14

Architecture

• “Policy script” processes event stream, incorporates:

• Context from past events

• Site’s particular policies

Network

libpcap

Event Engine

Policy Script Interpreter

Packet Stream

Filtered PacketStream

TcpdumpFilter

EventStream

EventControl

Real-time NotificationRecord To Disk

PolicyScript

… and takes action: Records to disk

Generates alerts via syslog or paging

Sends events to other Bro’sExecutes programs as a form of

response

Packet Filtering

•capture via pcap API (libpcap)

Filters expressed in same format as tcpdump(8).Also support off-line traces

port finger or port ftp or (tcp[13] & 7 != 0)

• Using libpcap gains significant advantages: it isolates Bro from details of the network link technology (Ethernet, FDDI, SLIP, etc.Network .

it greatly aids in porting Bro to different Unix variants (which also makes it easier to upgrade to faster hardware as it becomes available). Berkeley Packet Filter (BPF), found on some Unix, it allows libpcap to bring Bro’s network filter into the kernel (filters aren’t in userland!).

FreeBSD has a Zero-copy Buffer Extension, which makes it Bro’s OS of choice!

Event Engine

Written in C++.

Distills filtered stream into high-level, policy-neutral events reflecting underlying network activity!

E.g., connection_attempt, http_reply, user_logged_in, etc.

Protocols supported: 30 protocols

ARP, IP, ICMP, TCP, UDP

DCE-RPC, DNS, FTP, Finger, Gnutella, HTTP, IRC, Ident, NCP, NFS, NTP, NetBIOS, POP3, Portmapper, RPC, Rsh, Rlogin, SMB, SMTP, SSH, SSLCiphers, SunRPC, Telenet

Event Engine, cont.

Those protocol analyzers generate over 300 events!

new_connection(c: connection), new_packet(c: connection, p: pkt_hdr), connection_established(c: connection), tcp_contents(c: connection, is_orig: bool, seq: count, contents: string), icmp_echo_reply(c: connection, icmp: icmp_conn, id: count, seq: count, payload: string), ...

Originally, event engine analyzers were written in C++ Class libraries.

Now BinPAC (Binary Protocol Analyzer Compiler) is used.

binpac generates the C++ code.

Event Engine, cont.• When the packets arrive, the Event Engine checks if the

headers of these packets are well formed or not. Packages with not well formed headers are discarded and events of this problem are created.

• The packets with well formed headers gets a connection state which can be connection_attempt,connection_established, connection_rejected or connection_finished for TCP and for UDP the state is defined as udp_request or udp_reply.

• The packets are then collected to their corresponding connection. These collections use different protocols e.g. HTTP, FTP, SMTP and is therefore handled by the analyzers specialized for each protocol.

• Whether or not the connection are handled by any analyzer depends on which analyzers that are enabled in the system.

Event Engine, cont.

• The analyzer for each protocol then examines the content of the connection.

• When this procedure is done, the Event Engine has created high-level events describing the activity and these are now ready for the Policy Script Interpreter.

• As analyzers observe activity, generate events

• Event = name + typed parameter list

• Queued for asynchronous execution

• All events triggered by a given packet executed prior to processing next packet (Along with timers that expire at that point).

Policy Script Interpreter

• The Policy Script Interpreter takes the high-level events generated by the Event Engine and compares these with the policy scripts in the system.

• The events are sorted in a FIFO list which means the first that comes along are the first that is processes.

• Policy Script Interpreter takes action if it detects any suspicious and dangerous actions or it discards other events not defined in the policy scripts.

• Traffic that seems like attacks but aren’t (false negatives), can be detected at this point, but if the policy scripts are good, this will be minimal.

Policy LanguagePolicy analysis scripts which track network activity written in Bro’s language

Procedural You write event handlers = functions asynchronously executed that don’t return a value).

Strongly typed (to detect inconsistencies at compile-time).

Rich in types:

USUAL : bool, int, count, double, string, time, interval, port, addr, hostname, pattern.

• Domain-specific types, such as addresses, ports, subnets

Aggregates: record, table, set, file.

Real-time communication with other Bro instances and Extensive state management support (Timers; automatic expiration; persistence

Policy LanguagePolicy analysis scripts written in Bro’s language:

Procedural (you provide event-handlers that are executed asynchronously and that don’t return a value).

Strongly typed (to detect inconsistencies at compile-time).

Rich in types:

bool, int, count, double, string, time, interval, port, addr, hostname, pattern.

Aggregates: record, table, set, file.

Flex-like regular expressions built into language (pattern)!

Language, cont.

Operators (+, -, *, /, %, !, &&, ||, ?:, relationals, infix, --, ++).

Static, global & const scoping of variables.•const allowed_services: set[addr, port] = {

•[ftp_serv, [ftp, smtp, 20/tcp]], •[nttp.lbl.gov, nttp]•};

• Statements (if-then-else, when, return, print, add, delete, and function).

•No for loops! But recursion through functions is allowed.

Notices & Alarms

Convention is to express much detection in terms of “notices”

• I.e., “I’ve noticed the following activity”

• Notices do not necessarily convey a problem

• Distributed scripts detect 78 different types of notices

• Scripts provide number of ways to decide when to promote a notice to an “alarm” These can include syslog, sending email, paging

• When you see in a script something like NOTICE([$note=OutboundTFTP, $conn=u, $msg=fmt("outbound TFTP: %s -> %s", src, dst)]);it’s just a call to a function named “NOTICE”, using the language’s mechanism for constructing a record

• Note: Entire mechanism is add-on to language, not built in

Sample Bro Policy

• Using the Bro language, sites can write custom policy scripts to generate alarms on any policy violation.

• For example, if a site only allows external http and mail to a small, controlled lists of hosts, they could do this:

const web_servers = { www.lbl.gov, www.bro-ids.org, };

const mail_servers = { smtp.lbl.gov, smtp2.lbl.gov, };

redef allow_services_to: set[addr, port] += {

[mail_servers, smtp],

[web_servers, http], };

• Bro can then generate an Alarm or even terminate the connection for policy violations:

if ( service !in allow_services) NOTICE([$note=SensitiveConnection, $conn=c,]);

if ( inbound && service in terminate_successful_inbound_service )

terminate_connection(c);

Sample Bro Alarms

FTP buffer overflow attack

Bro in Practical Use• Primary IDS for LBNL/NERSC since 1996

• Primary IDS for SC00-03 conferences

• No specialized hardware needed

• Low cost allows for multiple deployment

• Requirements

• FreeBSD

• Intel platform

• Fiber tap

• Disk space to archive data

Sites using Bro (LBNL)

Use multiple Bros across a 10Gb/s link, where each Bro is only executing a minimal set of polices.

Operational 247 since 1996. Monitors traffic for suspicious behavior or policy violations: incoming/outgoing/internal

In conjunction with blocking routers, Bro acts as a dynamic and intelligent firewall

Blocks access from offending IP addresses

Blocks high risk ports

Blocks known high-risk activity

Terminates connections and/or sends alarms

Locates site policy violations (e.g.: Kazaa and gnutella)

Sites using Bro• IDS controls a blocking router

• IDS blocks dynamically when an intrusion attempt is detected or alerts upon suspicious activity

• Router blocks statically like a firewall “Intrusion Prevention”

November 20, 2003 SC2003, Poenix, AZ

Use of Bro Within NERSC

ESNet

NERSC

Filtering BorderRouter

Network Traffic

Tapped Traffic

Multiple Bro Systems

ACL Insertion

• Real Time Analysis• Redundant Backup

• Test Box• Bulk Traffic Recorder

Multiple IDS

• Snort• Bro Heavyweight Protocol Analysis

• Bro GRID / SSL AnalysisTapped Traffic

• Internal Traffic Bro Monitor• Wireless Network Bro Monitor

Wireless Network

Tapped Traffic

Bro at (NERSC)•Tied into a paging system for on-call security person

•Bro check pointed at set intervals (Reports generated whenever checkpointed)

• Clears out ‘orphaned’ sessions

• Allows for offline data analysis

Data archiving

• Maintain traffic data for about 3 months

• Maintain network

• Automatic ACL injection has very low false positive rate

•At NERSC average about 1 every 6 months

NERSC Network Traffic3 Week Period

Type of Traffic Number of

Connections

Overall Percentage

of Traffic

Bulk Data Transfer 666,529 83.73%

Grid Services 74,178 7.19%

Web Related 288,3754 5.30%

Database 620,1730 .27%

Mail 200,484 .04%

System Services 185,272 .04%

Interactive 116 <.1%

Hardware Support Significant Bro research effort these days focuses on

hardware to accelerate network analysis• Note, quite different problem than fast signature matching!

• Approach #1: Bro Cluster• Set of PCs use inter-Bro coordination for joint analysis

• Scales to x23 performance for 24 nodes (then flattens)

• Approach #2: Shunting• Custom (simple) FPGA provides intelligent front-end

• Per-flow decisions on forward/block/divert through IDS/IPS

• 1 Gbps prototype working, undergoing upgrading

• Future: Multicore Bro• Will require extensive analysis to understand/tune memory

locality & intra-analysis messaging

Bro Future Directions

Grid related technologies

Ability to detect Grid related protocols

X.509 Certificate Analyzer

• SSL Analyzer

• Verify certificates are legitimate

Router Shunting

Primary bottleneck in moving packets into user space

• Leverage router based hardware filtering to analyze “packets of interest”

• Proof of concept demo at SC01-03

1)Utilizing Bro and Juniper router

2) Hardware based BPF to filter traffic

Port Mirroring

External Network

JuniperGigE

Interface Bro

InternalNetwork

Mirrored Traffic

Filter-based Forwarding

Bro

Juniper

GigE Interface

Filtered Traffic

External Network

InternalNetwork

Filter

Pros

Wow, where do we begin?

Separation of event generation and event handling (mechanism vs. policy).

Code is/was written for a hostile environment.

Bro’s language (to write policy scripts) is quite tractable.

Resilience in the presence of flooding(Via connection compressor)

An active community (mailing list: Bro@bro-ids.org).

Reaffirms that open-source solution are possible!

Support for plugging in external analyses

Pros : Bro Noteworthy Features

Broccoli = Bro Client Communications Library

• C interface for external programs to transmit & receive values and events

• E.g., syslog reader sends log messages to Bro

Dynamic adaptation to load

• Via policy/load-levels.bro

Changes current packet filter to activate more/less analysis

User-level state management

• Entries in tables, sets automatically expire T sec. after Creation / last read / last write

• Advanced detection algorithms (Backdoors: protocols running off-port (obviated by DPD)

• )

ConsEncrypted Traffic is Bro’s bane:

Act like a SSL proxy (keep *all* necessary certificates?)

IPv6 ... I think it (finally) made it into development code and asymmetry .

Pretty complex, script-based system

• Requires understanding of the network

• No GUI, just ASCII logs

Only partially documented

• Lacking resources to fully polish the system

Thank You

The End