distributed intrusion detectionintrusion detection with

VASCAN ConferenceVASCAN ConferenceVASCAN ConferenceVASCAN Conference

October 21, 2010October 21, 2010October 21, 2010October 21, 2010

Information Technology Security Office

Distributed Intrusion DetectionIntrusion DetectionIntrusion DetectionIntrusion Detection

with Open Source SoftwareOpen Source SoftwareOpen Source SoftwareOpen Source Software

and Commodity HardwareCommodity HardwareCommodity HardwareCommodity Hardware

Will Urbanski

[email protected]

Philip Kobezak

[email protected]

++++

++++

++++

++++

Distributed Intrusion Detection with Open Source Software and Commodity Hardware

Information Technology Security Office 2

• High IPS maintenance costs

• Wanted more distributed view

• Had never put IPS in-line

• Wanted IPv6 support

• Wanted root access to components

for troubleshooting

• Wanted standard or common

hardware for compatibility and

maintenance

The Start of the ProjectThe Start of the ProjectThe Start of the ProjectThe Start of the Project



Concept of What We WantedConcept of What We WantedConcept of What We WantedConcept of What We Wanted

• Commodity hardware

• Multiple distributed sensors

• Open source software

• Open data formats

• For our own tools

• Low initial and ongoing cost

• Sold network group on access

to sensors



Network TopologyNetwork TopologyNetwork TopologyNetwork Topology



Hardware: Sensor DesignHardware: Sensor DesignHardware: Sensor DesignHardware: Sensor Design

• Kept under $700 each

• Dual port NIC for monitoring

• Original plan to use fiber taps -

switched to copper

• Dual Core, 4GB RAM

• Small HD

• On motherboard NIC



Hardware: Sensor DesignHardware: Sensor DesignHardware: Sensor DesignHardware: Sensor Design

Adapter Model Connector Cabling Slot Type Est. Price

EXPX9502AFXSR LC Fiber $2,500.00

EXPX9501AFXSR LC Fiber $1,500.00

E10G42AFDA $600.00

EXPX9501AT RJ45 Copper $900.00

EXPI9402PF LC Fiber $700.00

EXPI9400PF LC Fiber $500.00

EXPI9402PT RJ-45 Copper Cat5 up to 100m $163.00

10 Gigabit XF SR Dual Port

MMF 62.5/50 µm up to 300m

PCIe 2.0 x 8 lanes

10 Gigabit XF SR MMF 62.5/50 µm up to 300m

PCIe 2.0 x 8 lanes

10 Gigabit AF DA Dual Port

SFP+ Direct Attach Copper

SFP+ Direct Attach Cable up to 15m

PCIe 2.0 x 8 lanes

10 Gigabit AT Cat6 up to 55mCat6A up to 100m

PCIe 2.0 x 8 lanes

Pro/1000 PF Dual Port

MMF 62.5/50 µm up to 275m

PCIe 2.0 x 4 lanes

Pro/1000 PF MMF 62.5/50 µm up to 275m

PCIe 2.0 x 4 lanes

Pro/1000 PTDual Port

PCIe 2.0 x 4 lanes

Partial Listing of 1 and 10 Gigabit Interfaces from Intel


We use FreeBSD 8.0 64-bit

Why not Linux?

• K.I.S.S.

• Sensors run a ‘minimal’ FreeBSD install

• FreeBSD natively supports DMA between the NIC and the Kernel

• Kernel module via NTOP’s PF-RING

• Phil Wood’s libpcap implementation

Sensor DesignSensor DesignSensor DesignSensor Design



Combined IDS software configs into logical packages called snort instances

An instance contains:

• Rulesets (VRT, ET, or custom rules)

• Configurations for Snort and other IDS tools

System ArchitectureSystem ArchitectureSystem ArchitectureSystem Architecture



Snort

Daemonlogger

Barnyard2

Instance SoftwareInstance SoftwareInstance SoftwareInstance Software



“Only show IPv4 traffic going to my database servers”

Physical NIC Daemonlogger Virtual NIC

Snort Instance WorkflowSnort Instance WorkflowSnort Instance WorkflowSnort Instance Workflow



“Identify DB

attacks, brute force

attempts, and

network recon”

Virtual NIC Snort RAMDISK



RAMDISK

Save alerts to DB

Barnyard2 MySQL



Granularity

• Monitor for specific attack types against specific services, on specific

machines.

• Care less about viruses in student dorms

• Care more about PII leaked from misconfigured systems

Performance

Why use snort instances?Why use snort instances?Why use snort instances?Why use snort instances?



Granularity

Performance

• Running Snort on the physical NIC results in a large number of dropped

packets (60%+)

• unless you run a very very very very small number of rules

• Snort may be configured to look for attacks against web services only

but still sees P2P, streaming media, email traffic, etc

• Through the use of a snort instance we limit the traffic snort must

process.

• The fewer packets there are to process, the fewer packets there are

to drop

Why use snort instances?Why use snort instances?Why use snort instances?Why use snort instances?



Scale Up!Scale Up!Scale Up!Scale Up!



Average CPU usage per application per snort instance:

• Snort: 50% - 60%

• Daemonlogger: 20% - 25%

• Barnyard: < 1%

Because of this we can easily run one snort instance per core, without increasing

the load on the system to unacceptable levels.

Scale Up!Scale Up!Scale Up!Scale Up!



Two additional servers required for deployment:

• Database server for storing alerts

• Management server for pushing rules and monitoring sensors

DeploymentDeploymentDeploymentDeployment



Beefy physical machine:

• Multicore, running MySQL server

• Big Drives:

146GB for OS

1TB SAS drives in RAID10 for storage

Since June 1, 2010, we’ve recorded 22 million alerts.

Database ServerDatabase ServerDatabase ServerDatabase Server



• Rule management with Oinkmaster

• Manages and automatically configures rulesets

• Configuration propagation

• Configuration files propagated via secure copy.

• Monitoring

• Uptime monitored by NAGIOS

• Analytics and Reporting

• Alert management and reporting provided by BASE

Management ServerManagement ServerManagement ServerManagement Server




SummarySummarySummarySummary

ProsProsProsPros

• Minimal cost to implement

• No recurring annual costs

• Easy access to IDS data

• Easier to upgrade at a later date

• We are ready for IPv6 support

Cons

• Requires expertise and many

person-hours

• Must manually maintain software

updates

• Waiting on BY2 IPv6 support



Questions?Questions?Questions?Questions?

Philip Kobezak IT Security Analyst

[email protected]

Will UrbanskiIT Security Analyst

[email protected]

www.security.vt.eduwww.security.vt.eduwww.security.vt.eduwww.security.vt.edu

Contact Information:Contact Information:Contact Information:Contact Information:

Randy MarchanyIT Security Officer

[email protected]

distributed intrusion detectionintrusion detection with

Documents