Intrusion Detection and Malware AnalysisSignature-based IDS

Pavel LaskovWilhelm Schickard Institute for Computer Science

Misuse detection systems

Expert systems (NIDES, Emerald): rule-based decisions,rather slow, binary decisions only.Signature matching (Snort, Bro, Cisco Secure IDS, ISSRealSecure): pattern matching, policy scripting.State transitions (STAT suite): description of attacks byassertions over a state transition diagram.Data mining: automatic extraction of attack description rulesfrom labeled examples.Machine learning: automatic extraction of “black-box”classifiers from labeled examples.

Snort highlights

Initial open source release (December 1998):plain sniffer, no rules

1.0 release (June 1999)basic rules language, stateless packet processing

1.5 release (December 1999)packet pipeline architecture used up-to-date

1.8 release developed by Sourcefire (mid-2001):IP fragmentation and TCP reassembly

Current release (2.9, 3.0beta):highly stateful, 3000+ rules, protocol anomaly detection

Snort design criteria

A lightweight intrusion detection toolcross-platform portabilitysmall footprinteasy installation and configuration

A simple language for rulesHigh efficiency and low memory and CPU consumption

packet-level detection (with no support for TCP streamreassembly in early versions)packet filtering using BPF and rule hierarchy

An open source alternative to expensive commercial IDS

Snort architecture2. Then it sends them through a chute to determine if they are coins and how they should

roll (the preprocessor).

3. Next, it sorts the coins according to the coin type.This is for storage of quarters, nickels,dimes, and pennies (on the IDS this is the detection engine).

4. Finally, it is the administrator’s task to decide what to do with the coins—usually you’ll rollthem and store them (logging and database storage).

Figure 4.1 Snort Architecture

The preprocessor, the detection engine, and the alert components of Snort are all plug-ins. Plug-ins are programs that are written to conform to Snort’s plug-in API.These programs used to be partof the core Snort code, but they were separated to make modifications to the core source code morereliable and easier to accomplish.

Packet SnifferA packet sniffer is a device (either hardware or software) used to tap into networks. It works in a sim-ilar fashion to a telephone wiretap, but it’s used for data networks instead of voice networks.A net-work sniffer allows an application or a hardware device to eavesdrop on data network traffic. In thecase of the Internet, this usually consists of IP traffic, but in local LANs and legacy networks, it canbe other protocol suites, such as IPX and AppleTalk traffic.

Because IP traffic consists of many different higher-level protocols (including TCP, UDP, ICMP,routing protocols, and IPSec), many sniffers analyze the various network protocols to interpret thepackets into something human-readable.

Packet sniffers have various uses:

■ Network analysis and troubleshooting

■ Performance analysis and benchmarking

■ Eavesdropping for clear-text passwords and other interesting tidbits of data

Encrypting your network traffic can prevent people from being able to sniff your packets intosomething readable. Like any network tool, packet sniffers can be used for good and evil.

www.syngress.com

Introducing Snort • Chapter 4 189

PreprocessorSniffer DetectionEngine

Alerts/Logging

Rulesets

Network Backbone

PacketsLog Files/Database

441_HTC_OS_04.qxd 4/12/07 9:45 AM Page 189

Packet sniffer interacts directly with a network card usinglibpcap.Preprocessing, detection and alert components areimplemented as plugins.Various front-ends are available for logging (DB, Preludemeta-IDS, GUI).

Snort sniffer

Operates in promiscuous mode: passes all traffic to OS.Performs basic packet filtering using BPF.Decodes packet headers using pointer casts.typedef struct _EtherHdr {

u_int8_t ether_dst[6];

u_int8_t ether_src[6];

u_int16_t ether_type;

} EtherHdr;

/* lay the ethernet structure over the packet data */

p->eh = (EtherHdr *) pkt;

Snort preprocessors

Plugin architecture enablesdynamic plugin configuration.Preprocessor functions:

Stream reassembly (stream5)Packet defragmentation (frag3)Protocol decoding/normalization(HTTP, RPC, telnet, etc.)Alternative (non-rule) detectionmodes (portscan, arpspoof)

Plugin API enables development ofcustom preprocessor plugins.

Figure 4.3 Snort’s Preprocessor

Earlier in this chapter, we described Snort as a signature-based IDS.The signature-based IDSfunction is accomplished by using various rulesets.The rulesets are grouped by category (Trojanhorses, buffer overflows, access to various applications) and are updated regularly.

The rules themselves consist of two parts:

■ The rule header The rule header is basically the action to take (log or alert), type ofnetwork packet (TCP, UDP, ICMP, and so forth), source and destination IP addresses, andports

■ The rule option The option is the content in the packet that should make the packetmatch the rule.

The detection engine and its rules are the largest portion (and steepest learning curve) of newinformation to learn and understand with Snort. Snort has a particular syntax that it uses with itsrules. Rule syntax can involve the type of protocol, the content, the length, the header, and other var-ious elements, including garbage characters for defining butter overflow rules.

Once you get it working and learn how to write Snort rules, you can fine-tune and customizeSnort’s IDS functionality.You can define rules that are particular to your environment and customizehowever you want.

The detection engine is the part of the coin sorter that actually rolls the coins based on the type.The most common American coins are the quarter, dime, nickel, and penny. However, you might geta coin that doesn’t match, like the Kennedy half-dollar, and discard it.This is illustrated in Figure 4.4.

www.syngress.com

Introducing Snort • Chapter 4 191

Preprocessor Detection Engine

Packets

HHTP Encoding Plug-in

Port Scanning Plug-in

441_HTC_OS_04.qxd 4/12/07 9:45 AM Page 191

Snort detection engine

Rules are parsed into an internaldata structure.Rule matching is prioritizedaccording to matching complexity:

IP header rulesTCP header rulesApplication protocol header rulesContent rules

Multiple matches are possible: thehighest priority alert is reported.

Figure 4.4 Snort’s Detection Engine

Alerting/Logging ComponentAfter the Snort data goes through the detection engine, it needs to go out somewhere. If the datamatches a rule in the detection engine, an alert is triggered.Alerts can be sent to a log file, through anetwork connection, through UNIX sockets or Windows Popup (SMB), or SNMP traps.The alertscan also be stored in an SQL database such as MySQL and Postgres.

You can also use additional tools with Snort, including various plug-ins for Perl, PHP, and Webservers to display the logs through a Web interface. Logs are stored in either text files (by default in/var/log/snort) or in a database such as MySQL and Postgres.

Like the detection engine and the preprocessor, the alert component uses plug-ins to send thealerts to databases and through networking protocols such as SNMP traps and WinPopup messages.See Figure 4.5 for an illustration of how this works.

Additionally, with syslog tools such as Swatch, Snort alerts can be sent via e-mail to notify asystem administrator in real time so no one has to monitor the Snort output all day and night.

Table 4.1 lists a few examples of various useful third-party programs and tools.

www.syngress.com

192 Chapter 4 • Introducing Snort

Detection Engine

Packets

Rule

PacketsMatch?

No

Discard

If Yes, Send to Logging/Alerting

Logging/Alert

Do the

441_HTC_OS_04.qxd 4/12/07 9:45 AM Page 192

Snort logging

Various output modules are provided for handling alerts, e.g.:

Writing to a log fileSending message to a syslog facilityLogging to a database like MySQL or OracleGenerating an XML outputModifying configuration on routers and firewallsSending SMB messages to Windows machines

GUI tools (ACID, SnortSnarf, Barnyard, SGUIL) are available formanaging alerts.

Snort rules

General format: header (options)Header:

fixed formatpresent in every rule

Options:variable formatnot always necessary

Example:alert tcp $BAD any -> $GOOD any

(flags: SF; msg �SYN-FIN scan�;)

Snort rules: header

General format: action proto srcaddr srcport dir dstaddr dstport

Example: alert tcp $BAD any -> $GOOD anyActions

alert, pass, log

Protocolstcp, udp, icmp, ip

Directions-> (unidirectional), (bidirectional)

Variables, wildcards and expressions can be used, e.g.!$HOME, any, etc.

Snort rules: options

General format: ( keyword: value; )Example: (flags: SF; msg �SYN-FIN scan�;)Basic options:

content: pattern matcherpcre: Perl-compatible regular expressionmsg: alert messageflow: test for TCP connection state, traffic direction

Content options can be combined using the conjunction.

Snort rules: advanced options

Informational optionssid (Snort ID), priority, ref (reference, usually to CVE’s)

Advanced payload optionsbyte_jump, byte_test, distance, within, depth, offset, nocase,rawbytes, uricontent

Protocol analysis optionsKeywords for IP, TCP, ICMP protocols

Stateful rulesflowbits, threshold, flow

Regular expressionsuse with care: expensive!

Snort rule examples

alert tcp any any -> any any \

(flow: established, to_server; \

content: "foo"; msg: "detected foo";)

alert tcp $EXTERNAL_NET any -> $SMTP_SERVERS 25 \

(msg:"SMTP exchange mime DOS"; flow:to_server,established; \

content:"charset = |22 22|"; nocase; reference:bugtraq,1869; \

reference:cve,2000-1006; reference:nessus,10558; \

reference:url,www.microsoft.com/technet/security/bulletin/MS00-082.mspx; \

classtype:attempted-dos; sid:658; rev:11;)

alert tcp $EXTERNAL_NET 80 -> $HOME_NET any \

(msg:"EXPLOIT Netscape 4.7 client overflow"; flow:to_client,established; \

content:"3|C9 B1 10|?|E9 06|Q

Snort summary

A de-facto standard IDS in the practical security communityMore than 3,000,000 downloadsAbout 200,000 users

A light-weight, easily configurable IDSGood performance and reliabilityModerate expressivity of rule languageNumerous appliances available

Database, logging and alert interfacesGUI toolsIntrusion prevention and firewall interfacesShared object rules: more complex functionality

Bro highlights

Introduction of Bro at the USENIX Security Symposium(January 1998):

TCP and fragment reassembly, scripting language, initial set ofapplication-level protocol analyzers

First public release available for download:extended set of protocol analyzers including HTTP, SMTP

Protocol parser binpac (October 2005):generic protocol analysis tool, support for further protocols(SSL, RPC, NetBIOS)

Dynamic protocol detection (October 2006):services running at unusual ports can be detected

Broccoli: the Bro client communication library:event management for distrubuted intrusion detection

Bro design criteria

Handling of large volumes of datadeployment on a high-speed link of up to 4Gb/s

Separation of detection and policy mechanismsnecessity to adjust policy-based response to traffic volumessound software design

Extensibiltyadjustment to new services and kinds of attacks

Robustnessno packet drops especially in the presence of attacks

Bro architecture

Snifferlibpcap with standard BPF filters

Event engineTCP stream and fragment reassemblyGenerates meta-events for TCP and UDPprotocols (e.g. connection_attempt,udp_request, udp_reply, etc.)Application-specific events

Policy script interpreterEvent handlers are written in a specializedBro language: much more powerful thanSnort signatures.Policy scripts are compiled: efficiency gain!

Network

Event stream

Record to disk

Real-time notification

Filtered packet stream

Event Engine

Policy script

Event control

Tcpdump filter

Packet stream

Policy Script Interpreter

libpcap

Figure 1: Structure of the Bro system

Bro does for six Internet applications: FTP, Finger, Portmap-per, Ident, Telnet and Rlogin.

�7 gives the status of the im-

plementation and our experiences with it, including a briefassessment of its performance.

�8 offers some thoughts on

future directions. Finally, an Appendix illustrates how thedifferent elements of the system come together for monitor-ing Finger traffic.

2 Structure of the system

Bro is conceptually divided into an “event engine” that re-duces a stream of (filtered) packets to a stream of higher-levelnetwork events, and an interpreter for a specialized languagethat is used to express a site's security policy. More gener-ally, the system is structured in layers, as shown in Figure 1.The lower-most layers process the greatest volume of data,and hence must limit the work performed to a minimum. Aswe go higher up through the layers, the data stream dimin-ishes, allowing for more processing per data item. This ba-sic design reflects the need to conserve processing as muchas possible, in order to meet the goals of monitoring high-speed, large volume traffic flows without dropping packets.

2.1 libpcap

From the perspective of the rest of the system, just above thenetwork itself is libpcap [MLJ94], the packet-capture li-

brary used by tcpdump [JLM89]. Using libpcap gainssignificant advantages: it isolates Bro from details of thenetwork link technology (Ethernet, FDDI, SLIP, etc.); itgreatly aids in porting Bro to different Unix variants (whichalso makes it easier to upgrade to faster hardware as it be-comes available); and it means that Bro can also operateon tcpdump save files, making off-line development andanalysis easy.

Another major advantage of libpcap is that if the hostoperating system provides a sufficiently powerful kernelpacket filter, such as BPF [MJ93], then libpcap down-loads the filter used to reduce the traffic into the kernel. Con-sequently, rather than having to haul every packet up to user-level merely so the majority can be discarded (if the filteraccepts only a small proportion of the traffic), the rejectedpackets can instead be discarded in the kernel, without suf-fering a context switch or data copying. Winnowing downthe packet stream as soon as possible greatly abets monitor-ing at high speeds without losing packets.

The key to packet filtering is, of course, judicious selec-tion of which packets to keep and which to discard. For theapplication protocols that Bro knows about, it captures everypacket, so it can analyze how the application is being used.In tcpdump's filtering language, this looks like:

port finger or port ftp or tcp port 113 orport telnet or port login or port 111

That is, the filter accepts any TCP packets with a sourceor destination port of 79 (Finger), 21 (FTP), 113 (Ident),23 (Telnet), 513 (Rlogin), and any TCP or UDP packets witha source or destination port of 111 (Portmapper). In addition,Bro uses:

tcp[13] & 7 != 0

to capture any TCP packets with the SYN, FIN, or RST con-trol bits set. These packets delimit the beginning (SYN) andend (FIN or RST) of each TCP connection. Because TCP/IPpacket headers contain considerable information about eachTCP connection, from just these control packets one canextract connection start time, duration, participating hosts,ports (and hence, generally, the application protocol), and thenumber of bytes sent in each direction. Thus, by capturingon the order of only 4 packets (the two initial SYN packetsexchanged, and the final two FIN packets exchanged), wecan determine a great deal about a connection even thoughwe filter out all of its data packets.

The final filter we use is:

ip[6:2] & 0x3fff != 0

which captures IP fragments, necessary for sound trafficanalysis, and also to protect against particular attacks on themonitoring system

�5.3.

When using a packet filter, one must also choose a snap-shot length, which determines how much of each packetshould be captured. For example, by default tcpdump uses

3

Bro language

Strongly typeddetection of type inconsistensies at compile timeguaranteed validity of variable values at run-time

Domain-specific features:host namesIP addressesport numbers

Bro language: atomic data types

Traditional data types:bool, int, count (unsigned int), double, string (with lengthchecking)

Timing data types:timeinterval

Networking data types:port (e.g. http, 80/tcp)addrhostname (a list of possible addresses corresponding to a hostname)

Bro language: aggregate data types

Record: a collection of elements of arbitrary typestype conn_id: record {

orig_h: addr;

orig_p: port;

resp_h: addr;

resp_p: port;

};

Record fields are accessed using a $ operator:conn_id$orig_h

Table: a set of elements indexed by an atomic type or arecord containing only atomic types:table[conn_id] of ftp_session_info

A set is a table whose indexation does not yield a value (butallows membership queries).

Bro language: operators

C-like operators:+, -, *, /, %, !, &&, ||, ?:,

Example Bro policy script

const worm_types: table[string] of pattern = {

["Code Red 1"] = /\.id[aq]\?.*NNNNNNNNNNNNN/,

["Code Red 2"] = /\.id[aq]\?.*XXXXXXXXXXXXX/,

["Nimda"] = /\/scripts\/root\.exe\?\/c\+tftp/ |

/\/MSADC\/root.exe\?\/c\+dir/ |

/cool\.dll.*httpodbc\.dll/, # 29Oct01 Nimda variant

} &redef;

event http_request(c: connection, method: string,

original_URI: string, unescaped_URI: string,

version: string) {

for ( wt in worm_types )

if ( worm_types[wt] in unescaped_URI )

event worm_instance(c, wt);

}

Bro signature language

Similar syntax with Snort rulesAdditional features:

Analyzer-specific content conditions:http-request-header /regular expression/

Dependency conditions:requires-signature [id]

Context conditions:eval policy_function

Snort rules can be converted to Bro rules using snort2bro

Bro conclusions

The leading research community IDSOver 10 years development (the longest existing IDS project)Extensive operation experience in LBNL / Berkeley

High performanceDistributed operation mode available via Broccoli

Elegant multi-tier designPowerful signature and policy scripting languageExtensive support for application layer protocolsFurther info and download: www.bro-ids.org

Recommended reading

T. Kohlenberg, editor.Snort IDS and IPS toolkot.Syngress Publishing, 2007.

R. Pang, V. Paxson, R. Sommer, and L.L. Peterson.binpac: a yacc for writing application protocol parsers.In Proc. of ACM Internet Measurement Conference, pages 289–300, 2006.

V. Paxson.Bro: A system for detecting network intruders in real-time.Computer Networks, 31(23–24):2435–2466, December 1999.

M. Roesch.Snort: Lightweight intrusion detection for networks.In Proc. of USENIX Large Installation System Administration ConferenceLISA, pages 229–238, 1999.