automated signature and policy generation
DESCRIPTION
Automated Signature and Policy Generation. Douglas S. Reeves. MURI Annual Meeting October 29, 2013. Past Work: NSDMiner. Automated discovery of network service dependencies, based on passive observation of network traffic. Recen t Work: MetaSymploit. Goals for malware analysis - PowerPoint PPT PresentationTRANSCRIPT
Automated Signature and Policy Generation
Douglas S. Reeves
MURI Annual MeetingOctober 29, 2013
2
Past Work: NSDMiner
•Automated discovery of network service dependencies, based on passive observation of network traffic
3
Recent Work: MetaSymploit• Goals for malware analysis
• Faster signature generation (less time from release of exploit to availability of signature)
• High quality signatures + efficient pattern matching
4
• All-in-one framework with built-in components provide rich attack-generation capabilities
• Written in scripting languages (Ruby, Python, PHP…)
• Development / deployment of attacks + variants + combinations much faster and easier than development of patches
Script-Based Attack “Factories”
5
Ex: Metasploit
6
Script Example1. Probe Target
Port scanning, Fingerprinting, etc.
2. Compose Attack PayloadIncludes shellcode, junk, target-specific vul bytes, etc.
3. Send PayloadTrigger vulnerability
4. Post ExploitWait for shellcode to be executed, backdoor channel created, etc.
7
• First system for attack script analysis– Automatic IDS signature generation from source code
• Features– Based on symbolic execution– Only a few minutes to analyze new attack scripts and
generate signatures: “day-one defenses”
MetaSymploit
MetaSymploitAttack Scripts
IDS Signatures
8
Script-based Attack Framework & Scripting Language Interpreter
Symbolic API Extension
Behavior & Constraint Logging
Output API Hooking
Constant Pattern Extracting
Pattern Refining & Consolidating
Pattern Context Deriving
Symbolic Execution Layer (SymExeLayer)
Attack Script
Attack Payloads Behavioral API Calls & Attack Constraints
Signature Generation (SigGen)
Extracted Patterns Pattern Context
IDS Signatures
Symbolically executedArchitecture
9
Symbolic Execution Layer• “Symbolize” APIs related to environment and
dynamic content• Record behavioral APIs and attack branching
conditions• Hook output API to capture the entire attack payload
10
Script ExampleSymbolic APIs:probe_ver()shellcode()rand_alpha()
Behavior & Constraint Logging:probe_ver()sym_ver == 5shellcode() & get_target_ret()
Hook output API:sock.put(payload)
11
• Extract signature patterns for specific attack payload (e.g., constant bytes, length, offset)
• Refine patterns to filter out benign/trivial patterns, avoid duplicates
• Derive semantic context of patterns by analyzing behaviors and constraints
Signature Generation Layer
12
alert tcp any any -> any 617 (
msg:“script: type77.rb (Win), target_version: 5, behavior: probe_version, stack_overflow, JMP to Shellcode with vulnerable_ret_addr";
content:"|e9 38 6c fb ff ff eb f9 ad 32 aa 71|";
pcre:"/[.]{1167}\xe9\x38\x6c\xfb\xff\xff\xeb\xf9\xad\x32\xaa\x71[a-zA-Z]{2917}/";
classtype:shellcode-detect; sid:5000656; )
Example of SignatureLine 23: payload => [ <sym_shellcode, len=sym_integer>,<sym_rand_alpha, len=(1167-sym_integer)>,<"\xe9\x38\x6c\xfb\xff\xff\xeb\xf9\xad\x32\xaa\x71", 12>,<sym_rand_alpha, 2917> ]
red is symbolic value, green is concrete value
13
• We developed a lightweight symbolic execution engine for Ruby– No modification to Ruby interpreter required
• Integrated MetaSymploit into Metasploit Console as a simple command
• Output is Snort rules (signatures)• Gecode/R & HAMPI used as constraint solvers• Currently applied to 10 popular built-in components in
Metasploit: Tcp, Udp, Ftp, Http, Imap, Exe, Seh, Omelet, Egghunter, Brute
Implementation
14
• Tested 548 attack scripts
• Average symbolic execution time:Less than 1 minute
Evaluation: Speed and Completion
Category Num Percentage Manual ModifyAutomatically Executed 509 92.88% NoSymbolic Loop 9 1.64% Avg 10 LOC/per scriptNon-Symbolic-Extended API Call 12 2.19% Avg 3 LOC/per scriptObfuscation & Encryption 13 2.37% Not SupportedMulti-threading 3 0.55% Not Supported
Bug in Scripts 2 0.37% Change 2 LOC
15
Evaluation: Detection Rate
16
Evaluation: Detection Rate
• Tested signatures on 45 Metasploit attack scripts targeting 45 vulnerable applications from exploit-db.com
• Results– 100% detection rate with generated signatures
– 0% false positive rate on “normal” network traffic (collected in our department)
17
Evaluation: Comparison with Public Ruleset
• From 11/2012 Snort ruleset, only 22 out of 45 scripts had corresponding official Snort rules (based on CVE analysis)
Pattern comparison between 53 MetaSymploit generatedrules and 50 official Snort rules for 22 Metasploit attack scripts
18
• Snort ruleset 07/2013 has more rules to cover Metasploit-generated exploits– Including Meterpreter shellcode– Example specific rules: exploit-kit.rulesmalware-tools.rules
• Good news?
Evaluation: Comparison (cont’d)
19
Discussion• Fast, successful, accurate automated signature
generation for scripting-based exploits• Limitations
• Requires source code
• Standard limitations of symbolic execution: loops, path explosion, constraint solvers
• Cannot handle multi-threaded attacks
20
Future Work: Test-Driven Security Policy Generation
• SEAndroid is currently being merged into AOSP–goal is to reduce the attack surface using least-privilege policy
• Challenge: (human) effort required to write suitable MAC policies for a particular platform and applications
21
Current status of SEAndroid Policies• Current policy ruleset is manually written by NSA SEAndroid team
–793 allow rules
• Categorizes apps in a very coarse-grained way for simplicity
• Difficult to adapt rules for new platforms (ex.: The current ruleset breaks “Enforcing” mode for Nexus 7)
• The community often argues whether a new rule is correctly written
22
Proposed Approach Automatically generate MAC policy from functional
tests provided by the developers Not intended to be comprehensive ruleset; instead, a major
head start on creating rules Writing test cases is already an essential step in app
deployment; policy generation is “free” Test cases exercise expected use and correct behavior of an
app System apps and middleware framework are already
equipped with rich tests in AOSP
23
Proposed WorkFlow
JUnit Test Suites
Static Parser SEAndroid Test Runner
JUnit Test Suites Android Middleware
The tested app
Linux Kernel SEAndroid (Audit Mode)
SEAndroid Policy Rule Generator
Semantics of Test Cases
Runtime Behaviors of Middleware/Kernel APIs
Auto-Generated SEAndroid Policy Rules for this App
24
Assumptions• Developers are benign, and conscientiously provide
test cases with high coverage– Should be true for system and platform developers, not
necessarily true for 3rd party application developers
• Generated policies should be sound, won’t be complete, but…– Too many policy rules?
25
Example We processed the test suite of the Gallery app that
invokes Camera functionality to take and store photos The test suite CameraTestRunner contains 3 test
classes (13 test methods) These tests cover all camera activities, including
image storage
26
Example (cont'd) The test code and audit trace logs were analyzed to
generate SEAndroid policy (only partially automated):
–allow gallery3d_app mediaserver:binder call;
–allow gallery3d_app servicemanager:binder call;
–allow gallery3d_app system_server:binder { transfer call };
–allow gallery3d_app media_app:binder { transfer call };
–allow gallery3d_app media_app:fd use;
– …(29 rules generated)
27
Challenges How to distinguish runtime contexts between the
execution of test code and target app How to handle mock / fake / isolated
Content / ContentProvider used in test cases How to aggregate / generalize policy rules derived
from test cases (reduce ruleset size)