automated exploit detection in binaries
DESCRIPTION
Automated Exploit Detection in Binaries. Finding exploitable vulnerabilities in binaries. Matt Hargett http://www.clock.org/~matt matt {hizzat} use {dizznot} net Luis Miras http://dwerd.blogspot.com lmiras {hizzat} gmail {dizznot} com. Agenda. Definition Architecture Challenges. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/1.jpg)
Automated Exploit Detection in Binaries
Matt Hargett
http://www.clock.org/~matt
matt {hizzat} use {dizznot} net
Luis Miras
http://dwerd.blogspot.com
lmiras {hizzat} gmail {dizznot} com
Finding exploitable vulnerabilities in binaries
![Page 2: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/2.jpg)
Agenda
• Definition
• Architecture
• Challenges
![Page 3: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/3.jpg)
bugreport
• Set of tests for analysis tools
• Proof Of Concept tool
• Not a product or real-world tool
• Released under GPLv3 draft 2– http://sf.net/projects/bugreport
• Enhancements as issues come up
![Page 4: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/4.jpg)
Why C#?
• Very similar to Java and C++
• Open ECMA standard
• 3 open source implementations
• It has specific features we like– high-speed generics– nullable value types– strong typing– high quality and simple open source tools
![Page 5: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/5.jpg)
Target of Detection
• Many vendors have their own definitions of exploitable bugs.
"depends on what you mean by exploit and by bug"
• Our definition is Out-Of-Bound (OOB) memory write using tainted data.
![Page 6: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/6.jpg)
Out-Of-Bound Write Tests
• C code
• x86 code
• Test
• C# code
![Page 7: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/7.jpg)
bugreport architecture
• Set of tests
• x86 emulator– Other processors will be added later.
• Analysis engine
![Page 8: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/8.jpg)
Challenges
• Branches
• Inter-function Analysis
• Non-Contiguous functions
• Self Modifying code
• Loops
![Page 9: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/9.jpg)
Dealing with Branches
• Known values– Results in one machine state
• Unknown values – Results in two machine states– Constraints are used
• x <= value <= y
![Page 10: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/10.jpg)
Dealing with Branches
• cmp, test, math instructions set flags based on input
• jxx, sbb, etc. instructions act on flags
![Page 11: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/11.jpg)
Dealing with Branches
1: cmp eax, 02: jne 4 3: ret4: cmp eax, 2555: jle 7 6: ret7: ...
![Page 12: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/12.jpg)
Choosing Branches
1. Cheat, take branches (follow jxx, sbb).
2. Randomly pick branches
3. Take all branches (drop through and follow jxx, sbb)
4. Take some branches (drop through and follow jxx, sbb)
![Page 13: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/13.jpg)
Choosing Branches
• Many functions have guards at the entry.
• Guards generally drop through on failure.
• Taking all branches increases code coverage
![Page 14: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/14.jpg)
Dealing with Branches
void main(int argc, char** argv){ if (argc < 2) { exit(-1); }
printf("23c3\n");}
![Page 15: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/15.jpg)
Dealing with Branches
cmp [ebp+argc], 1jg short postGuardpush 0FFFFFFFFh ; statuscall _exit
postGuard:push offset a23c3 ; ”23c3\n"call _printf
![Page 16: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/16.jpg)
Choosing Branches
• Prefix is a tool that randomly took branches.
– Found many bugs for customers.– Produced different results each run.
• Bought by Microsoft and shelved.• Many customers keep old versions
around.• Prefast comes with DDK.
– Does not do interfunction value tracking
![Page 17: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/17.jpg)
Choosing Branches
• Taking all branches results in multiple machine states.
• Taking a branch sets constraints on input.
• These constraints must not be broken.
![Page 18: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/18.jpg)
Dealing with Branchesint getSize(char *ch) { int size = 1; char x = *ch; if (x != 0) { if (x != '\n') { size++; } else { size += 2; } } else { size--; } return size;}
![Page 19: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/19.jpg)
Dealing with Branches
What are the potential states?1. (x <=-1 || x >= 1) && (x != ‘\n’) && (size
== 2)2. (x == ‘\n’) && (size == 3)3. (x == 0) && (size == 0)
Real world code will have many potentialstates.
![Page 20: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/20.jpg)
Inter-function: Top-down
• Start at an export or entry point.
• Traverse code through functions
![Page 21: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/21.jpg)
Inter-function: Top-down
main() {foo(); x(); bar();}foo() {x(); }bar() {y(); }x() {y(); z(); }y() {z(); }z() { return 0; }
// Code omitted for brevity
![Page 22: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/22.jpg)
Inter-function: Top-down
Function Count
main() 0foo() 0bar() 0x() 0y() 0z() 0
![Page 23: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/23.jpg)
Inter-function: Top-down
Function Count
main() 1foo() 0bar() 0x() 0y() 0z() 0
![Page 24: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/24.jpg)
Inter-function: Top-down
Function Count
main() 1foo() 1bar() 0x() 0y() 0z() 0
![Page 25: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/25.jpg)
Inter-function: Top-down
Function Count
main() 1foo() 1bar() 0x() 1y() 0z() 0
![Page 26: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/26.jpg)
Inter-function: Top-down
Function Count
main() 1foo() 1bar() 0x() 1y() 1z() 0
![Page 27: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/27.jpg)
Inter-function: Top-down
Function Count
main() 1foo() 1bar() 0x() 1y() 1z() 1
![Page 28: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/28.jpg)
Inter-function: Top-down
Function Count
main() 1foo() 1bar() 0x() 1y() 1z() 2
![Page 29: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/29.jpg)
Inter-function: Top-down
Function Count
main() 1foo() 1bar() 0x() 2y() 1z() 2
![Page 30: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/30.jpg)
Inter-function: Top-down
Function Count
main() 1foo() 1bar() 0x() 2y() 2z() 2
![Page 31: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/31.jpg)
Inter-function: Top-down
Function Count
main() 1foo() 1bar() 0x() 2y() 2z() 3
![Page 32: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/32.jpg)
Inter-function: Top-down
Function Count
main() 1foo() 1bar() 0x() 2y() 2z() 4
![Page 33: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/33.jpg)
Inter-function: Top-down
Function Count
main() 1foo() 1bar() 1x() 2y() 2z() 4
![Page 34: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/34.jpg)
Inter-function: Top-down
Function Count
main() 1foo() 1bar() 1x() 2y() 3z() 4
![Page 35: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/35.jpg)
Inter-function: Top-down
Function Count
main() 1foo() 1bar() 1x() 2y() 3z() 5
![Page 36: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/36.jpg)
Inter-function: Top-down
• Complexity can explode.
• Very time consuming.
• Hitting the same functions multiple times.
• z() visited 5 times.
• Larger programs can have very large call chains.
• “like playing with a yo-yo in the grand canyon”
![Page 37: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/37.jpg)
Inter-function: Bottom-up
• Describe each function in isolation
• Taint return value
• Store return values for a function based on constraints
• Use it when function call is evaluated
• Creating a machine state diff.
![Page 38: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/38.jpg)
Inter-function: Bottom-up
• With deeply nested calls
• Taint return value
• Requires multiple sweeps
![Page 39: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/39.jpg)
Inter-function: Bottom-up
main() {foo(); x(); bar();}foo() { x(); }bar() { y(); }x() {y(); z(); }y() { z(); }z() { return 0; }
// Code omitted for brevity
![Page 40: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/40.jpg)
Inter-function: Bottom-upPass #1
main() { foo(); x(); bar();}
Done: <None>
![Page 41: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/41.jpg)
Inter-function: Bottom-upPass #1
foo() { x(); }
Done: <None>
![Page 42: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/42.jpg)
Inter-function: Bottom-upPass #1
bar() { y(); }
Done: <None>
![Page 43: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/43.jpg)
Inter-function: Bottom-upPass #1
y() { z(); }
Done: <None>
![Page 44: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/44.jpg)
Inter-function: Bottom-upPass #1
x() { y(); z(); }
Done: <None>
![Page 45: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/45.jpg)
Inter-function: Bottom-upPass #1
z() { return 0; }
Done: z()
![Page 46: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/46.jpg)
Inter-function: Bottom-up
• One pass through call graph seems similar to top-down.
• What is the difference?
• The difference is z() is evaluated as a machine state diff.
• z()’s analysis is cached
![Page 47: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/47.jpg)
Inter-function: Bottom-upPass #2
main() { foo(); x(); bar();}
Done: z()
![Page 48: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/48.jpg)
Inter-function: Bottom-upPass #2
foo() { x(); }
Done: z()
![Page 49: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/49.jpg)
Inter-function: Bottom-upPass #2
bar() { y(); }
Done: z()
![Page 50: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/50.jpg)
Inter-function: Bottom-upPass #2
x() { y(); z(); }
Done: z()
![Page 51: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/51.jpg)
Inter-function: Bottom-upPass #2
y() { z();}
Done: z(), y()
![Page 52: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/52.jpg)
Inter-function: Bottom-up
• At pass #2 z() and y() are cached.
• Each pass caches more functions.
![Page 53: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/53.jpg)
Inter-function: Bottom-upPass #3
Done: z(), y(), x(), bar()
![Page 54: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/54.jpg)
Inter-function: Bottom-upPass #4
Done: z(), y(), x(), bar(), foo()
![Page 55: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/55.jpg)
Inter-function: Bottom-upPass #5
Done: z(), y(), x(), bar(), foo(), main()
![Page 56: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/56.jpg)
Inter-function: Bottom-up
• This method took 5 passes.• Less passes can be achieved by starting
at the bottom. • Optimizations can include:
– Leaf nodes first– Functions with low calls and high xrefs
• Ideally Top-down and Bottom-up are combined.
![Page 57: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/57.jpg)
Non-Contiguous functions
• Modern compilers commonly make non-contiguous functions.
• Does this matter for analysis?
• NO.
![Page 58: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/58.jpg)
Non-Contiguous functions
• Functions come from languages.
• In reality a function is a collection of basic blocks.
• Basic blocks are a transformation between an input state and an output machine state.
• Functions are a collection of these transformations or expressions.
![Page 59: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/59.jpg)
Self-modifying code
• We don’t care about self-modifying code.
• Does Microsoft or any large vendor use packers or self modifying code?
![Page 60: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/60.jpg)
Unbounded loops
• Trillian exploit example• Find a control flow block• That forms a closed loop on itself• Where a pointer is
– written to– incremented
• Exit from loop (if found) is – a tainted byte comparison– said byte was written to pointer
![Page 61: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/61.jpg)
Unbounded loops
Inline strcpy
while (*dst++ = *src++);
![Page 62: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/62.jpg)
Unbounded loops
// \\MachineName\GetMachineName(WCHAR *src, WCHAR *dst, int arg_8){ for(src++; *src != (WCHAR)'\'; ) *dst++ = *src++;
...}
// MS-RPC Blaster Overflow// Code Snippet from // The Art of Software Security Assessment: Identifying and Preventing
// Software Vulnerabilities
![Page 63: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/63.jpg)
Bounded loops
• Find control flow blocks• That form a closed loop on themselves• Where pointers are
– written to– incremented
• Exit from loop is – comparison with an input– a variable being (++/--)
• Inline wsncpy with wrong n.
![Page 64: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/64.jpg)
Summary
• basic value tracking
• out-of-bounds memory detection
• dealing with branches
• dealing with loops
• with real, working C# code– http://sf.net/projects/bugreport
![Page 65: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/65.jpg)
Questions ?
![Page 66: Automated Exploit Detection in Binaries](https://reader036.vdocuments.site/reader036/viewer/2022062309/56813ad4550346895da309f1/html5/thumbnails/66.jpg)
Shameless Self-Promotion
Automating Exploit Detection with Binary Code Analysis
RSA Conference 2007 TutorialsFeb 4th and 5th San Francisco
Automating Exploit Detection:Cutting-edge Tools and Techniques
Black Hat Europe 2007 TrainingMarch 27th and 28th Amsterdam