Temporal Reverse E i iEngineering

Danny QuistColin AmesColin Ames

Offensive Computing, LLC

1Blackhat USA 2008

Danny QuistDanny Quist

• Co-founder Offensive Computing LLCCo-founder Offensive Computing, LLC

• Ph D Candidate at New Mexico Tech• Ph.D. Candidate at New Mexico Tech

S i I R E i i• Senior Instructor – Reverse EngineeringInfosec Institute

• dquist@offensivecomputing.net

2

Colin AmesColin Ames

• Security Researcher Offensive ComputingSecurity Researcher, Offensive Computing• Steganography Research

i i• Penetration Testing• Reverse Engineering• Malware Analysis• amesc@offensivecomputing netamesc@offensivecomputing.net

3

Offensive Computing LLCOffensive Computing, LLC

• Malware CommunityMalware Community

Free access to malware samples– Free access to malware samples– Largest open malware site on the Internet

1million hits per month– ~1million hits per month

• Business Services• Business Services

4

Temporal Reverse EngineeringTemporal Reverse Engineering

• Observe program change over timeObserve program change over time• Analysis is very time dependant

l d l d k h i l i• Tools developed to make the time analysis easier

• Lower complexity for analysis

5

Overview of TalkOverview of Talk

• Current Reverse Engineering TechniquesCurrent Reverse Engineering Techniques– Where they work

Where they fail– Where they fail• What is Temporal Reverse Engineering?• Techniques and Strategies• Checkpointing as an analysis toolp g y• Visualization Methods• Applications and Demos

6

• Applications and Demos

Reverse EngineeringReverse Engineering

• Reversing is hardReversing is hard

• Goal: Figure out how program works in• Goal: Figure out how program works in minimal amount of time

• Expensive (We don’t work cheap)

• Time consuming

7

Dominant StrategiesDominant Strategies

• Static AnalysisStatic Analysis

IDA Pro– IDA Pro– Figure out program flow

String Searching– String Searching– API Call tracing

8

Dominant StrategiesDominant Strategies

• Dynamic AnalysisDynamic Analysis

W t h f h th t– Watch for changes on the system• Registry, files, network

M it S t ll t OS/Lib l l– Monitor System calls at OS/Library level– Tools more accessible to unskilled people

S i l– Sysinternals…

9

ProsPros

Static Analysis Dynamic AnalysisStatic Analysis• Details• Precision, full code reversal

Dynamic Analysis• Fast• Lower barrier to entryPrecision, full code reversal

possible• Good tools available

Lower barrier to entry• High level overview• Good tools

• Antivirus– It’s profitable

– Sysinternals– Winalysis– CWSandboxCWSandbox

10

ConsCons

Static Analysis Dynamic AnalysisStatic Analysis• Too much detail• Full code reversing not

Dynamic Analysis• Misses details

Full code reversing not necessary

• Tools cumbersome, takes i d i i l

• Encourages “next->next->next” analysis

time and training to learn• Source level analysis full of

false positives• Tools easily subverted

false positives• Antivirus

– Doesn’t scale

11

Which is Better?Which is Better?

• Each perform a task wellEach perform a task well• Tools need to be developed on both sides

G d b b id d b i• Gaps need to be bridged to better integrate the two methods

• This talk is about a partial solution to bridging that gap

12

Bridging the GapBridging the Gap

• Fundamental problems:Fundamental problems:

Knowing when to analyze– Knowing when to analyze

– Data and state changeData and state change• What causes these?• Are they important?y p• Automation

13– How do you detect important data changes?

Steps Toward SolutionsSteps Toward Solutions

• TechniquesTechniques

Debuggers– Debuggers

– Dynamic TranslationDynamic Translation

– Page-fault assisted debugging (Saffron)Page fault assisted debugging (Saffron)

– Sandboxing

14

g

DebuggersDebuggers

• The GoodThe Good– Excellent tools available:

• OllyDbg WinDBG Visual Studio Debugger gdb• OllyDbg, WinDBG, Visual Studio Debugger, gdb• APIs

– PyDbg (thanks Pedram!)– Windows Debug API

• Most have scripting support

All i l i ll i– Allow single-stepping as well as tracing– Scripting support allows automation

15

– Well tested, vetted, proven

Debuggers (cont )Debuggers (cont.)

• The BadThe Bad– Detectable!

• IsDebuggerPresent / MSFT DRM trickery• IsDebuggerPresent / MSFT DRM trickery• Timing attacks• Exception triggeringp gg g

– Still too fine-grained• Focus is on assembly levely• Need to integrate with static analysis tools

– Trivial packer changes confuse debuggers

16(and us)

What about program flow tracing?What about program flow tracing?

• Visualization should be able to answerVisualization should be able to answer questions quickly

• How can we apply this to reverse engineering?engineering?

Aid l i d di fl• Aid analyst in understanding program flow

17

Visualization StrategiesVisualization Strategies

• Program Flow ExecutionProgram Flow Execution– Process Explorer

FileMon– FileMon• Runtime Instruction Tracing

– Intel’s PIN Framework– Allows Analysis of Program Flow

• Call-Graph Tracing• Basic-Block Visualization

18

Monitoring Program ExecutionMonitoring Program Execution

• Intel PINIntel PIN

Dynamic instrumentation library– Dynamic instrumentation library– Extensible

Excellent API– Excellent API– Process attach and detach

19

PIN BasicsPIN Basics

20

PIN BasicsPIN Basics

21

PIN Instruction MonitoringPIN Instruction Monitoring

22

Determine Changed DataDetermine Changed Data

• Look for droppersLook for droppers• Modified files

i h• Registry changes• Unusual withdrawals from bank account• The Usual Suspects

23

ImplementationImplementation

• Setup basic instruction traceSetup basic instruction trace

• Monitor via instruction and basic block• Monitor via instruction and basic block

D L d• Data Logged:– Execution– Memory read– Memory write

24

Graph LayoutGraph Layout

• Each node (vertex) represents an address ofEach node (vertex) represents an address of a basic block

• Each line (edge) observed transition• Each line (edge) observed transition• Collected with custom PIN DLLs• Processed with Oreas Govisual Diagram

Editor (GDE)

25

Oreas GDEOreas GDE

• Automatic Graph Layout ToolAutomatic Graph Layout Tool• Renders large graphs (> 40,000 nodes)

A i l• Automatic layout– Tree– Circular– Symmetric– Hierarchical– Orthogonal

26• http://www.oreas.com

Why not Graphviz?Why not Graphviz?

• Fails on extraordinarily large graphsFails on extraordinarily large graphs(Memory leaks? Bad algorithm?)

• “Good” for trivial graphs

• Graphs lack visual appeal

27

Demo: Find the Unpacking LoopsDemo: Find the Unpacking Loops

• Simple hello world programSimple hello world program

int main(int argc, char **argv)( g g ){

printf(“Hello, world\n”);return 0;;

}

Fi t t t T h i t ti• First test: Trace each instruction

28

29Hello WorldInst., No Packing

Complexity in RepresentationComplexity in Representation

• Previous graph is pretty but too complexPrevious graph is pretty, but too complex

• Tracing instructions is too complicated• Tracing instructions is too complicated– This is a small program

M d l l f il– Moderately complex programs fail

D d i i• Data reduction important– Compression of instruction graph

30– Tracing at basic block

Basic BlocksBasic Blocks

• Set of instructions with• Set of instructions with branch at the endAll f i f• Allows for compaction of tracing data

• More useful analysis information

• Visualization results better

31

32Hello WorldBasic Block, No Packing

Adding PackersAdding Packers

• Should be able to find the following:Should be able to find the following:

Packing loop– Packing loop– Main program

• Minimize extraneous information• Reducing analyst time is the key• Packers: ASPack, PE Compact, UPX, FSG

33

, p , ,

34Hello WorldASPack 2.12

35Hello WorldASPack 2.12

Hello WorldPECompact 1.68

36

Hello WorldPECompact 1.68

37

Hello WorldUPX 1.20

38

39Hello WorldUPX 1.20

40Hello WorldFSG 1.0

41Hello WorldFSG 1.0

Visualization ResultsVisualization Results

• Unpacking loops easy to identifyUnpacking loops easy to identify• Useful for saving analysis time

i l l• Visual appeal• Narrowing in on relevant data is useful

42

Memory CheckpointingMemory Checkpointing

• Analyze changes between two executionAnalyze changes between two execution points of a program

• Useful for comparing differences at certain• Useful for comparing differences at certain pointsE l i f d• Entropy analysis of data

• Decryption analysis

43

CheckpointingCheckpointing

• Determine when to checkpointDetermine when to checkpoint

Relevant Events– Relevant Events

Instruction– Instruction

Basic block– Basic block

– Page access44

Page access

Determine Relevant EventsDetermine Relevant Events

• MalwareMalware– File write

Network communication– Network communication– Any system call

C i l S f• Commercial Software– Most probably decryption point– Software load

45

Relevant EventRelevant Event

• Use system monitoring tools:Use system monitoring tools:

Filemon regmon sysinternals tools– Filemon, regmon, sysinternals tools– Winalysis

Wireshark– Wireshark– WinDBG, SoftICE, Ollydbg, etc.

46

CheckpointingCheckpointing

• Preservation of statePreservation of state

Register contents– Register contents– Stack contents

CPU State– CPU State– Memory

47

What We Care AboutWhat We Care About

• State of memory at a certain eventState of memory at a certain event• Typical checkpoint systems wish to restore

l i h• We want to analyze prior to these events• Be able to develop a temporal view of

program as it changes

48

Existing Checkpointing ToolsExisting Checkpointing Tools

• OS SuspendOS Suspend

• Cryopid• Cryopid

M P i• Memory Paging

• OS Scheduler

49

Isolating Important DataIsolating Important Data

• Memory mapsMemory maps

• Memory hotspots• Memory hotspots

E A l i• Entropy Analysis

• Manual exploration

50

Enabling AnalysisEnabling Analysis

• Prepare for analysisPrepare for analysis– Pausing and suspending execution

• Debugger• Debugger• Pagefault Debugger (Saffron)• System cally

– Copy running process space to disk– Reproduce Memory PE view and file PE formatReproduce Memory PE view and file PE format

• Repair Imports

51

Rebuilding PE files for IDARebuilding PE files for IDAHow IDA creates its import section .idata and

populates subviews Imports Namespopulates subviews Imports, Names

IMAGE DIRECTORY ENTRY IMPORT– IMAGE_DIRECTORY_ENTRY_IMPORT• RVA (Relative Virtual Address) to Import Directory

– IMAGE IMPORT DESCRIPTOR’sIMAGE_IMPORT_DESCRIPTOR s• OriginalFirstThunk

– RVA to INT (Import Names Table)Fi Th k• FirstThunk

– RVA to IAT (Import Address Table)

– Scan’s Code for call’s in INT52

Scan s Code for call s in INT• Prepends internal functions to .idata section

Rebuilding PE files for IDARebuilding PE files for IDASteps to Recovering

INT from packed or encrypted PE

Unpack using Saffron– Unpack using Saffron• Discover OEP

– Enumerate Loaded ModulesEnumerate Loaded Modules• CreateToolhelp32Snapshot, Module32First

– Scan Process heaps for Module Address p• Translate Virtual Address into RVA

– Rebuild INT and IAT

53

• Dump Process memory

Malware ExampleMalware Example

• Start with Storm/Nuwar/Peed SampleStart with Storm/Nuwar/Peed Sample• Found in spam folder e-card.exe

A l i i /d i h d• Analyze using static/dynamic methods• 591258adc48b422c86730214aef81989• Download on Offensive Computing

54

What does IDA Say?What does IDA Say?

Signs say: Packed with something weird

55

WinalysisWinalysis

56

OC ResultsOC Results

57

Virus Total ResultsVirus Total Results

58

Dynamic AnalysisDynamic Analysis

Debugging Tarpit?

No basic blocks in storm unpacker == viz fail

59

Initial AnalysisInitial Analysis

• Wireshark shows network trafficWireshark shows network traffic• Debugging reveals that originating source is

services exeservices.exe• System call tracing Showed a new file

dcreated:diperto-4417-e33.sys

60

Diperto-XXXX-xxx sysDiperto XXXX xxx.sys

• Device driver loaded by dropper executableDevice driver loaded by dropper executable• File is not packed!

G d di bl• Good disassembly

61

Diperto FeaturesDiperto Features

• Attaches to running process spaceAttaches to running process space• Rootkit finds services.exe

j i h• Injects into the process space

62

Diperto FeaturesDiperto Features

• Copies into services exe address spaceCopies into services.exe address space• Program runtime actually xor obfuscated

63

PayloadPayload

64

DecodingDecoding

• Could manually run through decryptionCould manually run through decryption• Could make IDA script

O i i i h 86• Better Option: Let it run with x86emu

65

Decoded DataDecoded Data

66

Injected FileInjected File

• Does the real work of the storm wormDoes the real work of the storm worm• Has full working unpacked code

S d i d d A C h d• Started via undocumented APC methods• Starts code running in remote process• Good obfuscation

67

Resulting Unpack is GoodResulting Unpack is Good

68

ConclusionConclusion

• Dynamic Runtime Visualization showsDynamic Runtime Visualization shows process change over time

• Multiple checkpoints allow for analysis l i lover multiple program states

• Leverage existing tools with time dependant data

69

ReferencesReferences• Visualization Grand Challenges: Illuminating the Path

http://nvac.pnl.gov/docs/RD Agenda NVAC chapter1.pdfp p g _ g _ _ p p• Dynamic Data Visualization of Meteorological Data

ASA-JSM Data Exposition, 2006• Visual Signatures in Video Visualization

IEEE Transactions on Visualization and Computer Graphics, Vol.12, No. 5, September/October 2006• Static Visualization of Dynamic Data Flow Visual Program Executiony g

Proceedings of the Sixth International Conference on Information Visualization, IV 2002• Hoglound, G., McGraw, G., Exploiting Software: How to Break Code, Chapter 3, Addison Wesley,

2004• Amini, P., Process Stalker, OpenRCE, http://pedram.redhive.com/code/process_stalker/• Amini, P., PaiMei, OpenRCE http://www.openrce.org/downloads/details/208/PaiMei, , , p p p g• Eagle, C., x86emu, http://ida-x86emu.sourceforge.net/• C. Luck, R. Cohn, R. Muth, H. Patil, A. Klauser, G. Lowney, S. Wallace, V.J. Reddi, K. Hazelwood,

Pin: Building Customized Program Analysis Tools with Dynamic Instrumentation, Proceedings of the 2005 Conference on Programming and Language Design and Implementation, 2005

• Oreas GDE, http://www.oreas.com/index en.php, p _ p p

Latest slides and code can be found on offensivecomputing.net

70

p g