Jiang Wang, Joint work with Angelos Stavrou and Anup Ghosh

CSIS, George Mason University

HyperCheck:

a Hardware Assisted Integrity Monitor

Motivation Our approach Prototype Implementation Evaluation Future work

Outline

2

Virtualization is widely deployed for servers and desktops. ◦ In 2009, 18% server workloads were virtualized◦ Expected to grow to more than 50% by 2012 (Gartner

Inc.) Hypervisors (also called Virtual Machine Monitors

or VMM) are the core component to enforce policy.

Hypervisors are the new attack target.

Motivation

Hardware

Hypervisor (Virtual Machine Monitor)

Privileged Domain

Guest OS 2(Linux)

Kernel 2

Guest OS 1(Windows)

Kernel 1…

Kernel 0

3

Xen vulnerabilities:◦Allow the attacker to run arbitrary code

in the privileged domain. E.g. CVE-2007-4993, CVE-2007-1320,

DMA attack (Invisible Things Lab, Blackhat 08)◦Modify the device driver to write

arbitrary data to the hypervisor via DMA◦E.g. Interrupt Descriptor Table (IDT).

HyperCall table.

VMMs vulnerabilities and attacks

4

Modify IDT directly

5

IDT hook

1

2

3

. . .

Org IDT

Code in memory

Copy and change attack

6

IDT hook

1

2

3

. . .

1

2

3

. . .

Org IDT New

IDT

Code in memory

Copilot (Petroni et al. USENIX security ‘04 ). ◦ Cannot get execution state. Copy and change attack.

◦ Can be subverted by DMA remapping attack.

HyperGuard (Rutkowska, Blackhat ‘08). ◦ Use SMM to get the execution state.◦ The OS is frozen when CPU in SMM –- high overhead.

HyperSentry, (Azab et al. CCS’ 10)◦ Use SMM to monitor the hypervisor integrity.

DeepWatch (Bulygin, Blackhat ‘08). ◦ Based on micro-controller existed on some motherboard.◦ Need the signature of the malware.

7

Out-of-VMM defense mechanisms

HyperSafe, (Wang, Oakland ’10)

◦ Method: non-modifiable memory lockdown and restricted pointer indexing

◦ Drawbacks: need to modify the kernel; aliasing problem

8

In-VMM defense mechanisms

Design goals: ◦ To monitor the hypervisor code and static data

◦ Complete execution view

◦ Low performance overhead

◦ No hardware modification

◦ No software changes to the hypervisor or kernel

◦ Provide out-of-box view that cannot be subverted

Our approach: HyperCheck

9

◦ SMM + COTS network card (NIC)◦ SMM existed in all x86 CPU after 486.

10

HyperCheck architecture

Hardware

Hypervisor

OS 1 OS 2

Monitor Machine PCI NIC

(1)SMM(2)

AnalysisModule

(1) Acquiring module(2) Register Checking module

System Management Mode (SMM) is another CPU mode for x86

To enter SMM, a System Management Interrupt (SMI) is required.

SMM has a special RAM—SMRAM, and can be LOCKED.

SMM code is included in the BIOS.

SMM Background

Real-address mode

Protected mode

SMM

SMRAM

11

SMRAM cannot be modified:◦ Locked by hardware in flash and memory

Can be integrated with BIOS code Can be set up by a trusted boot module

Other software on the target machine is not trusted.◦ Network card driver is put into SMM.

The attacker will modify some portion of the hypervisor kernel in the memory.

Assumptions & threat model

12

Logic steps of HyperCheck

PCI NIC triggers SMI

SMM check the CPU registers

SMM send the memory out via NIC

Analysis module rcv the data

Different from the previous?

Alarm

YES

13

• Two prototypes:– HyperCheck-I : QEMU based, easy debugging– HyperCheck-II: on real hardware. For performance

evaluation.• Protect static part of the VMM or OS

– VMM code– Dom 0 code– Linux or Windows kernel code– Static control data (such as Interrupt Descriptor

Table)

Implementation

14

PCI devices with DMA support

Use commercial network cards

◦ Challenge: they need drivers, and drivers normally reside in untrusted OS, Driver Domain, or VMM.

◦ Solution: put the driver into SMM.

We used Intel e1000 NIC.

Memory acquiring module

15

Resides in SMM

Previous CPU registers are saved in SMRAM before switching to SMM.

Check two registers:

◦ IDTR (Interrupt Descriptor Table Register): static

◦ CR3: page directory base register. Used to translate virtual addresses to physical ones.

Register checking module

16

Receive the packets from the acquiring module.

Compare the current memory snapshot with the clean state (obtained when the system just boot).

If different, potential attack.

Analysis module

17

Verifying the static property:

◦ Monitored the target code and data for one hour and didn’t find any changes.

◦ They do change when the system is booting

Detection

◦ Detected all the simulated attacks to the Xen hypervisor, Dom0, Linux and Windows kernels.

Evaluation

18

static Detected modification

Xen

IDT table Y YHypercall table Y YException table Y YHypervisor code Y Y

Dom 0 System call table Y YKernel code Y Y

Linux IDT table Y Y

System call table Y YKernel code Y Y

Windows IDT table Y YSystem call table Y Y

Summary of the static properties and detection test

19

Network Overhead (1)

Network overhead for variable packet size when sending 2.7MB data.

1 3 5 7 9 11 13 150

20

40

60

80

100

120

140

packet size( K bytes )

Tim

e (

million

CP

U c

ycl

es)

20

Network Overhead (2)

10 50 100 150 2000

50010001500200025003000350040004500

Data size (MB)

Tim

e (

million

CP

U

cycl

es)

Network overhead for variable data size.

21

SMM operation overhead

22

code

size(MB)time(ms)

HyperCheckSMM-only TPM

Linux 2 31 203 1022Xen+Dom0 2.7 40 274 >1022Window XP 1.8 28 183 >972

Hyper-V+root 2.4 36 244 >1022VMwareESXi 3.5 2.2 33 223 >1022

Comparing with other methods

Memory Registers OverheadHyperCheck x x low

SMM x x highPCI x low

TPM x x high

Table 1, CPU overhead comparison

Table 2, features comparison23

Scrubbing attack◦ Modify the hypervisor between two scans

interval and recover before the next scan.◦ -- Randomize scan interval.

Dynamic data◦ Current analysis module does not know how to

check them, such as stack, heap.◦ -- Syntax analysis.

Limitations & Future work

24

Thank you!

Questions?

25