isolation and integrity management in dynamic virtualized ... · isolation and integrity management...

IBM T. J. Watson Research Center

© 2007 IBM Corporation

Isolation And Integrity Management In Dynamic Virtualized Environments

Reiner Sailer <[email protected]>Manager Security Services (GSAL) TeamIBM Thomas J Watson Research Center, NY

Joint work with: See next slide


© 2005 IBM [email protected]

Research Collaborators

Stefan Berger

Ramon Cáceres (now AT&T)

Kenneth Goldman

Dimitrios Pendarakis

Ronald Perez

Eran Rom (HRL)

Sivan Tal (HRL)

Enriquillo Valdez

Mihai Christodorescu

Josyula R Rao

Reiner Sailer

Douglas Lee Schales

Wietse Venema

Andreas Wespi (ZRL)

Diego Zamboni (ZRL)



Virtualization Unleashes Security ValueVirtualization Unleashes Security Value

iTVDc - Infrastructure SecurityIntegrated, policy-driven isolation management for competing data center workloads

Continuous audit and compliance guarantees for dynamic cloud environments

Phantom - Integrated Security ServicesOn-demand network intrusion and host malware prevention for virtualized workloads

Transparent, effective and low overhead monitoring of dynamic virtual environments

Building security foundations using isolation and integrity managementBuilding security foundations using isolation and integrity management

TVDcTVDc

Hypervisor

Hypervisor

Hypervisor

Systems ViewSystems ViewTVDc ViewTVDc View

Prod. admin

Dev. admin

VLAN

VLAN

VLAN

SVMSVM VMVM VMVM VMVM

HypervisorHypervisor

HardwareHardware

Hosting mission critical applications and sensitive data in highly dynamic virtualized environmentsHosting mission critical applications and sensitive data in highly dynamic virtualized environments



High Utilization Benefits Power Consumption

Relative Power Consumption: Lowest at High Server Utilization

RPC

0% 100%

Virtualization



Collocating Customers Raises Isolation ConcernsComplication: Moving different customers onto the same platform raises concerns related to their isolation

Customer feedback suggests that insufficient isolation can be a disruptive force hindering virtualization

“Just pretend I’m not here”



We Must Strengthen Isolation Three-fold!Trusted Virtual Datacenter =

Adding controls on data sharing between VMs to improve isolation

Continuously monitoring isolation mechanisms and protecting integrity

Automating security management to account for increasing dynamics of ‘Cloud Computing’

+

+

TVD admin TVD admin



Virtualization-based Security Management

Virt

ual R

esou

rces

Phys

ical

Res

ourc

es

Blue Workload Green Workload

DB2IHS

WAS

WAS

WAS

WAS

WAS

WAS

DB2IHS



Classic Type 1 Hypervisor

Hypervisor

Guest Kernel Guest Kernel Guest Kernel

Application

Application

Application

Application

Application

Application

Application

Application

Application

HardwareCPU, Memory, and I/O devices

Virtualizes hardware

Virtual Machines



Trusted Virtual Data Center Value Proposition

TVDcTVDc

Systems ViewSystems View TVD ViewTVD View

321

Hypervisor

654

Hypervisor

987

Hypervisor

121110

Hypervisor

3

7 9

11

5

1

8

12

2

64

10

Radically simplifies security Management

Reduces the risk of security exposures through consistent, policy-driven enforcement

Leverage virtualization through centralized security services



Isolation and Integrity Management

Isolation Services Integrity Services

Enforces restrictions on administration and data sharing

Who can manage whatWhich customers can run togetherHow virtual machines can share data

Maintains software inventory and acts as early warning system for anomalies

What is running in each VM (TC, N/H-IDS)If VMs/Systems are correctly configuredIf VMs are up-to-date with patches

Extrusion/Intrusion ProtectionExtrusion/Intrusion Protection Malware Prevention and FidelityMalware Prevention and Fidelity



Security Services in Virtualized Environments

Isolation Services

Static Integrity Services(Load-time root of trust)• Configuration validation• Load-time code guarantees

Dynamic Integrity Services(Continuous root of trust)• Network Intrusion Detection• Host Intrusion Detection



Isolation Management

Holistic workload protection

• Run-time isolationIsolate VMs of different colors

• Network isolationIsolate traffic of different colors

• Storage isolationIsolate storage of different colors

• Management isolationSeparate tenant administrators responsible for different colors

Virtual Domain ViewVirtual Domain View



TVDc – Centralized Policy-Driven Workload Isolation

Three Layers of Workload Isolation

Physical Isolation: TVDc System authorization enables flexible partitioning of workloads onto different systems

Temporal isolation: TVDc Anti-collocation disables selective workloads from running concurrently on the same platform

Logical Isolation: TVDc Access control prevents sharing between concurrently executing workloads

2. Anti-Collocation

3. Controlled Sharing

t

-

1. System Authorization



sHype Access Control Architecture (Example: Xen)

Hardware

Xen / sHype ACM

Hypervisor securityhooks

Callbacks

Linux

Application

Application

MS Windows

Application

Application

SecureServices

Dom

0 (M

anagement)

VMFlexible framework: Supports Multiple Policies

Access Control Module Implements Policy Model

Hypervisor Security Hooksmediate inter-VM communication + resource accessinteract with ACM foraccess decision

Implemented for Xen, PHYP, rHype in various stages



Virtual LAN 1

Virtual LAN 2Virtual LAN 1

Virtual LAN 2Virtual LAN 1

Virtual LAN 2

TVDc – Centralized Policy-Driven Network isolation

1. Label VMs + VLANs

2. VMM enforces: VMs ↔ VLANs

3. Hardware VLAN switch enforces:

Blades ↔ VLANs

1. Label VMs + VLANs

2. VMM enforces: VMs ↔ VLANs

3. Hardware VLAN switch enforces:

Blades ↔ VLANs

VM1 VM VM4 VM5

VMM VMM

Blade 1 Blade 2

Network Switch

X

VM2 VM3



dom1 dom2

Implemented Network Isolation on Xen/sHype

VLAN Switch

vlan 100 vlan101

br0.100 br0.101

dom2dom0 (Management VM)

peth0.100 peth0.101

eth0

vif1.0 vif2.0

dom1

Physical LAN

eth0

Physical Machine

eth0

peth0

Other Machines/SwitchesOther Machines/Switches

eth0 vif1.0 vif2.0 eth0



TVDc – Centralized Policy-Driven Storage Isolation

Two Layers of TVDc Storage IsolationPhysical Isolation: TVDc System authorization enables flexible mapping of storage (Volumes) onto distinct physical systemsLogical Isolation: TVDc controls access of concurrently executing workloads to locally virtualized storage - Virtual Block Device (VBD)

Non-intrusive Storage Access Enforcement PointsPolicy-driven storage management ensures that storage is only accessible to authorized systems (A)Extensions of the local virtual storage management to mediate VM device access and manage security (B)

StorageSystem

Dom0

SAN

(B)

(A)

PI:



System/Service Management Solutions

TVDc: Orchestrating Server, Network & Storage Isolation

System x w. Xen

Dom

U

Xen/sHype

Dom

0

Blue Bridge

GreenBridge

System P

LPA

R

PHYP/sHype

Blue Bridge

Green Bridge

XenAPI Mgmt

Green VLAN

Blue VLAN

System x (Xen)

Virt

ual I

O S

erve

r

SVC SVC

Green VLAN

Blue VLAN

System P (PHYP)

Dom

U

Dom

U

LPA

R

LPA

R

“Blue” Trusted Virtual Domain

DB2IHS

WAS

WAS

WAS

Data Center Administrator



TVDCManagement

Use Cases For Protected Infrastructure VMs

Policy and host management:manage TVDc access control policy & virtualization settings

Crypto / vTPM server: keep keys and credentials out of Guest-VMs

Supervision/Introspection: monitor and protect Guest VMs from a secure place

…

VM

Hypervisor

VM VM

Crypto/vTPMServer

IntrusionDefenseServices



IsolationManagement

Infra-structure

Tenant Workloads

IntegrityManagement

Tenant Workloads

Virtualization-based Isolation and Integrity Management



GuestVM

GuestVM

GuestVM

GuestVM

SecureVM

SecureVM

Hypervisor




Isolation Services





Wanted: Structure and Trusted Foundations!

Status quo approach to IT and business security is too complex, not measurable, does not scale

Lack of robust trusted foundation erodes security

II. Trusted Computing:Creates foundation

I. Virtualization: brings ORDER



How Trusted Computing Fits In

Physically controlled room, usually closed run-time environment, information flows sanitized manually across isolation boundariesSecure coprocessors, physical protection, largely closed environment, signed executables, active device, usually server side add-onTrusted Platform Module, protected from software, open environment, passive, suitable for client sideHope for the best

absolutely secure108$

secure104$

trusted100$

hopefor thebest



Trusted Computing – Integrity Measurement Architecture

Inferred System

SHA1(Boot Process)SHA1(Kernel)SHA1(Kernel Modules)SHA1(Program)SHA1(Libraries)SHA1(Configurations)SHA1(Structured data)…

MeasurementsDeduce System

Properties

KnownFingerprints

Real System

Program

Kernel Kernelmodule

Config data

Boot-Process

Data

TPM-Signed PCR Integrity Value

(1) Measurement (2) Attestation

Attesting System Verifying System

(3) Verification

Analysis

IMATCGGrub



Remote Attestation Prototype

1. Submit Request and

Nonce

2. Receive:Sig(Nonce, PCR)

Measurement List

3. Check:Signature

Nonce 4. Validate:PCR Value

5. Evaluate:Individual

Measurements

6. Infer: High-Level

System Properties



VMM Integrity Verification Example (Xen)

KnownFingerprints Acceptable += Malicious

Hypervisor

VMM Measurement List | Fingerprint DB===============================================+============================#000: BC55F0AFE013C3402F00E0AA11EE6CFAA2B4D2AB | aggregate (bios + grub stages)#001: A8A865C7203F2565DDEB511480B0A2289F7D035B | grub.conf (boot configuration)#002: 1238AD50C652C88D139EA2E9987D06A99A2A22D1 | xen.gz#003: 84ABD2960414CA4A448E0D2C9364B4E1725BDA4F | isolation_policy.bin#004: 9ECF02F90A2EE2080D4946005DE47968C8A1BE3D | linux-2.6.18.8-xen…

#317: BC55F0AFE013C3402F00E0AA11EE6CFAA2B4D2AB | /bin/login#318: A8A865C7203F2565DDEB511480B0A2289F7D035B |/usr/bin/httpd#319: 1238AD50C652C88D139EA2E9987D06A99A2A22D1 | /usr/bin/java#320: 84ABD2960414CA4A448E0D2C9364B4E1725BDA4F | /usr/bin/sshd#321: 9ECF02F90A2EE2080D4946005DE47968C8A1BE3D | /usr/bin/python…

VMM Measurement List | Fingerprint DB===============================================+============================#000: BC55F0AFE013C3402F00E0AA11EE6CFAA2B4D2AB | aggregate (bios + grub stages)#001: A8A865C7203F2565DDEB511480B0A2289F7D035B | grub.conf (boot configuration)#002: 1238AD50C652C88D139EA2E9987D06A99A2A22D1 | xen.gz#003: 84ABD2960414CA4A448E0D2C9364B4E1725BDA4F | isolation_policy.bin#004: 9ECF02F90A2EE2080D4946005DE47968C8A1BE3D | linux-2.6.18.8-xen…

#317: BC55F0AFE013C3402F00E0AA11EE6CFAA2B4D2AB | /bin/login#318: A8A865C7203F2565DDEB511480B0A2289F7D035B |/usr/bin/httpd#319: 1238AD50C652C88D139EA2E9987D06A99A2A22D1 | /usr/bin/java#320: 84ABD2960414CA4A448E0D2C9364B4E1725BDA4F | /usr/bin/sshd#321: 9ECF02F90A2EE2080D4946005DE47968C8A1BE3D | /usr/bin/python…

+ Out ofPolicy

Hypervisor

SecureVM

SecureVM



Virtual TPMs Enable VM Integrity Attestation

Hardware

Secure Hypervisor

Guest Kernel Guest Kernel

Application

Application

Application

Application

Application

Application

Core Root of Trust

IMA-enabled OS IMA-enabled OS

Application

Application

IMA

-enabled Application

Application

IMA

-enabled Application

IMA

-enabled Application Measure HW,

hypervisor, and critical services

Virtual TPMs

Policy Manager

Support current IMA via vTPMs(flexible, scalable)

ACM




Isolation Services





Weinberg’s Second Law of Programming

If builders built buildings the way programmers write programs, …

the first woodpecker to come along would destroy civilization.



X-Force® 2008 Trend Statistics

Conclusions

… the following [malware] behaviors are included in the top ten list:

• Hides a file from folder listings by setting the hidden file attribute

• Injects code into processes

• Disables security software

… one of the most common actions malware takes upon installation is an attempt to evade detection …

Same day exploits

Public exploits



Virtualization enables on-demand, centralized services• Selective network intrusion and host malware protection delivered

on-demand with Virtual Machine/Guest granularity

• Consolidation of security services into a single enforcement point

Centralized protection using a Security-VM (SVM) means• Non-bypass, highly effective security functionality in Ring 0

• Efficient amortization of fixed security cost across workloads

• Reduction of security sprawl across virtual infrastructures

• Simplified management of security updates

• Minimal or no per-OS footprint

Virtualization: On-Demand, Centralized Security Services

SVMSVM VMVM VMVM VMVM


HardwareHardware



Usage Case: Intrusion Prevention System (IPS)Summary• Security virtual machine (SVM)

provides analysis of all virtual network traffic using intrusion prevention system

Detail• Attacker sends network exploit• Attack is routed to Guest VM• SVM monitors network traffic• Detects attack via IPS system• Attack Prevented!• Disables or firewalls Attack VM


HardwareHardware

IntegratedSecurity

VM

IntegratedSecurity

VM

GuestVM

GuestVM

SecurityServices vSwitch

AttackVM

AttackVM



Usage Case: Anti-Rootkit System (ARKS)

Summary• Security virtual machine (SVM)

uses virtual machine introspection to monitor critical OS data structures for changes made by rootkits and other types of malware

Detail• Rootkit strikes• Attempts to hide itself• SVM detects OS tampering• Attack Prevented!• Performs clean-up of rootkit


IntegratedSecurity

VM

IntegratedSecurity

VM

GuestVM

GuestVM OS Data

HardwareHardware

SecurityServices



CPU & Memory Introspection Types I

Passive Introspection• Non-intrusive reading of

Guest memory

Limited by• Data consistency (guest

is running)• Polling interval could

miss manipulations in guest memory

• Restricted to detection (no rollback)

VMM / HypervisorVMM / Hypervisor

SecurityAgent

SecurityAgent Guest

VMGuest

VM

Read Memory(Polling)SVM



CPU & Memory Introspection Types II

Trigger-based Introspection(e.g., Anti-Rootkit)

• available triggers: memory page execute, write, read

Limited by:• overhead reading guest pages• VM-VM event overhead

In-guest Security Agent(e.g., Anti-Virus)

• Minimize useless events• Create new or semantically

richer events• Protect context agent


SecurityAgent

SecurityAgent Guest

VMGuest

VM

2. Response

1. Events


SecurityAgent

SecurityAgent

Guest VMGuest VM2. Response

1. Events

ContextAgent



The Semantic Gap – From OS to Physical Semantics

Guest physical memoryIntrospection Event

Trigger

OS Semantics

Phys PageSemantics

Gap

Set Trigger

OS / Process Structures

Guest virtual memory

r/w/x



Secure VMSecure VM Protected Guest VM

Rootkit Protection SystemRootkits in action• User space:

Exploit the way ‘in’Create persistent trap doors

• Kernel space:Manipulate data/code to hide from HIDS, AV, etc.Tap into control flow

Rootkit detection/prevention• Introspect Guest Kernel space:

Security Agent: Detect / prevent changes to critical kernel data structures (‘anti-stealth’)

• Instrument Guest User Space:Existing HIDS, AV: Undo visible user space changes

KernelRootkit

DetectorSA

Hypervisor Introspection



Rootkit Demo Setup

Introspection Hypervisor

DS

SVMSVM GuestGuest

DS

Written

Reset D

S

Start VM

Lock Down DS

Anomaly Detected* DS change reverted (QuerySystemInformation)

Guest Physical Memory

Rootkit strikes* installs backdoor* hides by rewriting DS



Great Opportunities in Overcoming the Challenges

Opportunities Through Centralized Security Services• Marginal additional security cost per protected Guest

(utilization, maintenance, OS support,…)• Protected Security Agent run-time environment

‘Reclaiming Ring-0’ against Guest-VM rootkits and other malware• Differentiation through correlating events across data center

e.g., Cloud-Antivirus, Cloud-IMA: Check once, run everywhere

Challenges• Performance of SVM/out-of-guest event

processing (trigger precision/overhead)• Semantics of the introspection interface

(OS dependencies finding trigger targets)• VMM integrity (see BH08 and ISS: more

than 150 VMM vulnerabilities since 1999)

ISS X-Force 2008



Summary

Need for Virtualization is driven by energy saving potential• Introduces need for isolating collocated customers

TVDc isolation management can mitigate the risk of collocating customers in dynamic virtualized data centers• Introduces need for integrity management

Integrity management in virtualized environments poses significant challenges, many of which can be addressed• Run-time Integrity Attestation (Trusted Computing)• Malware Protection Services (Virtual Introspection)

#



References and Related WorkTVDc: Managing Security in the Trusted Virtual Datacenter. Stefan Berger, Ramón Cáceres, Dimitrios Pendarakis, Ronald Perez, Reiner Sailer, Wayne Schildhauer, Deepa Srinivasan, Enriquillo Valdez. ACM SIGOPS Operating Systems Review, Vol 42, Issue 1, January 2008.

Retrofitting the IBM POWER Hypervisor to Support Mandatory Access Control. Enriquillo Valdez, Reiner Sailer, Ronald Perez. 23rd Annual Computer Security Applications Conference (ACSAC), Florida, December 2007.

Capability based Secure Access Control to Networked Storage Devices. Michael Factor, Dalit Naor, Eran Rom, Julian Satran, Sivan Tal. Mass Storage Systems and Technologies, 2007. MSST 2007. 24th IEEE Conference on Volume , Issue , 24-27 Sept. 2007 Page(s):114 - 128

Shamon -- A System for Distributed Mandatory Access Control. Jonathan M McCune, Stefan Berger, Ramón Cáceres, Trent Jaeger, Reiner Sailer. 22nd Annual Computer Security Applications Conference (ACSAC), Miami Beach, Florida, December 2006

vTPM: Virtualizing the Trusted Platform Module. Stefan Berger, Ramón Cáceres, Kenneth Goldman, Ronald Perez, Reiner Sailer, Leendert van Doorn. 15th USENIX Security Symposium, Vancouver, Canada , July 2006.

Building a MAC-based Security Architecture for the Xen Opensource Hypervisor. Reiner Sailer, Trent Jaeger, Enriquillo Valdez, Ramón Cáceres, Ronald Perez, Stefan Berger, John Griffin, Leendert van Doorn. 21st Annual Computer Security Applications Conference (ACSAC), Tucson, Arizona, December 2005.

Design and Implementation of a TCG-based Integrity Measurement Architecture. Reiner Sailer, Xiaolan Zhang, Trent Jaeger, Leendert van Doorn.13th Usenix Security Symposium, San Diego, California, August, 2004.

In the interest of space, please refer to the references of the cited papers for further related work.

isolation and integrity management in dynamic virtualized ... · isolation and integrity management...

Documents