![Page 1: Process Out-Grafting: An Efficient “Out-of-VM” …...Process Out-Grafting: An Efficient “Out-of-VM” Approach for Fine-Grained Process Execution Monitoring Deepa Srinivasan,](https://reader033.vdocuments.site/reader033/viewer/2022041601/5e30e509af069308b52d6d80/html5/thumbnails/1.jpg)
Process Out-Grafting:
An Efficient “Out-of-VM” Approach for
Fine-Grained Process Execution Monitoring
Deepa Srinivasan, Zhi Wang, Xuxian Jiang, Dongyan Xu*North Carolina State University, Purdue University*
Workshop for Frontiers of Cloud Computing, Dec 1, 2011, IBM T.J. Watson Research Center, NY
![Page 2: Process Out-Grafting: An Efficient “Out-of-VM” …...Process Out-Grafting: An Efficient “Out-of-VM” Approach for Fine-Grained Process Execution Monitoring Deepa Srinivasan,](https://reader033.vdocuments.site/reader033/viewer/2022041601/5e30e509af069308b52d6d80/html5/thumbnails/2.jpg)
Malware Infection Trend
New malware samples collected by McAfee Labs, by month*
*Figure source: McAfee Threats Report: Second Quarter 2011, McAfee Labs 2
![Page 3: Process Out-Grafting: An Efficient “Out-of-VM” …...Process Out-Grafting: An Efficient “Out-of-VM” Approach for Fine-Grained Process Execution Monitoring Deepa Srinivasan,](https://reader033.vdocuments.site/reader033/viewer/2022041601/5e30e509af069308b52d6d80/html5/thumbnails/3.jpg)
Anti-Malware Isolation
� Traditional anti-malware tools are not well-isolated
� Virtual Machine (VM) introspection
� Isolate tool by placing it outside a VM
� Analyze states and events externally
User-mode Applications
…
Hypervisor
Virtual
Machine
Monitor
3
OS Kernel
![Page 4: Process Out-Grafting: An Efficient “Out-of-VM” …...Process Out-Grafting: An Efficient “Out-of-VM” Approach for Fine-Grained Process Execution Monitoring Deepa Srinivasan,](https://reader033.vdocuments.site/reader033/viewer/2022041601/5e30e509af069308b52d6d80/html5/thumbnails/4.jpg)
Anti-Malware Isolation
� Traditional anti-malware tools are not well-isolated
� Virtual Machine (VM) introspection
� Isolate tool by placing it outside a VM
� Analyze states and events externally
User-mode Applications
…
Hypervisor
Virtual
Machine
VM
Introspection
Monitor
4
OS Kernel
![Page 5: Process Out-Grafting: An Efficient “Out-of-VM” …...Process Out-Grafting: An Efficient “Out-of-VM” Approach for Fine-Grained Process Execution Monitoring Deepa Srinivasan,](https://reader033.vdocuments.site/reader033/viewer/2022041601/5e30e509af069308b52d6d80/html5/thumbnails/5.jpg)
Out-of-VM Solutions
� Livewire (Garfinkel et al., NDSS ‘03)
� XenAccess (Payne et al., ACSAC ‘07)
� VMScope (Jiang et al., RAID ‘07)
� Lares (Payne et al., Oakland ‘08)
� …
5
![Page 6: Process Out-Grafting: An Efficient “Out-of-VM” …...Process Out-Grafting: An Efficient “Out-of-VM” Approach for Fine-Grained Process Execution Monitoring Deepa Srinivasan,](https://reader033.vdocuments.site/reader033/viewer/2022041601/5e30e509af069308b52d6d80/html5/thumbnails/6.jpg)
Semantic Gap in Introspection
� What we want to observe
� High-level states and events (e.g. system calls, processes)
� What we can observe
� Low-level states and events (e.g. raw memory, interrupts)
OS Kernel
User-mode Applications
…
Hypervisor
Virtual
Machine
Internal
Monitor
External
Monitor
Semantic
Gap
6
![Page 7: Process Out-Grafting: An Efficient “Out-of-VM” …...Process Out-Grafting: An Efficient “Out-of-VM” Approach for Fine-Grained Process Execution Monitoring Deepa Srinivasan,](https://reader033.vdocuments.site/reader033/viewer/2022041601/5e30e509af069308b52d6d80/html5/thumbnails/7.jpg)
Addressing the Semantic Gap
� Guest view casting
� VMWatcher (Jiang et al., CCS ‘07)
� Automatic generation of introspection-based tools
� Virtuoso (Dolan-Gavitt et al., Oakland ‘11)
� Issues
� Sensitive to internal OS changes or updates
� Incompatible with existing anti-malware tools
7
![Page 8: Process Out-Grafting: An Efficient “Out-of-VM” …...Process Out-Grafting: An Efficient “Out-of-VM” Approach for Fine-Grained Process Execution Monitoring Deepa Srinivasan,](https://reader033.vdocuments.site/reader033/viewer/2022041601/5e30e509af069308b52d6d80/html5/thumbnails/8.jpg)
Our Goal
Support existing in-host process monitors
out-of-VM without semantic gap!
8
![Page 9: Process Out-Grafting: An Efficient “Out-of-VM” …...Process Out-Grafting: An Efficient “Out-of-VM” Approach for Fine-Grained Process Execution Monitoring Deepa Srinivasan,](https://reader033.vdocuments.site/reader033/viewer/2022041601/5e30e509af069308b52d6d80/html5/thumbnails/9.jpg)
In-host strace
9
![Page 10: Process Out-Grafting: An Efficient “Out-of-VM” …...Process Out-Grafting: An Efficient “Out-of-VM” Approach for Fine-Grained Process Execution Monitoring Deepa Srinivasan,](https://reader033.vdocuments.site/reader033/viewer/2022041601/5e30e509af069308b52d6d80/html5/thumbnails/10.jpg)
In-host Monitors
� Process-level monitoring
� System calls
� Library calls
� Instruction execution traces
10
![Page 11: Process Out-Grafting: An Efficient “Out-of-VM” …...Process Out-Grafting: An Efficient “Out-of-VM” Approach for Fine-Grained Process Execution Monitoring Deepa Srinivasan,](https://reader033.vdocuments.site/reader033/viewer/2022041601/5e30e509af069308b52d6d80/html5/thumbnails/11.jpg)
Process Out-Grafting
� Isolation
� Monitor protection from malware
� Compatibility
� Natural support for fine-grained user-mode process
monitoring tools (strace, ltrace, …)
� Efficiency
� No significant performance overhead due to isolation
11
![Page 12: Process Out-Grafting: An Efficient “Out-of-VM” …...Process Out-Grafting: An Efficient “Out-of-VM” Approach for Fine-Grained Process Execution Monitoring Deepa Srinivasan,](https://reader033.vdocuments.site/reader033/viewer/2022041601/5e30e509af069308b52d6d80/html5/thumbnails/12.jpg)
Rest of This Talk
� Motivation
� System Design
� Implementation & Evaluation
� Related Work
� Conclusion
12
![Page 13: Process Out-Grafting: An Efficient “Out-of-VM” …...Process Out-Grafting: An Efficient “Out-of-VM” Approach for Fine-Grained Process Execution Monitoring Deepa Srinivasan,](https://reader033.vdocuments.site/reader033/viewer/2022041601/5e30e509af069308b52d6d80/html5/thumbnails/13.jpg)
Assumptions
� Trusted hypervisor
� Untrusted monitored VM
� Non-goals
� OS kernel-level monitoring
� Stealthy monitoring
13
![Page 14: Process Out-Grafting: An Efficient “Out-of-VM” …...Process Out-Grafting: An Efficient “Out-of-VM” Approach for Fine-Grained Process Execution Monitoring Deepa Srinivasan,](https://reader033.vdocuments.site/reader033/viewer/2022041601/5e30e509af069308b52d6d80/html5/thumbnails/14.jpg)
Key Techniques
Hypervisor
Production VM Security VM
User
Kernel
User
Kernel
Suspect ProcessStub
Technique I: On-Demand Grafting
Technique II: Mode-sensitive Split Execution
Monitor
14
![Page 15: Process Out-Grafting: An Efficient “Out-of-VM” …...Process Out-Grafting: An Efficient “Out-of-VM” Approach for Fine-Grained Process Execution Monitoring Deepa Srinivasan,](https://reader033.vdocuments.site/reader033/viewer/2022041601/5e30e509af069308b52d6d80/html5/thumbnails/15.jpg)
Key Techniques
Hypervisor
Production VM Security VM
User
Kernel
User
Kernel
Technique I: On-Demand Grafting
Monitor
15
![Page 16: Process Out-Grafting: An Efficient “Out-of-VM” …...Process Out-Grafting: An Efficient “Out-of-VM” Approach for Fine-Grained Process Execution Monitoring Deepa Srinivasan,](https://reader033.vdocuments.site/reader033/viewer/2022041601/5e30e509af069308b52d6d80/html5/thumbnails/16.jpg)
On-Demand Grafting
![Page 17: Process Out-Grafting: An Efficient “Out-of-VM” …...Process Out-Grafting: An Efficient “Out-of-VM” Approach for Fine-Grained Process Execution Monitoring Deepa Srinivasan,](https://reader033.vdocuments.site/reader033/viewer/2022041601/5e30e509af069308b52d6d80/html5/thumbnails/17.jpg)
On-Demand Grafting
� Relocate suspect process to security VM
� Enable efficient, native inspection
� Eliminate hypervisor intervention
� Support existing process monitoring tools
� Initiate out-grafting as needed
� Restore process after monitoring
17
![Page 18: Process Out-Grafting: An Efficient “Out-of-VM” …...Process Out-Grafting: An Efficient “Out-of-VM” Approach for Fine-Grained Process Execution Monitoring Deepa Srinivasan,](https://reader033.vdocuments.site/reader033/viewer/2022041601/5e30e509af069308b52d6d80/html5/thumbnails/18.jpg)
When to Out-Graft?
18
� Process in user-mode
� Process in kernel-mode
� Hypervisor notified when user-mode execution resumes
Hypervisor
Production
VM
User
Kernel
NXPhysical
Memory
Frames
Suspect Process
![Page 19: Process Out-Grafting: An Efficient “Out-of-VM” …...Process Out-Grafting: An Efficient “Out-of-VM” Approach for Fine-Grained Process Execution Monitoring Deepa Srinivasan,](https://reader033.vdocuments.site/reader033/viewer/2022041601/5e30e509af069308b52d6d80/html5/thumbnails/19.jpg)
What to Out-Graft?
19
Hypervisor
Production
VM
User
KernelEIP
EAX
EBX
…
…
� Architecture-specific resources
� No OS kernel-specific resources
� Main root cause in semantic gap
� Continued execution of out-grafted process
Network
SocketDisk File
Physical
Memory
Frames
Suspect Process
![Page 20: Process Out-Grafting: An Efficient “Out-of-VM” …...Process Out-Grafting: An Efficient “Out-of-VM” Approach for Fine-Grained Process Execution Monitoring Deepa Srinivasan,](https://reader033.vdocuments.site/reader033/viewer/2022041601/5e30e509af069308b52d6d80/html5/thumbnails/20.jpg)
How to Out-Graft?
Hypervisor
Production VM Security VM
User
Kernel
User
Kernel
Suspect ProcessStub
NX X
Helper Module
Monitor
VCPU Register State
Page Mappings
Stub protected
by hypervisor
20
Physical
Memory
Frames
![Page 21: Process Out-Grafting: An Efficient “Out-of-VM” …...Process Out-Grafting: An Efficient “Out-of-VM” Approach for Fine-Grained Process Execution Monitoring Deepa Srinivasan,](https://reader033.vdocuments.site/reader033/viewer/2022041601/5e30e509af069308b52d6d80/html5/thumbnails/21.jpg)
Mode-Sensitive Split Execution
![Page 22: Process Out-Grafting: An Efficient “Out-of-VM” …...Process Out-Grafting: An Efficient “Out-of-VM” Approach for Fine-Grained Process Execution Monitoring Deepa Srinivasan,](https://reader033.vdocuments.site/reader033/viewer/2022041601/5e30e509af069308b52d6d80/html5/thumbnails/22.jpg)
Mode-Sensitive Split Execution
� All user-mode execution occurs in security VM
� All kernel-mode execution occurs in production VM
� Out-grafted process considers itself running in production
VM
22
![Page 23: Process Out-Grafting: An Efficient “Out-of-VM” …...Process Out-Grafting: An Efficient “Out-of-VM” Approach for Fine-Grained Process Execution Monitoring Deepa Srinivasan,](https://reader033.vdocuments.site/reader033/viewer/2022041601/5e30e509af069308b52d6d80/html5/thumbnails/23.jpg)
System Call Redirection
� Smooth, continued execution of out-grafted process
� Monitor isolation
Hypervisor (e.g., KVM)
Production VM Security VM
User
Kernel
User
Kernel
Suspect ProcessStub
Helper Module
Monitor
Disk File
Network
Socket
System
Call
- System call
number
- Arguments
No hypervisor
intervention!
23
![Page 24: Process Out-Grafting: An Efficient “Out-of-VM” …...Process Out-Grafting: An Efficient “Out-of-VM” Approach for Fine-Grained Process Execution Monitoring Deepa Srinivasan,](https://reader033.vdocuments.site/reader033/viewer/2022041601/5e30e509af069308b52d6d80/html5/thumbnails/24.jpg)
Page Fault Forwarding
� Consistent process memory mapping between VMs
� No semantic knowledge in security VM (e.g. memory-mapped
file)
Hypervisor (e.g., KVM)
Production VM Security VM
User
Kernel
User
Kernel
Suspect ProcessStub
Helper Module
Monitor
- Faulting
address
- Read/Write
fault
Page
Fault
Page Tables
Page
Fault
24
![Page 25: Process Out-Grafting: An Efficient “Out-of-VM” …...Process Out-Grafting: An Efficient “Out-of-VM” Approach for Fine-Grained Process Execution Monitoring Deepa Srinivasan,](https://reader033.vdocuments.site/reader033/viewer/2022041601/5e30e509af069308b52d6d80/html5/thumbnails/25.jpg)
Page Fault Forwarding
� Consistent process memory mapping between VMs
� No semantic knowledge in security VM (e.g. memory-mapped
file)
Hypervisor (e.g., KVM)
Production VM Security VM
User
Kernel
User
Kernel
Suspect ProcessStub
Helper Module
Monitor
Page
Fault
Page Tables
Page
Fault
25
Physical
Memory
Frame
![Page 26: Process Out-Grafting: An Efficient “Out-of-VM” …...Process Out-Grafting: An Efficient “Out-of-VM” Approach for Fine-Grained Process Execution Monitoring Deepa Srinivasan,](https://reader033.vdocuments.site/reader033/viewer/2022041601/5e30e509af069308b52d6d80/html5/thumbnails/26.jpg)
Rest of This Talk
� Motivation
� System Design
� Implementation & Evaluation
� Related Work
� Conclusion
26
![Page 27: Process Out-Grafting: An Efficient “Out-of-VM” …...Process Out-Grafting: An Efficient “Out-of-VM” Approach for Fine-Grained Process Execution Monitoring Deepa Srinivasan,](https://reader033.vdocuments.site/reader033/viewer/2022041601/5e30e509af069308b52d6d80/html5/thumbnails/27.jpg)
Implementation
� Hypervisor: KVM (2.6.36.1)
� Out-grafting support: +1309 SLOC
� Extended page tables support (Intel Core i7)
� Host OS: Ubuntu 10.04 (kernel 2.6.28)
� Guest OS: Ubuntu 9.04, Fedora 10
27
![Page 28: Process Out-Grafting: An Efficient “Out-of-VM” …...Process Out-Grafting: An Efficient “Out-of-VM” Approach for Fine-Grained Process Execution Monitoring Deepa Srinivasan,](https://reader033.vdocuments.site/reader033/viewer/2022041601/5e30e509af069308b52d6d80/html5/thumbnails/28.jpg)
Security Analysis
Hypervisor (e.g., KVM)
Production VM Security VM
User
Kernel
User
Kernel
Suspect ProcessStub Monitor
Disk File
Network
Socket System
Call
28
� Monitor isolation and effectiveness
� System call forwarding
� Security VM protected from kernel attacks
![Page 29: Process Out-Grafting: An Efficient “Out-of-VM” …...Process Out-Grafting: An Efficient “Out-of-VM” Approach for Fine-Grained Process Execution Monitoring Deepa Srinivasan,](https://reader033.vdocuments.site/reader033/viewer/2022041601/5e30e509af069308b52d6d80/html5/thumbnails/29.jpg)
Security Analysis
Hypervisor (e.g., KVM)
Production VM Security VM
User
Kernel
User
Kernel
Suspect ProcessStub Monitor
29
� Stub protection
� Hypervisor-protected memory and page tables
![Page 30: Process Out-Grafting: An Efficient “Out-of-VM” …...Process Out-Grafting: An Efficient “Out-of-VM” Approach for Fine-Grained Process Execution Monitoring Deepa Srinivasan,](https://reader033.vdocuments.site/reader033/viewer/2022041601/5e30e509af069308b52d6d80/html5/thumbnails/30.jpg)
Security Analysis
30
� Out-grafting detection
� Strong policy for random, arbitrary out-grafting
![Page 31: Process Out-Grafting: An Efficient “Out-of-VM” …...Process Out-Grafting: An Efficient “Out-of-VM” Approach for Fine-Grained Process Execution Monitoring Deepa Srinivasan,](https://reader033.vdocuments.site/reader033/viewer/2022041601/5e30e509af069308b52d6d80/html5/thumbnails/31.jpg)
Case Studies
� thttpd as test process for out-grafting
� Both disk and network usage
� Support existing in-host tools, out-of-VM
� strace, ltrace, gdb
� OmniUnpack (Martignoni et al., ACSAC ‘07)
31
![Page 32: Process Out-Grafting: An Efficient “Out-of-VM” …...Process Out-Grafting: An Efficient “Out-of-VM” Approach for Fine-Grained Process Execution Monitoring Deepa Srinivasan,](https://reader033.vdocuments.site/reader033/viewer/2022041601/5e30e509af069308b52d6d80/html5/thumbnails/32.jpg)
Case Studies
� thttpd as test process for out-grafting
� Both disk and network usage
� Support existing in-host tools, out-of-VM
� strace, ltrace, gdb
� OmniUnpack (Martignoni et al., ACSAC ‘07)
32
![Page 33: Process Out-Grafting: An Efficient “Out-of-VM” …...Process Out-Grafting: An Efficient “Out-of-VM” Approach for Fine-Grained Process Execution Monitoring Deepa Srinivasan,](https://reader033.vdocuments.site/reader033/viewer/2022041601/5e30e509af069308b52d6d80/html5/thumbnails/33.jpg)
strace
33
In-VM strace Out-of-VM strace
![Page 34: Process Out-Grafting: An Efficient “Out-of-VM” …...Process Out-Grafting: An Efficient “Out-of-VM” Approach for Fine-Grained Process Execution Monitoring Deepa Srinivasan,](https://reader033.vdocuments.site/reader033/viewer/2022041601/5e30e509af069308b52d6d80/html5/thumbnails/34.jpg)
Code Unpacking Detection
� OmniUnpack (Martignoni et al., ACSAC ‘07)
� Track page writes and executions
� Detect unpacking when executing a previously-written
page
� Faithful reproduction of algorithm in Linux
� Kernel module in security VM
34
Localized overhead in
out-of-VM monitoring
Thorough test of page
fault forwarding
NX bit support only in
security VM kernel
![Page 35: Process Out-Grafting: An Efficient “Out-of-VM” …...Process Out-Grafting: An Efficient “Out-of-VM” Approach for Fine-Grained Process Execution Monitoring Deepa Srinivasan,](https://reader033.vdocuments.site/reader033/viewer/2022041601/5e30e509af069308b52d6d80/html5/thumbnails/35.jpg)
Performance
� Dell T1500, Intel Core i7, 4 cores, 2.6 GHz, 4 GB
RAM
� VM configuration
� Production VM: 1 VCPU, 2047 MB RAM
� Security VM: 1 VCPU, 1 GB RAM
35
![Page 36: Process Out-Grafting: An Efficient “Out-of-VM” …...Process Out-Grafting: An Efficient “Out-of-VM” Approach for Fine-Grained Process Execution Monitoring Deepa Srinivasan,](https://reader033.vdocuments.site/reader033/viewer/2022041601/5e30e509af069308b52d6d80/html5/thumbnails/36.jpg)
Performance
36
� Inter-VM system call (getpid): ~11 µs
� Process state identification: ~250 µs
� Slowdown to out-grafted process
� File-copy time: 35.42%
� thttpd throughput: 7.38%
![Page 37: Process Out-Grafting: An Efficient “Out-of-VM” …...Process Out-Grafting: An Efficient “Out-of-VM” Approach for Fine-Grained Process Execution Monitoring Deepa Srinivasan,](https://reader033.vdocuments.site/reader033/viewer/2022041601/5e30e509af069308b52d6d80/html5/thumbnails/37.jpg)
Performance
Production VM Slowdown with a Contending Process Out-grafted
-6%
-4%
-2%
0%
2%
4%
6%
8%
37
![Page 38: Process Out-Grafting: An Efficient “Out-of-VM” …...Process Out-Grafting: An Efficient “Out-of-VM” Approach for Fine-Grained Process Execution Monitoring Deepa Srinivasan,](https://reader033.vdocuments.site/reader033/viewer/2022041601/5e30e509af069308b52d6d80/html5/thumbnails/38.jpg)
Rest of This Talk
� Motivation
� System Design
� Implementation & Evaluation
� Related Work
� Conclusion
38
![Page 39: Process Out-Grafting: An Efficient “Out-of-VM” …...Process Out-Grafting: An Efficient “Out-of-VM” Approach for Fine-Grained Process Execution Monitoring Deepa Srinivasan,](https://reader033.vdocuments.site/reader033/viewer/2022041601/5e30e509af069308b52d6d80/html5/thumbnails/39.jpg)
Related Work
� VM introspection
� Livewire (Garfinkel et al., NDSS ‘03), XenAccess (Payne et al., ACSAC ‘07), VMScope (Jiang et al., RAID ‘07), Lares (Payne et al., Oakland ‘08), VMWatcher (Jiang et al., CCS ‘07), Virtuoso (Dolan-Gavitt et al., Oakland ‘11)
� Efficient, isolated monitoring
� SIM (Sharif et al., CCS ’09)
� Process migration
� BLCR (Smith, UCB-TR ‘08), Zap (Osman et al., OSDI ‘02), Migration survey (Nuttall et al., ACM OS Review ’94)
� System call forwarding
� Application protection (Ta-Min et al., OSDI ‘06), Monitoring fidelity (Martignoni et al., ICISS ‘09)
� Sandboxing, isolation techniques
� Ostia (Garfinkel et al., NDSS ‘04), Janus (Goldberg et al., Security ‘96)
39
![Page 40: Process Out-Grafting: An Efficient “Out-of-VM” …...Process Out-Grafting: An Efficient “Out-of-VM” Approach for Fine-Grained Process Execution Monitoring Deepa Srinivasan,](https://reader033.vdocuments.site/reader033/viewer/2022041601/5e30e509af069308b52d6d80/html5/thumbnails/40.jpg)
Conclusion
Hypervisor
Production VM Security VM
User
Kernel
User
Kernel
Suspect ProcessStub
Technique I: On-Demand Grafting
Technique II: Mode-sensitive Split Execution
Monitor
40
Process Out-grafting: An Efficient Out-of-VM Approach for
Fine-grained Process Execution Monitoring (CCS ‘11)
![Page 41: Process Out-Grafting: An Efficient “Out-of-VM” …...Process Out-Grafting: An Efficient “Out-of-VM” Approach for Fine-Grained Process Execution Monitoring Deepa Srinivasan,](https://reader033.vdocuments.site/reader033/viewer/2022041601/5e30e509af069308b52d6d80/html5/thumbnails/41.jpg)
Thank you!
Questions?