virtual machine introspection · virtual machine introspection isolation interpretation...
TRANSCRIPT
![Page 1: Virtual Machine Introspection · Virtual Machine Introspection Isolation Interpretation Interposition. Isolation From in-guest kernel/userspace • Provided by Xen • Buggy emulation](https://reader030.vdocuments.site/reader030/viewer/2022040215/5f0625667e708231d41685ab/html5/thumbnails/1.jpg)
![Page 2: Virtual Machine Introspection · Virtual Machine Introspection Isolation Interpretation Interposition. Isolation From in-guest kernel/userspace • Provided by Xen • Buggy emulation](https://reader030.vdocuments.site/reader030/viewer/2022040215/5f0625667e708231d41685ab/html5/thumbnails/2.jpg)
Virtual Machine Introspection
● Isolation
● Interpretation
● Interposition
![Page 3: Virtual Machine Introspection · Virtual Machine Introspection Isolation Interpretation Interposition. Isolation From in-guest kernel/userspace • Provided by Xen • Buggy emulation](https://reader030.vdocuments.site/reader030/viewer/2022040215/5f0625667e708231d41685ab/html5/thumbnails/3.jpg)
Isolation
● From in-guest kernel/userspace• Provided by Xen
• Buggy emulation blurres the line
● From trusted computing base (TCB)• Possible via Xen Security Modules
• Move introspection system out from dom0!
![Page 4: Virtual Machine Introspection · Virtual Machine Introspection Isolation Interpretation Interposition. Isolation From in-guest kernel/userspace • Provided by Xen • Buggy emulation](https://reader030.vdocuments.site/reader030/viewer/2022040215/5f0625667e708231d41685ab/html5/thumbnails/4.jpg)
![Page 5: Virtual Machine Introspection · Virtual Machine Introspection Isolation Interpretation Interposition. Isolation From in-guest kernel/userspace • Provided by Xen • Buggy emulation](https://reader030.vdocuments.site/reader030/viewer/2022040215/5f0625667e708231d41685ab/html5/thumbnails/5.jpg)
![Page 6: Virtual Machine Introspection · Virtual Machine Introspection Isolation Interpretation Interposition. Isolation From in-guest kernel/userspace • Provided by Xen • Buggy emulation](https://reader030.vdocuments.site/reader030/viewer/2022040215/5f0625667e708231d41685ab/html5/thumbnails/6.jpg)
Interposition
● Trap to Xen when something of interest happens within the guest• Enable optional hardware traps• CLTS, HLT, LGDT, LIDT, LLDT, LTR, SGDT, MOV from
CR3, MOV from CR8, MOV to CR0, MOV to CR3, MOV to CR4, MOV to CR8, MOV DR, MWAIT, INT3, INT0, MTF, etc..
• See full list in Intel SDM 3c 25.1.3
![Page 7: Virtual Machine Introspection · Virtual Machine Introspection Isolation Interpretation Interposition. Isolation From in-guest kernel/userspace • Provided by Xen • Buggy emulation](https://reader030.vdocuments.site/reader030/viewer/2022040215/5f0625667e708231d41685ab/html5/thumbnails/7.jpg)
Interposition
● Change access permissions in EPT
● Trap violation into Xen
● R/W/X
● With some caveats
![Page 8: Virtual Machine Introspection · Virtual Machine Introspection Isolation Interpretation Interposition. Isolation From in-guest kernel/userspace • Provided by Xen • Buggy emulation](https://reader030.vdocuments.site/reader030/viewer/2022040215/5f0625667e708231d41685ab/html5/thumbnails/8.jpg)
EPT caveats
“An EPT violation that occurs during as a result of execution of a read-modify-write operation sets bit 1 (data write). Whether it also sets bit 0 (data read) is implementation-specific and, for a given implementation, may differ for different kinds of read-modify-write operations.” - Intel SDM 3c
![Page 9: Virtual Machine Introspection · Virtual Machine Introspection Isolation Interpretation Interposition. Isolation From in-guest kernel/userspace • Provided by Xen • Buggy emulation](https://reader030.vdocuments.site/reader030/viewer/2022040215/5f0625667e708231d41685ab/html5/thumbnails/9.jpg)
EPT caveats
● “Why can't the hardware report the true characteristics right away?” - Jan Beulich
● “when spec says so, there is a reason but I can't tell here. :-)” - Kevin Tian
● Well.. let's just mark all write volation as read violation too..
● Patched in Xen 4.5
![Page 10: Virtual Machine Introspection · Virtual Machine Introspection Isolation Interpretation Interposition. Isolation From in-guest kernel/userspace • Provided by Xen • Buggy emulation](https://reader030.vdocuments.site/reader030/viewer/2022040215/5f0625667e708231d41685ab/html5/thumbnails/10.jpg)
![Page 11: Virtual Machine Introspection · Virtual Machine Introspection Isolation Interpretation Interposition. Isolation From in-guest kernel/userspace • Provided by Xen • Buggy emulation](https://reader030.vdocuments.site/reader030/viewer/2022040215/5f0625667e708231d41685ab/html5/thumbnails/11.jpg)
![Page 12: Virtual Machine Introspection · Virtual Machine Introspection Isolation Interpretation Interposition. Isolation From in-guest kernel/userspace • Provided by Xen • Buggy emulation](https://reader030.vdocuments.site/reader030/viewer/2022040215/5f0625667e708231d41685ab/html5/thumbnails/12.jpg)
altp2m
● Add support for multiple EPTs for second stage lookup!
● One table for “restricted view”
● One table for “normal view”
![Page 13: Virtual Machine Introspection · Virtual Machine Introspection Isolation Interpretation Interposition. Isolation From in-guest kernel/userspace • Provided by Xen • Buggy emulation](https://reader030.vdocuments.site/reader030/viewer/2022040215/5f0625667e708231d41685ab/html5/thumbnails/13.jpg)
altp2m
● EPT pointer can be swapped in the VMCS
● No need to change EPT PTE permissions all the time
● No race condition
![Page 14: Virtual Machine Introspection · Virtual Machine Introspection Isolation Interpretation Interposition. Isolation From in-guest kernel/userspace • Provided by Xen • Buggy emulation](https://reader030.vdocuments.site/reader030/viewer/2022040215/5f0625667e708231d41685ab/html5/thumbnails/14.jpg)
Interposition
● Once trapped to Xen, forward events• Formerly known as mem_event
• Renamed and reworked as vm_event in 4.6
● Request/response via shared memory ring• Monitor page used for VMI related events
• Two additional pages: memory sharing and paging
![Page 15: Virtual Machine Introspection · Virtual Machine Introspection Isolation Interpretation Interposition. Isolation From in-guest kernel/userspace • Provided by Xen • Buggy emulation](https://reader030.vdocuments.site/reader030/viewer/2022040215/5f0625667e708231d41685ab/html5/thumbnails/15.jpg)
vm_event & mem_access & monitor
● Let's keep track of subsystem names
● vm_event is the underlying request/response mechanism
● mem_access memops control EPT
● monitor_op domctls control all other optional VM execution traps
![Page 16: Virtual Machine Introspection · Virtual Machine Introspection Isolation Interpretation Interposition. Isolation From in-guest kernel/userspace • Provided by Xen • Buggy emulation](https://reader030.vdocuments.site/reader030/viewer/2022040215/5f0625667e708231d41685ab/html5/thumbnails/16.jpg)
Event delivery structures in 4.6
● Defined in xen/vm_event.h public header
● Easily extendable and versioned
● No more hackery
● Event response can trigger specific behavior without additional hypercalls• Trigger emulation, singlestepping, swap altp2m...
![Page 17: Virtual Machine Introspection · Virtual Machine Introspection Isolation Interpretation Interposition. Isolation From in-guest kernel/userspace • Provided by Xen • Buggy emulation](https://reader030.vdocuments.site/reader030/viewer/2022040215/5f0625667e708231d41685ab/html5/thumbnails/17.jpg)
VMI with Xen on ARM
● ARM has two-stage paging similar to EPT
● mem_access implemented for 4.6
● Some caveats:• No singlestepping?
• Can be worked around but it's a pain
• Split-TLB ambiguities
![Page 18: Virtual Machine Introspection · Virtual Machine Introspection Isolation Interpretation Interposition. Isolation From in-guest kernel/userspace • Provided by Xen • Buggy emulation](https://reader030.vdocuments.site/reader030/viewer/2022040215/5f0625667e708231d41685ab/html5/thumbnails/18.jpg)
ARM mem_access
● ARM PTEs have fewer software programmable bits as compared to EPT
● ARM mem_access requires maintaining a Radix-tree to keep track of PTEs with custom permissions
● Radix-tree keyed with GPA
![Page 19: Virtual Machine Introspection · Virtual Machine Introspection Isolation Interpretation Interposition. Isolation From in-guest kernel/userspace • Provided by Xen • Buggy emulation](https://reader030.vdocuments.site/reader030/viewer/2022040215/5f0625667e708231d41685ab/html5/thumbnails/19.jpg)
ARM mem_access
● For a 2nd stage violation ARM provides the faulting GVA
● GPA only provided if fault happened during 1st stage pagetable walk
● Xen needs to translate GVA to GPA to perform Radix-tree lookup
![Page 20: Virtual Machine Introspection · Virtual Machine Introspection Isolation Interpretation Interposition. Isolation From in-guest kernel/userspace • Provided by Xen • Buggy emulation](https://reader030.vdocuments.site/reader030/viewer/2022040215/5f0625667e708231d41685ab/html5/thumbnails/20.jpg)
ARM mem_access
● Native CPU instructions to perform GVA to GPA translation
● Performs lookup as data-fetch access
● What if we trapped an instruction-fetch access?• In-guest translation hits iTLB
• Xen hits dTLB
![Page 21: Virtual Machine Introspection · Virtual Machine Introspection Isolation Interpretation Interposition. Isolation From in-guest kernel/userspace • Provided by Xen • Buggy emulation](https://reader030.vdocuments.site/reader030/viewer/2022040215/5f0625667e708231d41685ab/html5/thumbnails/21.jpg)
● Split-TLB is a real rootkit problem• ShadowWalker, MoRE, etc..
● Guest can load the iTLB with rootkit page and dTLB with benign page
● Flushing the TLB does not help, iTLB translation may be lost if PT no longer represents the cached translation
ARM Split-TLB problem
![Page 22: Virtual Machine Introspection · Virtual Machine Introspection Isolation Interpretation Interposition. Isolation From in-guest kernel/userspace • Provided by Xen • Buggy emulation](https://reader030.vdocuments.site/reader030/viewer/2022040215/5f0625667e708231d41685ab/html5/thumbnails/22.jpg)
● Execution tracing with mem_access may be problematic
● Use Secure Monitor Call (SMC) instruction injection!
● Similar to 0xCC injection on x86
● TODO
ARM future work
![Page 23: Virtual Machine Introspection · Virtual Machine Introspection Isolation Interpretation Interposition. Isolation From in-guest kernel/userspace • Provided by Xen • Buggy emulation](https://reader030.vdocuments.site/reader030/viewer/2022040215/5f0625667e708231d41685ab/html5/thumbnails/23.jpg)
● altp2m is primarily designed to be used with Intel #VE
● VMCALL instruction to perform EPTP switching from the guest
● Hybrid VMI
x86 future work
![Page 24: Virtual Machine Introspection · Virtual Machine Introspection Isolation Interpretation Interposition. Isolation From in-guest kernel/userspace • Provided by Xen • Buggy emulation](https://reader030.vdocuments.site/reader030/viewer/2022040215/5f0625667e708231d41685ab/html5/thumbnails/24.jpg)
● Why aren't we using git pulls?• Patches in mailinglist without branch-off point
specified
• Carving patches from mbox is a pain
• Start providing a public git branch for your series!!
Lessons learnt
![Page 25: Virtual Machine Introspection · Virtual Machine Introspection Isolation Interpretation Interposition. Isolation From in-guest kernel/userspace • Provided by Xen • Buggy emulation](https://reader030.vdocuments.site/reader030/viewer/2022040215/5f0625667e708231d41685ab/html5/thumbnails/25.jpg)
● Provide build-testing for the community• It's a waste of time to wait for review on
something that's broken
• Check for style issues automatically?
• Travis-CI is OK but can time-out on large series
• https://github.com/tklengyel/xen/tree/travis
Lessons learnt
![Page 26: Virtual Machine Introspection · Virtual Machine Introspection Isolation Interpretation Interposition. Isolation From in-guest kernel/userspace • Provided by Xen • Buggy emulation](https://reader030.vdocuments.site/reader030/viewer/2022040215/5f0625667e708231d41685ab/html5/thumbnails/26.jpg)
Thanks!
Tamas K Lengyel [email protected] [email protected] @tklengyel
LibVMI http://libvmi.comDRAKVUF http://drakvuf.com