trustworthy services built on event based probing …...trustworthy services built on event based...

Trustworthy Services Built on Event Based Probing for Layered

DefenseRead Sprabery, Zak Estrada, Zbigniew Kalbarczyk, Ravishankar Iyer,

Rakesh Bobba*, Roy CampbellUniversity of Illinois at Urbana-Champaign, *Oregon State University

IEEE International Conference on Cloud Engineering 2017

Virtualization Provided Security

2

Virtualization Provided Security

3

Passive Monitoring

4

Presenter

Presentation Notes

Security via Introspection techniques began with passive monitoring As highlighted by LibVMI

Passive Monitoring

5

Presenter

Presentation Notes

LibVMI provides a bridge

Passive Monitoring

6

Presenter

Presentation Notes

Allowing the VVM to inspect guest memory

Passive Monitoring for Intrusion Detection

7

Presenter

Presentation Notes

And detect malware The issue is that this relies on a full scan of guest memory to be effective. We’d really like to catch events as they happen

Active Monitoring (Dynamic Probing)

8

Presenter

Presentation Notes

Led to the rise of active monitoring Via dynamically inserted probes


9

Presenter

Presentation Notes

Only monitor critical components


10

Presenter

Presentation Notes

Upon hitting those components trap to the VMM

Dynamic Probing for Intrusion Detection

11

Presenter

Presentation Notes

At this point, introspection or logging can be performed

Dynamic Probing for Intrusion Detection

Existing Techniques: only monitored some Δt after guest boot sequence.

12

Presenter

Presentation Notes

Critical locations on monitored sometime after boot Dynamic probes may miss select events and do not have a good root of trust because of insertion time We want to build a trusted log from this data Don’t want to miss even a single event The guest kernel is trusted because we know that it boots into a known, trusted state. The probe must be inserted before the first user space process runs to build on that trust Of course, these VMMs are being used to deploy real applications in the clouds, many of which conform to the “micro-service” design pattern

Micro-Services and Virtual Appliances

13

Presenter

Presentation Notes

Our system is tuned to these services, so we’ll review them here quickly.


14


15


16


17


18


19

Presenter

Presentation Notes

A micro-service is a service that can stand on it’s own And is often packed into an individually distributable unit But makes up part of a larger service or application


20


21

Presenter

Presentation Notes

Micro-services is a deployment paradigm to make deploying large services easier by breaking them up into manageable pieces We extend this concept to security policies for intrusion detect services as the system layer

From Unmanageable Rule Sets

• Complex deployments• Many components per virtual

machine• Too many valid events occurring to

know which are security sensitive

22

Presenter

Presentation Notes

The problem with IDS is that it can be hard to quantify normal behavior Reduced behavior of micro-services allow us to move from unmanageable rulesets

To Manageable Rule Sets

• Simple deployments• Single service per machine• Normal behavior is easier to

classify

23

Presenter

Presentation Notes

To clearly defined behaviors that are easy to reason about

Overview

• Cloud Tuned Trustworthy Services• Contributions• Attack Model• Logged Events• Policy Recording Mechanism• Intrusion Detection System for Micro-Services• Evaluation• Conclusion and Future Work

24

Presenter

Presentation Notes

Review Background Highlight our techniques and attack model End with an evaluation and future direction

Cloud Tuned Trustworthy Services

• What does it mean to be cloud “tuned”?• There are two key observations

• Service Oriented Architectures• Existing Monitoring Frameworks (with extensions)

25

Presenter

Presentation Notes

Cloud tuned means that we can leverage SOA and extend existing monitoring frameworks to Provide more effective and easier to manage security

Trustworthy Services• Built on a trustworthy logging solution• Trustworthy logging can be the centerpiece of trusted services in the

VMM if:• The log is not missing data• And is difficult to tamper with

• Limitations of event-based probing• How can we place trust in a log generated by dynamically inserted probes?• How can we guarantee completeness – guaranteeing every invocation of an

event under inspection must appear in the log?

• Which events should be logged to provide the most coverage and least performance impact?

• exec• open 26

Presenter

Presentation Notes

Completeness is important because we may miss certain events after boot that could be malicious Attacks are scripted and could be executed quickly We want the same root of trust guarantees for dynamic probes Trustworthy services are important so that we can place trust in forensic tools and in IDS built

Monitoring Architectures for Cloud Computing

• Virtual Machine Introspection Techniques• Passive Monitoring – monitoring as a side-effect of the VMM/virtualization

technology (early approaches).• Event based probing – placing probes at specific instructions in the guest

• Can trigger events on any location• HyperProbes [11], DRAKVUF [12], Xenprobes [22], Spider [23]• These techniques use instruction replacement, causing a trap to exit to the VMM

• Running beneath existing workloads• In containers, this would be the host kernel

27

Presenter

Presentation Notes

References: [11] - Z. J. Estrada, C. Pham, F. Deng, L. Yan, Z. Kalbarczyk, and R. K. Iyer, “Dynamic VM Dependability Monitoring Using Hypervisor Probes,” in Dependable Computing Conference (EDCC), 2015 Eleventh European. IEEE, 2015, pp. 61–72. [12] - T. K. Lengyel, S. Maresca, B. D. Payne, G. D. Webster, S. Vogl, and A. Kiayias, “Scalability, fidelity and stealth in the DRAKVUF dynamic malware analysis system,” in 30th Annual Computer Security Applications Conference on (ACSAC), 2014. [22] - N. A. Quynh and K. Suzaki, “Xenprobes, a lightweight user-space probing framework for xen virtual machine,” in USENIX Annual Technical Conference (ATC ’07). [23] - Z. Deng, X. Zhang, and D. Xu, “Spider: Stealthy binary program instrumentation and debugging via hardware virtualization,” in Proceedings of the 29th Annual Computer Security Applications Conference. ACM, 2013, pp. 289–298

int3 probe forwarder

Dynamic Probing Mechanisms

KVM Hypervisor

Host Linux Kernel

Unmonitored Guest

Guest Kernel Address Space

0xFFFFC08c|sys_exec

0xFFFFC060|sys_open

Unmonitored Guest


0xFFFFC08c|sys_exec

0xFFFFC060|sys_open

Unmonitored Guest


0xFFFFC08c|sys_exec

0xFFFFC060|sys_open

SysOpenProbeSysExecProbe

Monitored Guest


0xFFFFC08c|sys_exec|int3

0xFFFFC060|sys_open|int3

Monitored Guest


0xFFFFC08c|sys_exec|int3

0xFFFFC060|sys_open|int3Existing Techniques: Probes inserted after guest boot sequence.

28

Presenter

Presentation Notes

Inserting probes after boot removes ability to know that no malware has been run between when the system was booted and when the probe was inserted If you do not have this, then attack detection becomes probabilistic

Contributions

• Algorithm for inserting probes at time of boot before probed instruction is invoked

• Methods to protect against of attacks that attempt to circumvent the logging system

• An IDS built on top of the logging system, tuned to service oriented Virtual Appliances

• Semi-automated policy-recording mechanism • Evaluation of the performance in terms of impact to applications and

in terms of detection capabilities of the IDS

29

Presenter

Presentation Notes

Our main contributions include ensuring the output of the logging system can be trusted We do this by providing guarantees for when probes are inserted And by exploring an attack model that may try to execute logged actions while going unlogged Our policy recording layer captures normal behavior with minimal administrator involvement This logging system has enhanced trust from the additions with the dynamic probes The logging is then used to build an IDS for micro-services

Integrity of Dynamically Inserted Probe

Hypervsior:17643c29e3c4609818f26becf76d29a3

09355fb2e43cf5169 7a15421816899be14

VM1

8533f92c020f68bf

VM2

fbd16a4a0d84e124

30

Presenter

Presentation Notes

Attested to integrity Hypervisor Virtual disk images Guest kernel How can we extend to dynamic probes? The guest kernel is only trusted until the first user space process executes We can use this knowledge to ensure probes are inserted while we still trust the kernel Why not just compile probes in to guests? Hard to adopt

Log Completeness

• Probes must be placed in their respective locations before instructions at those locations have executed even once.

• System calls being probed are loaded at a predictable location in guest physical address space

• Determines the page number containing the target instruction• Induce EPT violations on the page of interest to determine when to load

probe

31

Presenter

Presentation Notes

Timing of placement of probes Can use predictability of locations being probed Next I will go into the details of our approach by first introducing extended page talbes A form of hardware accelerated virtualization we leverage in our solution

Extended Page Tables

• 2-Level pages tables from Intel• Allows for direct translation from guest physical address to host

physical address in hardware• Per page RWX-permissions can be configured within the VMM• Hardware accelerated virtualization technology

• AMD’s equivalent is Nested Paging

32

Presenter

Presentation Notes

We leverage the VMM controlled permissions to induce an EPT violation pattern to signal when a probe should be inserted.

Induced EPT Sequence for Probe Insertion

RWX110

Ensure eXecute bit is disabled, enable

writes

Guest loads kernel page

RWX100

Initial write violation during

guest boot

RWX110

Execute violation on any instruction

on page

RWX101

Load probes on page, disable writes, enable

execute

33

Presenter

Presentation Notes

Introduce sequence Now let’s talk about our attack model

HardwareInvocation

Hardware Software

IDTR/MSRs sys_call_tableSYSTEM_CALL SYS_EXECVE

System Call Stack Trace

34

Presenter

Presentation Notes

Motivated by a stack trace leading up to the probed location The specific system calls being monitored

Attack Model

Invocation

Hardware Software

IDTR/MSRs sys_call_tableSYSTEM_CALL SYS_EXECVEHardware

SYSTEM_CALL: …

…

enable_interrupts()……

SYS_EXECVE:

call *sys_call_table(,%rax,8)

Interrupts Disabled

Risk of Timing Attack

Event Logged

12 Instructions

1

2

3

4 5

6

NEFARIOUS CODE BLOCK

35

Presenter

Presentation Notes

Circumvention Means an attacker wants to perform an “open” or “exec” system call and go unlogged Assume that the attacker has compromised the kernel via an unmonitored system call What is the extend of the damage that can be done? What can we do to combat it (on next slides) (2) – performance overhead changes depending on where you place the probe.

Logged Events & Probe Placement

• Increase the cost of compromising log completeness:• Use existing trusted boot techniques for both the hypervisor [13] and guests

[14], [15].• Write protect the probed location.• Monitor specific hardware registers.

• Log minimal number of locations to provide meaningful information for higher level services

• To reduce TCB and performance impacts

36

Logged Events & Probe Placement

OPEN

EXEC

• lidt• wrmsr

• Writes to probes

37

Presenter

Presentation Notes

At the beginning we talked about monitoring critical components Part of this work has been identifying what critical components must be monitored to raise cost of log circumvention On certain hardware events Only called during operating system boot lidt wrmsr Configured within the VMM Should only occur once during boot, not used after that Page table violations on probed locations Write to pages Trigger of actual probes We are only monitoring two system calls. We believe these can be used effectively to catch malicious behavior. Monitoring only two system calls allows to to keep TCB low Keep perf. Impact low

Output Log

TIME: VMID: X, …, LOGGER: SYS_EXEC, BEGIN

TIME: VMID: X, …, LOGGER: SYS_EXEC, ARG

TIME: VMID: X, …, LOGGER: SYS_EXEC, END

…

TIME: VMID: Y, …, LOGGER: SYS_EXEC, BEGIN

…

Event Parsing

Log Buffer

{B1,A1,…,E1} -> Event 𝜀𝜀1

Hypervisor Probes

SysExecProbe

SysOpenProbe{Bn,An,…,En} -> Event 𝜀𝜀n

…

Policies

{exec: {filename: /sbin/dhclient-script}}

{open: {read, filename: /sbin/resolvconf}}

Alert System

Policy Reader

Event Monitor & Policy Alerts

Policy Recorder

Event 𝜀𝜀1 -> Policy P1

…

Event 𝜀𝜀2 -> Policy P2

Overview of Trustworthy Services Architecture

38

Example Policy: $(which) command

39

Policy Stacking

dhcp.policy

wordpress.policy mysql.policy

40

Evaluation

• Performance• Cloud workloads:

• OpenSSL – request encryption• Apache – web serving• Redis – key/value store

• Attacks• Can the system detect attacks on scale out cloud applications?

41

Evaluation – Performance (Apache & OpenSSL)

42

Presenter

Presentation Notes

Performance tradeoff depends on where the probe was placed Up to the user depending on how comfortable they are with the tradeoffs

Evaluation – Performance (Redis)

• A – no probes• B – specific handlers• C – general handler 43

Evaluation – Attack Scenario

• Wordpress vulnerability• AJAX Load More Plugin

• Allowed a user to inject arbitrary PHP code• User passwords for WordPress accounts are easy to guess• Full control over the www-data user (web server user)

• Linux Mint – ISO’s compromised through WordPress vulnerability• Attackers used a similar vulnerability to get shell as the www-data user• URL’s for ISO’s were changed to a malicious image

44

Evaluation – Under Attack

45

Presenter

Presentation Notes

We want to highlight the IDS Stack policies Detection of attacks using automated policies


46


47


• Attacker opens a shell

48


• Attacker opens a shell

49

Presenter

Presentation Notes

An alert is generated because the attacker used an open or exec (exec in this case) system call in a way that was not recorded when we ran our policy recording mechanism More trust can be placed in this because it is being run in the VMM and is harder to circumvent

Limitations

• Limited detection for intra-process attacks• Attack(er) must perform an action outside of defined behavior

• Access a file• Execute a binary

• Attack will go undetected if it remains entirely inside of the compromised binary

• Inside of the PHP interpreter in the given example.

50

Conclusion & Future Work

• User configurable tradeoff between resilience and performance• Via probe location

• Algorithm for trustworthy insertion of dynamic probes• Low performance overheads in many cases• Future work:

• Runtime integrity checking (detecting intra-process attacks)• Improving log resilience at lower performance costs

51

[email protected]

52

trustworthy services built on event based probing …...trustworthy services built on event based...

Documents