VMM Based Rootkit Detection on Android

Class PresentationPete Bohman, Adam Kunk, Erik Shaw

Motivation

The Increase of Mobile Malware Variants (2004 – 2010)

• Smartphone malware on the rise• Increased security implications (compared to PC)• Sensitive information: GPS, contacts, SMS, call log• Constantly connected to the Internet• Naïve users, limited use of anti-virus protection

Defensive Rootkit Approaches• User mode rootkits

– Process infection, binary patching, lib hooks

• User mode integrity checkers– tripwire, chkrootkit, rkthunter, AV scanner

• Kernel mode rootkits– malicious device drivers and LKMs– sys call hooking, kernel data structure manipulation (DKOM)

• Kernel level inspection– behavioral analysis, data structure integrity checkers, hook detection

• But… Any kernel level inspection mechanisms can be subverted by kernel level rootkits!

Our Approach

• Two Pronged• KM security mechanisms • System call integrity checks• Hidden process detection• ACLs for Android personal data

• VMM inspection • Ensures integrity of static KM• Isolated from host OS

• We exercise a “layer-below” level of security in which we establish trust beneath the kernel

Android Software Stack

Overview

• Design• VMM Interface Design• Protection KM Design• Implementation• Results• Demo Presentation• Conclusion• Q&A

VMM Interface Design

Android VMM

Hardware (Emulator)

Linux Kernel

Trusted KM <ISR>

Libraries and Runtime

Application Framework

1. Hardware Timer Interrupt

2. Validate Protected KM 3. Raise Monitor Interrupt

4. Invoke KM

Protection KM Design

Linux KernelTrusted KMSystem Call WhitelistOriginal Sys Call Table

Libraries and Runtime

Maps

Application Framework

Contacts SMS App

Content Provider

Location Provider

Activity Manager

Open

Malicious Native

ApplicationSystem Calls

Open SocketRead GPSSQL QuerySys Call Table

…

Protection KM Design

Linux KernelTrusted KM

System Call WhitelistOriginal Sys Call Table

Libraries and Runtime

X

MaliciousLKM

System Calls

Sys Call Table

Y Z<ISR>

Android VMM

Monitor Interrupt

Linux Kernel

Implementation• VMM Interface• Implemented VMM security functionality in an emulated

hardware device within QEMU• Protected KM data and text compiled into QEMU emulator

(VMM)

Linux Kernel Source with

Protection KMCompilation

Kernel Image

Protected Text Sect.

Protected Data Sect.

QEMU Emulator(VMM) Compilation

QEMU Emulator(VMM)Protected TextProtected Data

Implementation

• Protection KM• Checks system service function pointers

in sys_call_table

• Reports problem and restores them ifthey have been altered

• Hooks sys_open and prevents access to mmssms.db and contacts2.db• Unless process name and parent process name are found

in the access control list

• Likewise restricts socket and sys_read of GPS data according to the access control list

Implementation

• Malicious native mode application• Reads contacts database file• Reads GPS location• Transmits data using network sockets• Runs as Linux user mode process,

underneath the Android ApplicationFramework

• Malicious LKM• Implemented by following the MindTrick Rootkit techniques• Intercepts read system calls to access GPS location• Attempts to access the SMS and contacts database files

Results• We are able to detect and correct modifications to the

sys_call_table• Malicious LKM system call table hooking was thwarted• Malicious LKM unable to obtain GPS coordinates

• We are able to prevent malicious access to sensitive resources• Malicious application fails to read GPS data, fails to open sensitive

database files, and fails to open network sockets• Malicious LKM also fails to open sensitive database files

• Demo (short): http://www.youtube.com/watch?v=IR9yDyT50HQ

• Demo (long):• http://www.youtube.com/watch?v=4M5YgsGU9Bw

Conclusion

• “Layer-Below” Protection• Security of the Linux kernel must be rooted in a layer

below the kernel to withstand kernel level attacks

• Trusted “layer-below” protection mechanisms can serve as the secure foundation for enabling additional advanced protection mechanisms in the kernel

Questions?

Backup – Android Architecture

Backup – Protection by Layer

App PermissionsDalvik VM IsolationApp signatures

Limits application abilities in order to prevent malicious behavior.

Virus ScannersRemote Lockout

Modified system binariesTrojan’d servicesStolen device

Linux user and group permissions

Access control

Architecture Layer Security Mechanism Threat Mitigation

Backup – Problem Statement

• Rootkit detection and prevention on the Android platform with specific regards to the sensitive resources Android provides.• Kapersky 2011: 1046 unique malware strains

targeting mobile platforms• Android platform built on Linux Kernel, a well known

target.• Sensitive information on smart phones• GPS, contacts, text messages, call log