virtualizing java for the cloud using a jvm hypervisor

© Copyright Waratek 2013

!

Virtualizing Java for the Cloud… John Ma(hew Holt | NY JavaSIG, November 2013


John Matthew Holt!•  CTO and co-‐Founder of Waratek •  Java Virtual Machine (JVM) engineer for over 10 years

•  Invented “Java VirtualizaEon” to transform how JVMs operate in virtualized and cloud-‐compute environments

•  Lead the design of the first Type-‐1 VMM (hypervisor) for a JVM


What will we cover?!•  First some background…

– Why virtualize INSIDE the JVM?

•  Virtualizing Java for the Cloud – Think “VMware for Java” – Case study 1: Java’s (bad) legacy – Case study 2: Java’s (in)security


First some background…!

• Why virtualize INSIDE the JVM?


Because…!•  Because JVMs make bad bedfellows with HW hypervisors (HV)

•  A JVM thwarts HW hypervisor features like: –  The Garbage Collector (GC) in a JVM thwarts memory-‐overcommit in a HW HV

–  The GC’s heap memory, and the Just-‐In-‐Time (JIT) permgen/codecache, thwarts ‘transparent page sharing’ (memory de-‐dup) in a HW HV

– Dedicated GC and JIT for EVERY Java applicaEon wastes lots and lots of CPU Eme and memory capacity doing the same thing over, and over, and over, and …


Todayʼs JVMs are OLD!•  JVMs were designed ~20 years ago, and haven’t changed design since then

•  JVM design PREDATES the major 2 trends of last decade: virtualizaEon and cloud-‐compuEng

•  Only 2 innovaEons in JVMs in 20 years: – Becer JIT compilers – Concurrent GCs – ….all blindly focused on single-‐tenant performance at the expense of mulE-‐tenant efficiency


Server Java has problems!•  JVMs are grossly inefficient users of CPU and memory

–  A JVM assumes it can use all available CPU and memory capacity for itself and its single applicaEon, eg:

–  1 JBoss can consume up to 500MB RSS with no apps –  1 Tomcat can consume up to 250MB RSS with no webapps –  Running mulEple JVMs on a server rapidly exhausts physically

memory while CPU uElizaEon averages 5% •  >100 different JVM versions makes compliance/support a

NIGHTMARE •  Security vulnerabiliEes and compromises are EVERYWHERE

inside Java


Existing virtualization does not virtualize Java well!

•  Real world stats using hardware virtualizaEon: –  >80% of Server JVMs have <=2GB of heap memory

•  Yet actual applicaEon heap uElizaEon averages <20% •  …so that means 1.6GB of heap memory per app is wasted

–  A 2GB JVM heap becomes ~3GB of OS RSS memory, which in turn becomes ~4GB of hypervisor tenant memory

•  …reducing actual applicaEon memory uElizaEon to <10%

–  CPU uElizaEon running Java on hardware virtualizaEon averages <7% •  Yet physical memory is exhausted because running mulEple JVMs has

consumed all of RAM

–  …Hardware VirtualizaEon SoluEons (like Vmware, KVM, XEN) were meant to solve exactly this kind of server underuElizaEon in 2002!

•  …but the problem sEll exists for Java in 2013


Java is a compliance nightmare!•  Real world stats:

–  93% of organizaEons are running versions of Java at least 5 years old

–  The average organizaEon has 51 disEnct versions of Java installed

–  <1% of installed Java is upgraded to the latest version •  Recurring enterprise conversaEon:

– QuesEon: “what version of Java do you run?” – Answer: “every version except the latest version!”


Java is a security nightmare!•  Real world stats:

–  In last 12 months, 250+ CVE alerts issued by US-‐CERT/NIST for Java

– One version of Java alone has 96 “perfect 10” vulnerabiliEes

–  “There are virtually no modern versions of Java without any known severe vulnerabiliEes” (source: Bit9)

•  CriEcal Security Patch Updates are released so oxen that developers and admins can’t keep up


Virtualization is the answer…!•  …but HIGHER up the Server Stack •  VirtualizaEon is creeping up the Server Stack already –  x86 HW got virtualized in 1999 (VMware et al) –  x86 OS got virtualized in 2006 (OpenVZ, LXC, et al) – …now it’s Java’s turn in 2013

•  Virtualizing Java means virtualizing at the TOP of the Server Stack, whereas exisEng hardware virtualizaEon, virtualizes at the BOTTOM


Cost-per-tentant to zero !•  The higher up virtualizaEon goes up the Server Stack, the cheaper a “tenant” becomes

•  Virtualizing INSIDE the JVM gets close to “zero-‐cost tenants”: – Helloworld in <1MB of tenant memory – Examples.war (Tomcat demo webapps) in <5MB of tenant memory

– …Java becomes CHEAPER than python/ruby/perl when the JVM itself gets virtualized!


Virtualization == Security!•  Virtualizing the JVM makes Server Java SECURE and COMPLIANT –  Virtualized Java Apps run as “tenants” in a shared JVM –  Tenants run in isolated Java Virtual Container (JVC) “jails” –  JVCs “lock-‐down” Java’s security vulnerabiliEes –  JVCs can run any Java version

•  Doesn’t macer what “Java version” a tenant uses, it gets the latest host JVM

•  Means a “Java version” becomes just a tenant API for the convenience of developers

•  Admins control the REAL version at the host JVM level so that tenants (developers) can stop caring about “versions” altogether


Java Virtualization!

• Think “VMWare for Java”


A hypervisor INSIDE a JVM!•  Waratek has wricen a type-‐1 VMM that runs INSIDE the

Oracle HotSpot and OpenJDK JVMs –  >300,000 lines of Java code, <5K lines of C-‐code –  Can easily run on any OS/CPU pair (only Linux/x86 supported so

far) –  Does EVERYTHING VMware does, but INSIDE the JVM

•  Virtual FS •  Virtual networks/IPs/NICs •  CPU pinning and quotas •  Memory overcommit, elasEc memory, memory quotas •  I/O QoS, rate-‐limiEng •  Etc…

–  …all with ZERO code change to Java applicaEons


A real VMM inside the JVM!


JVC

Java Code

C/C++ Code

VMWare for Java!

Waratek JVM

java.* APIs

SecurityManager

Waratek Java Hypervisor

Java App

•  Waratek virtualizes Java exactly like VMware virtualizes HW –  Waratek inserts a “Java

Hypervisor” between the JVM and the Java App + java.* APIs + SecurityManager

–  The Java app PLUS java.* APIs and SecurityManager run inside a Java Virtual Container (JVC) “jail”

–  For a Java App to go outside of its JVC “jail” it has to “trap” to the Java Hypervisor

JNI Libs

Process


JVC

Java Code

C/C++ Code

VMWare for Java!

Waratek JVM

java.* APIs

SecurityManager

•  Every incoming and outgoing access by a JVC is intercepted and checked by the Java Hypervisor, e.g: –  Class loading intercepted and checked –  JNI invocaEon intercepted –  Parameter and return value checks –  File access (delete/copy/move) checks –  Network operaEon checks –  IO rate-‐limiEng and flow-‐control –  CPU Eme control –  Memory consumpEon control –  Buffer overflow checks –  Null pointer checks –  Type safety checks –  OS access checks –  JNI Libraries isolated –  …


Java App

JNI Libs

Process


Slots into existing VM tools!•  You can manage the Java hypervisor exactly like you manage

exisEng HW hypervisors: –  SSH CLI –  RedHat LibVirt/virsh integraEon (beta) –  VMWare vCAC integraEon shortly –  OpenStack integraEon shortly

•  Plus other management interfaces –  HTTP REST API –  Local and remote JMX APIs –  /proc/PID/* pseudo-‐filesystem for JVM and each JVC –  Vmware CloudFoundry integraEon (beta) –  RedHat OpenShix integraEon soon


A HelloWorld JVC in LibVirt XML!


“Come in, weʼre open”!•  The Java Hypervisor has an open API for unlimited customizaEon and extensibility

•  Anyone can write their own PURE-‐JAVA plugins to the Java hypervisor to change ANYTHING about JVCs, eg: – Make a DropBox.com account the virtual filesystem for a JVC

–  Create a virtual LAN between distributed JVCs anywhere in the world

–  Clone live network I/O of an acEve JVC to a test JVC for side-‐by-‐side applicaEon tesEng


Example hypervisor plugin!•  Waratek wrote ElasECat as one example of a Java Hypervisor plugin

•  ElasECat is open source, is <1000 lines of Java code, and loads into the Java hypervisor at startup

•  ElasECat virtualizes Apache Tomcat so it becomes the world’s first mulE-‐tenant JavaEE container: –  Tomcat runs in one JVC and each webapp runs in it’s own dedicated JVC

–  If one webapp crashes it won’t crash any other webapp –  …all this achieved with ZERO code change to Tomcat or webapps, and without them knowing they’ve been virtualized!


Java Virtualization!

• Case study 1: Java’s (bad) legacy


Java (non)compliance!•  Real world stats:

–  93% of organizaEons are running versions of Java at least 5 years old

–  The average organizaEon has 51 disEnct versions of Java installed

–  <1% of installed Java is upgraded to the latest version •  Recurring enterprise conversaEon:

– QuesEon: “what version of Java do you run?” – Answer: “every version except the latest version!”


Case Study 1 Profile!•  A Global InsEtuEon •  Approx 40,000 Java apps on approx 20,000 Linux Servers –  Average app-‐to-‐server raEo of 2:1 –  Approx 25,000 apps running Java SE 6 –  Approx 10,000 apps running Java SE 5 –  Approx 3,000 apps running Java SE 4 –  Less than 40 apps running Java SE 7 –  2 apps sEll running Java 1.0.0 – … this is NOT unusual!


How old is that old app?!•  Java 7 is already 2 years old! •  Java 6 is 7 years old! •  Java 5 is 9 nears old! •  Java 4 is 11 years old! •  …that old app you wrote 5 years ago could sEll be running somewhere!

•  …and that NEW app you’re wriEng NOW could sEll be running in 5 years Eme!


Case Study: Before Waratek!•  With <40 apps on Java 7, 99.9% of their Java estate is unsupported and EOL –  Those Java CriEcal Patch Updates are precy criEcal axer all

•  Before adopEng Waratek, they were going to pay external consultants to migrate all of their legacy apps to Java 7 at an enormous cost-‐per-‐app –  And it would have taken 1 year or more


Why did they care?!•  Here’s one really simply example why:

–  Did you know that Eme zones in the world are not staEc and change frequently?

–  In general Eme zone updates can be released 4-‐6 Emes a year!

•  So if you don’t upgrade your old JVMs then you’re running a broken API –  …this can be a Ecking Eme-‐bomb (no pun intended) which only gets discovered when something goes really wrong and it’s too late!


Hereʼs another reason…!•  Every version of Java has “perfect 10” security vulnerabiliEes

•  One version of Java 6 alone has 96 “perfect 10” vulnerabiliEes from US-‐CERT/NIST!

•  Those old apps your running have a whole host of security vulnerabiliEes RIGHT NOW:

•  Remote code injecEon and inclusion •  DoS •  Buffer overflows and trigger-‐able SIGSEGV faults


Java legacy is EXPENSIVE!•  The cost of supporEng old soxware increases over Eme… –  …and so does the risk and damages of that old soxware failing

•  The real costs are hidden though -‐ an old Java app is the Ep of a very expensive iceberg: –  That old Java app is running on an old JVM –  That old JVM is running on an old OS –  That old OS is running on an old server/CPU –  …so the legacy cost is 3x more than JUST the app!


Case Study: With Waratek!•  With Waratek, hundreds of old apps can be migrated to an

UP-‐TO-‐DATE JVM in hours not years –  Skips/lowers tesEng requirements because the Java API hasn’t

changed –  Things like serializaEon/persistence keep running flawlessly

because version API is unchanged

•  Every app keeps running with the Java API it was tested with –  Means Java versions become a programming convenience, not

an administraEve nightmare –  Whatever version of Java you boot in a JVC, the host JVM is

brand new and shiny and up-‐to-‐date and supported


Old Java in JVCs == savings!•  The more old apps, and the older those apps:

–  The greater the cost of app-‐failure –  The greater the operaEng and support costs

•  Ge}ng old apps off old JVMs and onto JVCs saves lots of $$$ because – Much smaller footprint (tenant cost) – Much faster performance –  Secure and quaranEned –  Compliant and supported –  Legacy OS/hardware can be reEred


In a customerʼs words!

•  “[With Waratek] applicaKons on older versions of Java can be immediately migrated to a secure, resource-‐contained and isolated JVC without any code changes or applicaKon transformaKon”


Case Studies!

• Case Study 2: Java (in)security


Java (in)security!•  Real world stats:

–  In last 12 months 250+ CVE alerts issued by US-‐CERT/NIST for Java

– One version of Java alone has 96 “perfect 10” vulnerabiliEes

–  “There are virtually no modern versions of Java without any known severe vulnerabiliEes” (source: Bit9)

•  CriEcal Security Patch Updates are released so oxen that developers and admins can’t keep up


Did you know?!•  Server Java has the same root security vulnerabiliEes as desktop Java

•  …but acack vectors are slightly different: – Remote and local code-‐injecEon and inclusion – DoS – Buffer overflows/segfaults – Logic bombs / Eme bombs / trojans – …and biggest of all: employee/consultant sabotage!!


Java Code

C/C++ Code

Java is insecure at multiple levels!

JVM JNI Libs

java.* APIs

SecurityManager

Java App

•  Java’s security vulnerabiliEes fall into 3 acack vectors: –  Acacks to take control of java.*

APIs •  E.g. to delete/copy/move files

–  Acacks to take control of Java’s SecurityManager

•  E.g. insert malicious code anywhere inside the JVM

–  Acacks on JNI Libraries •  E.g. trigger buffer overflows, null-‐pointer

segmentaEon faults, unsafe type access, and so on


Java Code

C/C++ Code

JVMs are insecure by design!

JVM JNI Libs

java.* APIs

SecurityManager

•  By Java’s design, a JVM performs no security checks of its own: –  The JVM relies on security

enforcement in the java.* API and SecurityManager levels

–  If the SecurityManager or java.* APIs are compromised, then the JVM can be instructed to do anything like:

•  Insert malicious code •  Load C/C++ JNI Libraries •  Fork OS processes •  Write directly to heap memory •  Open network connecEons •  And anything else…

Java App


Unsafe JARs == unsafe JVM!•  It’s not just the Java APIs that you need to worry about •  Unlike the Java APIs, security vulnerabiliEes in third-‐party

class libraries (JARs) are poorly researched and documented and are EVERYWHERE –  Some isolated examples: code-‐injecEon vulnerabiliEes exist in all

of Spring, JIRA, Struts, Webwork2, RMI, JBoss, WebLogic, and many more

•  Third-‐party JARs are: –  full of security vulnerabiliEes –  very poorly audited/tested/understood –  not patched frequently enough


Unsafe Java == unsafe OS!•  A compromised JVM becomes an acacker’s puppet, and can

be made to do things like: –  Give valuable intelligence about OS, network, and neighbors –  “Poison” JAR files on disk, inserEng Trojans –  Hide “logic bombs” and “Eme bombs” that explode when an

employee is fired, some anniversary passes, etc –  Change startup params for the JVM, like open remote RMI access –  Fork new OS processes –  Manipulate file descriptors in unintended ways –  Launch remote or local DoS acacks –  Hijack servers for malicious “bot” purposes


Case Study 2 Profile!•  A Fortune 500 company •  Running a DIY “Java PaaS” pla�orm

– Many JVMs per OS –  …with poor isolaEon between each JVM –  CriEcal Security Patch Updates for Java are released so oxen, they can’t keep up

•  In the customer’s words: –  “[We have many] security vulnerabiliKes that leave us open to a(ack across the estate”


Many apps, little oversight!•  With so many apps there’s limited oversight for each

individual app •  No fine-‐grained tenant control because many JVMs are

packed into a single VMware guest OS –  …so VMware’s can’t help them this high up the SW stack

•  Their nightmare today: any app could have a trojan and nobody knows

•  Biggest problem: the JVM itself provides no security, and once compromised, a JVM is an acacker’s puppet


Customerʼs wish!•  Capability to define per-‐app quaranEne rules AND able to be

set by the app-‐owner DIRECTLY and IMMEDIATELY •  …but not possible today because:

–  ExisEng JVMs don’t have any means to define applicaEon permissions outside of the (highly-‐compromised) Java API

–  …because of this, per-‐app security rules must be defined at the underlying OS or hypervisor or network Eers

–  …but the OS/hypervisor/network Eers are managed by different teams/departments from the app-‐owners

–  An app-‐owner wanEng to apply or change an app’s security policy has to go to the OS/net/VMM teams who are backlogged doing other things and are slow, distracted and unresponsive


JVC

Java Code

C/C++ Code

Case Study: with Waratek!

Waratek JVM

java.* APIs

SecurityManager

•  By running Java apps inside of JVCs, compromises to a Java app or Java.* APIs will NOT effect the OS or Java Hypervisor or JVM or other JVCs

•  JNI Libraries are isolatable in their own OS process

•  Thus a compromised app remains under quaranEne and control:

–  Cannot insert malicious code outside its JVC –  Cannot delete/move/copy files outside its VROOT –  Cannot trigger buffer overflows –  Cannot do null-‐pointer segmentaEon faults –  Cannot perform unsafe heap memory access –  Cannot fork OS processes –  etc…


Java App

JNI Libs

Process


JVCs quarantine Java!•  Waratek Java hypervisor can “lock down” Java API acack vectors regardless

of the Java API version running in a JVC, a few examples: –  Block class defines by a JVC that does NOT originate from a local registered JAR file –  Hide sensiEve OS/network/environment info from a JVC –  Do payload scanning of all inbound network IO for a JVC –  …and much more

•  Means that even a JVC running an old Java API version can sEll be up-‐to-‐date and secured

•  Most important of all... the app-‐owner can set and change security policies DIRECTLY and IMMEDIATELY without bothering the OS or infrastructure or network teams/departments: –  Can set fine-‐grained per-‐app network rules –  Can set fine-‐grained per-‐app CPU/memory/IO quota rules –  Can set fine-‐grained per-‐app classloading rules, file rules, …anything!


In a customerʼs words!

•  “[With Waratek] applicaKons can be quaranKned without need of code changes or transformaKon”

•  “[Using Waratek] the immediate risks posed by these applicaKons can be miKgated and a route for the upgrade and future sustained management of the applicaKons can be established”


Summary!

• Recap and links…


Biggest change to Java in yrs!•  A hypervisor inside the JVM transforms Java with: – Ultra-‐lightweight JVCs for near zero-‐cost tenants – Mixed-‐version JVCs for version consolidaEon, legacy compliance, and “virtual upgrades”

– JVC “jails” for Java security, containment and quaranEne


Links!•  General informaEon about the Waratek JVM

– hcp://www.waratek.com

•  jSleep, JVI (Java VirtualizaEon Interface), and JMX APIs – hcp://javadoc.waratek.com

•  ElasECat source code – hcps://github.com/waratek/elasEcat-‐driver

NOTICE: Waratek, Cloud VM, jSleep, vSleep, jMotion, Jirsh, JVC, JVI, and ElastiCat are TRADEMARKS of Waratek


!

virtualizing java for the cloud using a jvm hypervisor

Technology