monoliths are so 2001 – what you need is modularity

© 2015 IBM Corporation

Monoliths are so 2001 – What you need is Modularity Graham Charters, STSM, WebSphere Liberty Repository & OSGi Applications Architect

Ian Robinson, Distinguished Engineer, WebSphere Foundation Chief Architect

Agenda

• Monoliths & Modularity

• Modularity and Java

• Introduction to OSGi

• Dismantling the Monolith

• A Case Study – Acme Air

• What’s new in OSGi Applications

• Summary

1

Monoliths & Modularity

Why Monoliths

• No project sets out to create a monolith

• Projects evolve

• Teams evolve

• Architecture knowledge is lost leading to sub-optimal decisions

How do we preserve knowledge?

• Design documentation?

• Wiki?

• Email?

• Work items?

• IRC?

• Brain?

• Developers make the decisions, and where do they hang out?

4

We need to capture and enforce the architectural knowledge in the code

Modularity

• Concept of Modularity in Software dates back to the 1960

• For a module systems to preserve architectural knowledge it needs the follow

qualities

• Reflect the right level(s) of architectural granularity

• Support the Reuse/Release Equivalence Principle

• Capture modularity information in the code

• Enforce modularity

5

None of your

business

What I can

provide to you What I will need

from you

Modularity and Java

7

Base Java modularity is inadequate

• Classes too fine-grained

• Jars are packaging with no explicit requirements/capabilities

• Linear classpath

Java EE modularity is inadequate

• Across apps - each archive typically contains all required libraries

• Common libraries/frameworks get installed with each application

• Multiple copies of libraries in memory

• Within apps - 3rd party libraries consume other 3rd party libraries leading to conflicting versions on the application classpath.

webB.war

WEB-INF/classes/servletB.class

WEB-INF/lib/json4j.jar

WEB-INF/lib/commons-logging.jar

WEB-INF/lib/junit.jar…

webC.war

WEB-INF/classes/servletC.class

WEB-INF/lib/json4j.jar

WEB-INF/lib/commons-logging.jar

WEB-INF/lib/junit.jar…

plankton.v1

plankton.v2

Modularity Issues in Java EE

Introduction to OSGi

9

Bundle

Manifest-Version: 1.0

Bundle-ManifestVersion: 2


Bundle-ManifestVersion: 2

Bundle-Name: My Example Bundle

Bundle-SymbolicName: com.my.bundle

Bundle-Version: 1.0.0

Export-Package: com.something.i.provide;version="1.0.0"

Import-Package: com.something.i.need;version="[1.1,2.0)"

Bundle

Bundle

Bundle


Bundle-ManifestVersi





OSGi Modules (aka Bundles)

• A Jar plus OSGi Manifest, includes:

• Bundle Identity

• Exported Packages

• Imported Packages

• Dependency resolution

“wires” bundles into

a dependency graph

• Each gets its own

class loader

• Classloading

delegates via graph

Classloader

Classloader

Classloader

Classloader

10

Dynamic Bundle Lifecycle

• Bundles have a dynamic

lifecycle

• Can come and go

independently

• APIs enable graceful

reaction to changes

Bundle

INSTALLED

RESOLVED

UNINSTALLED

STARTING

ACTIVE

STOPPING

install update

refresh

un

insta

ll

up

da

te

refr

esh

stop

policy reso

lve

un

insta

ll

11

Local Services

• Publish/find/bind service model • Fully dynamic

• Local

• Non-durable

• Primary mechanism for bundle collaboration

• POJO advertized with properties and/or interface and/or class

service

Provider

Bundle

Consumer

Bundle

register get

listen

WebSphere OSGi Applications

Background

• First surfaced to applications in a WAS v7 Feature Pack

• http://www-01.ibm.com/software/webservers/appserv/was/featurepacks/

• Modular development, deployment and management

• Blueprint (Standardized Spring Component Model)

• Web applications (Java EE 5)

• Remote Services and Heterogeneous Assembly (SCA)

• Included in the WAS Base in v8 and continually extended:

• Java EE 6 Web technologies

• Post-deployment configuration

• Performance metrics

• In-place Update

• Application Extension

• Modular EJB

• Blueprint Role-based Security

• OSGi Applications Web Console

• Liberty Profile support

13

http://www-01.ibm.com/software/webservers/appserv/was/featurepacks/



Component Models

Category Full Profile Liberty Profile

Presentation Servlet, JSP, JSF Servlet, JSP, JSF

Business Blueprint*, EJB Blueprint*

Persistence JPA, JDBC JPA, JDBC

Integration JAX-RS, JMS, SCA JAX-RS, JAX-WS, JMS

14

Component models for enterprise Web applications

Re-use rather than re-invent

• Java EE, Spring

*some differences in supported namespaces

Web Application Bundle

• Web Application Bundle (WAB) = WAR + OSGi metadata

• Web-ContextPath: header used to specify default context root

• OSGi Manages

• Module Lifecycle

• Classloading (opportunity to split out WEB-INF/lib)

• Service Integration

• Web Container Manages

• Web component life-cycle, serving, etc...

webC.war

logging f/w

persistence f/w

MVC f/w

webB.war

logging f/w

persistence f/w

MVC f/w

webA.war

logging f/w

persistence f/w

...

WEB-INF/classes/servletA.class

WEB-INF/web.xml

WEB-INF/lib/…

webA.jar


WEB-INF/web.xml

META-INF/MANIFEST.MF

webA.jar


WEB-INF/web.xml


webA.jar


WEB-INF/web.xml


logging f/w

persistence f/w

...

These can

become shared

bundles

Bundle

Blueprint Components and Service

• Dependency Injection container

• Standardizing established Spring conventions

• Configuration and dependencies declared in Blueprint XML

• Standardization of Spring “application context” XML

• Extended for OSGi: publish/consume components as OSGi services

• Simplifies unit test outside either Java EE or OSGi r/t

• Integrated into server runtime to simplify deployment & support

16

dependencies injected

publishes

service consumes

service

Blueprint container scoped by a bundle (one per bundle). Multiple <blueprints /> per container.

A static assembly and configuration of beans

(POJOs)

Blueprint managed bundle

Brightness

Blueprint Extensions

• Namespaces extend core component model

• Transactions

• Resource References (Full Profile)

• JPA integration

• Security (Full Profile)

• Third-party (Liberty 8.5.5.3)

<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0"

xmlns:sec="http://www.ibm.com/appserver/schemas/8.0/blueprint/security"

xmlns:tx="http://aries.apache.org/xmlns/transactions/v1.0.0">

<service id="adjustmentService" ref="BrightnessDeltaBean"

interface="colors.adjustment.api.BrightnessService">

</service>

<bean id="BrightnessDeltaBean"

class="colors.brightness.BrightnessDelta">

<sec:access-constraint method="setBrightnessDelta" role="adjuster" />

<tx:transaction method="*" value="Required" />

<jpa:context property="em" unitname="brightnessUnit"/>

<property name="delta" value="100"/>

</bean>

</blueprint>

EJB Bundle

• EJB Bundle = EJB Jar + OSGi Metadata

• Export-EJB: Opt-in header for EJB Bundles

– Existence: process bundle for EJBs

– Absence: do not process bundle for EJBs, even if it contains them

• Header value governs registration of EJBs as OSGi Services

– Excludes Message-Driven and Stateful beans

– Best practice: only export EJBs to be shared outside bundle

• EJBs run in the same WAS EJB Container

• Uses OSGi for Classloading and Life-cycle

18

Example Meaning

Export-EJB: Process all EJBs and register them as OSGi services

Export-EJB: BlogBiz,

BlogPersistence

Process all EJBs, register BlogBiz and BlogPersistence as

services if they exist

Export-EJB: NONE Process all EJBs but don’t register them as OSGi services

Persistence Bundle

• Persistence Bundle = Persistence Archive + OSGi Metadata

• Meta-Persistence: opt-in header

• Identifies the location of the persistence xml file

• Defaults to META-INF/persistence.xml

• Datasource lookup mechanisms:

<jta-data-source>



blueprint:comp/blueprint_component_name

</jta-data-source>

<jta-data-source>

osgi:service/javax.sql.DataSource/(osgi.jndi.serviceName=

jndi_name_of_the_data_source)

</jta-data-source>

<jta-data-source>

jndi_name_of_the_data_source

</jta-data-source>

.eba

OSGi Application

• Isolated, cohesive collection

of bundles

• Defined by application

manifest

• Configuration by

exception

• Deployed as .eba archive (zip

file)

• Provisioning resolves

application against archive

contents and configured

bundle repositories

APPLICATION.MF Application-Name: Color Blender Application

Application-SymbolicName: colors.blender.simple.app

Application-ManifestVersion: 1.0

Application-Version: 1.0.1


Application-Content:

colors.blender;version="[1.0.0,2.0.0)",

colors.provider.green;version="[1.0.0,2.0.0)",

colors.provider.red;version="[1.0.0,2.0.0)",

colors.web;version="[1.0.0,2.0.0)",

colors.provider.ejb.blue;version="[1.0.0,2.0.0)"

Application-ImportService:

colors.adjustment.api.BrightnessService

Application-Name: Color Blender Application

Application-SymbolicName:

colors.blender.simple.app

Application-ManifestVersion: 1.0

Application-Version: 1.0.1


Application-Content:

colors.blender;version="[1.0.0,2.0.0)",

colors.provider.green;version="[1.0.0,2.0.0)",

colors.provider.red;version="[1.0.0,2.0.0)",

colors.web;version="[1.0.0,2.0.0)",

colors.provider.ejb.blue;version="[1.0.0,2.0.0)"

Application-ImportService:

colors.adjustment.api.BrightnessService

colors.web

colors.blender

OSGi Application Provisioning

• Application manifest allows

version ranges for contents

• Deployment Manifest

(optional) locks down exact

versions

• Provisioning resolves

application against archive

contents and configured

bundle repositories

• Non-content bundles satisfy

unresolved application

package and service

dependencies

• shared between

applications on same

server

.eba

APPLICATION.MF

DEPLOYMENT.MF

colors.api

colors.

web

colors.

blender colors.

provider.

blue

Shared Bundles

colors.

blender

OSGi Application

colors.

provider.

blue

colors.api

colors.

web

Dismantling the Monolith

Adoption Strategies

• Favoured Java EE Approach:

1. Replicate existing classloading in OSGi using bundle fragments

2. Incrementally separate out individual bundles

3. Adopt OSGi best practices

• Considers Java EE classloading and component models

• Surfaces OSGi early, and incrementally - a staged approach

23

beans

core

ejb3

entities

json-proxy

soap wsappclient

streamer web

app-host

log core ejb3

wab

web

app-host

log core ejb3

wab

web

log

http://www.slideshare.net/GrahamCharters/modularizing-existingenterpriseapplicationsos-gicommunityevent2012v01

Stage 1: Understanding the starting point

• Java EE prescribes a hierarchical classloading model

• Assuming “parent first” and “multiple”: • Each Application has own

classloader

• Each WAR has own class loader

• WAR has visibility to Application classes

• WAR prefers Application classs, Application prefers System classes, etc...

• OSGi modularity enforced through class visibility using classloaders

• Migration strategies need to consider the impact of this change

• e.g. replicate visibility relationships of existing application in OSGi

24

Bootstrap

Extensions

System

Application Application

WAR WAR WAR

Stage 1: Replicating Java EE classloading

• Preserve Application and WAR roles

• Application -> Application Host Bundle

• Add application modules to fragments of host bundle

• Web App Archive -> Web Application Bundle

• Add WEB-INF/classes to Bundle-Classpath

• Extract WEB-INF/lib jars and add as fragments of Web Application Bundle

• We now have two classloaders just as we did in Java EE

• We also have full visibility of the modules

25

System

Application

WAR

app-host

log core ejb3

wab

web

Stage 2: Factoring out Bundles

• Now the application’s running in OSGi we can start to split out the fragments as independent, re-usable bundles

• Strategy: • Do one fragment at a time

• Start with the leaf dependencies – Wab bundle contents first, then the app bundle

– Project build dependencies help with identification

– Third-party libraries

– “shared libraries” (if runtime supports this concept)

• Detach from host and calculate the package imports & exports

26

wab

web

wab

web

Fragment-Host: wab Export-Package: daytrader.web

Import-Package: daytrader.web

Stage 3: Incrementally Adopt Best Practices

• Stage 2 documents the ‘as-is’ architecture, warts-‘n’-all

• OSGi Best Practices show how to make the most of OSGi http://www.ibm.com/developerworks/websphere/techjournal/1007_charters/1007_charters.html

• Adopting best practices leads to:

• High cohesion - bundles with clear and distinct roles in the

architecture

• Loose coupling - flexible, but necessary, dependencies

• ...which all leads to greater agility, flexibility and re-use

• Development teams can understand and explain the architecture

and are no longer afraid to change the code

• Applications can evolve and new application can be created at a

pace that enables, rather than inhibits, the business

27

http://www.ibm.com/developerworks/websphere/techjournal/1007_charters/1007_charters.html

Case Study: Acme Air

Acme Air

• A Sample Java EE Web Application

• Key Technologies

• Web content - html, JavaScript

• REST APIs - JAX-RS

• Persistence – JPA

• Spring

• Built using Maven

• Deployed as a WAR with all the

dependencies inside (WEB-INF/lib)

acmeair-webapp.war

acmeair-common.jar

acmeair-service.jar

springxxx.jar

WEB-INF/classes/xxx.class

WEB-INF/web.xml

WEB-INF/lib/…

...

29

Stage 1:

30

Stage 1 - 1 = 0: Single Bundle

• Single monolithic bundle

• Build a WAB not a WAR

• Web–ContextRoot

• Bundle-Classpath

• Package imports

• Uses JPA so also make it a Persistence

Bundle

• Change use java:comp/env to

osgi:service

• Address classpath issues (e.g. Spring

@ComponentScan, file loading)

31

acmeair-webapp.war

acmeair-common.jar

acmeair-service.jar

springxxx.jar


WEB-INF/web.xml

WEB-INF/lib/…

acmeair-webapp-bundle.jar


WEB-INF/web.xml


...

<plugin> <groupId>org.apache.felix</groupId> <artifactId>maven-bundle-plugin</artifactId> <version>2.5.3</version> <extensions>true</extensions> <configuration> <instructions> <Web-ContextPath>/acme.air.osgi</Web-ContextPath> <Embed-Dependency>*;scope=compile</Embed-Dependency> <Bundle-Classpath>WEB-INF/classes</Bundle-Classpath> <Embed-Transitive>true</Embed-Transitive> <Meta-Persistence> WEB-INF/classes/META-INF/persistence.xml </Meta-Persistence> <Import-Package> !javax.resource, !javax.resource.cci, !javax.resource.spi, ... javax.*, org.xml.sax, org.w3c.dom </Import-Package> <_wab>src/main/webapp</_wab> </instructions> </configuration> </plugin>

acmeair-common.jar

acmeair-service.jar

springxxx.jar

...

Stage 1: OSGi Application

• Build an OSGi Application

(EBA) to deploy the WAB

• Eba-maven-plugin generates

application manifest based on

pom configuration

32


acmeair.eba

META-INF/APPLICATION.MF

<plugin> <groupId>org.apache.aries</groupId> <artifactId>eba-maven-plugin</artifactId> <version>1.0.0</version> <extensions>true</extensions> <configuration> <generateManifest>true</generateManifest> <instructions> <Application-SymbolicName> ${project.artifactId} </Application-SymbolicName> </instructions> </configuration> </plugin>

Application-ManifestVersion: 1 Application-SymbolicName: acmeair-eba Application-Version: 1.0.0.SNAPSHOT Application-Name: acmeair osgi application Application-Description: AcmeAir Application Application-Content: net.wasdev.wlp.sample.acmeair-webapp-bundle;version="1.0.0.SNAPSHOT"

Stage 1: Fragments

• Fragments used to separate out “modules”

from WAB without affecting classloading

• Update EBA to include Fragments

• First opportunity to see the true package

dependencies

• Opportunity for small refactoring (e.g.

persistence.xml from WAB to persistence

fragment)

33



WEB-INF/web.xml


springxxx.jar

acmeair-common-bundle.jar

acmeair-service-bundle.jar

acmeair-service-jpa-bundle.jar

<plugin> <groupId>org.apache.felix</groupId> <artifactId>maven-bundle-plugin</artifactId> <version>2.5.3</version> <extensions>true</extensions> <configuration> <instructions> <Fragment-Host> net.wasdev.wlp.sample.acmeair-webapp-bundle </Fragment-Host> </instructions> </configuration> </plugin>

persistence.xml

Stage 2: Bundles

• Detach Fragments from Host

• Move opt-in headers from Host to new bundles

• Export packages from Host, if necessary (e.g. Spring)

• Separation Issue: webapp bootstraps spring beans from service-jpa then looks them up

34



WEB-INF/web.xml


springxxx.jar

acmeair-service-bundle.jar



persistence.xml

<plugin> <groupId>org.apache.felix</groupId> <artifactId>maven-bundle-plugin</artifactId> <version>2.5.3</version> <extensions>true</extensions> <configuration> <instructions> <Meta-Persistence> META-INF/persistence.xml </Meta-Persistence> </instructions> </configuration> </plugin>

Stage 2: Bundles...not quite

• Webapp uses a singleton “ServiceLocator” to get the Spring beans

• Webapp ServiceLocator bootstraps the Spring application context

by trying to load config from the service-jpa bundle but can’t find it

• @ImportResource({"classpath:/spring-config.xml"})

• These modules are too tightly couple and so can’t function

independently.

35



WEB-INF/web.xml


springxxx.jar


@ImportResource({"classpath:/spring-config.xml"})

spring-config.xml

Stage 2 & 3: Bundles...almost

• ServiceLocator is acting as a Service Registry. OSGi has one of

those.

• Bundle Activator used to bootstrap Spring application context and

register bean services

• Also needed to register EntityManager service for Spring

• Webapp changed to load services from Service Registry

36



WEB-INF/web.xml


springxxx.jar


spring-config.xml

SpringActivator @ImportResource({"classpath:/spring-config.xml"})

ic.lookup("osgi:service/acmeair.CustomerService")

Stage 2 & 3: Bundles

• But... there’s still a slight

issues... lifecycle

• Can’t bootstrap Spring if the

Persistence support is not ready

• Could “sleep”, but for how long?

• Solution: use a service lifecycle-

aware component model

(Blueprint)

37



spring-config.xml


EntityManagerFactory

<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0"> <bean id="SpringContextInitializerBean" class="com.acmeair.jpa.service.config.SpringContextInitializer"> <property name="ctxt" ref="blueprintBundleContext" /> <property name="emf" ref="emf" /> </bean> <reference id="emf" interface="javax.persistence.EntityManagerFactory“ /> </blueprint>

blueprint.xml

Was it worth it?

• A set of modules re-usable at different levels

• Persistence

• Local Domain Services

• Remote REST Services

• Looser-coupling and higher cohesion

• No persistence content in the Web front-end

• Spring isolated to the module that uses it

• Flexible lifecycle

38

acmeair-webapp-bundle.jar acmeair-service-jpa-bundle.jar acmeair-common-bundle.jar

What’s New

What’s new in WebSphere OSGi Applications

Liberty had addressed a number of top customer requirements through 2014

• Customer Blueprint Namespace Handlers

• Use cases: CXF, Camel, Config Adnim, etc…

• External Bundle Repositories

• Use cases: Rational Asset Manager, Nexus integration

• Local Application to application integration

• Use cases: Application interaction, ‘extension’

• JAX-RS & JAX-WS

• Use cases: Integration & Interop

• Subsystems (beta)

• Uses cases: extension and in-place update

• Be sure to raise RFE to help us prioritize your needs

• https://www.ibm.com/developerworks/rfe/

https://www.ibm.com/developerworks/rfe/

https://www.ibm.com/developerworks/rfe/

Summary

Summary

• Enforce right-scale modularity is key to preserving architectural

knowledge

• Even relatively simple applications can become entangled in the

absence of a good modularity

• A methodical approach to adopting OSGi can deliver results

early and dismantle the monolith

• WebSphere Application Server Liberty Profile is now an even

better platform for Enterprise OSGi

Notices and Disclaimers

Copyright © 2015 by International Business Machines Corporation (IBM). No part of this document may be reproduced or

transmitted in any form without written permission from IBM.

U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with

IBM.

Information in these presentations (including information relating to products that have not yet been announced by IBM) has been

reviewed for accuracy as of the date of initial publication and could include unintentional technical or typographical errors. IBM

shall have no responsibility to update this information. THIS DOCUMENT IS DISTRIBUTED "AS IS" WITHOUT ANY WARRANTY,

EITHER EXPRESS OR IMPLIED. IN NO EVENT SHALL IBM BE LIABLE FOR ANY DAMAGE ARISING FROM THE USE OF

THIS INFORMATION, INCLUDING BUT NOT LIMITED TO, LOSS OF DATA, BUSINESS INTERRUPTION, LOSS OF PROFIT

OR LOSS OF OPPORTUNITY. IBM products and services are warranted according to the terms and conditions of the

agreements under which they are provided.

Any statements regarding IBM's future direction, intent or product plans are subject to change or withdrawal without

notice.

Performance data contained herein was generally obtained in a controlled, isolated environments. Customer examples are

presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual

performance, cost, savings or other results in other operating environments may vary.

References in this document to IBM products, programs, or services does not imply that IBM intends to make such products,

programs or services available in all countries in which IBM operates or does business.

Workshops, sessions and associated materials may have been prepared by independent session speakers, and do not

necessarily reflect the views of IBM. All materials and discussions are provided for informational purposes only, and are neither

intended to, nor shall constitute legal or other guidance or advice to any individual participant or their specific situation.

It is the customer’s responsibility to insure its own compliance with legal requirements and to obtain advice of competent legal

counsel as to the identification and interpretation of any relevant laws and regulatory requirements that may affect the customer’s

business and any actions the customer may need to take to comply with such laws. IBM does not provide legal advice or

represent or warrant that its services or products will ensure that the customer is in compliance with any law.

Notices and Disclaimers (con’t)

Information concerning non-IBM products was obtained from the suppliers of those products, their published

announcements or other publicly available sources. IBM has not tested those products in connection with this

publication and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM

products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products.

IBM does not warrant the quality of any third-party products, or the ability of any such third-party products to

interoperate with IBM’s products. IBM EXPRESSLY DISCLAIMS ALL WARRANTIES, EXPRESSED OR IMPLIED,

INCLUDING BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A

PARTICULAR PURPOSE.

The provision of the information contained herein is not intended to, and does not, grant any right or license under any

IBM patents, copyrights, trademarks or other intellectual property right.

• IBM, the IBM logo, ibm.com, Bluemix, Blueworks Live, CICS, Clearcase, DOORS®, Enterprise Document

Management System™, Global Business Services ®, Global Technology Services ®, Information on Demand,

ILOG, Maximo®, MQIntegrator®, MQSeries®, Netcool®, OMEGAMON, OpenPower, PureAnalytics™,

PureApplication®, pureCluster™, PureCoverage®, PureData®, PureExperience®, PureFlex®, pureQuery®,

pureScale®, PureSystems®, QRadar®, Rational®, Rhapsody®, SoDA, SPSS, StoredIQ, Tivoli®, Trusteer®,

urban{code}®, Watson, WebSphere®, Worklight®, X-Force® and System z® Z/OS, are trademarks of

International Business Machines Corporation, registered in many jurisdictions worldwide. Other product and

service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on

the Web at "Copyright and trademark information" at: www.ibm.com/legal/copytrade.shtml.

http://www.ibm.com/legal/copytrade.shtml

Thank You Your Feedback is

Important!

Access the InterConnect 2015

Conference CONNECT Attendee

Portal to complete your session

surveys from your smartphone,

laptop or conference kiosk.

monoliths are so 2001 – what you need is modularity

Technology

bundle bundleversion

bundle identity

bundle collaboration

osgi manifest

base java modularity

v2 modularity issues

modularity graham charters

java introduction