devoxx 2011 integration-camel-cxf-servicemix-activemq
DESCRIPTION
Presentation about Apache projects used for Integration (Apache Camel, CXF, ActiveMQ, ServiceMix) and new strategy about Cloud, Provisioning and Elastic Services (Fuse Fabric)TRANSCRIPT
Integration with Apache ServiceMix, Camel, CXF and ActiveMQ Charles Moulliard – Gert Vanthienen Apache Committer FuseSource
Speaker : Charles Moulliard
• Engineer in Agronomy & Master in Zoology – 18 years of experience in IT world development (J2EE, .NET), specialized in new technologies web/Internet & integraNon, Project manager in bank, financial, telco, insurance and transport world
• SoluNon Architect at FuseSource • CommiSer on projects : Apache ServiceMix, Apache Karaf (PMC) & Apache Camel
TwiSer : hSp://twiSer.com/cmoulliard
LinkedIn : hSp://www.linkedin.com/in/charlesmoulliard
My blog : hSp://cmoulliard.blogspot.com
Speaker : Gert Vanthienen
• @Apache SoYware FoundaNon – Apache Camel, Karaf, ServiceMix – Committer / PMC member – Apache Member
• @FuseSource – Member of the engineering team – Mainly working on FUSE ESB
Agenda
• IntegraNon – introducNon
• PresentaNon of projects -‐ Camel, CXF, AcNveMQ and ServiceMix
• Architectures presentaNon
• High availability, scalability and clustering
• PresentaNon of Fuse IDE
• Fabric new Strategy for provisioning and cloud
Par t I
Par t II
Agenda
• Integra(on – introduc(on
• PresentaNon of projects -‐ Camel, CXF, AcNveMQ and ServiceMix
• Architectures presentaNon
• High availability, scalability and clustering
• PresentaNon of Fuse IDE
• Fabric new Strategy for provisioning and cloud
Par t I
Par t II
IntroducNon (1)
• IntegraNon projects are really hard and pain`ul
• We have to integrate different technologies/systems/infrastructures
• Lot of standards/products/tools exist (SOA,EAI,…) and formats (EDI, Fixedlength, XML, CSV, FIX,
• Their implementaNons are resource and Nme consuming
• SOA, BPEL and EAI have not been successfull
IntroducNon (2)
Apache foundaNon proposes alternaNve projects to simplify « integraNon project development »
• Apache Camel – integraNon « Swiss knife » toolbox
• Apache CXF – webservice/REST stack
• Apache AcNveMQ – middleware layer
• Apache ServiceMix – Container for IntegraNon project
Apache ODE (BPEL),
Apache Tuscany (SCA)
Apache Synapse (ESB – XML)
Not covered
Agenda
• IntegraNon – introducNon
• Presenta(on of projects -‐ Camel, CXF, Ac(veMQ and ServiceMix
• Architectures presentaNon
• High availability, scalability and clustering
• PresentaNon of Fuse IDE
• Fabric new Strategy for provisioning and cloud
Par t I
Par t II
Camel
• Birthdate -‐ March 2007
• OpenSource IntegraNon Framework implemenNng
EIP paSerns (book published in 2005 by Gregor Hohpe)
• Why EIP To provide word definiNon, grammar and language when designing integraNon projects
• Camel uses a Domain Specific Language transposing the EIP paSerns into Java, XML
Camel
• Some of its features/characterisNcs – In memory bus AlternaNve to JBI using NMR
– Route objects can be XML, File, Stream, Bytes
– TransacNonal architecture
– Sync/Async exchanges
– Threads Management
– Error and excepNon handling
Camel (1)
• More than 50 paSerns are implemented
hSp://camel.apache.org/enterprise-‐integraNon-‐paSerns.html
Camel (2)
• More than 100 components
hSp://camel.apache.org/components.html
Camel (3)
• 18 data formats to simplify data exchange between layers, applicaNons connected
• Example : JAXB, Bindy, SOAP
• Uses marshaler (xml object) and unmarshaler (object xml)
hSp://camel.apache.org/data-‐format.html
Camel (4)
The model Content Based
Router
And its DSL language
Camel (5)
Camel (6)
Camel (7)
Camel (8)
URI corresponds to component name +
properNes
Camel (9)
• Transpose the DSL using Java
• First the configure() method must created
Camel (10)
• Next we create a java class extending RouteBuilder class
Camel (11)
• Language is flexible and proposes
different ways to define the Endpoints and predicates
Camel (12)
• Integrate easily with IntelliJ, Eclipse,
NetBeans
Camel (13)
• Instead of Java, you can use
Spring / XML DSL
Camel (14)
• IDE assistance
DEMO 1
Apache Camel
Agenda
• IntegraNon – introducNon
• Presenta(on of projects -‐ Camel, CXF, Ac(veMQ and ServiceMix
• Architectures presentaNon
• High availability, scalability and clustering
• PresentaNon of Fuse IDE
• Fabric new Strategy for provisioning and cloud
Par t I
Par t II
CXF
• Merge of 2 projects (CelNc and Apache Xfire)
• Simplify the creaNon and deployment of webservices
• 2 approaches “java to wsdl” or “wsdl to java”.
• Support the following standards : – JAX-‐WS : Web Services (XML/SOAP)
– JAX-‐RS : RESTfull service (JSON)
– SOAP 1.1, 1.2, WSDL 1.1
– WS-‐Security, WS-‐Addressing, WS-‐RM
CXF (1)
<plugin> <groupId>org.apache.cxf</groupId> <artifactId>cxf-codegen-plugin</artifactId> <version>${cxf-version}</version> <executions> <execution> <id>generate-sources</id> <phase>generate-sources</phase> <configuration> <sourceRoot> ${basedir}/target/generated/src/main/java </sourceRoot> <wsdlOptions> <wsdlOption> <wsdl> ${basedir}/src/main/resources/report.wsdl </wsdl> </wsdlOption> </wsdlOptions> </configuration> <goals> <goal>wsdl2java</goal> </goals> </execution> </executions> </plugin>
To use it, simply add cxf maven plugin to the pom of the project
CXF (2)
• … and configure a spring config file <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:cxf="http://camel.apache.org/schema/cxf" xsi:schemaLocation=" http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd http://camel.apache.org/schema/cxf http://camel.apache.org/schema/cxf/camel-cxf.xsd">
<import resource="classpath:META-INF/cxf/cxf.xml"/>
….
<cxf:cxfEndpoint id="reportIncident" address="http://localhost:9080/incident" wsdlURL="etc/report_incident.wsdl" serviceClass="org.apache.camel.example.ReportIncidentEndpoint"> </cxf:cxfEndpoint>
… </beans>
CXF (3)
• Plugin will generate Java classes – JAXB2
– Interface + @Webservice
@WebService(targetNamespace = "http://reportincident.devoxx.fusesource.org", name = "ReportIncidentEndpoint")!@XmlSeeAlso({ObjectFactory.class})!@SOAPBinding(parameterStyle = SOAPBinding.ParameterStyle.BARE)!public interface ReportIncidentEndpoint {!
@WebResult(name = "outputReportIncident",! targetNamespace = "http://reportincident.devoxx.fusesource.org",! partName = "out")! @WebMethod(operationName = "ReportIncident",! action = "http://reportincident.devoxx.fusesource.org/ReportIncident")! public OutputReportIncident reportIncident(! @WebParam(partName = "in”,! name = "inputReportIncident”,! targetNamespace = "http://reportincident.devoxx.fusesource.org")! InputReportIncident in! );!}!
DEMO 2
Apache CXF
Agenda
• IntegraNon – introducNon
• Presenta(on of projects -‐ Camel, CXF, Ac(veMQ and ServiceMix
• Architectures presentaNon
• High availability, scalability and clustering
• PresentaNon of Fuse IDE
• Fabric new Strategy for provisioning and cloud
Par t I
Par t II
AcNveMQ (1)
Clients connect to the broker using connectors
with simple URIs
File & JDBC-‐based message stores supported.
… all based on a flexible Spring-‐based
core.
Topics and queues created dynamically.
Network connectors control how the
broker interacts with other brokers for
discovery, replicaNon, failover, clustering and store
& forward.
• A high performance, reliable messaging fabric, supporNng JMS, C, C++,.Net, Stomp clients. Offers persistence for messages and bridge between ServiceMix instances
• Point to Point – Sender/producer sends messages to a queue.
– Receiver/consumer listens for messages from the queue
– Messages are stored unNl read (or expired)
– Messages can be persisted on disk, are read only once.
c1:Consumer p:Producer
<<jvm>>���frodo:ActiveMQ
Foo:File m
1: Producer sends a message to the broker
m 2: Consumer reads
the message
AcNveMQ (2)
• Publish to subscribe – A JMS client sends message to the topic.
– The JMS broker sends message to all subscribers that are currently alive. • AlternaNvely, durable subscrip@ons can be used so that all subscribers will receive message even if not alive at Nme of sending
– Messages are consumed x Nmes (relaNon 1 to many)
s1:Subscriber
p:Producer
<<jvm>>���frodo:ActiveMQ
Foo:Topic m
m s2:Subscriber
m
AcNveMQ (3)
DEMO 3
Apache Ac(veMQ
Agenda
• IntegraNon – introducNon
• Presenta(on of projects -‐ Camel, CXF, Ac(veMQ and ServiceMix
• Architectures presentaNon
• High availability, scalability and clustering
• PresentaNon of Fuse IDE
• Fabric new Strategy for provisioning and cloud
Par t I
Par t II
ESB – ServiceMix (1)
DemysNfy the concept Enterprise Service Bus
• → This is not
• → Or a
But an exchanging pla`orm of messages, objects moving within transport layers
ESB -‐ ServiceMix (2)
• Allow to separate the business layer from those that will transport the informaNon and transform (marshal) it
• Provide connectors(= endpoints) to collect informaNon and send it on the bus as messages
ServiceMix (3)
• ServiceMix – Project of the Apache FoundaNon
• Started in 2005 to implement the Java Business IntegraNon specificaNon
• Decoupling of the components (business, technical) by using XML messages
• Define a model to package/deploy applicaNons (SU/SA)
• Layer transport is called Normalised Messaging Router
ServiceMix (4)
Message is ACK or ERR
Routing is defined into
the SU
Normalized = XML
exchanges
ServiceMix (5)
• JBI approach is interesNng but restricNve
– Everything must be XML format
– EncapsulaNon of the rouNng – Components dvlpt (25 vs more than 100 for camel)
– Few support from IT actors (IBM, Oracle, …) to improve exisNng spec 1.0 2.0
– Parallel with EJB story ….
ServiceMix (6)
• Decision has been taken 3 years ago by members of project to
– Become independent of JBI specificaNon
– Use « modularity » for jars management, versioning based on OSGI, OSGI EE specificaNons
– Adopt Camel as the new rouNng engine
ServiceMix (7)
• ServiceMix 4
• Based on an OSGI kernel – Apache Felix – Eclipse Equinox
• Becomes an IntegraNon Pla`orm for open-‐source
– Camel
– CXF – AcNveMQ,
– Aries (JPA, TransacNon, JNDI, Web)
Karaf – Heart of ServiceMix (8)
• Birthdate - 16th of June 2010 • Runtime using OSGI • Provide a lightweight container where
Java applications Camel routes WebServices
can be deployed
Karaf – Heart of ServiceMix (9)
<<jvm>>���:ServiceMix4
Java Boot (Register as service - Wrapper)
Lightweight container (Apache Karaf)
OSGI Kernel (Apache Felix / Eclipse Equinox)
Container Container Container Container Container Container
Karaf – Heart of ServiceMix (10)
• The pla`orm is « modular » because we can select the librairies, containers that we would like to use
Integration
WebService
Web
IoC
Middleware Java
Camel, JBI Jetty, Pax Web
CXF
ActiveMQ
Spring, Blueprint
Karaf – heart of ServiceMix (11)
• CharacterisNcs :
AdministraNon console (local, remote, ssh, web, jmx)
Provisioning system (features)
Hot deployment and configuraNon management
Instances management
Security integraNon (JAAS → ldap, jdbc, file)
Logging management (log4j, logger, commons logging, ….)
DEMO 4
Apache Karaf/ServiceMix
ServiceMix
ActiveMQ
Camel
CXF
<<supports jbi endpoints>> <<deploy into>>
<<supports cxf endpoints>>
<<supports activemq endpoints>> <<deploy into>>
<<deploy into>>
<<supports transport>> <<supports transport>>
<<implements flow with>>
The big picture
Agenda
• IntegraNon – introducNon
• PresentaNon of projects -‐ Camel, CXF, AcNveMQ and ServiceMix
• Architectures presenta(on
• High availability, scalability and clustering
• PresentaNon of Fuse IDE
• Fabric new Strategy for provisioning and cloud
Par t I
Par t II
<<jvm>> ServiceMix4
Architecture -‐ Messaging
FTP Web Service
Bean<<MyBean>>
from(«Yp://server/dir») .beanRef(«myBean»);
from(«cxf:bean:myWS») .beanRef(«myBean») ;
• The bus operates the exchanges between the endpoints using Camel, NMR, JBI
BUS
<<jvm>> ServiceMix4
Architecture -‐ Java
• The OSGI pla`orm proposes a Service Registry (comparable to JNDI) where "interfaces" are registered
• Clients access to the service using lookup mechanism
« Interface » DAO
A
OSGI registry B
<bean id="incidentServiceTarget" class="org.apache.camel.service.impl.IncidentServiceImpl"> <property name="incidentDAO"> <osgi:reference interface="org.apache.camel.dao.IncidentDAO"/> </property> </bean>
Service PoJo
C
Hibernate/JPA
<<jvm>> ServiceMix4
Architecture -‐ Messaging + Java
• Decoupling of the "services" from integraNon layer
from(« cxf:bean:myWS ») .beanRef(« myBean », « validate ») .beanRef(«org.devoxx.jpaService») .beanRef(« myBean », « generateResponse »);
C
Web Service
B
Bean A
« Interface » DAO
OSGI registry
Service PoJo Hibernate/JPA
<<jvm>> ServiceMix4
« Interface » DAO
OSGI registry
Service PoJo
Hibernate/JPA
Architecture -‐ Messaging + Java + Web
• Can be web pla`orm, it uses JeSy Web ApplicaNon Server, Pax Web – a tooling for war and OSGI HTTP Service to register Servlet, …
Web ApplicaNon
OSGI RunNme
Bundle
Web Service Bean
A
Architecture – OSGI limitation • CamelContext are isolated due bundles
classloader • They can’t communicate directly using direct://,
vm:// component
56
<<jvm>> :ServiceMix4
BUS
FTP
from(Yp) .to(direct://order)
from(direct://order) .beanRef(“order”)
WebService
Bundle A Bundle B
<<jvm>> :ServiceMix4
Architecture -‐ LimitaNon
• NMR is required to bridge routes
NMR
Camel Context
FTP
from(Yp) .to(NMR://order)
from(NMR://order) .beanRef(“order”)
Bean Order
Bundle A Bundle B
Architecture -‐ Extend
• When we need asymetric deployment because CPU, memory is not sufficient then we use AcNveMQ to interconnect the instances and persist messages
Agenda
• IntegraNon – introducNon
• PresentaNon of projects -‐ Camel, CXF, AcNveMQ and ServiceMix
• Architectures presentaNon
• High availability, scalability and clustering
• PresentaNon of Fuse IDE
• Fabric new Strategy for provisioning and cloud
Par t I
Par t II
High-‐availability -‐ AcNveMQ
• 2 mechanisms pure Master/Slave & Shared message store
<<jvm>> samwise:ActiveMQ
Master state replicated with the slave
<<jvm>> frodo:ActiveMQ
<<jvm>> samwise:ActiveMQ
The brokers compete to acquire lock on the shared system (file or DB)
<<jvm>> frodo:ActiveMQ
Server 2 Server 1
Master
• A clustered, highly available approach with 2 machines
Slave
<<jvm>> Frodo:ActiveMq
<<jvm>> Frodo:ActiveMq
<<jvm>> Frodo:ActiveMq
<<jvm>> Frodo:ActiveMq
Network Of
Brokers
AcNveMQ -‐ Network of brokers
High Availability -‐ ServiceMix
• Only one mechanism is available – Lock based (File, DB)
• Bundles can be started or not
Master
<<jvm>>���frodo:SMX4
<<jvm>> samwise:SMX4
Slave
Master
<<jvm>>���frodo:SMX4
<<jvm>> samwise:SMX4
Slave
AcNve/Passive
AcNve/AcNve
High-‐Availability – All
• Combine AcNveMQ & ServiceMix + Network of brokers
Slave
Headquarter
queue:order BUS
Camel -‐ FTP
from(“amq:queue:order”) .beanRef(“order”)
BUS
Bean
queue:order
Master
queue:order
BUS
SMX4 -‐ Master
queue:order
from(“Yp”) .to(“amq:queue:order”)
Regional offices
Camel -‐ WS
BUS queue:order
from(“cxf”) .to(“amq:queue:order”)
from(“amq:queue:order”) .beanRef(“order”)
Bean
SMX4 – Slave
NetWork of Broker
Fuse Fabric Strategy
• IntegraNon – introducNon
• PresentaNon of projects -‐ Camel, CXF, AcNveMQ and ServiceMix
• Architectures presentaNon
• High availability, scalability and clustering
END OF PART I
A break is welcome !
Par t I
Fuse IDE & Fabric New Strategy to build integration projects
Fuse Fabric Strategy
• IntegraNon – introducNon
• PresentaNon of projects -‐ Camel, CXF, AcNveMQ and ServiceMix
• Architectures presentaNon
• High availability, scalability and clustering
• Presenta(on of Fuse IDE
• Fabric new Strategy for provisioning and cloud
Par t I
Par t II
DEMO
Fuse IDE
Fuse Fabric Strategy
• IntegraNon – introducNon
• PresentaNon of projects -‐ Camel, CXF, AcNveMQ and ServiceMix
• Architectures presentaNon
• High availability, scalability and clustering
• PresentaNon of Fuse IDE
• Fabric new Strategy for provisioning and cloud
Par t I
Par t II
Fuse Fabric Strategy
• Why Fuse Fabric ? – “Integration” projects are really hard to install, configure, run and specially on OSGI platforms
– Distribution of workload/process is not an easy task
– Packaging and deployment must be simplified
– We want to make it easy!
What is Fuse Fabric? • http://fabric.fusesource.org/ • Open source software for configuring, provisioning
& running Fuse and Apache software on any machines – physical, virtual, private, public, private+public cloud or not
etc
• Keeps you DRY from those rainy clouds :) • Weave your containers into an easy to manage
fabric
Fuse Fabric: What does it solve? • Fact :
– Pay twice “dependencies” creation (maven and OSGI)
– maven-felix-pluglin simplifies bundle creation (*)
BUT this is still a complex process • Solution :
• Fab (Fuse Archive Bundle) • Use pom.xml dependencies instead of
MANIFEST to deploy bundles
P A C K A G I N G
Fuse Fabric: What does it solve?
• Fact – OSGI : Karaf – ServiceMix allows to create new instances and administrate them
BUT
DEPLOYMENT
• Instances are not “cloned”
• Configurations must be updated (manually)
• No Central Deployment platform
• Fabric will help us to create instances and manage them – Locally – Remotely (ssh) – In the cloud
DEPLOYMENT
Fuse Fabric: What does it solve?
Fuse Fabric: What does it do? • Scalability of HTTP, WebServices, RESTfull
services depend on external infrastructure (web farm, loadbalancer, hardware, …)
• Configuration changes are required (IP address, port number, …) to deploy new servers/services
• Solution : – Virtualization of services, endpoints, brokers – Discovery of services, endpoints and
message brokers
SCALAB I L I TY
What do we need ?
• Registry = repository of information(s) organised in a tree model
• Agent = process waiting instructions about things to do on servers (karaf, servicemix, activemq)
• Profile = list of tasks to do like – Insert component(s) – Setup a repository – Configure endpoint
Registry • Based on Apache ZooKeeper
– Is a Centralized Coordination Service, distributed and highly reliable
– Uses a directory & file based abstraction for looking at entries
Agent
• Feature deployed per Karaf/ServiceMix container • Agent connects to the registry to discover “tasks,
instructions” • Agent registers important informaNon into the registry • its locaNon, how to manage it via JMX etc
• Agent can control child Karaf containers or child processes
Profiles • Allow for easy group configuration & control
– like using Access Control Lists for security or LDAP trees
• Contains artefacts definition • An agent is started with one or more profiles
associated with it • You can then easily control a group of processes • Profiles support multiple inheritance to simplify
configuration work • Profiles are versioned for easy rolling update /
rollback
Profiles • Allow for easy group configuration & control
– like using Access Control Lists for security or LDAP trees
• Contains artefacts definition – Bundle – Features = collection of bundles – Repository = features xml file – Configuration = parameters of the endpoints, …
• An agent is started with one or more profiles associated with it
Profiles • Registry entries for a given agent:
– /fabric/configs/agents/{agent} – /fabric/configs/versions/{version}/agents/{agent} – /fabric/configs/versions/{version}/profiles/{profile}
• Profiles support multiple inheritance to simplify configuration work
• Profiles are versioned for easy rolling update / rollback
Camel Fabric • Expose / Virtualise a endpoint into the fabric space • Allow to invoke a remote endpoint • Loadbalance requests
DEMO
Fabric, Zookeeper. Provisioning and loadbalancing (ElasNc services)
D-OSGi Fabric • Implementation of distributed OSGI defined in
Enterprise OSGI Specification v4.2 • Allow to register “Interfaces” and look-up them
from remote instances • Use a TCP channel to send bytes over the wire
between jvm of fabric karaf instances • Services can be distributed everywhere and
easily faciltating the scalability
D-OSGi Fabric (next)
DEMO
Distributed OSGI services
ActiveMQ Fabric • Message brokers automatically discovery each
other • Clients use connection URI “fabric:default” to
connect to a message broker • If you have multiple logical ActiveMQ clusters just
name them in the URI e.g. – fabric:us-west, fabric:us-east, fabric:europe
• Workload can be shared between brokers (Apache Apollo)
ActiveMQ Fabric (next)
Other Fabrics • CXF Fabric
– adds a feature to the CXF bus for discovery & load balancing
• ServiceMix Fabric – allow local NMR to use remote endpoints when no local
NMR endpoint available
• Process Fabric – start & monitor processes – keep processes running across machines
References
• IntegraNon -‐ Camel : hSp://camel.apache.org
• ESB -‐ ServiceMix : hSp://servicemix.apache.org
• RunNme -‐ Karaf : hSp://karaf.apache.org
• WebService -‐ CXF : hSp://cxf.apache.org
• Broker -‐ AcNveMQ : hSp://acNvemq.apache.org
• OSGI EE -‐ Aries : hSp://aries.apache.org • FuseSource : hSp://fusesource.com
• EIP : hSp://www.enterpriseintegraNonpaSerns.com
References • Fabric web site :
– hSp://fabric.fusesource.org/ – hSps://github.com/fusesource/fabric
• Apache Zookeeper project – hSp://zookeeper.apache.org/
• Demo
– hSps://github.com/fusesource/fabric/tree/master/fabric-‐examples/fabric-‐camel-‐cluster-‐loadbalancing
– hSps://github.com/fusesource/fabric/tree/master/fabric-‐examples/fabric-‐camel-‐dosgi
A Progress SoYware Company
Any Questions?
• twitter: @cmoulliard@gertv #fusenews • http://fusesource.com