integration solution patterns

Integration Solution Patterns

© WSO2 2013. Not for redistribution. Commercial in Confidence.

May 2013 Nadeesha Gamage Dakshitha Ratnayake

About WSO2

•  Providing the only complete open source componentized cloud platform •  Dedicated to removing all the stumbling blocks to enterprise agility •  Enabling you to focus on business logic and business value

•  Recognized by leading analyst firms as visionaries and leaders

•  Gartner cites WSO2 as visionaries in all 3 categories of application infrastructure

•  Forrester places WSO2 in top 2 for API Management •  Global corporation with offices in USA, UK & Sri Lanka

•  200+ employees and growing •  Business model of selling comprehensive support & maintenance for our products

150+ globally positioned support customers

Integration

Image Source - http://blog.bootstraptoday.com/2013/01/03/considering-saas-3-tips-for-your-data-integration-strategy/

Integration – Why?

•  Business applications rarely live in isolation.

•  Users expect instant access to all business functions an enterprise can offer.

•  Functionality may reside in different systems.

•  Requires disparate applications to be connected into a larger, integrated solution.

Integration – How?

•  Architecting integration solutions is a complex task.

•  There are many conflicting drivers and even more possible 'right' solutions.

•  No "cookbook" for enterprise integration solutions.

•  Asynchronous messaging architectures have proven to be the best strategy for enterprise integration because they allow for a loosely coupled solution that overcomes the limitations of remote communication, such as latency and unreliability.

Why Patterns?

Patterns are an excellent way to ensure: 1. Everyone understands the model taken – poses a design

problem and presents a solution 2. Common vocabulary 3. Use of best practices - experience base that architects have

gained by repeatedly building solutions and learning from mistakes.

4. Faster development time – reuse of patterns and not reinvent patterns

Outline §  Enterprise Integration Patterns (EIP) – Gregor Hohpe

§  Message Translator, Content-Based Router, etc.

§  Implementation of EIPs in WSO2 ESB §  Low level patterns §  Using functional components such as mediators, sequences

etc.

§  High level solution patterns §  ESB Patterns – Concentrator, Security Proxy etc. §  Overall Integration Patterns

§ e.g. API Gateway, Trusted subsystem etc. § Also see Integration Styles from EIP §  Implementation of integration patterns using WSO2

products

Enterprise Integration Patterns

http://www.eaipatterns.com/

"The core language of EAI, defined in the book Enterprise Integration Patterns by Gregor Hohpe and Bobby Woolf, is also the core language of defining ESB flows and orchestrations, as seen in the ESB's developer tooling."

Enterprise Integration Patterns

§  EIPs are patterns using asynchronous messaging as the style of integration.

§  They are technology-agnostic which means it does not matter what programming language or operating system you use.

§  Categorization of Patterns •  Messaging Channels •  Message Constructions •  Message Routing •  Message Transformation •  Messaging Endpoints •  System Management

WSO2 ESB is…

•  A lightweight, high performance ESB •  Feature rich and standards compliant

•  SOAP and WS-* standards •  REST support •  Domain specific protocol support (eg: FIX)

•  User friendly and highly extensible •  100% free and open source with commercial support

WSO2 ESB - Functional Components

•  Mediators

•  Sequences

•  Endpoints

•  Proxy Services

•  Event Sources

•  Priority Executors

•  Message Stores

•  Tasks

•  Local Entries

•  Registry

More on Functional Components

•  Each functional component serves a specific purpose

•  Functional components can be mixed and matched to implement various integration scenarios and patterns

•  Configuring WSO2 ESB for a given scenario requires: •  Identifying the right set of components •  Putting them together in the optimal manner

Mediators

http://docs.wso2.org/wiki/display/ESB460/Mediators

Rich mediator library, wherein most cases a single mediator can be directly mapped to an EIP.

List of Mediators

•  Send •  Log •  Property •  Sequence •  Event •  Drop •  Enrich •  Filter •  Out •  In •  Switch •  Router •  Conditional Router

•  XSLT •  URLRewrite •  XQuery •  Header •  Fault •  Cache •  Clone •  Store •  Iterate •  Aggregate •  Callout •  Transaction •  Throttle

•  RMSequence •  DBReport •  DBLookup •  Rule •  Entitlement •  OAuth •  Class •  POJOCommand •  Script •  Spring •  Smooks

Sequences

•  A chain of mediators

•  Messages are sent through all the mediators in the sequence, in the order they appear

Enterprise Integration Patterns with WSO2 ESB

Complete Guide - http://docs.wso2.org/wiki/display/IntegrationPatterns/Enterprise

+Integration+Patterns+with+WSO2+ESB The guide shows how each pattern in the patterns catalog can be

simulated using various constructs in WSO2 ESB. A selected few of the patterns will be discussed in this webinar.

Message Channel

A message channel is a logical channel which is used to connect the applications. One application writes messages to the channel and the other one (or others) reads that message from the channel. Message queue and message topic are examples of message channels.

•  Queues - •  With WSO2 ESB, a common pattern is to persist the

message from an incoming HTTP request into a persistent message queue, and then process from there.

•  WSO2 Message Broker, ActiveMQ etc. can provide the persistent queue.

•  Topics – Supported by ESB and Message Broker •  WSO2 ESB has an event distribution model and eventing

support. •  WSO2 Message Broker, the Apache QPid-based broker,

provides higher performance as well as supporting the JMS API.

Message Channel continued

Message Translator

Message translator transforms messages in one format to another. Can be implemented through several Mediators in WSO2 ESB – • Enrich • XSLT • Smooks • Payloadfactory • XQuery

Publish-Subscribe Channel

•  This type of channel broadcasts an event or notification to all subscribed receivers.

•  This is in contrast with a point-to-point channel. Each subscriber receives the message once.

•  The most common implementation of this pattern is messaging topic.

•  Event mediator. •  Topics – WSO2 ESB/ WSO2 Message Broker

Publish-Subscribe Channel continued

Dead Letter Channel

The Dead Letter Channel describes how messaging system determines how to deliver a dead message to the specified recipient. This may be caused by connection problems or other exception like overflowed memory or disc space.

Dead Letter Channel continued

Content-Based Router

Content-Based Router examines message contents and route messages based on data contained in the message.

Content-Based Router continued

Message routing is simulated by Switch and Send mediators of the WSO2 ESB. The Switch Mediator depicts the Router and observes the content of the message while the Send Mediator is used to send the message to a selected recipient. Each case defined should decide on routing the message to the appropriate service.

Content Enricher

Content Enricher as the name suggests enrich message with missing information. Usually an external data source like a database or a web service is used.

Content Enricher continued

Message Filter

Message Filter EIP checks an incoming message against a certain criteria that the message should adhere to. If the criteria is not met, the filter will discard the message. Else, it will route the message to the output channel. The Filter mediator in the ESB will check on the criteria.

Splitter/Aggregator

The Splitter EIP - Processes messages containing multiple elements. Breaks out the composite message into a series of individual messages. The Aggregator EIP - Combines the results of individual messages so that they can be processed as a whole.

Splitter/Aggregator continued

Splitter

Aggregator

Composed Message Processor

The Composed Msg. Processor EIP is used to process a composite message. It maintains the overall message flow when processing a message consisting of multiple elements.

Composed Message Processor continues

•  Use of Sequences •  Named Sequences •  Sequences in Proxy Services – In Sequence, Out

Sequence, Fault Sequence

WSO2 ESB Configuration Documented as EIPs

ESB Patterns

•  It’s worth understanding the way the ESB is being used: •  ESB are not the same as network routers •  Can be used in multiple ways for Routing, Traffic filtering,

Transformation, Protocol Switching, Load balancing. •  More like a Swiss-army knife •  Be clear about how you intend to use it •  The same ESB can be implementing multiple patterns

•  Anti-Patterns •  Implement all your business logic in the ESB

The Concentrator Pattern

EAI Hub

•  Request Transformation •  ESB can split and clone messages.

•  Response Transformation •  ESB can aggregate messages.

•  Service chaining •  ESB can act as a lightweight orchestration engine.

Federated ESB Pattern

Enterprise ESB

Routing and Audit

Department ESB

Department ESB

Department ESB

Overall Integration Patterns

•  API Gateway •  Service Broker/Trusted Subsystem •  Service Firewall •  Master Data Management •  Security Gateway •  Dual Channel/ Claim Check

WSO2 Product Stack

API Gateway

Business Problem •  Different Service platforms run on a variety of server hardware,

operating systems and development platforms. •  Services expose different communication and security patterns. •  Complexity in exposing the SOA to internal and external

consumers.

API Gateway

Solution •  An Approach to simplify and normalize interaction with

heterogeneous services. •  Define a unified generic service layer on-top of the existing

services to expose to the consumers. •  API Gateway Pattern is able to solve this business problem.

API Gateway

Service Broker/ Trusted Subsystem

Business Problem •  Backend services need to be relieved from authentication of

service clients. •  Security authentication needs to be handled before the call

reach the backend service.

Solution •  User authentication to be handled at the ESB layer. •  Backend service trust the ESB. •  ESB access backend service on-behalf of the authenticated

user.

Service Broker/ Trusted Subsystem

Service Firewall Pattern

Business Problem •  To expose internal business services to external service

consumers. •  To ensure that only authorized users access the services

exposed. •  To validate the user credentials before reaching the internal

network. Solution •  Deploy a separate ESB to validate user requests. •  Achieve this by using a Service Firewall Pattern.

Service Firewall Pattern

Security Gateway Pattern

Master Data Management (MDM)

Business Problem •  Business organizations have legacy applications and databases

that store and manage master data. •  Managing master data is critical to make successful business

decisions. •  Present IT infrastructure is depended on SOA. •  Need a way to connect the Master Data with the SOA.

Master Data Management (MDM)

Traditional Approach

Master Data Management (MDM)

Solution •  To expose Master data as services in a SOA. •  Master Data Management Pattern is able to solve this problem.

Dual Channeling

Business Problem •  When large files are required to go through a workflow process. •  Moving files through the workflow is inefficient, creates

additional traffic, reduce the throughput.

Dual Channeling Solution •  Dual Channel pattern avoids the above mentioned problems by

having two channels. •  For the metadata on file •  For the actual file

•  A light-weight message with the file related metadata, to process within the workflow.

•  Metadata contains the pointer reference to the actual file.

Dual Channeling

Dual Channeling

Resources

•  http://wso2.com/blogs/architecture

•  http://www.eaipatterns.com/

•  http://docs.wso2.org/wiki/display/IntegrationPatterns/Enterprise+Integration+Patterns+with+WSO2+ESB

Questions?

Engage with WSO2

•  Helping you get the most out of your deployments •  From project evaluation and inception to development

and going into production, WSO2 is your partner in ensuring 100% project success

59

lean . enterprise . middleware