event driven architecture andré hoekzema enterprise technology architect microsoft nederland...
TRANSCRIPT
Event Driven ArchitectureAndré HoekzemaEnterprise Technology ArchitectMicrosoft [email protected]
Agenda• Introduction• Standardize on Event Based Interaction• Use Event Processing Effectively• Patterns of Response
Introduction
Aspirations in the Global Event Cloud
• Situational Awareness• Sense & Respond• Track & Trace
.NET Linux
J2EEUnix
OS/390
MQ
DB2
Finance
PeopleSoft
SAP
Siebel Dir
Outlook
CustomerEmployee
ProductSales
Digital Archeology
CBDI
Passwd change
new auto pay
account login
account login
deposit
activity history
withdrawal
logout
account login
account balance
transfer
deposit
new auto pay
enquiry
enquiry
logout
Passwdchange
new auto pay
account login
account login
deposit
activity history
withdrawal
logout
account login
transfer
deposit
new auto pay
enquiry
enquiry
logout
Passwd change
new auto pay
account login
account login
deposit
activity history
withdrawal
logout
account login
account balance
transfer
deposit
new auto pay
enquiry
enquiry
logout
Passwdchange
new auto pay
account login
account login
deposit
activity history
withdrawal
logout
account login
transfer
deposit
new auto pay
enquiry
enquiry
logout
Passwd change
new auto pay
account login
account login
deposit
activity history
withdrawal
logout
account login
account balance
transfer
deposit
new auto pay
enquiry
enquiry
logout
Passwdchange
new auto pay
account login
account login
deposit
activity history
withdrawal
logout
account login
transfer
deposit
new auto pay
enquiry
enquiry
logout
Passwd change
new auto pay
account login
account login
deposit
activity history
withdrawal
logout
account login
account balance
transfer
deposit
new auto pay
enquiry
enquiry
logout
Passwdchange
new auto pay
account login
account login
deposit
activity history
withdrawal
logout
account login
transfer
deposit
new auto pay
enquiry
enquiry
logout
Passwd change
new auto pay
account login
account login
deposit
activity history
withdrawal
logout
account login
account balance
transfer
deposit
new auto pay
enquiry
enquiry
logout
Passwdchange
new auto pay
account login
account login
deposit
activity history
withdrawal
logout
account login
transfer
deposit
new auto pay
enquiry
enquiry
logout
/
EDA Characteristics
Aggregate events across multiple sources; compare reality with expectations
Analyze
Detect events across extended environment in real-time
SenseUpdate expectations; Invoke
distributed services in real-time
Respond
Standardise on Event-Based InteractionSense
Some Definitions
• Bank Depostit• Stock Trade• Customer Order
• Password Change• Shipment Delivery
• Business Event: An event relevant to a commercial enterprise
• Event object (sometimes also just called “event”): A report or record of an event happening
• XML document describing the bank deposit• Shipping notification in e-mail• Row in a database table
What Makes EDA Different from Most Conventional Business Applications? • Event-driven Architecture (Gartner)
– Events are pushed not pulled (recipient need not poll repeatedly)
– Recipients act immediately upon event arrival, not when a request is made or on a pre-planned schedule (runs sooner)
– Event source does not specify what action the recipient will perform – the recipient determines the operation (less coupling)
Observer pattern• One-to-many dependency - when one object changes state, all
dependents are notified and updated
Event-Based Interaction
Notification Service Decouples Producer and Consumer with Publish Subscribe Functionality
Communications
Consumers (Sink)
Event
Producers
Notification Service
Notification Service
Notification Notification
Publish Subscribe
Event BasedInteraction
Act
Service Oriented
Service Oriented Communication• Two fundamental aspects of a service-oriented
(SO) world are:
– A common protocol for communication between SO applications
– A common foundation for creating SO applications
Application Y
Application X
Application Y
Windows Communication
Foundation
Service Oriented Infrastructure• SOAP is not enough:
– Must connect with existing applications that don’t speak SOAP
– Need to translate among different data formats– No agreement on a queued messaging protocol
Notification
Service
SOAP
Messages
Application X
Application Y
Standards EmergingWS-Notification
WS-Base Notification
WS-BrokeredNotification
WS-Topics
approved OASIS standard as of October 2006
WS-EventNotification (new)
W3C Submission in March 2006 WS-Eventing
WS-Resource Transfer (new)
Event Passing
• Benefit is experienced mostly in the IT department– The absence of control coupling – Easy to add new sinks, move a sink, or eliminate a sink
Publish Subscribe
Sink++Source++
SinkEvent
Event
Event
SourceEven
t
Notification Technologies• WS Protocol Based
– WCF + Customer Finance Framework (CFF),
– WCF + Robotics Studio
• Rules Based– BizTalk Rules Engine, – SQL Server Notification
Service, – SQL Server Service Broker
•Notification Protocols
•Notification Rules
Publish Subscribe
Sink++Source++
SinkEvent
Event
Event
SourceEven
t
Summary Event-Driven Interaction
CommunicationsRe-use of Service Oriented InfrastructureMicrosoft Platform: WCF, BizTalk Server, W2K3
Protocol Ws-Eventing, WS-Notification, (WS-EventNotification)
“Extreme” Decoupling with Notification Service
• Rules Based: BTS Rules Engine, SQL Service Broker, SQL Notification Service
• Protocol Based: not yet oo the box, WCF + CFF, WCF + Robotic Studio
Event Driven InteractionEvents are pushed,Immediate response by Consumer, Reaction unknown by Sender.
Benefits Flexibility and MaintainabilitySystems that possibly align with global event cloud
Use Event Processing EffectivelyAnalyse
Types of Event Processing
• Simple Event Processing• Event Stream Processing• Complex Event Processing
Event Stream Processing (ESP)• Multiple ordered events in one or a few event streams.
Typically high-volume and low-latency• Stock quotes, click streams, RFID
Event Streams
Response(ERP,BAM,
BPM, ..)Event
Event
Service Oriented Communication
ESP System
Event Processing Agent (EPA)
Fast filtering, calculate aggregates (sum, average), Boolean
Event Stream ProcessingRFID Inventory Management
Inventory < $4K
> $1,500
RFID Support in BizTalk Server R2
DistributionInventoryManufacturing Packaging Warehouse ReceptionQuality Assurance
Device Layer
Physical Layer
Business Applications
RFID Event Processes
Business IntelligenceSQL / SQL BI for data analytics
R2
R2
BizTalk RFID Server
DistributionInventoryManufacturing Packaging Warehouse ReceptionQuality Assurance
Event Processing Agent
Custom Event Handlers
Rules Engine Event Handlers
SQL Server Event Sink
BizTalk RFID API & Web Services
RFID Sink Database
Biztalk Server orBusiness Application
RFID Devices
Device Providers
Device Service Provider Interface (DSPI)
Device Layer
Physical Layer
Business Applications
RFID Event Processes
A Distributed Notification Architecture
Local Broker (middleware
)
Local Broker (middleware
)
Border Broker
InnerBroke
r
InnerBroke
r
Border Broker
Border Broker
Local Broker (middleware
)
Local Broker (middleware
)
Border Broker
Border Network
Filter-based or Flooding
Producer
Consumer
iGPS : Transforming the Retail Pallet Pooling Industry
Retail StorePallet Manufacturer Retail Supplier
Retail Regional DC
Retail Pallet Depot
• XAM RFID Portal
• Retailer RFID Portal
• Pallet Pool for Major Retailers• RFID Tags Embedded• End-to-End Solution
Microsoft.com/rfid
Deployments
Deployments
4/2007 - Central & 2 Sites
6/2007 - 6 Sites
9/2007 - 15 Sites
12/2007 - 30 Sites
Q1 2008 > 50 Sites
-Central Server- Edge Server (Mfg)- Edge Server (Depot)
TBH
M Allen
TBH
D Ray
Q3 2007: >2,000,000 transactions/month
Q2 2008: >10,000,000 transactions/month
RFID Readers RFID Readers RFID Reader Mobile
External WMS, ERP
Central Server
Edge Server Edge ServerEdge ServerCustomer ERP
Web ServicesWCF
Complex Event Processing• Cloud may contain many Event Types, Streams, Channels• The events may be unrelated• CEP System issues pattern detection using
Rules and Algorithms
Service Oriented Communication
CEP System
Event Processing Agent (EPA)
Complex-event Objects
Response(BAM, BPM)Event
Event
CEP System
Event
Event Cloud
Complex Event Processing B2B Scenario
Multiple txns on same credit card within 10 min at locations over 20 miles apart?
Suspicious purchase? (Notify cardholder)
Suspicious or multiple txns
Patterns for Complex Event Processing
Filtering
Database Lookups
In-memory Caching Hierarchal Events
Correlations (Joins)
Event Pattern Matching
Dynamic Queries
Aggregation over Windows
State Machines
http://www.coral8.com/system/files/assets/pdf/Coral8DesignPatterns.pdf
Summary Use Event Processing Effectively
Technologies BizTalk Server 2006/R2 + RFID Edge ServerSQL Server 2005, .NET Framework
Event Processing CapabilitiesPatterns for Event Processing leverage full Platform Capabilities: SOA/BPM, Data/BI, Scalability
Basic Types • Simple Event Processing, • Event Stream Processing• Complex Event Processing
Benefits Earlier and Improved Businss Insight
Patterns of ResponseRespond
Sample Applications That Use CEP with Business Events• Algorithmic trading — Finance, energy• Compliance reporting and monitoring — RegNMS, others• Customer experience management — Call center and Web clicks• Financial controls — "Track and trace," surveillance• Fraud detection — Anti-money-laundering, credit cards, telco• Hospital monitoring — Emergency room management• Military — Intelligence, battlefield operations, satellite• Pharmaceutical — Supply chain management, pedigree tracking• Security — Intrusion detection, denial of service• Sensor networks — RFID, GPS and others• Transportation operations — Trucks, airlines, ships or trains• "Triple play" telco — Telephone, television and Internet
Patterns of Response• Invoke Service (Orchestration)• Update BAM Dashboard• Initiate Human Workflow• Web 2.0 Subscriptions
Notification
Service
Application X
Application Y
BAM
Web 2.0
BAM as Key Enabler for CEP
Predlogpripravljen
Predlaganogradivo
Obravnavagradiva
Služba Vladeza zakonodjo
Drug organ DU
Mnenje SlužbeVlade
Obravnava na DT Vlade
Delovna telesa Vlade
PripravapredlogaPredlagatelj DO
Obravnava gradiva
Mnenje DO
Obravnava zaključena
Mnenje DT
Obravnava zaključena
Obravnava naseji Vlade
mnenje pozitivno
Vlada
Predlaganogradivo
Dopoljnjevanje gradiva
Gradivo dopoljnjeno
Dopoljnjeno gradivo
Predlaganogradivo
Dopoljnjeno gradivo
Priprava mnenja
Obravnava zaključena
negativno mnenje
negativno mnenje
Besedilo predloga zakona
za prvo obravnavo
Matično delovno telo DZ
Prva obravnava
Mnenje pripravljeno
Priprava mnenja
Sekretariat zazakonodajo
Pripombe
pozitivno mnenje
Mnenjesekreteriata
Zapisnik seje
Sklepi in stališča
Državni zbor
Zakon ni sprejet
PredlagateljPriprava predloga za drugo obravnavo
Prva obravnavazaključena
Priprava dopolnitveSekretariat zazakonodajo
zakon se ne sprejme
Druga obravnava
potrebni večji popravki
potrebni manjši popravki
Amandmaji
Poslanci
Matično delovno telo DZ
Predlagatelj
Drugo zainter. DT
Priprava amandmajev
Amandmaji vloženi
Dopolnitevpripravljena
Državni zbor
Besedilo predloga za drugo obravnavo
Besedilo predloga za drugo obravnavo
Druga obravnavazaključena
besedilo
za prvo obravnavo
za drugo obravnavo
Besedilo predloga zakona za drugo
obravnavo
Besedilopredloga zakona
za tretjo obravnavo
število sprejetihamandmajev
Stališča in sklepi DZSprejeti amandmaji
Priprava predloga za tretjo obravnavo
velikoPriprava dopolnitvemajhno
Tretja obravnava
Dopolnitev pripravljena
Besedilo predlogaza tretjo obravnavo
Sekretariat zazakonodajo
za tretjo obravnavo
Državni zbor
Mnenje MDT DZ
Mnenjesekreteriata
Mnenje MDT DZ
Mnenje MDT DZ Mnenjesekreteriata
Besedilopredloga zakona
za tretjo obravnavo
Zakon ni izglasovan
Zapisnik seje
Zakon jeizglasovan
Sprejetzakon
Letni program ali
zahtevek Vlade
Predlog za tretjo obravnavo
Predlagatelj
Predlog za drugo obravnavo
Event Driven Process Design
BAM Dash Board
Event
Activity
Summary Patterns of Response
Benefits Ability to act on earlier and Improved Businss Insight
Where to StartEvent Driven Process Design + BAM as enabler for CEP
Patterns of Response Once you have gained business insight ..
Platform Architecture
User Experience
Composite Apps Office App DevicesWeb
Portal Collaboration
Composition Layer
Data Aggregation
Workflow Orchestration
Eventing
Service Agents Business Rules
Data Platform
Data ManagementData Integration
Core InfrastructureApplication Server
Applications
ERP SCM CICS CRM
Business Intelligence
Develo
pm
en
t an
d
Toolin
g
Iden
tity &
Access
Man
ag
em
en
t
Microsoft Platform Stack
User Experience
Composite Apps Office App DevicesWeb
Portal Collaboration
Composition Layer
Data Aggregation
Workflow Orchestration
Eventing
Service Agents Business Rules
Data Platform
Data ManagementData Integration
Core InfrastructureApplication Server
Applications
ERP SCM CICS CRM
Business Intelligence
Develo
pm
en
t an
d
Toolin
g
Iden
tity &
Access
Man
ag
em
en
t
Office Business
Applications (OBA)
Conlusion• Event Driven Architecture for Flexible and
Maintainable Systems
• Event Processing for improved Business Insight
• Leverage full Platform Capabilities to support Event Driven Architecture
Resources: Roy Schulte (Gartner), Brenda Michelson (EventLinks), David Luckham (Power of Events), Nico Copier (Microsoft), Eric van Bever (Microsoft), Richard Evdemon (Microsoft), Shy Cohen (Microsoft), Jack van Hoof (Dutch Railways), CBDI-Forum
Discussion, Questions & Issues
© 2007 Microsoft Corporation. All rights reserved.This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.