towards a model-based characterization of data and services integration

© 2011 TIBCO Software Inc. All Rights Reserved. Confidential and Proprietary.

Towards a Model-Based Characterization of Data and Services Integration

Paul C. BrownPrincipal Software Architect


2

Context: Interacting Systems

: System Bservice

: System A

reference

Response Request


3

Overlapping Information Views

System B Information

System A Information

: System Bservice

: System A

reference

Response Request


4

Multiple Schemas



: Message Schema

: System Bservice

: System A

reference

: A Schema : B Schema

Response Request


5

Domain Model: Conceptual Abstraction

Concepts, relationships, key attributes

: Domain Model



: Message Schema

: System Bservice

: System A

reference


Response Request


6

Mapping Concepts to Representational Schema

Concepts, relationships, key attributes

: Domain Model



: Message Schema

: System Bservice

: System A

reference


Response Request


7

Abstract

Conceptual

global : Domain Model



domain B : Domain Modeldomain A : Domain Model

: Message Schema

: System Bservice

: System A

reference


Response Request

Concepts are not Uniform: Mapping is Required


8

Abstract

Conceptual





: Message Schema

: System Bservice

: System A

reference


Response Request

Representational Mappings Derived from Concepts


9

Abstract

Conceptual





: Message Schema

: System Bservice

: System A

reference


Response Request

Conceptual Mappings Derived from Abstractions


10

Information has an Inherent Network Structure

Customer Service Order Fulfillment Service

Product Service

Sales Order Service

-quantity-price-orderLineID-/status

Sales Order Line Item

-quantity-shipmentLineID

Shipment Line Item-cardType-cardNumber-expiration-cardholderName

Credit Card Info

-date-orderID-/status

Sales Order

-name-customerID

Customer

-SKU-description

Product

-shipmentID-status

Shipment

Address

Address

Address Carrier

-shippedItem

-shipmentAddress

0..*1

-customerAddress

-trackingNumber

Carrier Shipment

Carrier Shipment

0..10..*-billingAddress

-shippingAddress

0..*1

-substituteFor

0..*

0..1

-orderedItem1

0..*


11

Communication Schemas Tend To Be Tree Structured

+getOrder( request : Get Order Request ) : Get Order Response...

Sales Order Service Interface+orderShipped( notification : Order Shipped Notification )

...

Sales Order Status Interface

b) orderShipped() Operationa) getOrder() Operation

Order Shipped Notification

-itemID : SKU-quantity-price-/status

Sales Order line Item

Get Order Response

-itemID : SKU-quantity-orderLineID-shipmentLineID

Shipment Line Item

-orderID

Get Order Request

-shipmentID-dateShipped-orderID

Shipment Notice-datePlaced-orderID-/status-customerID-customerName

Sales Order

Address

1..*

-billTo-shipTo


12

Information Realities

Canonical data model mythOne size does not fit all: complete concept vs. reference

Vocabularies varyThink about the multiple meanings of “attribute” “Procedure” to a physician vs. health insurance company

Information evolvesMedical procedure code change ICD-9 ICD-10Versioning is important

Information is replicatedConsistency is an issue


13

Data-Related Model Requirements

Network- and tree-structured representational schema (e.g. database schema, XSDs, JSON, record formats)

Mappings between representational schema Abstracted models of concepts and relationships

Independent of representational schema

Mappings between the abstracted concepts and representational schema

Flexible vocabulary One concept many terms One term many concepts

Versions of representations and mappings Mappings whose expression requires some form of

computation


14

Service Realities

Many operations are not pure functionsOperate on persistent state (information) managed by the

service

Operations are not independentSequencing constraints, business rules

Services often house cached dataCache update protocols and timing Impact on service operations

Vocabulary (terminology) varies and overloadedParticularly between organizations

Services evolveVersioning is important


15

Service-Related Model Requirements

Representation of: Services and their interfaces Service operations and their data structures State and state instances

Relationship between service operations and service state Dependencies between service operations Versions and mappings between versions Abstracted concepts related to services operations Mappings between concepts and concrete service models Flexible vocabulary Mappings whose expression requires some form of

computation


16

Process (Service Utilization) Context

display checkout page(s)

place order

display order acceptance

Accepted?

Sales Order Service Scope

record items shipped

send order for fulfillment

validate order and obtain payment

close order

record items received

all received?

deliver items

report delivery

ship items

send delivery notice

Customer Order Fulfillment ServiceWebSite Sales Order Service Carrier

shipment notice

delivery notice

goods

select checkout

receive goods

enter addresses,

credit card info, submit order

print order

Yes

Yes


17

Process (Service Utilization) Realities

Services may be involved in multiple processes Protocol semantics must be understood

REST, SOAP, XML over JMS, HTTP, file transfer, etc.

Coordination must be understoodFire and forget to distributed transactions


18

Process (Service Utilization) Model Requirements

Observable behavior of the service as viewed through its interfaces

Service usage scenarios Sequencing and dependency constraints between

service operations Coordination of activities between components Versions of observable behavior, usage scenarios,

sequencing, and coordination Mappings between versions


19

UML Can Satisfy Many Modeling Requirements

UML can represent concrete thingsData structure schemaServices, operations, state machines

UML can represent abstractionsConcepts and Relationships

How do we model mappings and multiple vocabularies?


20

Semantics of Business Vocabulary and Rules (SBVR)


21

Semantics of Business Vocabulary and Rules (SBVR)

Abstract concepts and relationships

Concrete schema and other representations


22

SBVR Can Complete the Modeling Picture

SBVR can represent mappings (simple and complex)

SBVR handles multiple vocabularies for concepts SBVR can be used to relate multiple UML

representations


23

What We Need

Uniform approach to modeling Abstract and concrete concepts and relationships

• Multiple domains with relationships Bi-directional Mappings

• Abstract concrete models• Concrete concrete models• Concrete models actual schema• Schema schema mappings

Versioning at all levels + mappings between versions

Tooling Schema Concrete model + schema-model mapping Concrete mappings schema mappings (e.g. XSLT) Abstract mappings concrete mappings

UML and SBVR seem to have the required building blocks!

towards a model-based characterization of data and services integration

Documents

tibco software

representational mappings

mappings mappings

mapping concepts

abstracted concepts

domain model

data structuresstate

conceptual mappings