dynamic coupling: pot of gold under semantic rainbow
DESCRIPTION
Dynamic Coupling: The Pot of Gold under the Semantic Rainbow When two people communicate they bring to the conversation a wealth of subtle messages and contextual implications. Such interactions are always dynamic -- never the same from one conversation to another. However, when two pieces of software communicate, we strive for fixed, static interfaces in order to facilitate the interaction while abstracting the underlying code -- what we call loose coupling. But in building such contracted APIs, we extinguish any hint of the conversational context so important in human communication. Such is the challenge for all semantic integration: loosely coupled interfaces, be they Web Services or RESTful APIs, fix the interaction context, eliminating the inherent dynamism of all true semantic interactions. No semantic standard, no traditional integration tool will ever overcome this problem. It's no wonder true semantic integration has always been just out of reach. Attendees of this session will learn: * The limitations of loose coupling and why we need dynamic coupling to address them * Why the battle between loosely coupled interfaces and semantic integration will never end without a rethink of integrationTRANSCRIPT
Copyright © 2014, Intellyx, LLC
1
Dynamic Coupling
The Pot of Gold under the
Semantic Rainbow
Jason BloombergPresident
@theebizwizard
Copyright © 2014, Intellyx, LLC2
About Jason Bloomberg
• President of Intellyx, Global Advisory Firm focused on Digital Transformation through Architecting Agility– Formerly with ZapThink 2001 - 2013
• Contributor to Forbes– http://www.forbes.com/sites/jasonbloomberg
• Created the Licensed ZapThink Architect (LZA) SOA course & associated credential
• Run the Bloomberg Agile Architecture Certification Course around the world
• Buy my newest book, The Agile Architecture Revolution
• Include @theebizwizard in tweets
Copyright © 2014, Intellyx, LLC3
Human Interactions Always Have Context
Ph
oto
Cre
dit
: T
he L
ex T
alio
nis
htt
ps:
//w
ww
.flic
kr.c
om
/ph
oto
s/6
52
51
86
1@
N0
7/5
98
80
80
13
1/s
izes/
o/
Every department, team, group, or interaction has its
own business context
Copyright © 2014, Intellyx, LLC4
Traditional Apps Have their Own Business Context
Copyright © 2014, Intellyx, LLC5
Don’t Shoehorn your Data
Shoehorning information into techie ideal of canonical data model woefully unrealistic
Copyright © 2014, Intellyx, LLC6
Why Semantic Interop has been Out of Reach
• Existing technology approaches have been too inflexible
• Require a more agile approach
• Not just about technology
We require Agile Architecture
Copyright © 2014, Intellyx, LLC7
What is Bloomberg Agile Architecture?
• Bloomberg Agile Architecture™ reinvents Enterprise Architecture (EA) to drive business agility across the organization
• New architecture paradigm• EA technique focused on
architecting agility
Copyright © 2014, Intellyx, LLC8
BAA Maturity Metamodel
Copyright © 2014, Intellyx, LLC9
BAA Maturity Metamodel
Copyright © 2014, Intellyx, LLC10
Loose Coupling
• Abstract underlying code with contracted interface
• Separation of concerns between consumers & providers
• Can change underlying code without breaking interaction, as long as endpoints comply with contract
Ph
oto
Cre
dit
: R
ay F
ost
er
htt
ps:
//w
ww
.flic
kr.c
om
/ph
oto
s/9
44
18
46
4@
N0
8/9
63
13
93
07
3/s
izes/
c/
Copyright © 2014, Intellyx, LLC11
Contracted Interfaces
• We must somehow contract interfaces in order to abstract the underlying functionality– Provides loose coupling– Some measure of flexibility
• Contracts (including REST’s uniform interface) are always a compromise – Interface itself now lacks
flexibility– IT becomes limiter of business
agility
Ph
oto
Cre
dit
: M
LB F
an
Cave h
ttp
://m
lb.m
lb.c
om
/fan
cave/
Copyright © 2014, Intellyx, LLC12
Pros & Cons of Loose Coupling
• Pros– Can make some changes
without breaking interaction– Increases flexibility of
distributed computing interactions
– Supports Service discoverability & late binding
• Cons– Inflexible interfaces limits
business agility– Challenges versioning
Services Ph
oto
Cre
dit
: S
teven
Dep
olo
htt
ps:
//w
ww
.flic
kr.c
om
/ph
oto
s/st
even
dep
olo
/44
82
49
12
95
/siz
es/
o/
Copyright © 2014, Intellyx, LLC13
The Importance of a Uniform Interface
• Web Services have custom operations– Unwanted tight coupling– Vendor incompatibilities
• REST has only HTTP verbs as operations (GET, POST, PUT, and DELETE)– HTTP verbs, URI structure,
and Internet Media Types fully define API
– Dramatic simplification of integration
Ph
oto
Cre
dit
: D
VID
SH
UB
htt
ps:
//w
ww
.flic
kr.c
om
/ph
oto
s/d
vid
s/6
34
36
84
93
9/s
izes/
z/
Copyright © 2014, Intellyx, LLC14
Deep Interoperability
Copyright © 2014, Intellyx, LLC15
Challenges of HATEOAS
• Central architectural constraint of REST• How to program client to obtain necessary
metadata via hyperlinks?• Hypermedia-based data
formats area of active research– Many immature options,
no clear best practice
Ph
oto
Cre
dit
: H
ori
a V
arl
an
htt
ps:
//w
ww
.flic
kr.c
om
/ph
oto
s/h
ori
avarl
an
/44
39
64
34
19
/siz
es/
z/
Hypermedia as the Engine of Application State
Copyright © 2014, Intellyx, LLC16
HATEOAS and Affordances
• Resolve URIs, process requests via a uniform interface
• Clients render representations that conform to standard Internet Media Types – Generalized affordances– Doesn’t matter what URI,
request, or representation as long as they conform to the constraints of the architecture P
hoto
Cre
dit
: E
raPh
ern
alia
Vin
tag
e h
ttp
s://
ww
w.fl
ickr
.com
/ph
oto
s/era
ph
ern
alia
_vin
tag
e/3
03
43
41
31
0/s
izes/
o/
Copyright © 2014, Intellyx, LLC17
Does REST Provide Deep Interoperability?
• Deep Interoperability– Endpoints automatically negotiate
basis for integration– Transport & message protocols, API
contracts, data schemas• HATEOAS potentially allows for
deep interoperability– Difficult to program in practice– How to program software client to
know how to follow hyperlinks to obtain necessary metadata?
Ph
oto
Cre
dit
: p
lais
an
ter
htt
ps:
//w
ww
.flic
kr.c
om
/ph
oto
s/p
lais
an
ter/
53
60
97
67
81
/siz
es/
z/
Copyright © 2014, Intellyx, LLC18
Custom Media Types
• REST loose coupling depends upon Standard Internet Media Types (formerly called MIME types)
• Internet Media Types designed for display metadata– HTML, PDF, etc.
• REST’s answer for custom data formats are Custom Media Types
• Introduce tight coupling– How to communicate, maintain, & version them? P
hoto
Cre
dit
: P
en
Wag
gen
er
htt
ps:
//w
ww
.flic
kr.c
om
/ph
oto
s/ep
w/4
34
15
54
77
1/s
izes/
z/
Copyright © 2014, Intellyx, LLC19
Automating HATEOAS
• Hypermedia affordances depend upon HATEOAS being handled manually
• A person changes the state of their application by clicking hyperlinks
• Code an arbitrary piece of software to serve as the RESTful client – May not have a user
interface– Knows how to follow
hyperlinks to gather all the metadata it needs to understand how to behave
Ph
oto
Cre
dit
: C
had
Mill
er
htt
ps:
//w
ww
.flic
kr.c
om
/ph
oto
s/ch
ad
mill
er/
13
67
00
90
5/s
izes/
o/
Copyright © 2014, Intellyx, LLC20
The Agility Platform
Your Application
Your Application
Application-Building
Tool
Your Application
Application-Building
Tool
Tool-Building Platform
Automated HATEOAS in Action
Copyright © 2014, Intellyx, LLC21
BAA Layers of Abstraction
META Dealing with Change (metaprocesses, metapolicies, etc. )
DYNAMICAbstract Models (dynamic schemas,
dynamic APIs, etc.)
ABSTRACTED (LOGICAL)
Abstracted Technology (schemas, software interfaces, etc.)
PHYSICALTechnology (software, middleware,
databases, etc.)
Copyright © 2014, Intellyx, LLC22
Working with Abstract Models
• Abstracts choice of schema
• Abstracts choice of API contract
• Abstracts semantic context
Ph
oto
Cre
dit
: T
on
y C
alr
ns
htt
ps:
//w
ww
.flic
kr.c
om
/ph
oto
s/to
nyca
irn
s/1
47
31
78
20
52
/siz
es/
l
Resolve to logical models at run time
Copyright © 2014, Intellyx, LLC23
Supporting Abstract Models in Software
• Tool-building platform (business agility platform)– All affordances, affords tool-
building capabilities• Application-building tool
– Affords application-building capabilities to non-developers
– Rich UI support– Non-imperative logic
• Event-driven, process-centric, and/or rules-based, for example
Ph
oto
Cre
dit
: elle
nm
1 h
ttp
s://
ww
w.fl
ickr
.com
/ph
oto
s/elle
nm
1/6
48
93
45
85
1/s
izes/
l
Copyright © 2014, Intellyx, LLC24
Assembling the Pieces of Agile Architecture
• Architecting for change requires working at higher levels of abstraction
• Abstract models resolvable to capabilities at run time– Dynamic coupling– Dynamic schemas– Extreme late binding
Architect for Change
Ph
oto
Cre
dit
: Pa
ulio
G4
eord
io h
ttp
s://
ww
w.fl
ickr
.com
/ph
oto
s/p
au
liog
eord
io/2
30
87
03
48
6/s
izes/
l
Copyright © 2014, Intellyx, LLC25
Dynamic Coupling
• Tight coupling– Must code all nodes at
once• Loose coupling
– Interfaces abstract underlying code
• Dynamic coupling– Interfaces resolved at run
time
Ph
oto
Cre
dit
: G
ord
on
Rob
ert
son
htt
ps:
//w
ww
.flic
kr.c
om
/ph
oto
s/g
ord
on
_rob
ert
son
/55
07
64
57
38
/siz
es/
l
Copyright © 2014, Intellyx, LLC26
Dynamic Schemas
• Rigid data schemas– Entity-Relational
Model, XML Schemas• Schemaless data
– Problem of implied schemas
• Dynamic schemas– Metadata modeled at
design time, resolved at run time
Ph
oto
Cre
dit
: Ju
han
Son
in h
ttp
s://
ww
w.fl
ickr
.com
/ph
oto
s/ju
han
son
in/2
31
71
85
23
4/s
izes/
l
Copyright © 2014, Intellyx, LLC27
Dynamic Schemas
• WSDL files, URIs, HTTP verbs, and Internet Media Types don’t contract message semantics
• Metadata describing such semantics scattered about– Reintroduces tight coupling &
inflexibility• Dynamic schemas abstract all
such semantic metadata in a consistent way
Ph
oto
Cre
dit
: n
eofito
htt
ps:
//w
ww
.flic
kr.c
om
/ph
oto
s/n
eofito
/40
37
43
75
04
/siz
es/
z/
Copyright © 2014, Intellyx, LLC28
How Late is your Binding?
• Early binding– Traditional integration
• Late binding– Web Service discovery– Content-based routing on
ESBs– DNS resolution for RESTful
endpoints• Extreme late binding
– Fully dynamic discovery– Resolve URIs & metadata
references in real time Ph
oto
Cre
dit
: R
alp
h R
ep
o h
ttp
s://
ww
w.fl
ickr
.com
/ph
oto
s/ra
lph
rep
o_p
hoto
log
/40
79
28
55
74
/siz
es/
l
Extreme Late Binding
• Metadata-driven, late-bound schemas
• Schemas (and all other metadata) bound in real time– Across the organization– Regardless of change
• Business entities maintain separate, dynamic business contexts
Copyright © 2014, Intellyx, LLC29
Ph
oto
Cre
dit
: g
eis
hab
oy5
00
htt
ps:
//w
ww
.flic
kr.c
om
/ph
oto
s/g
eis
hab
oy5
00
/29
15
15
59
98
/siz
es/
m/
Copyright © 2014, Intellyx, LLC30
Getting to Agile Architecture