Applying Agile Values to Enterprise Architecture
Software Architectural Trends
Past, Present, and Onwards
by: Benjamin Scherrey
Chief Systems ArchitectProteus Technologies, Co. Ltd.
Twitter: @proteusguyEmail: [email protected]://proteus-tech.com
Architecture
Architecture is Form not Structure
Keeps the effort focused on Value
Identifies the Stable Parts of the System
Introduces Constraints (Architectural Drivers)
The result of tough decisions made now in order to make future decisions easier
Debate: Does Form follow Function (Behaviour) or does Function follow Form?Answer: Form follows failure. Architectural styles evolve after years of trial and error.What do we mean by form then?Jim Coplien offers the example of a bottle plant. The bottle is the structure. The form (mold) could be injected with a variety of materials and still provide it's value. The FORM is the bottle's ESSENSE. The space created gives it's use. Pleasing asthetics makes us want to use it. This is a BIG DEAL!
Value = Crossing the Chasm
Stability != StaticCodify Business processes so we don't have to worry about them.
Embodies the critical design decisions that typify a system.Constraints paradoxically give us freedom!
Significance of design decisions need to be understood and assessed.
Goals of Architecture
Reduce Cost of Change
Reduce Coupling/Increase Cohesion
Support Locality of Decision Making
Shared Vision Upon Which Whole Team Can Proceed
Reduces the significance of decisions because the hard decisions have been made and make the easy ones easier.Making easy decisions first constrain your options and make hard decisions harder.
Larry ConstantineCoupling is the degree to which each program module relies on each one of the other modules.Cohesion is a measure of how strongly-related or focused the responsibilities of a single module are.In a highly-cohesive system, code readability and the likelihood of reuse is increased, while complexity is kept manageable.
Loosely coupled systems allow for decomposition based on Time (when we need to know/decide) rather than strictly based on Code Structure.
Enterprise Architecture
Pitfalls
Bigger Budgets, Lowered Expectations
Throw It Over the Wall Design
Beautiful Works of Fiction
People confuse the rules of drawing with the rules of construction not realizing that they are not the same. -- Leon Battista Alberti Italian Renaissance Architect (1404-1472)
Never the intention (except perhaps for Vendors!) but often the result. Attaching the word Global to it almost guarantees it will never be delivered.
Distributed development means you can't silo your architecture decision making yet that's what most companies do with their Ivory Tower Architects.If your architects aren't producing code to back up their designs then you have a problem.Coplien's version of Conway's Law - "If the parts of an organization (e.g. teams, departments, or subdivisions) do not closely reflect the essential parts of the product, or if the relationship between organizations do not reflect the relationships between product parts, then the project will be in trouble. ... Therefore: Make sure the organization is compatible with the product architecture."
Blueprints document the FINAL decisions once completed.Not only Customers but also Designers get enthralled by the beautify of large UML diagrams so that they forget the Business Justifications. This is not a new problem.
Heavy documentation actually eliminates the ability to determine the significance of a decision.
Enterprise Architecture
Pitfalls
Magazine Architecture
Don't forget about the people who will besubjected to your system!
Magazine Architecture is pretty pictures of empty rooms before you put the people into them.
Keep it REAL.
Make it sustainable!
How Much Architecture?
Big Upfront Designvs.No Upfront Designvs.Rough Upfront Design
BUFD Classic Waterfall FAILDesign Movement from 1980s. ISO-9001CASE Tools Knowledgeware's ADW/ TI's IEFProgrammers would become obsolete once this code generation stuff started working.Couldn't deal with EMERGENT requirements.Presumed to know things we can't possibly have known therefore doomed to fail to deliver VALUE to stakeholders from the beginning.
NUFD suggested by early XP/TDD proponents. Basically a myth. Having 10-20 years domain knowledge helps but what if it's the first time you're doing this?
RUFD Rough sketch of map. Point out land marks and things to watch out for.Explore options.Result must be working code.
Agile Values
"Programs must be written for people to read, and only
incidentally for machines to execute."
Hal Abelson
(Structure and Interpretation of Computer Programs, Second
Edition)
"Having the faculty of quick motion. Nimble. Active. Readiness." Oxford English Dictionary
An Architecture makes you Ready. James Coplien
Software Development is a Human Endeavor
Agile as a process recognizes and aligns itself with this realization better than any other.
So what is the CORRECT definition of Agile?
Agile is an adjective not a NOUN or VERB.
To be Agile you must first be strong.
How do we achieve readiness?
Agile Architecture
An Agile ModelThe Agile Manifesto Utah 2001
Individuals and interactions over processes and toolsWorking software over comprehensive documentationCustomer collaboration over contract negotiationResponding to change over following a plan
Agile is a Direction, not a Destination. There is more than one path!
Agile Architectural Values
Mayo-Smith Pyramid
Beware: Grandiose Designs Incur Grandiose RisksScope fixed, budget & schedule may suffer.
Focus: Immediate, practical functionality maximizes value of each increment while reducing risk. Scope flexible, budget, schedule under control.
Focus is on ALWAYS delivering VALUE to CUSTOMER.
To deliver value we must manage risk!
Classic Waterfall / Big Upfront Design
If you fix scope then budget & schedule become variables (and they never go down).
Pharoh StoryPharoh got two levels down and died... huge waste.
Deliver value early & often.
Fix your budget & schedule then focus on delivering incremental value as early as possible.
LEAN vs AGILE (OPTIONAL)
For our purposes they are equivalent. Difference is priority of focus when executing.
Initial reaction of an Agile team when encountering an obstable is to go around it.
Initial reaction of a Lean team when encountering an obstable is to remove it.
Agile Architecture How-to
Agile is a reaction against methodological excesses"Plans are useless, but planning is everything." Dwight D. Eisenhower
Put a Stick in the Ground
Design Lives in Code
Fight methodolofical excess. Rejects Ivory Tower Architects & Silo'd teams.Focus on DOING.Piecemeal growth, local adaption.
Sometimes reactions can go too far. Don't abandon planning when you abandon over detailed plans.
How to get started doing?It's OK to know stuff. Leverage local domain knowledge.Document your assumptions & be prepared to change them as you learn new things.Find the stable parts of the system (typically data)
Don't build big documents.Build shared (executable!) vision.Make docs that summarize what the code provides details for.Write an email don't call it a document!
Agile Architecture How-to
SCRUM (An actual discipline!)Everybody all together from the beginning.
Test Driven Development & Continuous IntegrationYou can't implement what you can't test.
SCRUM (Jeff Sutherland 1993) inspired by LEAN and the Toyota Way. High Innovation, High parallelism. Working on the edge of chaos.Time boxing, incremental & iterative development + standup meetings from Bell LabsDon't work overtime.Can't fit arbitrary work to an arbitrary schedule.
Small Cross functional teams - no silosWe will deliver what we can to an agreed upon level of quality.
Constant design. Everybody's an architect.Automation and frequent execution & feedback critical.
How else can you actually measure what value is being delivered or that you've met your agreed-upon quality levels?No more optional than double entry ledgers for accountants or clean room procedure for surgeons.
SPIKEsIdentify with certainty those things that are uncertain.As they are also User Stories and prioritized together there is local control over when this value/risk must be identified. Not all risk is equal. Managing risk mitigation is essential.
Iteration ReviewsFrequent reviews and feedback loops gets the project on track and also revaluates how well the process is working for us in the first place.
Architecture without the Architect
No Software Architects in Asia (Few Anywhere)
Think Like an Architect (identify value stream/fit into ecosystem)
Send out Scouting Parties (Fail Fast)
Hire a Guide (proper use of consultants)
(SKIP IF LESS THAN 15 MINUTES REMAINING)Takes 10-15 years of solid experience in a variety of technologies to become an architect.Zero Technical Career Paths == Lots of incredibly smart people with little experience and little exposure managing 20 other people one chapter behind them. This is an issue that will take 10-20 years to fix if we start today. No movement towards that however.Have to import them until then. Demand is insanely high. (Auto mechanic anecdote)
ADD MORE HERE
21st Century Architecture Challenges
FAT Object Interfaces (Enterprisey)
Brittle Distributed Dependencies (SOAP)
How to Test for Value?
General Loss of Cohesion
OO has been an unqualified success in terms of improving our ability to model and implement systems.Unfortunately the most successful objects in our systems get used by many parts of their system so their interfaces get fat & complex, especially Enterprise objects like Javabeans.
Distributed systems have really helped break up some of the issues of monolithic architectures and generally increased cohesion. Coupling, however, has not been significantly reduced and, in fact, can be increased because you're tightly coupled with a system that you don't have control over. Remote objects exposed as SOAP services suffer much of the same brittleness issues as CORBA did in the 90s. RESTful systems help this and it's nice to see Microsoft getting on board now and dropping SOAP.
Original proponents of TDD suggested it as an alternative to ANY upfront design. The concept of YAGNI and refactoring using a 100% discovered approach replaced thinking and designing as an activity. This may be fine for systems where the entire business value model and entities can fit in your head but fails for anything larger than that. Not saying that anyone should be coding LARGE systems. The Dependency Horizon of anything a team is coding on should be quite visible and small. But the integration of these small systems to compose LARGE systems absolutely requires an architecture that clearly identify VALUE STREAMS and how each part fits into the ECOSYSTEM.To wit TESTs should demonstrate value delivered so where and how we test is just as important as the fact that we do test. This is hard to identify with current architecture & design processes.
I just said distributed systems improved cohesion, right? Yes but abandoning procedure & structured coding practices has also made it difficult to identify the actual ALGORITHMs we are implementing. They are broken up and strewn across multiple classes & object methods so as to be unidenfifiable. Functional languages (for all their advantages and promising applicability to parallel systems) swing the pendulum too far by prohibiting any side effects and most stateful coding. There is a clear loss of cohesion for any processes that must cross multiple class boundaries.
New Architecture on the Horizon
Failures of the Object Oriented ModelProgramming in Classes rather than Objects
Lost our Use Cases
Responsibility Driven Design
Design & Architecture hasn't stopped. There are serious issues to address and a promising new option to consider.
Object Oriented Revolution of late-80s & early 90's originated in late 60s with Simula 67 and fully realized with SmallTalk.Objects should be independent, anthropomorphic entities acting as self-aware computers.GUIs exposes objects to Users MVC (Trygve Reenskaug 1978) Unifies programmer's vision of program with data model of program.Languages expressed objects solely as classes good at modeling data and fairly static.Unfortunately classes are poor at modeling behaviour Users vision of the program. Behaviour often does not fall along class/data lines.
Computers hold data in classes. Intelligence is in the Use Case.Ivar Jacobson 1986Use Cases best model behaviour using Actors or Roles (which we interpreted as Class Objects) but we kinda lost touch with this fact because even class oriented OO was paying big dividends.
My OO Experience
(Responsibility Driven Design 1989) - Rebecca Wirfs-Brock
Domain Context Interaction
Trygve Reenskaug & James Coplien
What Object Oriented Intended
What the System Isvs.What the System Does
Orthogonal to MVC
So Trygve meets Rebecca on a boat in Norway and figures out that classes don't have responsibilities, Roles do! Coplien meets him and helps take it forward as a discipline.
OO is supposed to engage the end user & capture their mental model. Got it half right (data not behaviour). ROD was an attempt to fix this.Focus on class based got us off track.
Classes model data and structure What the system is the Domain Objects. Mental model of how the system works on the inside. What about usage behaviours?
Behaviours set the Context for how the Domain objects are Used. Described in Use Cases.A use case should:Describe what the system shall do for the actor to achieve a particular goal.Include no implementation-specific language.Be at the appropriate level of detail.Not include detail regarding user interfaces and screens. This is done in user-interface design, which references the use case and its business rules.
Roles/Interfaces come from UXHelping programmer separate Use Case part of code from Data model.MVC is matching Objects from bottom with Roles from TopSo if Classes aren't properlly Roles where do we get those?
Domain Context Interaction
Roles(vs. Classes)
Proc TransferMoney( SrcAccount, DestAccount )Class SavingsAccountClass Checking Account
StatelessImplement Behaviour of Algorithm
Taxonomy: Data (Class) + Algorithm (Role) -> Object
Are Injected & Removed from Classes at RUNTIME
A Role is simply a name or label
Source Account & Destination Account in the Context of a TransferMoney Transaction.What are the class types for the Accounts?
Who here has a Source Account?
Classes talk about Objects in terms of Form/Structure.Roles talk about Objects in terms of Behaviour (how it's used).
Role is the name of an object according to its Use.
If I have a chair can I call it a hat?A Class says No. A Role says Yes so long as it supports the necessary interface.
I have one set of DNA that identifies me.I'm a man, son, father, programmer, speaker. How many objects can I be? These are roles not objects! (All depend on CONTEXT)Can't use parts of me during those operations, it's still just me.Think of Class Ben with all those methods for all those roles! Remember our FAT interfaces and poor COHESION? Here it is!
Roles are Stateless
Roles implement the behaviour of the Algorithm
Taxonomy: Data (Class) + Algorithm (Role) -> Object
Roles are injected into Classe instances at RUNTIME in order to create a fully formed Object.
Like Neo downloading Martial Arts or Helicopter Pilot skills in the Matrix.
Domain Context Interaction
Contexts
Implement the Use CasesCast the RolesInstantiate the ObjectsExecute the Use CasesRemove the Roles from Objects
Let's talk more about Contexts
Contexts are where our Use Cases live.
Besides implementing Use Case, Contexts also perform casting of roles.
Know how to map classes to roles (building objects) or instantiate new objects to play those roles.Roles are defined within the context.
Algorithms are procedures in the Context utilizing local Roles and their methods. COHESION is increasing again!
when the Context has finished, it removes the Role from that Object instance.
Domain Context Interaction
Behaviour gets First Class status
Separates rapidly changing Behaviour from slowly changing Domain Knowledge
Help developers reason about system-level state and behavior
Explicitly supports a style of thinking that is closer to People's Mental Models
The End.
To improve the readability ofobject-orientedcode by giving system behavior first-class status;
To cleanly separate code for rapidly changing system behavior (what the systemdoes) from code for slowly changing domain knowledge (what the systemis), instead of combining both in one class interface;
To help software developers reason about system-level state and behavior instead of only object state and behavior;
To support an object style of thinking that is close to peoples' mental models, rather than the class style of thinking that overshadowed object thinking early in the history of object-oriented programming languages.
The End
References
Essential (Abstract) Use CasesScott Amblerhttp://www.agilemodeling.com/artifacts/essentialUseCase.htm
Coupling & Cohesionhttp://en.wikipedia.org/wiki/Coupling_(computer_science)
The DCI Architecture: A New Vision of Object-Oriented Programmingby Trygve Reenskaug and James O. Coplienhttp://www.artima.com/articles/dci_vision.html
Why DCI is the Right Architecture for Right NowJim Coplienhttp://www.infoq.com/interviews/coplien-dci-architecture
To improve the readability ofobject-orientedcode by giving system behavior first-class status;
To cleanly separate code for rapidly changing system behavior (what the systemdoes) from code for slowly changing domain knowledge (what the systemis), instead of combining both in one class interface;
To help software developers reason about system-level state and behavior instead of only object state and behavior;
To support an object style of thinking that is close to peoples' mental models, rather than the class style of thinking that overshadowed object thinking early in the history of object-oriented programming languages.
The End
