buzzword deathmatch: agile vs soa

Alberto B

randolini IT ServicesA

lberto Brandolini IT Services

Buzzword Buzzword Deathmatch:Deathmatch:Agile vs SOAAgile vs SOA

formerly “Agile Development with SOA”

Alberto B



About meAbout me• 10 years experience in IT, mainly as a consultant• Took part in many large scale projects– Government(s)– Banking– Insurances

• A foot in the process, the other one in the architecture.

• My blog: http://ziobrando.blogspot.com

http://ziobrando.blogspot.com/

Alberto Brandolini

Finish

Alberto B



AgendaAgenda

• Starting from the dinosaurs…Starting from the dinosaurs…– The Agile LandscapeThe Agile Landscape– The SOA Landscape

Alberto B



The Agile RationaleThe Agile Rationale

• Waterfall software development proved itself inefficient and unsatisfactory– Waterfall is “traditionally” associated with • high cost, • long development time• Result unpredictability and low success ratio• Difficulties in managing and accommodating change

• If asked, nobody is doing waterfall anymore (…or so they think)

Alberto B



Unified ProcessUnified Process

• The unified process changed the scenario– IterationsIterations as a fundamental part of the process– Fine grained rolesroles and artefactartefact definition– UMLUML as the official representation language– A comprehensive definition of all development comprehensive definition of all development

process activitiesprocess activities• Unfortunately, it also set the ground for– A family of UML modelling tools– A lot of documentation templates

Alberto B



The Dark side of Unified ProcessThe Dark side of Unified Process

• The tools became more and more important, ultimately twisting the process

• Analysis and design turned into solo activities

• Paper, paper, paper, and more paper

Alberto B



DevelopersDevelopers

Developers were considered “interchangeable resources” whose only goal was to implement the specification, though iteratively.

Alberto B



Roles and responsibilitiesRoles and responsibilities

• The fine grained roles framework The fine grained roles framework eventually turned into a slow down eventually turned into a slow down factor:factor:– People took over only “the right tasks”People took over only “the right tasks”– (implicit waterfall)(implicit waterfall)– Slower responseSlower response– BottlenecksBottlenecks– Blame gameBlame game– ......

Alberto B



Rebel forces gatheredRebel forces gathered

Alberto Brandolini

colonna sonora

Alberto B



The three flavours of AgilityThe three flavours of Agility

• XP made a breakthrough focusing on extreme software development practices

• Scrum defined a framework in which Agile practices could be applied within an organization

• Lean software development introduced concepts and principles from the manufacturing industry into software development (defining also a theoretical background)

Alberto B



eXtreme ProgrammingeXtreme Programming• User Stories

– Less formal than Use cases, act like placeholder for a real discussion

• Frequent small releases– Iterations are shorter and targeted to production,

• Frequent planning and estimation– Each iteration is re-planned according to the currently available information

• No anticipated development– No frameworks or layered architecture

• Test first– Test suites run preserving application integrity, and producing better interfaces

• Customer availability– Real feedback is the key to make the right thing

• No code ownership• Continuous integration

– The whole system is frequently integrated and tested

• Pair programming– Programmers work in pairs, in coding sessions

• No overtime• ...

Alberto B



XPXP

• Feedback seems to be the driving factor for many of the proposed practices– From the code– From peers– From the customer– From the project– From the team

• The team is largely empowered and encouraged to propose original solutions

• Process itself might be modified or improved

Alberto B



ScrumScrum

• Scrum does not refer strictly to software development, but provides a framework for adaptive project management

• Unlike UP, only 3 roles are defined in Scrum

– The Team

– The Product Owner

– The Scrum Master

Alberto B



Development team landscapeDevelopment team landscape

Alberto B



The agile teamThe agile team

• Small units• Cross-functional• The team is free to take any design decision• In Scrum, the team reports only to the

product owner– A well defined ceremony and set of actions to

prevent the team to drift in dangerous directions

Alberto Brandolini

Does this apply to yeor team

Alberto Brandolini

Is this what vendors provide?

Alberto B



Dealing with several stakeholdersDealing with several stakeholders

The Product Owner is the ONE and ONLY responsible for the development team.

Alberto B



XP vs ScrumXP vs Scrum

XP• Frequent iterations of

working software• Frequent status update

and re-planning• Focus on the development

team internal organization and practices

Scrum• Frequent iterations of

working software• Frequent status update

and re-planning• Focus on the relations

between the development team and the organization

Alberto B



XP and ScrumXP and Scrum

• The two perspectives are largely complementary:– XP does not say much about what happens

outside of the development team– Scrum lets the team free to organize, and XP

might be a sensible choice

• Scrum has definitely a better marketing

Alberto B



Lean software development Lean software development principlesprinciples• Eliminate waste

– Everything that is not delivering any value to the user

• Amplify learning– Developing is discovery,

• Decide as late as possible– Avoid irreversible decisions or take them only when you have as much

information as needed to decide

• Deliver as fast as possible– Fast development cycles help feedback. Speed is better than quality

• Empower the team– The team is or will become the maximum expert on the matter

• Build integrity in– Software must be useful, used and right for the job

• See the whole– Failing to see the whole picture will lead to local optimizations

Alberto Brandolini

sistemare

Alberto B



WasteWaste• Instances of waste that can show up in different

shapes– Unnecessary documentation– Anticipated design– Over engineering– Waiting– Communication leaks– Defects– Handoff– Complexity– Blame– Quality (?)– ...

Alberto Brandolini

siostemare

Alberto B



The lo-fi approachThe lo-fi approach• As a consequence, some tools were

deprecated, favouring a non-tech approach:– Paper– Post-it– Whiteboards

• Information radiators took advantage of physical proximity to allow a more efficient exchange of meaningful information (where possible)

• Some tools were basically banned (es. MSProject), others entered the show (es. Wikis)

Alberto B



The bottom-up The bottom-up revolutionrevolution• Agile sets the ground for

interesting proposal to emerge from the team

• The team learns and focuses on a given problem space eventually turning into the best available expert on the matter

Alberto Brandolini

Ancora qualcosa?

Alberto B



Breeding collaborationBreeding collaboration

• Isolation happens seldom• Many activities are

performed in groups, or pairing, or in a clearly visible way

• Transparency enables trust and a more effective form of collaboration

• Information exchange happens in both ways

Alberto B



The Ideal Agile scenarioThe Ideal Agile scenario

• Not every condition is optimal to turn to agile: to fully benefit of agile’s potential we should (ideally) – Be employed by a company in whose top

priority is software development. – Be located in the same place– Have access to customers (whatever this means)– Be free to grow as a team and assume

responsibilities– Free to arrange logistics, hardware, etc.

Alberto B



AgendaAgenda

• Setting the BackgroundSetting the Background– The Agile Landscape– The SOA LandscapeThe SOA Landscape

Alberto B



SOA RationaleSOA Rationale• Large organizations were burdened with the

weight of– Different legacy systems– Mergers and acquisitions

• Necessity to integrate heterogeneous systems• Duplicate and redundant systems

– High (unnecessary) complexity• Operation costs• Evolution costs

– Extremely slow reaction times, or ...basically being stuck, or failing to deliver value

• ...Does all this sound familiar?

Alberto B



Pursuing uniformityPursuing uniformity

Standards, frameworks, rules and tight integration failed to deliver the needed business agility, resulting in a heavier burden to take into account, before even designing a possible solution to a given problem.

Alberto B



Service Oriented ArchitectureService Oriented Architecture

• SOA aims to dramatically reduce coupling between applications in a given landscape:– Language coupling XML– Protocol coupling WS, SOAP– Network coupling ESB– Availability coupling messages

• Standards and low coupling defined an enterprise-level architecture made up of replaceable componentsreplaceable components

Alberto B



Low couplingLow coupling

• Services are meant to talk to each other, with the lowest possible reciprocal knowledge En

terp

rise

Serv

ice

Bus

Alberto B



The ideal goalThe ideal goal

• SOA was a tool to– Allow the long awaited necessary cleanup– Allow IT to become a driving factor for the

business instead of a burden– “It’s a mess”– “I’ll schedule it for 2010”– “We have no time right now”– “We can do that”– “Why not doing that instead?”– “...We have an idea”

– Allow enterprises to reduce vendor lock-in recovering control on IT budget.

Alberto B



Put in another wayPut in another way

• SOA is a tool to allow large enterprises to achieve business agility– Shorter products release cycles– Getting feedback straight from the market– Experimenting new ways to make business

• SOA is not a way to recreate an existing structure with a newer technology

Alberto Brandolini

Qui ci pouò stare l'esempio di Amazon

Alberto B



The “classical” SOA scenarioThe “classical” SOA scenario

The agile greenfield• Be employed by a company

in whose top priority is software development.

• Be located in the same place• Have access to customers

(whatever this means)• Be free to grow as a team

and assume responsibilities• Free to arrange logistics,

hardware, etc.

The “classical” SOA• Company’s top priority is

generally not software development

• Consultants, contractors (and sub-contractors) are the rule

• Development takes often place in multiple locations, remote and offshore teams are possible.

• Smaller incentives to grow and assume responsibilities

• Low control over logistics, hardware, etc.

Alberto B



Unleashing freedomUnleashing freedom

• Loose coupling and language neutral standards offered the possibility to develop applications in the most appropriate technology

Enterprise Service Bus

Alberto B



... Maybe we’re still coupled...?... Maybe we’re still coupled...?

• Technical coupling was only one of the many coupling factor on the stage:– Licence budget– Operations & Support– Standards, Rules and existing prescriptions– HR strategies– Architecture– Corporate culture

Alberto B



A new JargonA new Jargon

• UML was not enough for SOA needs• A new Jargon eventually emerged as well as

new notations, new diagrams, new stencils, new layers ...

Alberto Brandolini

Somethiong to finish

Alberto B



AgendaAgenda

• Setting the Background– The Agile Landscape– The SOA Landscape

• Putting things togetherPutting things together

Alberto Brandolini

Mmmm togliere

Alberto B



Pairing Agile and SOAPairing Agile and SOA

“Enterprises experience more success with SOA when they eschew big top-down delivery projects and instead get down in the trenches with an evolutionary approach. Agile processes provide a basic structure for this kind of incremental delivery.”

-Carey Schwaber, Forrester Research

Interestingly, not so many articles tell how Agility benefits from SOA.

Alberto B



Agile as a Cooperative GameAgile as a Cooperative Game

• Software development might be classified as a finite, goal-directed, cooperative game (Cockburn)

Finite,Finite,Non-goal-directedNon-goal-directed

Finite,Finite,goal-directedgoal-directed

Carpet wrestling

InfiniteInfinite

Jazz

Tennis Software Software DevelopmentDevelopment

Dolls

Competitive Cooperative

Organization Survival

Career management

Alberto B



Software development as a GameSoftware development as a Game

• Finite: a project starts and ends (somehow) • Goal directed: es. deliver on time• Collaborative: we’re playing together within

a team

• … but we’re not only doing that:– We’re playing career game– Playing family game– Etc. etc.

Alberto B



A successful teamA successful team

Alberto B



Some key issuesSome key issues

• The Team– Best of breed selection– Team goals before individual ones– High motivation

• The goal– Clearly defined goal– Time-framed experience– Measurable outcome

Alberto B



A “not so successful” teamA “not so successful” team

Alberto B



Key issuesKey issues

• The team– Best of breed?– Individual goals before common goals

• The Goal– Not so clearly defined– Random time-frame– Non measurable outcome (only history will tell)

Alberto B



Limited resources gameLimited resources game

• A Software Development scenario is bounded on several key areas:– Budget– Time– Expertise– Talent– Manageable Information

Alberto B



SOA game?SOA game?

• SOA is generally a different type of game played at a different level:– SOA’s goal might be of a longer term strategy– Mid term results might not be easily observable

and measurable by the team.• Es. Budget• Suppression of an external system

Alberto B



The SOA GameThe SOA Game

A common critic to Agile methods is that they focus on a short-term strategy, in a given project scope.

SOA aims to see “the whole picture” focusing on somewhat obscure long-term goals

Alberto B



The dominant cultureThe dominant culture

• Enterprise culture might be far from agility• SOA might have management’s sponsorship

but agile might not.• Careers, roles and specialities might be put

under pressure

Alberto B



Project SizeProject Size

• SOA scope calls for many development teams at a time.

Alberto B



A more realistic scenarioA more realistic scenario

Alberto B



Team’s rewardTeam’s reward

• Individual bonuses might be counterproductive or out of control– Consultants and contractors might not be

reached• We’ve got to work on something else– Increase the team’s knowledge– Improve working conditions– Bring (honest) positive feedback– ...

Alberto B



Top – Down repriseTop – Down reprise

• Some tools and artifacts re-introduce a top-down philosophy– Role based game – Tool driven design– Specification based development

• Another vicious effect is that ...– Bottom up feedback is discouraged– Possible good ideas get lost

Alberto B



Starting Agile and SOA?Starting Agile and SOA?

• Though “orthogonal”, two revolutions at the same time are probably too much for any team

• But ...agile performs well where a lot of explorations and uncertainty is involved:– Adaptive process– spikes

• For a good Agile team SOA is “Just another Technical Hassle”

Alberto B



Read the scenarioRead the scenario

• Different roles within the organization read buzzwords differently:– Team might value agility more • (and blame SOA if something goes wrong)

– Architects might value SOA more• (and blame agility if something goes wrong)

– Management couldn’t care less• And value only results

Alberto B



Set up the sceneSet up the scene

• Don’t raise expectations you cannot fulfil• Keep management aware of potential

emerging problems– Transitions to agile practices (especially Scrum

and lean) stress the organization, exposing problems that lived long “under the carpet”.

– Problems have always being there, but pointing them out might result impolite.

Alberto B



Choose a close target firstChoose a close target first

• Can’t wait months to prove that choices were good.– Look to close targets– Achieve them– Build on this• Confidence• Team jellying• Reputation

Alberto B



Travel lightTravel light

• Availability of tools to manage SOA-related complexity does not mean that complexity has to be encouraged

• The dark side of SOA would not be better than it was before

Alberto B



Adhere to standardsAdhere to standards

• As a corollary to “Decide as late as possible”:– Develop on standards whenever possible• Generally better documentation• More easily replaceable• ... Think long term

– Avoid temptations from vendor-specific features– Keep deviation from standards under control– Don’t buy anything until proves necessary.

Alberto B



Rethink wasteRethink waste

• Some obvious form of waste “doing the same thing twice” might be preferable to “document what you’ve been doing”

• Large scale changes priority• Company and location boundaries impact– Communication cost– Handoff cost

Alberto B



The cost of communicationThe cost of communication

• Information does not propagate with documentation, but by knowing what others are doing.– Keep multiple teams in sync• Scrum of Scrums• Information radiators• Proximity• End of iteration demo

• When written words are better, favour Wikis and on-demand documentation.

Alberto B



Share the big planShare the big plan

• Preventing developers to see the whole picture severely harms the possibility of a bottom-up contribution– Only local optimizations are possible

Alberto Brandolini

Alberto B



High level planningHigh level planning• SOA is a long term transformation process• Every step changes the scenario– More knowledge– Different weight– Different opportunities

• Frequent high-level re-planning is necessary to achieve the goals (not necessarily the original ones) – Measure, don’t expect– Assess risks, don’t make predictions– Don’t initiate anything that’s not needed now

Alberto Brandolini

fioniore

Alberto B



Testing a SOATesting a SOA

• SOA overall design is based on Design by Contract principles– Clean interface definition based upon standards– “Official” expected service behaviour

• Let’s test on the contract!

Alberto B



Testing a SOA: challengesTesting a SOA: challenges

• Language independent test suite • Mocks and Stubs implementing services • Selection of key areas • Test environment management

Alberto B



Environment definitionEnvironment definition

• Staging environment might be hard to afford in certain context– Keep continuous integration tools running and

testing the app– Extend your integration capabilities as far as you

can get– Find a balance between correctness and

manageability– Keep the tests updated

Alberto B



Put in two words...Put in two words...

• Don’t let the “old bad habits” strike back• Be prepared to compromises, in the short

term (it’s a limited resource game)– Keep track the price you pay– Be ready to change them if opportunities arise– Always aim to your long term goals

• Involve the right sponsors• Communicate!• Be ready for a rollercoaster ride!

Alberto B



ReferencesReferences

• Agile Software Development – Alistair Cockburn (Addison Wesley)

• Extreme Programming Explained – Kent Beck (Addison Wesley)

• The Unified Software Development Process – Ivar Jacobson, Grady Booch, James Rumbaugh (Addison Wesley)

• Agile Modeling – Scott Ambler (Wiley)

Alberto B



ReferencesReferences• Lean Software Development: An Agile Toolkit

– Mary Poppendieck, Tom Poppendieck (Addison Wesley)

• User Stories Applied – Mike Cohn (Addison Wesley)

• Enterprise SOA: Service-Oriented Architecture Best Practices – Dirk Krafzig, Karl Banke, Dirk Slama (Prentice Hall)

• Service-Oriented Architecture: A Field Guide to Integrating XML and Web Services – Thomas Erl (Prentice Hall)

buzzword deathmatch: agile vs soa

Technology