How to break apart a monolithic system safely

without destroying your teamMatthew Skelton, Skelton Thatcher Consulting

@matthewpskelton

LondonCD meetup: 11 Oct 2016#londoncd

TodayCognitive load for teams

‘Monolith’Code Forensics

Team-first boundariesMonolith-splitting recipe

For now, let’s forget:Microservices

CQRS / Event Sourcing

Queues(Architectural changes)

Continuous Delivery / etc…30+ organisations

UK, US, EU, India, China

How to break apart a monolithic system safely without destroying your

team

Safer, more rapid changes to software systems

(Business Agility)

A ‘team-first’ approach to software subsystem

boundaries

TEAM

TEAMcapabilities

appetite & aptitude

understanding

responsibilities

Conway’s Law

‘Reverse Conway’Tobbe Gyllebring (@drunkcod)

homomorphic force

(#Conway #Yawnoc)

HT @allankellynet

(same) (shape)

Cognitive load for teams

We have SCIENCE!

Cognitive load for teams• Driskell et al, 1999 ‘Does Stress Lead to a Loss of Team Perspective?’

Group Dynamics: Theory, Research, and Practice 3, no. 4 (1999): 291.• Fan et al, 2010 ‘Learning HMM-Based Cognitive Load Models for

Supporting Human-Agent Teamwork’. Cognitive Systems Research 11, no. 1 (2010): 108–119.

• Ilgen & Hollenbeck, 1993 ‘Effective Team Performance under Stress and Normal Conditions: An Experimental Paradigm, Theory and Data for Studying Team Decision Making in Hierarchical Teams with Distributed Expertise’. DTIC Document, 1993.

• Johnston et al, 2002 ‘Application of Cognitive Load Theory to Developing a Measure of Team Decision Efficiency’. DTIC Document, 2002.

“stress impacts teamperformance … by narrowing or weakening the team-level

perspective required foreffective team behavior.”

– Driskell et al, 1999Group Dynamics: Theory, Research, and Practice 1999, Vol. 3, No. 4,291-302

‘Monolith’

“Don’t start with a monolithwhen your goal is a

microservices architecture”

– Stefan Tilkov, innoQ

http://martinfowler.com/articles/dont-start-monolith.html

“Start monolithic and extract”

– Tammer Saleh, Pivotal

https://www.infoq.com/presentations/cloud-anti-patterns

A ‘team-first’ approach to software subsystem

boundaries

Types of software monoliths•Application monolith•Joined at the DB•Monolithic releases (coupled)•Monolithic thinking (standardisation)…

Application monolith

Single block of codeDeployed as a unit

Joined at the DB

Difficult to change separately (but not impossible)Risk is (probably) elevated

Chris Collyer, http://www.stone-circles.org.uk/stone/pentreifan.htm

Monolithic releases

Smaller components bundled together into a ‘release’

Monolithic thinking

‘One-size-fits-all’ for teamsAssumption that minimising variation is A Good Thing

Dangers of splitting a monolith•Reduced domain consistency (UX, Architecture, Data)

•Data duplication (unintentional)•Additional operational complexity due to distributed system and async messaging

•Degraded UX across the product

Splitting a monolith

Reiner Flassig - CC BY-SA 2.0 de - Wikipedia

Choose the right technique for splitting Understand the nature of the monolith

‘Fracture planes’ for code•Business domain bounded context•Regulatory compliance•Change cadence•Risk•Performance isolation•User personas•Team location

Code Forensics

ForensicsYour Code as a Crime SceneAdam Tornhill

Adam TornhillCode, Crime, Complexity: Analyzing software with forensic psychologyAdam TornhillTEDxTrondheim

youtube.com/watch?v=qJ_hplxTYJw

‘Code Maat’ tool

Adam Tornhill, http://www.adamtornhill.com/articles/crimescene/codeascrimescene.htm

Code City plus Code Maat forensics

Beware of badly-named subsystems

"information-poor abstract names are magnets for extra [unwanted] responsibilities"

– Adam Tornhillp.185, Your Code as a Crime Scene

Team-first boundaries

DevOpsTopologies.com

Team types

Component team

Platform / ’substrate’ team

Supporting / ‘productivity’ team

Product/Feature team

Team typesde

vops

topo

logies

.com

Component team

Platform / ’substrate’ team

Supporting / ‘productivity’ team

Product/Feature team

Code repositoriesAlign repositories to subsystem boundaries

Avoid monolithic-y repos like TFS** Don’t get me started on TFS, grrr…

Code repositories

Repo 1 Build Test Deploy Run

Repo 2 Build Test Deploy Run

Repo 3 Build Test Deploy Run

“You can use a monorepo only if your organisation has published a

scientific paper on Computer Science. Otherwise, use one repo

per separate deployable thing.”

– Matthew SkeltonLondonCD meetup group, 11 Oct 2016

Find natural or available ‘fracture planes’ for splitting

a monolith

(with the team in mind)

Monolith-splitting recipeTried and tested!

How to break apart a monolith without destroying your team

1. Instrument the monolith – logging 2. Grok data flows and fault responses3. Align teams to available segments

4. Split off segments one-by-one

Instrument the monolith

Instrument the monolith

Instrument the monolith

Instrument the monolithCorrelation ID Logs

Event ID

use logging as a channel/vector to

make distributed systems more testable

use logging as a channel/vector to

make distributed systems more testable

Grok data flows and fault responses

Grok data flows and fault responsesCorrelation ID

Event ID

Unexpected collaborating subsystems

Undetected fault condition

Grok data flows and fault responsesCorrelation ID

Event ID

Adjust subsystem boundaries

Fix poor fault responses

runbooktemplate.info

Run Book dialogue sheets

Align teams to available segments

Align teams to available segments

Align teams to available segmentsMap to business domain

Align teams to available segmentsIdentify likely components or ‘platform’ elements

Split off segments one-by-one

Split off segments one-by-one

Split off segments one-by-oneSeparate:- Builds- Infrastructure- Deployments- Versions- Lifecycle

Team needs / responsibilities / capabilities come first

How to break apart a monolith without destroying your team

1. Instrument the monolith – logging 2. Grok data flows and fault responses3. Align teams to available segments

4. Split off segments one-by-one

How to break apart a monolith without destroying your team

1. Instrument the monolith – logging 2. Grok data flows and fault responses3. Align teams to available segments

4. Split off segments one-by-one* plus a few other things

Further material

Books & articles• Working Effectively with Legacy Code, by Michael Feathers• Building Microservices by Sam Newman• ‘Managing Cognitive Load for Team Learning’ by Jo Pearce

http://12devsofxmas.co.uk/2015/12/day-3-managing-cognitive-load-for-team-learning/

Training• From Monolith to Microservices (online training) – Sam Newman,

author of Building Microserviceshttp://www.oreilly.com/live-training/from-monolith-to-microservices.html

Talks & slides• What is cognitive load theory and why should you care?, by Jo

Pearce - http://www.slideshare.net/JoPearce5/what-is-cognitive-load-theory-5mins

• Building Microservices by Sam Newman

Research papers• Driskell, James E., Eduardo Salas, and Joan Johnston. ‘Does Stress Lead to a Loss of Team

Perspective?’ Group Dynamics: Theory, Research, and Practice 3, no. 4 (1999): 291.• Fan, Xiaocong, Po-Chun Chen, and John Yen. ‘Learning HMM-Based Cognitive Load Models

for Supporting Human-Agent Teamwork’. Cognitive Systems Research 11, no. 1 (2010): 108–119.

• Ilgen, Daniel R., and John R. Hollenbeck. ‘Effective Team Performance under Stress and Normal Conditions: An Experimental Paradigm, Theory and Data for Studying Team Decision Making in Hierarchical Teams with Distributed Expertise’. DTIC Document, 1993. http://oai.dtic.mil/oai/oai?verb=getRecord&metadataPrefix=html&identifier=ADA284683.

• Johnston, Joan H., Stephen M. Fiore, Carol Paris, and C. A. Smith. ‘Application of Cognitive Load Theory to Developing a Measure of Team Decision Efficiency’. DTIC Document, 2002. http://oai.dtic.mil/oai/oai?verb=getRecord&metadataPrefix=html&identifier=ADA525820.

thank you

Matthew Skelton@matthewpskelton

skeltonthatcher.com