from monoliths to microservices at realestate.com.au
TRANSCRIPT
From monoliths to microservices
@evanbottcher@bethesque @joneaves
Why?
Why? What?
Why? What? How?
Residential Commercial Media & Developer International
Group
Global A
rchitecture
Property
The Money
People & Experiences
Why?
1. Long release cycle
1. Long release cycle
2. Coupled releases
1. Long release cycle
2. Coupled releases
3. Long defect fix time
Teams at REA are;
self-empowered
Teams at REA are;
self-empowered
value independence
Teams at REA are;
self-empowered
value independence
trusted
Teams at REA are;
self-empowered
value independence
trusted
Vision
rapid deployment
rapid deployment fault tolerant
rapid deployment fault tolerant discoverable
rapid deployment fault tolerant discoverable independent
rapid deployment fault tolerant discoverable independent
?Microservices
1. Narrow responsibility
Microservices
1. Narrow responsibility
2. Small codebase
Microservices
Microservices
1. Narrow responsibility
2. Small codebase
3. There is no #3
Starting the journey
What?
Residential Commercial Media & Developer International
Group
Group
(source: flickr)
(source: flickr)
CRM ERP
Customer Relationship Management
Enterprise Resource Planning
CRM ERP
CRM ERP
Billing Payment
CRM ERP
Billing Payment
Billing
Contract
Sales automation
Product rules
Finance
Pricing
Business Systems
Line of Business
Line of Business
Line of Business
Customers
Line of Business
Line of Business
Line of Business
Customers
Ouch!
Billing
Contract
Sales automation
Product rules
Finance
Pricing
Billing
Sales automation
Product rules
Finance
Pricing
Contract
Billing
Product rules
Finance
Pricing
Contract
Billing
Product rules
Finance
Pricing
Contract
New Apps
New Apps
Billing
Product rules
Finance
Pricing
Contract
New Apps
New Apps
New team boundaries?
(source: wikimedia)
Strategic Goals Architectural
Principles Design and Delivery
Practices
Enable scalable business More customers/transactions Self-service for customers
Support entry into new markets Flexible operational processes New products and operational processes
Support innovation in existing markets Flexible operational processes New products and operational processes
Reduce inertia Make choices that favour rapid feedback and change, with reduced dependencies across teams.
Eliminate accidental complexity Aggressively retire and replace unnecessarily complex processes, systems, and integrations so that we can focus on the essential complexity.
Consistent interfaces and data flows Eliminate duplication of data and create clear systems of record, with consistent integration interfaces.
No silver bullets Off the shelf solutions deliver early value but create inertia and complexity.
Standard REST/HTTP
Encapsulate legacy
Eliminate integration databases
Consolidate and cleanse data
Published integration model
Small independent Services
Continuous deployment
Minimal customisation of COTS/SAAS
Strategic Goals Architectural
Principles Design and Delivery
Practices
Enable scalable business More customers/transactions Self-service for customers
Support entry into new markets Flexible operational processes New products and operational processes
Support innovation in existing markets Flexible operational processes New products and operational processes
Reduce inertia Make choices that favour rapid feedback and change, with reduced dependencies across teams.
Eliminate accidental complexity Aggressively retire and replace unnecessarily complex processes, systems, and integrations so that we can focus on the essential complexity.
Consistent interfaces and data flows Eliminate duplication of data and create clear systems of record, with consistent integration interfaces.
No silver bullets Off the shelf solutions deliver early value but create inertia and complexity.
Standard REST/HTTP
Encapsulate legacy
Eliminate integration databases
Consolidate and cleanse data
Published integration model
Small independent Services
Continuous deployment
Minimal customisation of COTS/SAAS
Strategic Goals Architectural
Principles Design and Delivery
Practices
Enable scalable business More customers/transactions Self-service for customers
Support entry into new markets Flexible operational processes New products and operational processes
Support innovation in existing markets Flexible operational processes New products and operational processes
Reduce inertia Make choices that favour rapid feedback and change, with reduced dependencies across teams.
Eliminate accidental complexity Aggressively retire and replace unnecessarily complex processes, systems, and integrations so that we can focus on the essential complexity.
Consistent interfaces and data flows Eliminate duplication of data and create clear systems of record, with consistent integration interfaces.
No silver bullets Off the shelf solutions deliver early value but create inertia and complexity.
Standard REST/HTTP
Encapsulate legacy
Eliminate integration databases
Consolidate and cleanse data
Published integration model
Small independent Services
Continuous deployment
Minimal customisation of COTS/SAAS
Strategic Goals Architectural
Principles Design and Delivery
Practices
Enable scalable business More customers/transactions Self-service for customers
Support entry into new markets Flexible operational processes New products and operational processes
Support innovation in existing markets Flexible operational processes New products and operational processes
Reduce inertia Make choices that favour rapid feedback and change, with reduced dependencies across teams.
Eliminate accidental complexity Aggressively retire and replace unnecessarily complex processes, systems, and integrations so that we can focus on the essential complexity.
Consistent interfaces and data flows Eliminate duplication of data and create clear systems of record, with consistent integration interfaces.
No silver bullets Off the shelf solutions deliver early value but create inertia and complexity.
Standard REST/HTTP
Encapsulate legacy
Eliminate integration databases
Consolidate and cleanse data
Published integration model
Small independent Services
Continuous deployment
Minimal customisation of COTS/SAAS
Martin’s first law of distributed objects
Don’t distribute!
Microservices Monoliths
Single service maintainability Fit-for-purpose technology choice Integration Complexity
Environment Provisioning Operational Complexity
Testing Complexity
How?
Netscaler configs. Firewall rules. Name all the things consistently.
Configured package repository. Cron jobs. Service healthcheck
Run the specs. Run code hygiene. Keep all the gems up to date.
Configure monitoring. Enable NRPE. Configure apache.
Build an RPM. Configure dependencies. Ruby version. Configure
Passenger configs. Logrotate. Splunk forwarder. Logger
Create users. Configure NewRelic. Load rake tasks. Load
Connect to a database. Make connection pooling work.
Externalise configuration. Run database migrations. Chkconfigs.
#noops
(source: wikimedia)
If you build it, you support it
Integration Complexity Environment Provisioning Operational Complexity
Testing Complexity
Single service maintainability Fit-for-purpose technology choice
Team managed infrastructure If you build it, you support it
Microservices Monoliths
Time
# of
Ser
vice
s
3 months
Time
# of
Ser
vice
s
3 months
Time
# of
Ser
vice
s
3 months
2 services!
Condor
Contract Proposals
Pricing
Account Manager
Assignments
Condor
Contract Proposals
Pricing
Account Manager
Assignments
CondorCustomerAcceptance
AcceptedContracts Adapter
Contracts CDM
Contract Proposals
Pricing
Account Manager
Assignments
CondorCustomerAcceptance
AcceptedContracts Adapter
Contracts CDM
Contract Proposals
Pricing
Account Manager
Assignments
CondorCustomerAcceptance
AcceptedContracts Adapter
Contracts CDM
Contract Proposals
Pricing
Account Manager
Assignments
Anti-corruption
‘Stencil’ (Service Template)
Stencil
Stencil ServiceClone
Stencil Service
Service
Service
Service
Service
Service
Stencil Service
Service
Service
Service
Service
Service
Stencil Service
Service
Service
Service
Service
Service
Patch
Harvest a service template(but don’t start by writing one)
Success brings its own problems
Test Prod
Prod
Test
Test
Test
Prod
Test
Test
Test
Slow Easy to break
Hard to fix Scales BADLY
Lots of infrastructure
False negativesLots of set up
Integration Tests
Why do we do integration testing?
Why do we do integration testing?
BugsWhy do we do integration testing?
BugsWhy do we do integration testing?
Bugs ConsequencesWhy do we do integration testing?
Bugs Consequences Time Don’t knowWhy do we do
Consequences Time Don’t know
Time Don’t know Monitoring
Time Don’t know Monitoring
Time Don’t know Monitoring
Communication
Time Don’t know Monitoring
Communication
Cause
Time Don’t know Monitoring
Communication
Cause Logging
Time Don’t know Monitoring
Communication
Cause Logging
Production
Time Don’t know Monitoring
Communication
Cause Logging
Production Approval process
Time Don’t know Monitoring
Communication
Cause Logging
Production Approval process
Integration tests
Monitoring
Communication
Logging
Approval process
Integration tests
Monitoring
Communication
Logging
Approval process
Integration tests
✓✓✓✓
If you build it, you support it
Monitor meaningful things
Just deploy it
Throw away integration tests
A BC P
The problem with mocks
???
???
mock
A BC P
Test symmetry
mock
A BC P
Test symmetry
mock
C P
Pact… how it works
mock
C P
Pact… how it works
mock
C P
pact
Pact… how it works
mock
C P
pact
Pact… how it works
mock
C P
pact
Pact… how it works
mock
Slow Easy to break
Hard to fix Scales BADLY
Lots of infrastructure
False negativesLots of set up
Integration Tests
Fast Stable
Easy to debug Reliable
No extra infrastructure
Low setupScales linearly
Pact
Arrange - set up mock server
Arrange - set up mock server
Arrange - set up mock server
Arrange - set up expectations
Arrange - set up expectations
Arrange - set up expectations
Act and Assert
Act and Assert
Act and Assert
Act and Assert
Act and Assert
Set up test data
Set up test data
Consumer assumed:
Set up test data
Consumer assumed:
Provider complies:
Set up test data
Consumer assumed:
Provider complies:
Verify: Fail
Verify: Success!
Fast feedback in isolation +
Early detection +
Rapid recovery
Single service maintainability Fit-for-purpose technology choice
Team managed infrastructure If you build it, you support it
Service Template Consumer Driven Contracts
Early detection Rapid Recovery
Integration Complexity Environment Provisioning Operational Complexity
Testing Complexity
Microservices Monoliths
Time
# of
Ser
vice
s
9 months
15 services
One ‘Poppenservice’
= Two weeks(source: flickr.com)
To the Cloud!
(source: wikimedia)
Tribe
Tribe
Squad Squad
Squad
Delivery Engineering
“We aim to remove hurdles … and ensure the whole team has the knowledge to responsibly operate the system.”
- Hydra’s Vision
Single service maintainability Fit-for-purpose technology choice Team managed infrastructure++
If you build it, you support it Service Template
Consumer Driven Contracts Early detection (MTTD)
Rapid Recovery
Integration Complexity Environment Provisioning Operational Complexity
Testing Complexity
Microservices Monoliths
One ‘Poppenservice’ = Two hours
(source: flickr.com)
Time
# of
Ser
vice
s
2 years
60 services
Sharing things is hard
Service
Service Service
Service Service Service
Service Service Service
Library
Service Service Service
Library
ServiceService
Service Service Service
Library
ServiceService
Service Service Service ServiceService
Service Service Service ServiceService
Service Service Service ServiceService
Service Service Service
Library
ServiceService
Service Service Service ServiceService
Service Service Service ServiceService
Service Service Service ServiceService
Sharing == Coupling
My Tribe
All of REA
My Squad
Be cautious about shared stuff
(copy/paste may not be wrong)
Service Custodians
Establish Service Custodianship*
(*it’s REALLY hard to make work)
Why?
Why? What?
Why? What? How?
Caution.
Time
# of
Ser
vice
s
Microservices is a long term strategy
Fast feedback in isolation Rapid Recovery
Delivery Engineering Deliberate sharing
Service Custodianship
If you build it, you support it Harvest a service template Early detection of failures
Microservices is a long term strategy
Thank You!
@evanbottcher@bethesque @joneaves