microservices 5 things i wish i'd known code motion
TRANSCRIPT
Microservices; 5 things I wish I’d knownVincent Kok
AMSTERDAM 16 - 17 MAY 2017
Lives in Sydney Move to the other side of the
world to join Atlassian
About me: @vincentkok
Confluence Development manager on
the Confluence team
Dutch Lived most of my live 30 mins from
Amsterdam
Microservices Everybody seems to want them. Do we really know the impact of our choices?
Why do we want them so badly? Microservices are messy!
https://flic.kr/p/9u5pDA
http://geek-and-poke.com/geekandpoke/2013/7/13/foodprints
Grow Fat Code base grows. All
the things slow down.
Age Your code base will become a jurassic
park introducing new tech becomes hard
Ownership Who is responsible for which part and
more important: who has the pager
Economies of Scale
The bigger the team the more they
interrupt each other
Monolithical issues
81000Build jobs ran last week
31992Automated tests
Cause of issues can be extremely hard
INCIDENT RESPONSE
Who is having the pager?
Remember, we’re not all webscale
Optimise for rapid and sustainable flow of value
DAN NORD
Small The size will be reasonable and
manageable
Independent lifecycle
Nothing will hold the team back. Go as
fast as you can
Optimise for the problem Pick solution and tech based on the problem at hand
Replaceable It is easier to replace if there is a need for
it
The microservice promise
CONFLUENCE EXAMPLES
CONFLUENCE EXAMPLES
Scheduler
Attachments
Operational Transformation
Platform Services
CONFLUENCE EXAMPLES
Scheduler
Attachments
Operational Transformation
Platform ServicesFront end
CONFLUENCE EXAMPLES
Core functionality
Scheduler
Attachments
Operational Transformation
Platform ServicesFront end
5 patterns
Basics
Deployments
Testing
Security
Operations
https://flic.kr/p/9t2138
Creating a call-out Watch the tutorial in the Presentation Guidelines to learn how to create call-outs on screenshots within this template.
Treat them as cattle, not pets
BILL BAKER
A MINIMAL SERVICE
Health check 200 app is alive. 500 app is unhealthy, destroy the node
Stateless* Run as many nodes as you need
Expose a port Only access to the service
DEEPCHECK
Deep check Quickly discover if a service
fails to connect to a dependency
DEEPCHECK EXAMPLE
{ "avatar": { "details": { "avatarRepository": { "isHealthy": true }, "crowd": { "isHealthy": true }, "deadlock": { "isHealthy": true
CODE & BUILDS
1 repository 1 build
Strict separation of config from code
12 FACTOR APP
Redeploy Part of the service
configuration.
Configuration lifecycles
Instant change Switches you would like to
enable/disable straight away
Rebuild Rebuild to apply changes
Only one person There is only one person in
the team that owns it
Deployment smells
Takes more then 15 mins
Setting it up should be quick and initial deployment should
quick
Requires a ticket A ticket for the deployment
team
Always deploy an empty service into production…
ME; AND PROBABLY OTHERS
Developers in control
Artifact What is the artifact we’re running. We’re mostly standardising on Docker
Resources What resources are requires: RDS, SQS, Dynamo etc..
Compute What EC2 instance do we want how many of those and when to scale
Alarms What are the alarm thresholds for this service
Ownership Who is owning the service
Configuration We will be adding more icons as need arises. Speak up if in need!
DECLARATIVE DEPLOYMENT
name: Confluencedescription: Confluence Vertigolinks: binary: type: docker name: docker.atlassian.io/confluence tag: latest healthcheck: uri: /wiki/internal/healthcheck deepcheck: uri: /wiki/internal/deepcheck semanticCheck: dockerImage:
CONFIGURATION
config: environmentVariables: ASAP_AUDIENCE: "foo" ASAP_ISSUER: "foo" CONFLUENCE_VERTIGO_SMTP_HOST: "smtp.foo.com" CONFLUENCE_VERTIGO_SMTP_PORT: "587" LOG4J_EXTRA_RULES: "log4j.logger.org.hiberate=DEBUG"
environmentOverrides: staging: config: environmentVariables: ASAP_PUBLIC_KEY_FALLBACK_REPOSITORY_URL: "https://s3.amazonaws.com/keysto
RESOURCES
resources: - type: sqs name: default attributes: MaxReceiveCount: 20 VisibilityTimeout: 60 scaling: instance: m3.xlarge min: 7
500Services in production
Testing microservices
Testing microservices
TESTING MONOLITHS IS EASY
Unit
Integration
UI
TESTING
Live service Test agains a real service
TESTING
Mock service Test against a mock service
In process A local implementation of
your client
Out of process Use tools like WireMock and
MockServer
Two options
MOCKING SERVICES - IN PROCESS
<beans profile=“integration-test"> <bean id="attachmentService" class=“c.a.attachment.AttachmentStub”/></beans>
MOCKING SERVICES - WIREMOCK
{ "request": { "url": “/rest/api/content“, "method": “POST” "Accept": { "matches": “application/json” } }, "response": { "status": 200 }}
Stable API If it is external it already
should have a CTK so rely on it
How to trust your mock?
Contract testing Internal fast moving API’s an
benefit from this
Rely on monitoring Small service, low MTTR
therefore low impact
Semantic Check Automated test that runs against a node before it will be added to the load balancer
OAuth 2.0 Grant a client access to
resources based on a newly created set of credentials
Common standards
OpenID Connect Identity on top of OAuth 2
OpenID Allows identity and some
metadata only
SECURING SERVICES
How to secure a set of many services
ASAPAtlassian Service Authentication Protocol
HOW DOES IT WORK
Foo BarJWT
WHATS INSIDE?
Foo Bar
{ "typ": "JWT", "kid": "foo/key1", "alg": "RS256"}{ "sub": “32769:87e…” "aud": "bar", "nbf": 1494284564, "iss": "foo", "exp": 1494284624, "iat": 1494284564, "jti": “961253cf-ac…”}
50kg
Uptime of a system with 30 services of 99.99
WHAT A MICROSERVICE ARCHITECTURE
2 hours99.99 = 99.7
30
RESILIENCE
Failure is imminent!
Circuit breakers Write code with failure in
mind
Three must haves
Request tracing Don’t spend hours debugging
Log aggregations Stream all logs into one
place.
DO YOU KNOW YOUR SYSTEM?
CREATE INSIGHT: AGGREGATED LOGGING
Response times How much time do spend calling other services.
Back pressure Stop putting pressure on a system that is in trouble and fail fast
Fallback How do you handle failure. A mandatory step in the programming model.
Circuit breakers
CREATE INSIGHT: CIRCUIT BREAKERS
Request Tracing
Request TracingX-B3-TraceId : 1X-B3-SpanId : 1
Request TracingX-B3-TraceId : 1X-B3-SpanId : 1
X-B3-TraceId : 1X-B3-SpanId : 2X-B3-ParentSpanId : 1
Request TracingX-B3-TraceId : 1X-B3-SpanId : 1
X-B3-TraceId : 1X-B3-SpanId : 2X-B3-ParentSpanId : 1
X-B3-TraceId : 1X-B3-SpanId : 3X-B3-ParentSpanId : 2
Request TracingX-B3-TraceId : 1X-B3-SpanId : 1
X-B3-TraceId : 1X-B3-SpanId : 2X-B3-ParentSpanId : 1
X-B3-TraceId : 1X-B3-SpanId : 3X-B3-ParentSpanId : 2
X-B3-TraceId : 1X-B3-SpanId : 4X-B3-ParentSpanId : 3
TRACE ID’S
You Build It You Run It The team who builds it looks after it.
Ops Team Handover your services and let them
deal with the fun. Don’t do this.
What should you take home?
Basics Services are cattle not pets.
Testing Testing a monolith is “easy” what’s think about your service testing strategy
Deployment Deploying a service shouldn’t take longer then 15 minutes
Operations You build it you run it.
Security Think how you would like to secure service to service communications
Focus on value Optimise for rapid and sustainable flow of value
VINCENT KOK - ATLASSIAN - @VINCENTKOK
Thank youCodeMotion Amsterdam 2017