microservices in der cloud - software architecture summit berlin 2016

PowerPoint-Prsentation

Christian Deger @cdeger | AutoScout24Microservices in der CloudMit AutoScout24 auf der berholspur

1

Workshop: Questions at any timeInteractive discussionsShare your experiencesEveryone can contribute?!

Please dont let me bore you!

2

Christian DegerChief [email protected]@cdeger

AutoScout24 in Munich, 6 YearsSoftware Engineer, Team Lead, Architect, Chief Architect

3

2,4 Million Vehicles

Who does not know AS24?Listings business, completley digital.Our focus is the search for the vehicle and everything around that4

Microservices in the cloud adoption?

Who is doing CI?Who is doing CD?Who is doing Infrastructure as Code?Who is in the cloud?Who plans on moving to the cloud?Who knows about microservices?Who is doing microservices?5

AgendaBackgroundWhy change?PreparingThe journeyContinuous deliveryWhy AWS?Technical migrationStatus quo and learnings

MonolithSVN Monorepo.NET WebformsSwimlanesGit repos.NET MVCShared databaseSwimlanes:Teams wanted to be faster and more autonomousRough extraction of a capabilityCode duplication of base functionalitySeparate delivery pipelinesFrequent releasesComposition using subdomainsCash-Stack $$

Autonomy:Teams decided when to releaseFaster and more local technical decision.Still:Mostly same delivery process.Delivered to existing VMs.Bad:Added additional pressure on operations team. Devs just went ahead.

7

2000 Servers2 Data Centers

MTBF optimized

Proven delivery engineHighly optimized, but of last decadeIT platform supported growth for >6 yearsAvailability = MTBF / (MTBF + MTTR)Proven agile and lean principles8

Dev and Ops Silos

DevelopmentChangeOperationsStability

The split at CTO level.Reflexes:Protection: Ticket systemBlaming: Devs ships bad software. Ops is to slow.Introduced proxies between silos: DevOps as role anti-pattern.QA handover, Product handover.

9

Ops are on call for services written by devs.Performance degradation: Just buy more servers.Devs unaware of response time and availability.10

Triggering slide for me years ago.11

In AutoScout24 there was resistance against change.We are not like NetflixThat is to expensiveWe have built a private cloudsunken cost fallacyabsurd lift and shift cost calculation.12


NewCEO

Scout24 was sold end of 2013New CEO Greg Ellis beginning of 2014Are you ready for the future?14

Talent?Do you attract

We got good, agile .NET developers.But from banks and insurrances.We were not getting the talents for internet business.15

We started think about our ecosystem and the flywheel that drives it. .NET small, slowLinux/JVM larger, faster

21st CenturyWhat does a

tech company look like?

17

We want to learn and participate from the advances the unicorns in our industry make.Visited Netflix, LinkedIn, AirBnb, etc.

Innovation for us no longer comes from previous enterprise suppliers.Oracle, IBM, Microsoft

18

Great DesignUniversally ConnectedMobile FirstInstant Business ValueMassive Data InsightHighly Available

19

good, but not greatHmm, we are

We are not bad, but we are not great.The new questions triggered something...20

Rebooteverything

Escape the gravity of status quo.Perhaps its a pattern: Just rebuild it.

21

Project

Tatsu

Fly at the speed of fear - DisruptiveJapanese dragon: Flying beast

Started Nov. 2014 with one team, now at 4 teams.

Rollercoaster: Sixflags Magic Mountain23


.NET / Windows to JVM / LinuxMonolith to MicroservicesData center to AWSDevs + Ops to Collaboration cultureInvolve product people

Windows is not used in 21st century companiesLinux is easier to automateCloud native architecture, use AWS services. Closing feedback loopProductNot a pure technical transformation.Leaner version of AS24.Shiny new cut.

Major JVM Languages

We settled on the JVM very early.Alternatives where to young and risky.Many companies we try to learn from use the JVM.Go was favored by some.26

No traction in major internet companiesMajor JVM Languages

Major JVM LanguagesNo traction in major internet companiesNot accepted by C#developers

Major JVM LanguagesNo traction in major internet companiesNot accepted by C#developersAttracts talentIs a starting point

Twitter, Gilt, SoundCloud, Zalando.Polyglot: Starting point onlyNo endless language war.

Would we make the same choice again?+ It does attract talent+ Ecosystem: Play, Akka, SparkLong learning curveNo idiomatic Scala / many concepts

29

Our/My top influencers.Martin Fowler, Adrian Cockcroft, Stefan Tilkov

30

Martin Fowlers blog: March 25, 2014Founded Meetup: May 8, 2014Microservices Selbsthilfegruppe.

31

Why Microservices?

32

Why Microservices?SpeedIndependent deployableFast local decisionsAutonomous teamsStrong boundariesLoosely coupledTechnology diversityScale the organization

Scale the organization + SpeedSpeed: Not waiting for others. Technology diversity: I dont want to end up with another migration.Strong boundaries:Modularized monolith degrades.Process boundary as a strong architectural force.Bounded Context

Strong module boundarieshttp://martinfowler.com/articles/microservice-trade-offs.html#boundariesIndepent deploymenthttp://martinfowler.com/articles/microservice-trade-offs.html#deploymentTechnology diversityhttp://martinfowler.com/articles/microservice-trade-offs.html#diversity

33

Microservices The bad parts

Eventual consistencyhttp://martinfowler.com/articles/microservice-trade-offs.html#consistencyDistributionhttp://martinfowler.com/articles/microservice-trade-offs.html#distributionOperational Complexityhttp://martinfowler.com/articles/microservice-trade-offs.html#ops

34

Microservices The bad partsOperational complexityDistributed systemDifficult to change boundaries

Operational complexity:More of everythingDistributed system:Communication patterns, resilience, eventual consistencyDifficult to change boundaries:Adjusting bounded contexts involves services and their interactions now.

Eventual consistencyhttp://martinfowler.com/articles/microservice-trade-offs.html#consistencyDistributionhttp://martinfowler.com/articles/microservice-trade-offs.html#distributionOperational Complexityhttp://martinfowler.com/articles/microservice-trade-offs.html#ops

36

Loosely coupled service oriented architecturewith bounded contexts.Adrian Cockcroft

Microservices are small, autonomous servicesthat work together.Sam Newman

Loosely coupled service oriented architecturewith bounded contexts.

Microservices are small, autonomous servicesthat work together.

CA Book:A microservice is an independently deployable component of bounded scope that supports interoperability through message-based communication. Microservice architecture is a style of engineering highly automated, evolvable software systems made up of capability-aligned microservices.

37

Death Star Diagrams

Amazon 2008Twitter 2013

https://blog.twitter.com/2013/observability-at-twitterhttps://twitter.com/Werner/status/74167351456714342438

http://scs-architecture.org/Self-Contained Systems = Microservices Flavor

Team 1Team 2Team 3

One business capability is owned, built and run as an SCS by one team.Self-Contained System are vertical slices integrated at the UI.

Netflix style vs. SCS:We are not that experienced in distributed systems.No API Gateway monolith.No UI monolith.

39


samedirection

First team was tightly aligned.Wanted to the right thing, they were prepared.When ramping up to four teams, we realized, that we need41

STRATEGICGOALSGoals of the business sideARCHITECTURALPRINCIPLESHigh-Level PrinciplesDESIGN AND DELIVERY PRINCIPLESTactical measuresREDUCE TIME TO MARKETEstablish fast feedback loops to learn, validate and improve. Remove friction, hand-offs and undifferentiated work.MOBILE FIRSTStart small and use device capabilities.SUPPORT DATA-DRIVEN DECISIONSProvide relevant metrics and data for user and market insights. Validate hypothesis for problems worth solving.

YOU BUILT IT, YOU RUN ITThe team is responsible for shaping, building, running and maintaining its products. Fast feedback from live and customers helps us to continuously improve.ORGANIZED AROUND BUSINESS CAPABILITIESBuild teams around products not projects. Follow the domain and respect bounded contexts. Make boundaries explicit. Inverse Conway Maneuver.LOOSELY COUPLEDBy default avoid sharing and tight coupling.No integration database. Dont create the next monolith.MACRO AND MICRO ARCHITECTUREClear separation. Autonomous micro services within the rules and constraints of the macro architecture. AWS FIRSTFavor AWS platform service over managed service,over self-hosted OSS, over self built solutions.DATA-DRIVEN / METRIC-DRIVENCollect business and operational metrics. Analyze, alert and act on them.ELIMINATE ACCIDENTAL COMPLEXITYStrive to keep it simple. Dont over-engineer.Focus on necessary domain complexity.AUTONOMOUS TEAMSMake fast local decisions. Be responsible. Know your boundaries. Share findings.INFRASTRUCTURE AS CODEAutomate everything: Reproducible, traceable, auditable and tested. Immutable servers.CROSS-FUNCTIONAL TEAMSEngineers from all backgrounds work together in collaborative teams as engineers and share responsibilities. No silos.BE BOLDGo into production early. Value monitoring over tests.Fail fast, recover and learn. Optimize for MTTR not MTBF.SECURITY, COMPLIANCE AND DATA PRIVACYBuild with least privilege and data privacy in mind.Know your threat model. Limit blast radius.COST EFFICIENCYRun your segment in the right balance of cost and value.

ONE SCOUT ITFoster collaboration. Harmonize and standardize tools.Pull common capabilities into decoupled platform services.

Version 2.0Icons made by Freepik from www.flaticon.com are licensed under CC BY 3.0

BEST TALENTAutonomy, Purpose and Mastery: We know why we do things, we decide how to approach them and deliberately practice our skills.

DisclaimersStrategic Goals include business and IT specific goals, because the target audience for the principles is IT.Dont be stupid! is relevant for all principles. Principles not Dogmas.Obvious things that are already part of our core beliefs are note mentioned. For example Lean, Agile and Continuous Integration.

Mobile FirstSync state between platforms

Tech CultureFor a better understanding of those terms, see Dan Pink: https://www.youtube.com/watch?v=u6XAPnuFjJcZalando and their Radical Agility: https://tech.zalando.de/working-at-z/

Cost efficiencyIncludes infrastructure

One Scout ITApplience model included as concept for a platform service

Organized around Business CapabilitiesIncludes internal products in the business or infrastructure platform.Business capabilities typically map to bounded contexts known from DDD.Inverse Conway Maneuver: Loosely coupled teams responsible for business capabilities lead to corresponding services aligned with those business capabilities.Expert exchange is not in violation: For example platform experts can help co-creating a service in a product team.

Eliminate Accidental ComplexityEssential complexity is the core of the problem we have to solve. It is based on true functional and cross-functional requirements.Accidental complexity is all the other stuff that doesnt directly relate to the solution derived from the requirements.Boy scout rule

Loosely coupledAvoid sharing includes shared infrastructure.Using a multi-tenant capable platform API is not sharing. No premature generalization. First have the problem and solve it. With the next incarnation of the same problem, start generalizing it.

AWS FirstFavor over as in the agile manifesto: There are use cases for things on the right, we start with and prefer things on the left.Document your decisions (ADR).Be aware of NIH.

You build it, you run itOwn the product and services lifecycles includes also retiring them.

Be bold

MTBF: Mean time between failuresMTTR: Mean time to recoveryAvailability = MTBF / (MTBF + MTTR)

Autonomous TeamsKnow your boundaries: Be thoughtful, when you are stepping into the realms of the macro architecture or outside your business capability.

Macro and Micro ArchitectureFor example, Scala is our default product engineering language. With good reasons you are free to deviate from that.Your freedom is not my responsibility - Quote from a Netflix engineer: When a team makes a decision like that, it needs to able to support it in the long run.

Infrastructure as codePhoenix servers and immutable containersReport and alert on security and conformity violations.

BuildMeasureLearn

Time to market: the core goal, fast feedbackWhy fast feedback: Build Measure Learn43

http://de.slideshare.net/adriancockcroft/microxchg-microservices

OODA Loop Military strategy developed by US Air Force Colonel.It is vital to cycle through the loop faster than to opponent.

44

Conways Law

organizations which design systems ... are constrained to produce designs which are copies of the communicationstructuresof these organizations

Intro to next slide: Inverse Conways Maneuver.45

Autonomous teamsbusiness capabilitiesorganized around

Have empowered, self organizing cells, to get teams that build decoupled services.Collective code ownership. Does not scale.We want to scale the organization and still be fast.Products not projects.http://martinfowler.com/bliki/BusinessCapabilityCentric.html

Follow the domain and respect bounded contexts.Make fast local decisions.Be responsible.

46

Two Pizza RuleJeff Bezos

Single digit number of team members.47

collaborationculture

DevOps for us is not a team, a role or a tool is collaboration.No silos, no handovers, no fingerpointing.No software that is hard to operate.No infrastructure nobody needs.But also to use tools like Slack, Github

48

We are all engineers!

No more devs and ops -> engineersPut people together in one stable team. All other experiments failed.Do not fall back into old behaviours:Zoom in you find 4 devs on one side and 2 ops on the other -> rotate pairsNot all T-shapes are the same

49

You build it,you run it.

Freedom and Reponsibility.The team is responsible for shaping, building, running and maintaining its products.Fast feedback from live and customers helps us to continuously improve.The one who feels the pain of being woken up at nightthe one to be able to make the changes-> real resilient services-> Customer first50

Monitoring is the new testing

Tests only run on delivery. Monitoring runs all the time.

Performance.CD Pipelines.Open OpsGenie Alerts.Costs per day.Page Speed.

Next step: Monitor business KPIs.51

How (not) to shareshared nothing as defaultloosely coupledfast local decisionsvoluntary adoptionexception: macro concerns

Sharing implies dependencies.Share only with good reason.Availability over shared nothingNo side effects: Story of shared state in dashingUse over re-useCopy npaste, OSS, libraryFast local decisions over committeeRe-use only after hardening

Follow thetrail

Convenience offerings.People will stick with decisions, when they are good enough for them.They will create different solutions, when it is important for them.CD Tool, Language decision, Service Template.53

GuildsSelf-organizing; common interests; across teams Macro Architecture, Infrastructure, Frontend, QA...Beware of mandelbrot teams

Need because market segments and teams are getting the high bandwith communication and this are cross segment expert groups.

Mandelbrot TeamsIn some teams, this leads to the same separation within the team. Former ops engineers working on infrastructure stories and software engineers working on product stories.This leads to the usual defects: Infrastructure products can have a very high bus factor and bottlenecks, because not the whole team feels responsible, but is waiting for the Ops guy to be available.This is not the default, but observable.


ContinuousDelivery

Fast feedback from real users. Build, Measure, LearnFast feedback on quality of service.Creating value for user is daily business.

We started with one release per month......to many releases per day.

But only code changes were delivered.56

DevOps Survey

Continuous delivery vs continuous deployment?Who does continuous delivery?Reasons not to do continuous delivery?57

Forsgren, Nicole and Humble, Jez, The Role of Continuous Delivery in IT and Organizational Performance (October 27, 2015). Forsgren, N., J. Humble (2016). "The Role of Continuous Delivery in IT and Organizational Performance." In the Proceedings of the Western Decision Sciences Institute (WDSI) 2016, Las Vegas, NV. . Available at SSRN: http://ssrn.com/abstract=2681909 or http://dx.doi.org/10.2139/ssrn.2681909DevOps Science

Jezz Humble: What We Learned from Three Years Sciencing the Crap out of Devops

58

Application code in one repository per service.

CIDeployment package as artifact.CDDeliver package to servers

Delivery Pipeline Data Center

Commit stage: Unit tests etc.Additional database migration scripts.Blue/ Green delivery on the instance.

59

Application code and infrastructure specification in one repository per service.

CIDeployment package and infrastructure declaration as artifact.CD 1. Create or update service infrastructure.

2. New instances pull down package and start application.

Delivery Pipeline AWS

Every change goes through the delivery pipeline.High traceability.Delivery: CloudFormation + ASGDependencies: Global stack and Base AMI.

60

TraceableRepeatableReliable(Faster)

61

Cattle,not pets

Phoenix servers.No configuration drift.Security: Alerting on instances that are to old

Tradeoff cycle time:Current cycle time: commit to production is 20-30 minutes: To slow!

62

Separatecode deploymentfeature releasefrom

Who is using feature branches and does CI?No merging of branches anymore.You are not doing CI, when you are branching.Short lived branches < 1 day are ok.Dynamic Feature Toggles:Canary releases.Product can switch on a feature for acceptance.

63

http://martinfowler.com/articles/feature-toggles.html

64

Nostagingenvironment

We only maintain one environment where all services integrate: ProductionBe bold!MTTR over MTBF

65

Consumer driven contractsSmoke testsCanary releasesShadow trafficSemantic monitoring

Integrate in production

Shadow traffic: Best in combination with flagging test data.Semantic monitoring:For example continuous user journey tests.Dont monitor the cell, listen to the heartbeat.Monitor for invariants in your system: Leads : Homepage visits

66


http://archive.oreilly.com/network/2006/12/20/web-20-bezos.html

Jeff Bezos 2006Avoid undifferentiated heavy liftingUndifferentiatedYour IdeaUndifferentiatedYour IdeaOn Premise:AWS:

68

https://aws.amazon.com/resources/gartner-2016-mq-learn-more/Gartner Magic Quadrant forCloud IaaS - 2016

Consolidation is already happening, compared to 2015 and 2014

Azure status: https://azure.microsoft.com/en-us/status/history/http://status.aws.amazon.com/

Gartner: Magic Quadrant for Cloud Infrastructure as a Service, Worldwide report,Lydia Leong et al, published 3 August, 2016

69

http://www.slideshare.net/AmazonWebServices/introduction-to-microservices-66320469/

Number of new services and updates to service is still increasing.

70

SQS + S3Kinesis + S3Kinesis + DynamoDBSQS + DynamoDBProxy + DynamoDBDynamoDBEvolution

Fast evolutionWorking code and infrastructure for all options within 2 weeks

SQS + S3 Inspired by ImmobilienScout24Kinesis + S3 Feedback from AWSKinesis + DynomaDB Queryable, Storage costs not prohibitive (7x)SQS + DynamoDB Queue better than LATEST or TRIM_HORIZONProxy + DynamoDB Changes are collected via queue table in OracleDynamoDB DynamoDB is global service

Unlimited Infrastructure with APIs

Automated conformity and security:Alarms for unexpected production changes: Changes not through the pipeline.Alarms for old base AMIs, guarding against missing security patches.

72

Right-SizingCost OptimizationElasticityReservationsCost TransparencyCost Driven Design

Cost transparency: CloudHealth, Dashboards.Designing with cost in mind.Resource utilization: Example searchRefactoring to reduce cost: Example GTM pixel

Stacked: Yellow = AWS, Blue = DC, Line = 1GB/s73


Migration strategy

Migration strategy. Vertical slices

Frontend integrationLoosely coupledAutonomous teamsHigh optimization

Loosly coupledNo shared asset pipeline.No origin specific behaviour.Autonomous teamsMicroservices imply frontend integrationSelf contained, generic MS over UI monolithHigh optimisationGoogle pagespeedCaching

Allows hybrid approach: Routing back traffic to DC

PageSpeed Module

css (page+fragment)js (page+fragment)

ngx_pagespeed

css (page)js (page)

css (fragment)js (fragment)

No shared asset pipeline.

Pagesare accessible via (localised) URLare owned by one teamcould be cacheableFragmentsare parts of a pagedont know the original request should send cache headersAssetsShould be combined and minifiedCachingCloudFront Caching: Caching on edge locations. Respects Cache Headers from Jigsaw.PageSpeed Caching: Caches combined assets.Backend Caching: Respects Cache Headers from microservices.

Event Streaming

Microservices challange: Collect data for BI from all services.Application events are written directly to Kinesis as Json.Ingesting 1.4 TB per day.Intended to be used for real time processing later.

Event Sourcingone way data highway and data pumps

One way data highwayEvent Sourcing - History of all changes

Design decision: No queries against DC. Data needs to be pushed into AWSAbility to replay all events from beginning of time.Events = All changes to a classifiedDatabase change propagation = poor mans event sourcing, no intent captured

Kafka Connect, LinkedIn DataBus

Templates

Faster bootstrappingCopied not inheritedCollect and share best practices

Example of Follow the trailCopied not inherited:Typically not required to retrofit everything from the templateLoose couplingDisadvantage: No Collect and share best practices: Survival of the fittest.81

Everything fails,all the time.Werner Vogels, CTO Amazon, 2008

82

Resilience, AvailabilityChaos engineeringTimeouts/ Circuit BreakerBulkheadshttp://techblog.netflix.com/2014/09/introducing-chaos-engineering.html

Chaos engineering: Continuous failure injectionBasic HA patterns given by AWS: ASG, Multi AZ.

83


Commit to Production20 Minutes Cycle Time

New Service1 Day Service Bootstrapping3 Days Frontend4 Days Backend

OEM Testdrive: https://www.autoscout24.de/testdriverService Bootstrapping: CD Pipeline to production, Monitoring, AlertsFrontend: Move the pixel. Reuse a lot of other services via ssi and ui compositionBackend: Persistence, Validation, Integrations

Cycle Time: From commit to production

85

015 Teams025 Lambda Functions200 Repositories040 Microservices009 Systems

Status Quo

Status in June 2016.Systems in terms of Self-Containted Systems

86

Roman Riding via Adrian Cockcroft and Simon Wardley.Private Cloud is no fun. Same goes for Public+Private Hybrid.87

LearnersExperiencedRamp up

First 8 teams: Split teams to share knowledge

Initially we wanted to distribute Tatsu Knowledge as fast and efficient as possible.

88

https://rework.withgoogle.com/blog/five-keys-to-a-successful-google-team/

Who is on a team matters less than how the team members interact, structure their work, and view their contributions.The five keys to asuccessful Google team

Splitted teams to often during ramp-up-> Favor stable teams.

Pro Split:Experienced and learning engineers collaborate: Faster learning

Pro Stable:Stable team can become high performing team.

89

LearnersExperienced

Ramp up

Plan for next teams: Coach dont split

Coaching

90

Tribes/Segments/Departments

Conways Law, there are more communication structures than only teams.Alignment, autonomy and cohesion apply also to tribes and also to the next organizational structure. For example verticals.Following that, for teams within a tribe tight coupling of services might make sense. In contrast to the communication outside of the tribe.Job rotation: Make sure not to make this structures to rigid.91

ProductPlatformProduct vs Platform Mind the gapPull vs Push?Disconnect?

92

An act of Deliberate Collective LearningBig PictureDesign Level

http://eventstorming.com/

An act of Deliberate Collective Learning

93

https://www.thoughtworks.com/insights/blog/build-your-own-technology-radar

94

?

Picture CreditsTatsu Sign by Martin Lewison from The Hague, Zuid-Holland, The Netherlands under CC BY-SA 2.0Martin Fowler by Webysther Nunes under CC BY-SA 4.0Werner Vogels by Guido van Nispen under CC BY 2.0"HotWheels - '69 Ford Torino Talladega by Leap Kye, licensed under CC BY-ND 2.0Differences between Traditional vs Next Generation by Simon Wardley under CC BY-SA 3.0Enterprise IT Adoption Cycle by Simon Wardley under CC BY-SA 3.0And the future is private by Simon Wardley under CC BY-SA 3.0Leosvel et Diosmani by Ludovic Pron under CC BY-SA 3.0Spare wheel by Brian Snelson under CC BY 2.0Wandergeselle by Sigismund von Dobschtz under CC BY-SA 3.0Wheel clamps Texas by Richard Anderson from Denton, United States (Boots.) under CC BY-SA 2.0Sharing Sucks (4536747557) by eyeliam from Portland, United States under CC BY 2.0Traffic Jam by Doo Ho Kim under CC BY-SA 2.0Puzzling by Bernd Gessler (Own work) CC BY-SA 3.0Amazon16 by Neil Palmer/CIAT under CC BY-SA 2.0Pizza by Jakob Dettner, Rainer Zenz under CC BY-SA 2.0 deBezos Iconic Laugh by Steve Jurvetsonunder CC BY 2.0

96