devops 101+: from collaboration to microservices

DevOps 101+:From collaboration to microservicesDonnie Berkholz, Ph.D.Research Director — Development, DevOps, & IT Ops

Open Source North, June 2016

2

microservices

3Source: 451 Research/Microsoft Cloud+Hosting commissioned research

Minimizing risk, maximizing agility

4

The need for speed

Source: 451 DevOps study, Q3 2014; n=237

63% want more

5

Technology adoption

is increasingly bottom-up

Wikipedia: G.dallorto

6Source: 451/Microsoft Hosting + Cloud Study 2015

7

The new stack?

An infinite array of possible stacks.

8

Polyglot programmingThere’s no obvious choice for the right language, based on community adoption.

Donnie Berkholz Source: http://redmonk.com/dberkholz/2014/05/02/github-language-trends-and-the-fragmenting-landscape/

9

Polyglot databases

https://orchestrate.io/blog/2013/09/11/11polyglot-persistence-and-nosql-more-flexibility-more-complexity/

10

Polyglot frameworksTa

gged

que

stio

ns/m

onth

DevOps:Putting IT into high gear

11

12

Agile, truly tip to tail

Business to customer

13

3 pillars of DevOps

Culture

Automation

Measurement

14

Culture:Tear down all the silos

Flickr: kalandrakas

15Flickr: respresFlickr: hartvig, snapeverything, roymaloon

Automation:Pets vs Cattle

16

Automation: Infrastructure as code

Wikipedia: Magnus Manske

17

Knight Capital and the $460 million bug

Wikipedia: Jericho

18

Continuous delivery

Source: continuousautomation.com

19

Where are we today?

Highly Manual

Manual with Limited Automation Tools

Automated with Manual Exception Handling

Policy Based Automation and Orchestration

Other

10.0%

54.7%

27.9%

6.8%

0.7%

n = 843Source: 451 VotE Cloud, Q3 2015

20

Measurement: #monitoringsucks/monitoringlove

21

< 250 employ-

ees

250-999 employees

1,000-9,999 employees

>10,000 employees

0%10%20%30%40%50%60%70%80%90%

Agile adoption: still not universal

451 Research, Voice of the Enterprise: Software-Defined Infrastructure, Q4 2015 (n=670)

22

< 250 employ-

ees

250-999 employees

1,000-9,999 employees

>10,000 employees

0%

10%

20%

30%

40%

50%

60%

DevOps adoption: resource-dependent?

451 Research, Voice of the Enterprise: Software-Defined Infrastructure, Q4 2015 (n=568)

23

DevOps tools in use still vary widely

Infrastructure as a Service

Application Topology/Architecture and Management (e.g. service

modeling, application packaging)

Release management

QA planning and automation tools

Performance Monitoring and Analysis/Log Event Management

Testing

33.83%37.31%

39.30%39.30%

40.80%40.80%

44.28%45.77%

51.24%51.74%

63.18%

Source: 451 Research/Red Hat, Q1 2016, n=201

24

0%5%

10%15%20%25%30%35%40%

6.0%

27.9%

34.3%

23.4%

3.5%1.0%

3.5%0.5%

Release speed still lags demand

Source: 451 Research/Red Hat, Q1 2016, n=201

Enter containers:The future of virtualization

26

Aren’t they just like VMs? No.

Source: 451 Research, “Now Shipping: The Docker and containers ecosystem rapidly takes shape”

Containers vs VMs: no clear approach

27451 Research, Voice of the Enterprise: Software-Defined Infrastructure, Q4 2015

Containers Run Separately from VMs

Containers Run On Top Of VMs

Containers Are Replacing VMs

10.9%

14.6%

9.0%

n = 458

Automation, agility, empathy

28

Developers love Docker

29

Discovery and Evaluation

Running Trials/Pilot Projects

In Test and Development Environment

Initial Implementation of Production Applications

Broad Implementation of Production Applications

No Plans

56.1%

10.7%

3.9%

4.2%

2.1%

22.9%

31.5%

10.2%

8.4%

9.4%

4.7%

35.8%

Q3 2015 Q1 2015

Docker is not just a toy

30

14.1%}Source: 451 VotE Cloud, 2015; Q1 n=991; Q3 n=960

of cloud-using orgs

Prod in 3Q15:

Discovery and Evaluation

Running Trials/Pilot Projects

In Test and Development Environment

Initial Implementation of Production Applications

Broad Implementation of Production Applications

No Plans

56.1%

10.7%

3.9%

4.2%

2.1%

22.9%

31.5%

10.2%

8.4%

9.4%

4.7%

35.8%

Q3 2015 Q1 2015

Docker is not just a toy

31Source: 451 VotE Cloud, 2015; Q1 n=991; Q3 n=960

32.7%}of cloud-using orgs

Pilot+ in 3Q15:

Fragmentation drives microservices —enabled by containers

32

33

Loosely coupled services

“ The only communication allowed [at Amazon] is via service interface calls over the network.”– Steve Yegge, Google, Oct 2011,

paraphrasing Jeff Bezos memo

https://plus.google.com/+RipRowan/posts/eVeouesvaVX

34

Loosely coupled teams

“ One of the biggest changes is that we no longer have an official ‘architecture’ team. Instead, we have made ‘architecture’ an ‘ingredient’ on each of our teams.”

http://tech.gilt.com/post/102628539834/making-architecture-work-in-microservice

– Lauri Apple, Gilt Groupe, 14 Nov 2014

35

The foundation of microservices

36

Container-native OSs

Snappy Ubuntu

Container orchestration is limited (∴ adoption immature)

37451 Research, Voice of the Enterprise: Cloud, Q3 2015

Currently use

Considering using in the next two years

Not familiar with these tools

Have no plans to use in the next two years

9.4%

36.1%

39.9%

14.6%

n = 534

Real-world examples

38

39

Real-world example #1

http://www.slideshare.net/nathariel/scaling-microservices-architecture-on-aws

40

Hailo architecture

41

Hailo architecture

42

Hailo architecture

43

Complexity is the new normal

44

Real-world example #2: REA (realestate.com.au)

http://techblog.realestate.com.au/a-microservices-implementation-retrospective/

45

REA microservices timeline

0 6 12 18 240

20

40

60

Months

Mic

rose

rvic

es

http://yowconference.com.au/slides/yow2014/SkurrieBottcherEvans-MonolithsToMicroservices.pdf

“ Microservices is a long term strategy.”– Evan Bottcher,

ThoughtWorks/REA, 9 Dec 2014

46

Real-world example #3: Ctrip (Chinese travel site)

http://www.slideshare.net/yang75108/micro-service-architecture-c-trip-v11

47

Real-world example #3: Ctrip (Chinese travel site)

http://www.slideshare.net/yang75108/micro-service-architecture-c-trip-v11

48

Real-world example #3: Ctrip (Chinese travel site)

http://www.slideshare.net/yang75108/micro-service-architecture-c-trip-v11

49

The cloud-native movement is just about to take off

Developing and running web-based applications

Migrating legacy workloads and applications to the cloud

Developing and running cloud native applications

Managing legacy workloads, applications and assets on the cloud

Testing new technologies and methods

32%

32%

13%

13%

9%

Source: 451 Research/Red Hat, Q1 2016, n=201

50

From primitives to platforms

ServerlessPaaSCaaS

Container orchestrati

on

IaaS / Containe

rs

OpinionatedFlexible

How? DevOps (Culture, Automation, Measurement)

What? Microservices

Why? Survival

51

52

Thank you!Donnie BerkholzTwitter: @dberkholzdonnie.berkholz@451research.com

Some content from this presentation is Creative-Commons licensed.

https://creativecommons.org/licenses/by-sa/https://creativecommons.org/licenses/by/

53

Minimizing risk, maximizing agility

Architecture: Microservices, composable monitoringCode: Continuous integration, feature flagsServers: Continuous delivery, infrastructure as codeServices: Rolling updates, resilience engineeringProduct: Continuous deployment, restricted audience

54