cd@ yahoo 2015 ithome devops conf

Post on 14-Apr-2017

141 Views

Category:

Engineering

1 Downloads

Preview:

Click to see full reader

TRANSCRIPT

Continuous Delivery @ Yahoo Baiyi Ying⎪ Sep 01, 2015

Questions

Version Control

NASA's Marshall Space Flight Center@flickrcc

Code Change to Production

How Frequent ?

Deliver ?

Yuri Samoilov@flickrcc

Deliver ?

Yuri Samoilov@flickrcc

VALUE

照顧花園...

優雅吃飯

久坐不累

Katerina Kamprani - The Uncomfortable project All rights reserved

nisha@flickrcc

visualplun.ch@flickrcc

照顧花園...

優雅吃飯

久坐不累

Katerina Kamprani - The Uncomfortable project All rights reserved

Katerina Kamprani - The Uncomfortable project All rights reserved

nisha@flickrcc

visualplun.ch@flickrcc

照顧花園...

優雅吃飯

久坐不累

Katerina Kamprani - The Uncomfortable project All rights reserved

Katerina Kamprani - The Uncomfortable project All rights reserved

Katerina Kamprani - The Uncomfortable project All rights reserved

nisha@flickrcc

visualplun.ch@flickrcc

照顧花園...

優雅吃飯

久坐不累

Katerina Kamprani - The Uncomfortable project All rights reserved

Katerina Kamprani - The Uncomfortable project All rights reserved

Katerina Kamprani - The Uncomfortable project All rights reserved

花惹發?

Arturo J. Paniagua@flickrcc

nisha@flickrcc

visualplun.ch@flickrcc

Katerina Kamprani - The Uncomfortable project All rights reserved

Deliver ?

Yuri Samoilov@flickrcc

things customers want to use

How do you know you’re delivering values?

IDEAS

CODEDATA

build

lear

n

measure

VALUE

Katerina Kamprani - The Uncomfortable project All rights reserved

WASTE visualplun.ch@flickrcc

VALUE

Katerina Kamprani - The Uncomfortable project All rights reserved

WASTE visualplun.ch@flickr visualplun.ch@flickrcc

In the Good(?) Old Days…..

Development

Testing

Testing

Release Go/No-go

1 week before D day

Release Go/No-go

Release Day

Release Day

wolfgangfoto@flickr

Development Testing

Release Go/No-go

Release Day

wolfgangfoto@flickr

Why Change ?

VALUE

Katerina Kamprani - The Uncomfortable project All rights reserved

WASTE visualplun.ch@flickr

Speed

Small Batch

Easy toIdentifyIssues

Check&

Revise

Quality

Build the Right Thing

Build the Thing Right

Good Working Code

Lots Testing

Lots Auto Testing

Eliminate Human

Involvement

Continuous Delivery

From District 9 movie

COMMIT TO PRODUCTIONWITHOUT HUMAN INTERVENTION

Results

CD Adoption Rate

Months

Multiplier

Production Pushes

APAC Production Pushes

PO: 好開心!!設計的 features可以很快的上 production 得到 users 的反應

SE: release 成本變小,incidents 沒有變多

Dev: 開始習慣先寫測試。測試變得更重要了,因為如果沒測好,就直奔production了

QA: Incremental changes rock!

Dan Goodwin@flickrcc

IDEAS

CODEDATA

build

lear

nmeasure

Lessons Learned

CD is Simple

Version Control

NASA's Marshall Space Flight Center@flickrcc

Certify

Build

Build

Certify

Version Control

NASA's Marshall Space Flight Center@flickrcc

Certify

Build

Build

Certify

Version Control

NASA's Marshall Space Flight Center@flickrcc

Certify

Build

Build

Certify

Version Control

NASA's Marshall Space Flight Center@flickrcc

Certify

Build

Build

Certify

Not Cheap Nor Easy

We Thought

William Warby@flickrcc

In Reality ….

Tools v.s. People

sean@flickrcc

hakanforss.wordpress.com

● tools will never be perfect ● can implement CD with imperfect tools

hakanforss.wordpress.com

● Mindset Change● Culture Change● CD is the Norm● All-in

How

Executive Mandate

Lost Planet Airmen (1951)

NO CD, NO LAUNCH, NO FOOLING

sean@flickrcc Nick Royer@flickrcc

5 Stages of Adopting CD

5 Stage of Grief

Denial

Arturo J. Paniagua@flickrcc

AngerDavid K@flickrcc

Bargaining

mark engelbrecht@flickrcc

Depression

marc latermouille@flickrcc

Acceptance

Denial

Arturo J. Paniagua@flickrcc

AngerDavid K@flickrcc

Bargaining

mark engelbrecht@flickrcc

Depression

marc latermouille@flickrcc

Acceptance

5 Mental Stages of Adopting CD

Denial 否認

蛤?難道你的老闆的老闆的老闆... x N不是 Marrisa 嗎?

全面CD ? 怎麼可能!應該不包括我們吧?

別鬧了!!Projects 都作不完了,一堆 features 等著要上...都這麼忙了,還來搞這個!!

Anger 憤怒

Arturo J. Paniagua@flickrcc

Bargaining 討價還價

好啦好啦,我們會作啦!不過手上這個 project 超重要,沒有人力時間弄 CD,可不可以幫這個 project 申請 exception ?

David K@flickrcc

持續洗腦

Depression 沮喪

kris gabbard@flickrcc

● fear● misunderstanding● doubt

Version Control

NASA's Marshall Space Flight Center@flickrcc

Certify

Build

Build

Certify

uncle bob@Ruining your Test Automation Strategy.

Nate Steiner@flickrcc

天啊! 要怎麼把這 30K 隻manual tests 改成自動化測試程式?

Well, you don’t

Acceptance 接受

● No manual testings● No manual commands in production● Not just an engineering project

marc latermouille@flickrcc

Aggressive Audacious Goals by Phases

Git in 30

1800 repos

30 D

Build Farms in 30

130 Farms 4 Centralized Farms

30 D

40K Jobs65K Builds / days

CD in 2

Scale & Diversity

cellanr@flickrcc

Single Way

Common Terminology

Common Principles

Tools & Services

Teams to Figure Out

Common Terminology

■ Package● compiled software, scripts, configuration or other data● smallest granularity of deployment● versioned

Common Terminology

■ Image● a set of packages● determines the functionality a

node will have

■ Component● runtime entity● defined by an image

Common Terminology

■ System● one or more components ● certified, launched, and operated together.

Common Terminology

Principles

■ Repeatable & Reproducible ● Versioned● Immutable Artifacts

■ Traceable● Artifact -> all the changes made

■ Version Control● source code, launch scripts and configurations, test code, test data.

Version Control

Package Component System

Package

Certify

ArtifactStorages

Build Build

PackagePackagePackage

Image

Deploy

Certify

Tagger

Build

Prod

PC1

PPC2

P

Manifest

DeployCertify

NASA's Marshall Space Flight Center@flickrcc

ArtifactStorages

Tools & Services■ Build & Workflow

● Jenkins ■ Artifact Storages

● Proprietary ● Artifactory

■ Tagger● Proprietary

■ Visualizer● Proprietary

■ Software Delivery ● Proprietary● Chef

■ End-2-End examples■ STOP development and Do it■ Use Senior Engineers■ Pipeline Demos and Reviews

Other Tactics

Results Recap

■ 7X More Production Pushes ■ Quality

Test

able

Cod

e

Ref

acto

ring

CD

Dec

oupl

ed

Des

ign

Prin

cipl

es

Dat

a D

riven

TDD

BD

D

Ope

ratio

nal

■ First Domino ● Product Excellence● Engineering Excellence

……

…..

etc

After Adopting CD

Before Adopting CD

回不去了!誰想回去啊?

Summary

visualplun.ch@flickr

Jesse! S?@flickrcc

Nathan E Photography@flickrcc

sean@flickrcc

hakanforss.wordpress.com

top related