Download - CD@ yahoo 2015 iThome devops conf
Continuous Delivery @ Yahoo Baiyi Ying⎪ Sep 01, 2015
Questions
Version Control
NASA's Marshall Space Flight Center@flickrcc
Code Change to Production
How Frequent ?
Deliver ?
Yuri Samoilov@flickrcc
Deliver ?
Yuri Samoilov@flickrcc
VALUE
照顧花園...
優雅吃飯
久坐不累
Katerina Kamprani - The Uncomfortable project All rights reserved
nisha@flickrcc
visualplun.ch@flickrcc
照顧花園...
優雅吃飯
久坐不累
Katerina Kamprani - The Uncomfortable project All rights reserved
Katerina Kamprani - The Uncomfortable project All rights reserved
nisha@flickrcc
visualplun.ch@flickrcc
照顧花園...
優雅吃飯
久坐不累
Katerina Kamprani - The Uncomfortable project All rights reserved
Katerina Kamprani - The Uncomfortable project All rights reserved
Katerina Kamprani - The Uncomfortable project All rights reserved
nisha@flickrcc
visualplun.ch@flickrcc
照顧花園...
優雅吃飯
久坐不累
Katerina Kamprani - The Uncomfortable project All rights reserved
Katerina Kamprani - The Uncomfortable project All rights reserved
Katerina Kamprani - The Uncomfortable project All rights reserved
花惹發?
Arturo J. Paniagua@flickrcc
nisha@flickrcc
visualplun.ch@flickrcc
Katerina Kamprani - The Uncomfortable project All rights reserved
Deliver ?
Yuri Samoilov@flickrcc
things customers want to use
How do you know you’re delivering values?
IDEAS
CODEDATA
build
lear
n
measure
VALUE
Katerina Kamprani - The Uncomfortable project All rights reserved
WASTE visualplun.ch@flickrcc
VALUE
Katerina Kamprani - The Uncomfortable project All rights reserved
WASTE visualplun.ch@flickr visualplun.ch@flickrcc
In the Good(?) Old Days…..
Development
Testing
Testing
Release Go/No-go
1 week before D day
Release Go/No-go
Release Day
Release Day
wolfgangfoto@flickr
Development Testing
Release Go/No-go
Release Day
wolfgangfoto@flickr
Why Change ?
VALUE
Katerina Kamprani - The Uncomfortable project All rights reserved
WASTE visualplun.ch@flickr
Speed
Small Batch
Easy toIdentifyIssues
Check&
Revise
Quality
Build the Right Thing
Build the Thing Right
Good Working Code
Lots Testing
Lots Auto Testing
Eliminate Human
Involvement
Continuous Delivery
From District 9 movie
COMMIT TO PRODUCTIONWITHOUT HUMAN INTERVENTION
Results
CD Adoption Rate
Months
Multiplier
Production Pushes
APAC Production Pushes
PO: 好開心!!設計的 features可以很快的上 production 得到 users 的反應
SE: release 成本變小,incidents 沒有變多
Dev: 開始習慣先寫測試。測試變得更重要了,因為如果沒測好,就直奔production了
QA: Incremental changes rock!
Dan Goodwin@flickrcc
IDEAS
CODEDATA
build
lear
nmeasure
Lessons Learned
CD is Simple
Version Control
NASA's Marshall Space Flight Center@flickrcc
Certify
Build
Build
Certify
Version Control
NASA's Marshall Space Flight Center@flickrcc
Certify
Build
Build
Certify
Version Control
NASA's Marshall Space Flight Center@flickrcc
Certify
Build
Build
Certify
Version Control
NASA's Marshall Space Flight Center@flickrcc
Certify
Build
Build
Certify
Not Cheap Nor Easy
We Thought
William Warby@flickrcc
In Reality ….
Tools v.s. People
sean@flickrcc
hakanforss.wordpress.com
● tools will never be perfect ● can implement CD with imperfect tools
hakanforss.wordpress.com
● Mindset Change● Culture Change● CD is the Norm● All-in
How
Executive Mandate
Lost Planet Airmen (1951)
NO CD, NO LAUNCH, NO FOOLING
sean@flickrcc Nick Royer@flickrcc
5 Stages of Adopting CD
5 Stage of Grief
Denial
Arturo J. Paniagua@flickrcc
AngerDavid K@flickrcc
Bargaining
mark engelbrecht@flickrcc
Depression
marc latermouille@flickrcc
Acceptance
Denial
Arturo J. Paniagua@flickrcc
AngerDavid K@flickrcc
Bargaining
mark engelbrecht@flickrcc
Depression
marc latermouille@flickrcc
Acceptance
5 Mental Stages of Adopting CD
Denial 否認
蛤?難道你的老闆的老闆的老闆... x N不是 Marrisa 嗎?
全面CD ? 怎麼可能!應該不包括我們吧?
別鬧了!!Projects 都作不完了,一堆 features 等著要上...都這麼忙了,還來搞這個!!
Anger 憤怒
Arturo J. Paniagua@flickrcc
Bargaining 討價還價
好啦好啦,我們會作啦!不過手上這個 project 超重要,沒有人力時間弄 CD,可不可以幫這個 project 申請 exception ?
David K@flickrcc
持續洗腦
Depression 沮喪
kris gabbard@flickrcc
● fear● misunderstanding● doubt
Version Control
NASA's Marshall Space Flight Center@flickrcc
Certify
Build
Build
Certify
uncle bob@Ruining your Test Automation Strategy.
Nate Steiner@flickrcc
天啊! 要怎麼把這 30K 隻manual tests 改成自動化測試程式?
Well, you don’t
Acceptance 接受
● No manual testings● No manual commands in production● Not just an engineering project
marc latermouille@flickrcc
Aggressive Audacious Goals by Phases
Git in 30
1800 repos
30 D
Build Farms in 30
130 Farms 4 Centralized Farms
30 D
40K Jobs65K Builds / days
CD in 2
Scale & Diversity
cellanr@flickrcc
Single Way
Common Terminology
Common Principles
Tools & Services
Teams to Figure Out
Common Terminology
■ Package● compiled software, scripts, configuration or other data● smallest granularity of deployment● versioned
Common Terminology
■ Image● a set of packages● determines the functionality a
node will have
■ Component● runtime entity● defined by an image
Common Terminology
■ System● one or more components ● certified, launched, and operated together.
Common Terminology
Principles
■ Repeatable & Reproducible ● Versioned● Immutable Artifacts
■ Traceable● Artifact -> all the changes made
■ Version Control● source code, launch scripts and configurations, test code, test data.
Version Control
Package Component System
Package
Certify
ArtifactStorages
Build Build
PackagePackagePackage
Image
Deploy
Certify
Tagger
Build
Prod
PC1
PPC2
P
Manifest
DeployCertify
NASA's Marshall Space Flight Center@flickrcc
ArtifactStorages
Tools & Services■ Build & Workflow
● Jenkins ■ Artifact Storages
● Proprietary ● Artifactory
■ Tagger● Proprietary
■ Visualizer● Proprietary
■ Software Delivery ● Proprietary● Chef
■ End-2-End examples■ STOP development and Do it■ Use Senior Engineers■ Pipeline Demos and Reviews
Other Tactics
Results Recap
■ 7X More Production Pushes ■ Quality
Test
able
Cod
e
Ref
acto
ring
CD
Dec
oupl
ed
Des
ign
Prin
cipl
es
Dat
a D
riven
TDD
BD
D
Ope
ratio
nal
■ First Domino ● Product Excellence● Engineering Excellence
……
…..
etc
After Adopting CD
Before Adopting CD
回不去了!誰想回去啊?
Summary
visualplun.ch@flickr
Jesse! S?@flickrcc
Nathan E Photography@flickrcc
sean@flickrcc
hakanforss.wordpress.com