CONTINUOUS DELIVERYALEXEY TURCHANIKOV, INFRASCALE

oleksiy. turchanikov@gmail .com

#1 WHAT IS THIS?#2 HOW WE ACHIEVED THIS#3 CONSEQUENCES#4 DETAILS

WHAT IS THIS?

PIPELINEbuild unit test deploy to

feature env UI/func test manual QA merge to mainline

build unit test deploy to integration UI/func test manual QA

deploy to staging manual QA (optional) deploy to production manual QA

Story on production as a part of DoD 5 days per story on average 1-3 releases per day 2-4 hours to release hotfix 15 isolated QA environments

feature

integration

staging production

GOALSLower risksVisible progress, peace of mindFast feedback loop

INGREDIENTSFast feedback for every changeAutomationCollaborationContinuous improvement

HOW WE ACHIEVED THIS

BEGINNING

Ukraine

Backend

Frontend

Desktop

Mobile

QA / Regression

Sysadmins

Partners Afg

hani

stan

India10 h

ours

, lan

guag

eProduct Owner

USA

BOTTLENECK: COMPONENT TEAMS

CHANGE: FEATURE TEAM

BOTTLENECK: REGRESSION TESTING

CHANGE: SMALL STORIES, AVOID REGRESSION

BOTTLENECK: DEPLOYMENT VIA IT OPS

CHANGE: SELF-SERVICE DEPLOYMENT, EXTERNAL CONFIGURATION

BOTTLENECK: BUGS IN TRUNK

CHANGE: FEATURE BRANCHES

BOTTLENECK: QA ENVIRONMENT

CHANGE: VIRTUAL ENVIRONMENTS, CM AUTOMATION

BOTTLENECK: INFRASTRUCTURE

CHANGE: DEVOPS

BOTTLENECK: QA

CHANGE: AUTOMATION

UPCOMING AND ONGOINGDevOps: DSC-based

configurationProduction deployments by QABuild-Test-Deploy-Test speedupLearning Management System

DETAILS

FEATURE PIPELINE: BUILD, UNIT TEST

build unit test deploy to feature env UI/func test manual QA merge to

mainline

Full pipeline for every commitParallelizedTests:

Only fast testsUnit test coverage: <25%

Fail on metric changesOutput: custom package file

FEATURE PIPELINE: DEPLOY

build unit test deploy to feature env UI/func test manual QA merge to

mainline

Isolated environments, VM based on template One-step deployment of whole stack Includes DB migrations, IIS, MSMQ… Same package for Integration, Staging,

Production Environment-specific configuration in a single

file under source control + conventions PowerShell as a glue

FEATURE PIPELINE: UI / FUNCTIONAL TESTS

build unit test deploy to feature env UI/func test manual QA merge to

mainline

Test data auto-created Selenium & WebDriver for Web White for PC Priorities:

OS-dependent functions (time saver) Common workflows (safety net) New functionality

Optimizations: SSDs Run in parallel Headless browsers – no luck

Test execution (and failure!) recorded as video

FEATURE PIPELINE: MANUAL QA

build unit test deploy to feature env UI/func test manual QA merge to

mainline

Feature-levelScope defined with developerImmediately available for

testing

FEATURE PIPELINE: STAGING, PRODUCTION

One-by-oneSame packagePartial deployment – tier,

application or web serviceDeployment by developer or QARisk-based manual QA

deploy to staging manual QA (optional) deploy to production manual QA

staging production

PIPELINE ON BUILD SERVER

Jenkins

TeamCity

ORGANIZATIONFeature teamsOffset sprint schedulesArchitecture committeeCross-training, rotation, Lunch

& Learn, L3 support “shifts”

CONTROL CHART

Rally

Jira

CONSEQUENCES

PRODUCT MANAGEMENTGood: Clear DoD for business Faster integration No “surges” and overtimes

Watch for: Keep Sales, Support up to date Roadmap is a must Global (e.g. architectural) changes

SALES, MARKETINGFast response to changeVirtual “Big Bang” releases

SUPPORTFast turnaround for fixes and

feature requests2-4 hours to release hotfixNeed continuous learning

QUESTIONS