successful testing continuous delivery - coding serbia 2013

39
codecentric Nederland BV @mieldonkers SUCCESFUL TESTING THE CONTINUOUS DELIVERY PROCESS

Upload: miel-donkers

Post on 08-May-2015

288 views

Category:

Technology


1 download

TRANSCRIPT

Page 1: Successful testing continuous delivery - coding serbia 2013

codecentric Nederland BV

@mieldonkers

SUCCESFUL TESTING THE CONTINUOUS DELIVERY PROCESS

Page 2: Successful testing continuous delivery - coding serbia 2013

codecentric Nederland BV

INTRODUCTION

Miel Donkers Developer @mieldonkers

Page 3: Successful testing continuous delivery - coding serbia 2013

codecentric Nederland BV

• As a tester, do you need to wait for an acceptable release from developers?

• Loose time doing all repetitive manual stuff? • Maintain different environments and software versions manually,

over-and-over-and-over…

• Didn't we fix this issue three months ago? •  Is this really release 1.6.5?

All sounds familiar?

TYPICAL

Page 4: Successful testing continuous delivery - coding serbia 2013

codecentric Nederland BV

WHY CONTINUOUS DELIVERY

Page 5: Successful testing continuous delivery - coding serbia 2013

codecentric Nederland BV

Page 6: Successful testing continuous delivery - coding serbia 2013

codecentric Nederland BV

WHAT IS CONTINUOUS DELIVERY

•  Continuous Delivery pipeline

•  Automation

•  Manual control

•  Feedback

Page 7: Successful testing continuous delivery - coding serbia 2013

codecentric Nederland BV

GOALS

•  Predictability

•  Quality

•  Speed

Page 8: Successful testing continuous delivery - coding serbia 2013

codecentric Nederland BV

WHAT ARE THE MAIN STEPS OF CONTINUOUS DELIVERY?

•  Build – compile, unit test, version, package

•  Quality – metrics, documentation

•  Test – acceptance-, regression- and performance tests

•  Provision environments – deployment to test- and staging environment

•  Production – green/blue deployment to production

Page 9: Successful testing continuous delivery - coding serbia 2013

codecentric Nederland BV

PUSH VS. PULL

Page 10: Successful testing continuous delivery - coding serbia 2013

codecentric Nederland BV

STOP THE LINE!

Page 11: Successful testing continuous delivery - coding serbia 2013

codecentric Nederland BV

Prod

ROLE OF THE TESTER

TEST

Local Build

Integration server

Release build

DEV

Acc

Test

Users

DEV DEV

Page 12: Successful testing continuous delivery - coding serbia 2013

codecentric Nederland BV

ANATOMY OF A DEPLOYMENT PIPELINE

Release

•  Every change results in a trigger of the deployment process •  Software is build once and only once

•  The same deployment process for every environment •  Deployment in production-like environments

Commit Automated Acceptance Test

Automated Capacity Test

Manual Testing

Fast Feedback

Trust in Application Stability

Done

Page 13: Successful testing continuous delivery - coding serbia 2013

codecentric Nederland BV

COST OF TESTING

COST

FEATURES BUILT

Page 14: Successful testing continuous delivery - coding serbia 2013

codecentric Nederland BV

ASK YOURSELF

•  Which parts are tested how?

•  What remains to be tested?

Page 15: Successful testing continuous delivery - coding serbia 2013

codecentric Nederland BV

APPLICATION STACK

Apps / Services / Components

Application Configuration

Middleware Middleware

Configuration

Operating System

OS Configuration

Hardware

TESTING

Page 16: Successful testing continuous delivery - coding serbia 2013

codecentric Nederland BV

Page 17: Successful testing continuous delivery - coding serbia 2013

codecentric Nederland BV

WHAT DO WE TEST WHERE AND WHEN?

•  Automation is key

•  Do what we do best

•  Team effort

Page 18: Successful testing continuous delivery - coding serbia 2013

codecentric Nederland BV

MANUAL TESTING

Still required: •  "Show me" •  Exploratory testing

•  User acceptance testing

Page 19: Successful testing continuous delivery - coding serbia 2013

codecentric Nederland BV

Examples

Page 20: Successful testing continuous delivery - coding serbia 2013

codecentric Nederland BV

OUR SETUP

https://github.com/...

CD Server

Test Server

User Acceptance Test Server

Production Cluster Blue / Green

CD Slave

Page 21: Successful testing continuous delivery - coding serbia 2013

codecentric Nederland BV

EXAMPLE – DELIVERY PIPELINE 1/2

Push to Git Repo 1

Provision-Deploy Acceptance Commit

Trigger CD Pipeline 2

Build Source Code 3

Run Unit Tests 4

Start EC2 Image 1

Provisioning 2

DB Create Update 3

Deploy WAR 4

Start Tomcat 5

Start Selenium 1

Acceptance Test

Run AATs 2

Version Git Repo 6

Release WAR 7

Run Code Analyse 5

Page 22: Successful testing continuous delivery - coding serbia 2013

codecentric Nederland BV

EXAMPLE – DELIVERY PIPELINE 2/2

Run JMeter 1

Provision-Deploy UAT

Start EC2 Image 2

Provisioning 3

DB Create Update 4

Deploy WAR 5

Start Tomcat 6

Performance Test

Select Version 1

Provision-Deploy PRODBLUE/GREEN

Start EC2 Image 2

Provisioning 3

DB Update 4

Deploy WAR 5

Start Tomcat 6

Select Version 1

Page 23: Successful testing continuous delivery - coding serbia 2013

codecentric Nederland BV

STEPS VISUALIZED

Page 24: Successful testing continuous delivery - coding serbia 2013

codecentric Nederland BV

Page 25: Successful testing continuous delivery - coding serbia 2013

codecentric Nederland BV

Page 26: Successful testing continuous delivery - coding serbia 2013

codecentric Nederland BV

EXAMPLE

§ Elke stage een slide.

Page 27: Successful testing continuous delivery - coding serbia 2013

codecentric Nederland BV

ANATOMY IN DETAIL

Source Code

Commit Stage

Compile Commit Tests

Assemble Code Analysis

Artifact Repository

Acceptance Stage

Configure Environment Deploy Binaries

Smoke Test Acceptance Tests

UAT Configure Environment

Deploy Binaries Smoke Test

Capacity Stage Configure Environment

Deploy Binaries Smoke Test

Run Capacity Tests

Production Configure Environment

Deploy Binaries Smoke Test

Env.&App Config

Env.&App Config

Version Control

Testers Self-service deployments

Operations Perform push

Button releases

Developers See code metrics and test failures

reports binaries metadata binaries

reports metadata binaries

reports metadata

Page 28: Successful testing continuous delivery - coding serbia 2013

codecentric Nederland BV

Page 29: Successful testing continuous delivery - coding serbia 2013

codecentric Nederland BV

Page 30: Successful testing continuous delivery - coding serbia 2013

codecentric Nederland BV

Page 31: Successful testing continuous delivery - coding serbia 2013

codecentric Nederland BV

TOOLING THAT CAN HELP YOU

#!/bin/sh!

Page 32: Successful testing continuous delivery - coding serbia 2013

codecentric Nederland BV Angelo Storari of Ancona, Italy

Page 33: Successful testing continuous delivery - coding serbia 2013

codecentric Nederland BV

HIGHLIGHTS

•  Fast Feedback & Dashboards •  Manual testing is (still) necessary

•  But try to automate as much as possible •  Learn from each other

Page 34: Successful testing continuous delivery - coding serbia 2013

codecentric Nederland BV

THANK YOU FOR YOUR ATTENTION – ANY QUESTIONS?

Page 35: Successful testing continuous delivery - coding serbia 2013

codecentric Nederland BV

[email protected]

@mieldonkers

See our website for more information: http://www.codecentric.nl/portfolio/continuous-delivery/

CONTACT INFORMATION

Page 36: Successful testing continuous delivery - coding serbia 2013

codecentric Nederland BV

Brian Marick

TEST QUADRANT

Page 37: Successful testing continuous delivery - coding serbia 2013

codecentric Nederland BV

CONTINUOUS IMPROVEMENT CYCLE

Requires: Discipline & Responsibility

Page 38: Successful testing continuous delivery - coding serbia 2013

codecentric Nederland BV

WHAT ARE THE KEY BENEFITS OF CONTINUOUS DELIVERY?

•  Low-risk releases

•  Faster return on investment in software projects

•  Improvement of competitiveness and responsiveness

•  Quality improvement of new software versions

Page 39: Successful testing continuous delivery - coding serbia 2013

codecentric Nederland BV

EXAMPLE – TECHNOLOGY STACK

•  Amazon EC2 (Virtualization, Cloud) •  Jenkins (CI Server) •  Git/Github (Version Control) •  Sonar (Code Quality) •  jUnit (Unit Tests) •  jBehave (Acceptancetests) •  Selenium (UI Tests)

•  Puppet (Provisioning) •  Tomcat (Application Server) •  Artifactory (Artifact Repository) •  Maven (Build Tool) •  jMeter (Performance Tests) •  AppDynamics (Performance Tests) •  Liquibase (DB Versioning)