continuous delivery - anztb · continuous delivery & the changing role of the tester dr kelvin...
Post on 28-Jul-2020
3 Views
Preview:
TRANSCRIPT
21/05/15
1
Continuous Delivery &
The Changing Role of the Tester
Dr Kelvin Ross ANZTB 2015, Auckland 22 May 2015
2
Disruptive Technology
21/05/15
2
3
Disruptive Technology
4
Disruptive Technology
21/05/15
3
5
Disruptive Technology
CLOUD SOCIAL
BIG DATA MOBILE
DISRUPTIVE FORCES
21/05/15
4
Biggest Disruptors The Last 5 Years? Agile
Offshoring Virtualisation
8
Disruptive Forces
Customer Testing
DevOps
Big Data
21/05/15
5
10
Startup Corporate
“You can't just ask customers what they want and then try to give that to them. By the time you get it built, they'll want something new.”
Steve Jobs
21/05/15
6
12
14
Scientific Method for Customer Experiments
to Guide Software Development
21/05/15
7
15
FAIL FAST
16
21/05/15
8
17
3 – 6 Months
18
1 – 2 weeks
21/05/15
9
19
“You can't just ask customers what they want and then try to give that to them. By the time you get it built, they'll want something new.”
Steve Jobs
SPEED is the new black
20
21/05/15
10
21
DEV OPS
21/05/15
11
24
No Silos
25
CI CD CD
Continuous Integration
Continuous Deployment
Continuous Delivery
21/05/15
12
Source: Principles and Practices in Continuous Deployment Mike Brittain,Engineering Director, Etsy @mikebrittain mikebrittain.com/talks
Rapid Releases
Source: http://readwrite.com/2014/01/01/three-reasons-your-startup-needs-devops-or-else
21/05/15
13
CD Platform
Source Control
Build System
Pipeline Management
& Visualization
Artifact/Package
Repository
Automated Configuration Management
(IaC)
Test infrastructure
Automated Deployment
System
Source: Continuous Delivery at Scale John Esser, Director Engineering Productivity, Ancestry.com ALM Forum Seattle, 2014
Git Flow
Version & Configuration Management
21/05/15
14
30
Infrastructure
Application Code
Static Data
Test Data Version Everything
31
Data Versioning
Infrastructure/Platform Versioning
21/05/15
15
32
Cloud Computing
New Strategies • Duplicate test
environments
• Copy of production
• Grid based test automation
• Blue/Green Deployments
• Resilience testing
33
21/05/15
16
34
Elastic Computing
Time
Which is better?
Source: Principles and Practices in Continuous Deployment Mike Brittain,Engineering Director, Etsy @mikebrittain mikebrittain.com/talks
Release Pipeline
21/05/15
17
36
UNIT
INTEGRATION
SYSTEM
• Fast • Isolated • Aligned with Code
• Slower • UI focused • Later in the processes
• Focused on linkages • Harnesses, Mocks and
Stubs
Testing Pyramid – Right Way
Source: Steve Neely, sneely@rallydev.com @stenvoon http://bit.ly/cdpaper ALM Forum Seattle, 2014
Release Pipeline
21/05/15
18
Source: Advanced Topics in Continuous Deployment Mike Brittain,Engineering Director, Etsy @mikebrittain mikebrittain.com/talks
Deployment Gating
Source: Principles and Practices in Continuous Deployment Mike Brittain,Engineering Director, Etsy @mikebrittain mikebrittain.com/talks
Production Testing
21/05/15
19
40
Source: Principles and Practices in Continuous Deployment Mike Brittain,Engineering Director, Etsy @mikebrittain mikebrittain.com/talks
Failure Monitoring
41
FAIL FAST
21/05/15
20
42
FAIL FAST MTBF ✗
43
FAIL FAST MTTR ✓
21/05/15
21
45
Customer Testing
Secured $80m in VC funding
Testing In The Wild Applause
21/05/15
22
47
Elastic Resourcing
Time
Which is better?
48
Customer Testing
21/05/15
23
49
A / B Tests
Source: Advanced Topics in Continuous Deployment Mike Brittain,Engineering Director, Etsy @mikebrittain mikebrittain.com/talks E.g.“Google Analytics Content Experiments”
50
Feature Toggles
21/05/15
24
51
Source: Advanced Topics in Continuous Deployment Mike Brittain,Engineering Director, Etsy @mikebrittain mikebrittain.com/talks
Feature Toggles
52
Source: Advanced Topics in Continuous Deployment Mike Brittain,Engineering Director, Etsy @mikebrittain mikebrittain.com/talks
Feature Toggles Progressive Delivery
To Minimise Risk
21/05/15
25
53
Can We Skip Some Pre-Prod
Testing?
Early Customer Testing
55
BIG DATA Volume Velocity Variety
21/05/15
26
56
How To Test Prediction
57
Big Data Challenges Staging test data sets
Test data privacy Variation of test data sets Production data variability
Controllability & Observability
Algorithm Evaluation Heuristic Testing
Exploratory Testing
A/B Testing Customer Sampling &
Monitoring
21/05/15
27
New QA Skills
CI / CD Process
Data Analytics
Test Environment Provisioning
Production Monitoring Feature Architecture & Deployment Planning
Technical Influence
Automation
Experimentation
Canary Releasing
21/05/15
28
Blue Green Deployments
Feature Toggles
21/05/15
29
Feature Flippers
64
A / B Tests
Source: Advanced Topics in Continuous Deployment Mike Brittain,Engineering Director, Etsy @mikebrittain mikebrittain.com/talks E.g.“Google Analytics Content Experiments”
21/05/15
30
Dark Launch
21/05/15
31
68 In 5 Years???
21/05/15
32
69
Skill Changes
70
DevOps Teams Embedded, Self Organising, Functional
Business Tester Automated Tester DevOps Tester
Non-Technical Functional Tester
21/05/15
33
71
Reduced Test Management
Coordinate Resourcing Practice Support
Consultants Coaches Governance
Test Managers Test Leads
72
• Crowd Test Coordination • Customer Test Coordination • Feature Deployment Planning • User Experience and Failure
Monitoring
21/05/15
34
73
Q&A
top related