The journey from agile towards DevOps

April 2, 2015Kwo Ding, Hardy Seinhorst - RabobankCarlo Cadet – Perfecto Mobile

Faster and more stable delivery against reduced costs

Speakers

Hardy Seinhorst Kwo Ding

• Over 20 years of management and technical experience in IT.

• Experience in application development, running and optimizing data centers, IT strategy and information management.

• Rabobank’s program manager for implementing new digital engagement.

• 8+ years of software testing experience.• Experience with onsite and offshore team

management for both waterfall and agile methods.

• Specialized in chain integration (end-to-end) testing, continuous integration & delivery and test automation.

Test ManagerDigital Program Manager

Introducing the digital portfolioCurrent online presence

- Website for desktop and for mobile

- Native app for iOS, Android and Windows Phone

Development

- 1 new hybrid environment that will replace all native apps and the

website for desktop created by 10 scrum teams

- Total of 18 scrum teams, 200 FTE

- Core of a team: Development, PO, QA, Operations

- Extended team: BA, Appl Eng, project mngt

Platform #users # login / month

Desktop 3,6M 18,6M

App iOS 1,2M 30M

App Android 1,2M 30M

App Windows Phone 0,02M 0,3M

The waterfall methodology of software development; success rate as well as fit for use are low

What the Business wants from IT:

The fit for use is also low2:

Project size FTETime

(months)Success

rate

<750K€ 6 6 55%

750K€ to 1.5M 12 9 33%

1.5M to 3M 25 12 25%

3M to 6M 40 18 15%

6M to 10M +250 +24 8%

Over 10M +500 +36 0%

13%

often

always

7%

rarely

19%

never 45%

sometimes

And side effects occur:In reality success rate is low1:

Long cycle times

Silo mentality

Big bang delivery

Fragile process

Over-documentation

Stick to the plan seems to make sense, but…

Shorter time to marketMore relevant softwareLower cost to change

Higher qualityAlignment Business and IT

2) Source: CHAOS report 2004: Standish group

1) Source: The Standish Group International Inc. 2002

POLL:

What methodology are you using?

1. Waterfall Development

2. Agile Development

3. Continuous Integration

4. Continuous Delivery

5. DevOps

5

The journey ahead – Shrinking feedback cycles

6

We are about here in our journey

Achieving Agility Drives Serious Change

7

Complete Minimum Viable

Gradual ExposureBig Bang

Wave 1 Wave 2 Wave 3

Collaboration T-Shaping Collective Ownership/Delivery

Time Boxed “Just In Time” Continuous Delivery

Monitoring/Tools Inception Automation Testing Continuous Deployment

Coaching and Grooming Team Empowerment Initiate T-Shaping culture Integrated Team - Dev & Ops

Improved Happiness Score Pick up any task outside the

comfort zone Integrated Team - DevOps

Peak Happiness Score Shared capabilities, Pick up any task Integrated Team - DevOps

Time Boxed Visual Board in place Daily Prioritization based on

shared backlog Cycle and Lead time

Non Time Boxed/Continuous Real time usage of visual board “Just In Time” delivery Improved Cycle and Lead time

Non Time Boxed/Continuous Real time usage of visual board Lean implementation Proactive maintenance

Integrated tooling Application monitoring Start with automated testing Metrics Integration – holistic view

Full use of tooling (automated in flow of tickets)

Continuous Integration Move to fully automated tests Business Value measurement

Full use of tooling (complete use of reports as well)

Continuous Deployment Fully automated deployment Business Value measurement

Dev Ops

Peo

ple

Pro

cess

Tech

no

log

y

Measuring progress

Why test automation?

• Tests take less time to run

• Increase # test runs

• Improve accuracy and efficiency

• Increase test coverage

• Necessary for Continous Integration & Delivery (Agile, DevOps)

Test Automation

Environment

Testing

Deployment

Tool Stack

10

Cucumber UFT

Test Suite: Automated vs Manual

Automated

tests

Tests not

possible to run

manually

Manual tests

automated

Tests not worth

automating

Manual tests

Manual vs Automated

Source: Dorothy Graham

Automation Strategy

12

Technology facing

Business facingC

riticalp

roje

cts

Su

pp

ort

pro

gra

mm

ing

Functional tests

(system (integration) & chain, 200

portability testing – 10% x 10 devices)

Exploratory testing

User acceptance testing

Accessibility

Visuals (partially)

Unit testing

Unit integration testing

Non-functional testing

(performance, security, production

acceptance testing)

Regression testing is cross-category

Automated

Automated

Manual

Manual/automated

Test

Continuously

Continuous Quality(as-is)

Build

Unit Test

Check-out &

Build application

Security/Perf.

code analysis

Quality code

analysis

Check-in

Check-in code &

review

Deploy

to TST

Deploy to

TST

Smoke test

on TST

Regression

Test

System Test

Deploy

to ACC

Deploy to

ACC

Smoke test

on ACC

Acceptance

Chain Test

Performance

Test

PAT

UAT

Portability

Test

Pre-PAT

Security

Production

Schedule P-

deployment

Deploy to

PROD

PROD

verification

Automated

step

Manual step

Code review

Exploratory

test

3 week sprint

*

*

Once per Sprint

Test

Continuously

Continuous Quality(to-be)

Build

Unit Test

Check-out &

Build application

Security/Perf.

code analysis

Quality code

analysis

Check-in

Check-in code &

review

Deploy

to TST

Deploy to

TST

Smoke test

on TST

Regression

Test

System Test

Deploy

to ACC

Deploy to

ACC

Smoke test

on ACC

Acceptance

Chain Test

Performance

Test

PAT

UAT

Portability

Test

Pre-PAT

Security

Production

Schedule P-

deployment

Deploy to

PROD

PROD

verification

Automated

step

Manual step

Code review

Exploratory

test

3 week sprint

Deploy stories when done, not per sprint schedule

Recommendations

15

1. Just start, solve issues during the journey

2. Focused change

3. Work closely together with complete team, Behavior Driven Development (BDD) is a great approach to improve collaboration.

4. Perform discrete test levels