scaling test first for the enterprise

Scaling Test First for the Enterprise

February 24, 2016

#TestFirst @KevinDunneQARobust testing platform purpose-built to help agile teams create

higher quality software

ABOUT QASYMPHONY

#TestFirst @KevinDunneQA

Creating Visibility and Speed with Automation and Test ManagementThursday, March 24th at 2pm ET

– Guest Speaker: Joe Colantonio

Joe Colantonio is a test automation architect for a large Fortune 100 company with more than 15 + years of test automation and performance testing experience. Joe is also the founder of the popular test automation blog, joecolantonio.com and the host of TestTalks, a podcast dedicated to all things test automation related. He is also the author of the highly rated book The UFT API Testing Manifesto – A step-by-step hands-on testing guide for the masses.

UPCOMING WEBINAR


QUALITY JAM 2016 – APRIL 13TH IN ATLANTA

Learn more at qualityjam.com

Speakers Include:

• Scott Berkun, Best Selling Author

• Keith Klain, Software Testing Thought Leader

• Michael Cooper, Chief Quality Officer, Healthcare IT Leaders

• Adam Satterfield, VP of QA, BetterCloud


• Early bird pricing of $100. Pricing will go up to $200 on March 1st.

• For a limited time, you can get an additional $20 off using the promo code QASVIP when you check out.

• For the first 5 people who send an email to [email protected], you will get a complimentary pass to the event.

QUALITY JAM 2016

mailto:[email protected]


• This webinar will be recorded and available on demand next week

• If you have a question, type it into the Q&A module on the left. We will answer as many as we can towards the end of the webinar

• Join the conversation on Twitter using the hash tag #TestFirst

• At the end of the webinar, you will be asked to take a short survey

HOUSEKEEPING


Kevin Dunne, VP of Strategy and Business Development, QASymphony

Kevin Dunne is the VP of Strategy and Business Development at QASymphony. In this role, Kevin focuses on innovation, thought leadership and new global business opportunities. As one of the first employees at QASymphony, Kevin has seen many facets of the business working in sales, customer support, marketing, and product management.

Kevin comes to QASymphony from Deloitte, where he managed testing on large government and Fortune 500 engagements delivering ERP implementations and custom software development. Kevin holds a Bachelor of Science degree from Vanderbilt University.

OUR PRESENTER


What is TDD?

Why are we hearing about it now?

How is it different?

Why use TDD?

How will it benefit my organization?

What drawbacks if any exist?

How to Implement TDD?

How do I prepare my people?

What’s the new process?

Do I need new tools?

AGENDA

September 25, 2015

Introduction to QASymphony for [INSERT COMPANY

NAME]

What is TDD?


Past development cycles often modeled the Rational Unified Process:

Source: http://www.psa-software.com/_img/_knowledge_center/rup.jpg

WHERE WE CAME FROM

http://www.psa-software.com/_img/_knowledge_center/rup.jpg


We would have obviously chosen a more efficient process, but we were constrained by many limitations, including:

WHY WE CHOSE IT

Environment Creation

Code MergesOn-Premise Prevalence

Desktop Focus

Lack of Collaboration

Off-Shore Development


Many of our priori limitations have been replaced, based on macro trends around technology and industry:

WHAT’S CHANGED

• Containers have simplified the process dramaticallyEnvironment Creation

• Git has replaced Subversion as the industry standardCode Merges

• Cloud adoption is at an all time high, increased securityOn-Premise Prevalence

• Prevalence of Web, Mobile, Internet of thingsDesktop Focus

• Increase in teamwork, chat and collaboration technologyLack of Collaboration

• Shifts towards rural sourcing, onshoring of laborOff-Shore Development


Now that we have freed ourselves of past limitations, the process has been shifted to one that aligns more with our needs:

Traditional Approach

Test-First Approach

HOW THE PROCESS HAS ADAPTED

Design Requirements Code Test Deploy

Design (Automated)

TestCode Refactor Deploy


Test-First methodologies were coined “Test Driven Development”. Less technically focused versions called Acceptance Test Driven Development (ATDD) and Behavior Driven Development (BDD) also emerged:

TDD VS ATDD VS BDD

Test Driven Development (TDD)

Behavior Driven Development BDD

Acceptance Test Driven Development

(ATDD)

Unit Test Driven Development

(“Technical TDD”)


ATDD and BDD are similar in that they both try to make TDD more accessible to business users. The major functional difference comes down to how the tests are structured:

"I think this definition leaves out a key piece, we are focusing on collaboration and learning. Having worked on a project that was using 'ATDD', in 2005 I think, we had the same goals then as BDD

without the Given When Then language.“ - Wes Williams

WHAT’S THE DIFFERENCE?


ATDD/BDD offer benefits over more Unit Focused/Technical TDD, but also has its drawbacks:

PROS AND CONS

Pros

• Increased understanding of tests from business stakeholders

• Increased collaboration early in the cycle

• More focus on customer and business needs

• Higher involvement of business in development and quality

Cons

• Addition of more tooling in the development/delivery chain

• Greater time spent defining tests and specifications

• Demands stronger contributors in requirements, dev, test

• Often increases the automation needs in an organization

September 25, 2015


NAME]

Benefits of TDD


Test Driven Development brings several major benefits to organization, most notably:

1. Move Testing Up Front – prevents having to rush testing at the end of the cycle

2. Bake in Automation from Day 1 – protects against getting behind with test and automation coverage

3. Build More Testable Software – requires developers to think about testability, and create more robust software

4. Push to Customers When Ready – allows you to push software to customers just in time, as it is developed

HOW CAN TDD HELP US?


Moving Testing Up Front removes the risk of having to make compromises at the end of the cycle on quality or on-time delivery:

Traditional Development Timeline

Ends on: Day 1 Day 3 Day 14 Day 20 Day 21

There is risk in this process that any process, typically Code, will run over and either squeeze development, or push release dates. TDD removes it!

MOVE TESTING UP FRONT

Design Requirements Code Test Deploy


The cost of resolving a defect is far greater when found later in the cycle. TDD should help you to find defects as they are created, so they can be resolved quickly and at a lower cost:

Source: http://www.isixsigma.com/industries/software-it/defect-prevention-reducing-costs-and-enhancing-quality/

DEFECT COSTS INCREASE AS CODE MATURES

http://www.isixsigma.com/industries/software-it/defect-prevention-reducing-costs-and-enhancing-quality/


In traditional development, automation is often built after the code is developed, which has significant limitations:

• Test development is more costly, since we cannot access the code to make it more testable (more details to come)

• Tests are slower and more brittle, with higher levels of maintenance, if we can only access the UI

• Test coverage is incomplete, as we must chose strategically where to build out automation coverage

Moving to TDD flips the process and forces developers to write code to satisfy tests, increasing automation coverage, speed, and reducing cost

BAKE IN AUTOMATION UP FRONT


Moving towards BDD will force your developers to build an application that can be tested well at the Unit, Integration, and UI levels:

Source: http://zeroturnaround.com/wp-content/uploads/2015/12/PUZZLE-1-min.png

BUILD FOR TESTABILITY

http://zeroturnaround.com/wp-content/uploads/2015/12/PUZZLE-1-min.png


Moving towards BDD will also demand a more complete testing strategy focused on more than just UI testing:

BUILDING A COMPLETE TESTING STRATEGY

UI Tests

Integration Tests

Unit

Tests

Unit

Tests

Integration Tests

Unit Tests


TDD paired with continuous delivery will allow you to push features as they become ready, if you’d like to:

Old Way

New Way

PUSH FEATURES WHEN THEY ARE COMPLETE

Code Feature A

Code Feature B

Code Feature C

Test Deploy

Wait

Wait

Code Feature A

Code Feature B

Code Feature C

WriteTests

Deploy

Deploy

Deploy

September 25, 2015


NAME]

Shifting to TDD


A change in organizations is never successful without a complete strategy:

Source: http://www.consultia.co/wp-content/uploads/2015/07/security-technologies-processes.png

RECIPE FOR SUCCESSFUL CHANGE

http://www.consultia.co/wp-content/uploads/2015/07/security-technologies-processes.png


People are the largest driver and inhibitor in change, so there are a number of key changes needed to be made in human capital to move to TDD:

PEOPLE RELATED CHANGES

Automation Talent Gap Feature Centric Culture

Siloed Communication Style


Building automation competence is not simple, but it can be done with proper investment:

BUILDING AUTOMATION COMPETENCE

•Train Up•Train your existing employees around automation and new frameworks•Low cost, long time to benefit, scalable

•Hire In•Hire new employees with automation skill to augment your current team•Medium cost, medium time to benefit, scalable

•Outsource•Outsource your automation efforts to a team •High cost, low time to benefit, not scalable


Developers need to be educated that their time is not solely spent developing feature code:

Current Time Spent Future Time Spent

SHIFTING FROM A FEATURE CENTRIC CULTURE

•Feature Code

•Feature Code•Test code•Refactoring


Proper BDD is implemented with collaboration between development, test, and business/product – coined the “3 amigos”:

Source: http://image.slidesharecdn.com/bdd-in-action-140408000826-phpapp01/95/bdd-in-action-principles-practices-and-realworld-application-8-638.jpg?cb=1396916226

CREATING THE 3 AMIGOS CULTURE

http://image.slidesharecdn.com/bdd-in-action-140408000826-phpapp01/95/bdd-in-action-principles-practices-and-realworld-application-8-638.jpg?cb=1396916226




Process can be the source of the largest changes, as a process that has been in place for years may need to be extensively overhauled:

PROCESS RELATED CHANGES

Requirements Specification

Requiring Tests Before Code

Frequent Refactoring


BDD and ATDD requirements are typically written in a Given, When, Then format that differs from traditional requirements:

Source: https://github.com/cucumber/cucumber/wiki/Feature-Introduction

SHIFTING TO GIVEN, WHEN, THEN

https://github.com/cucumber/cucumber/wiki/Feature-Introduction


Developers are always eager to code – requiring them to write tests (or wait for tests to be written) before coding can be a paradigm shift

Source: http://www.tnooz.com/wp-content/uploads/2011/11/angry-computer.jpg

BUILDING TESTS BEFORE CODE

http://www.tnooz.com/wp-content/uploads/2011/11/angry-computer.jpg


Properly implemented TDD, BDD, or ATDD should provide additional focus on refactoring and creating more elegant solutions and manageable code:

Source: http://hanwax.github.io/assets/tdd_flow.png

FOCUSING ON REFACTORING

http://hanwax.github.io/assets/tdd_flow.png


Tools are often the first place organizations look for change, but they likely should be approached after process and technology:

TECHNOLOGY RELATED CHANGES

Automation Framework Testability / Interfaces

Monitoring


May teams struggle to pick an automation framework, but the reality is most frameworks are similar and differentiate in a few key areas:

CHOOSING AN AUTOMATION FRAMEWORK

•Dev Style•Is it more focused on BDD, TDD, or ATDD?

•Scalability•Does it have features to support large enterprises, as well as small teams?

•Language•Does it support dev centric languages (i.e. Java) or easier to use languages (i.e. Ruby)

•Workflow•Does it support the concept of features, or stories?


Moving to BDD process moves testing activity from solely UI based to focusing more testing at the services/API layer:

BUILDING TESTABLE SOFTWARE


For cloud and web based applications, it is likely wise to implement monitoring to guard against any potential bugs leaking into production while the TDD process is getting started:

MONITORING

What to monitor:• Scalability• Performance• Usability• Localization• …the list goes on


Kevin [email protected]

Twitter: @KevinDunneQA

QUESTIONS?



Thank You

February 24, 2016