ca - take a value-driven approach to better software

Take a Value-Driven Approach to Better Software

Jonathon WrightChief Data Therapist

20th of September, 2016

2 © 2016 CA. ALL RIGHTS RESERVED.

Agenda

WHAT IS A REQUIREMENT?

WHY FOCUS ON REQUIREMENTS?

HOW ABOUT SOFTWARE ENGINEERING?

HOW ABOUT REQUIREMENTS ENGINEERING?

INTRODUCING THE REQUIREMENT-DRIVEN APPROACH

BONUS: REQUIREMENTS ECOLOGY?

1

2

3

4

5

6

What is value?


Definition: Value

Value: denotes the degree of importance of some thing or action, with the aim of determining

what actions are best to do (deontology), or to describe the significance of different actions

(axiology)..

Wikipedia

Wikipedia

Business value: expands concept of value for the firm beyond economic value (also known as

economic profit, economic value added, and shareholder value) to include other forms of value.


So what is a value stream?

Dean Leffingwell, Scaled Agile Framework (v4.0.6), 2016, (http://www.scaledagileframework.com/value-stream-level/)

The primary purpose of the value stream:

To describe Lean-Agile approaches to system development that scale to

the challenge of:

– Defining, building, and deploying solutions

These solutions require additional constructs, artefacts and coordination.

Support for complex systems (ecosystems of ecosystems)

http://www.scaledagileframework.com/value-stream-level/


How to manage the value stream?

Dean Leffingwell, Scaled Agile Framework Foundations (v4.0.6), “Value Stream level”, 2016

Identify and organize around Value Streams

Establish local governance with Value Stream roles and the Economic Framework

Frequently integrate and validate Customer solutions

What is a requirement?


Definition: Requirement

Requirement: statement which translates or expresses a need and its associated constraints and

conditions.

Requirement: singular documented physical and functional need that a particular design, product

or process must be able to perform.

ISO/IEC/IEEE 29148:2011

Wikipedia


What is a requirement?

Erik Simmons, nuCognitive, “21st Century Requirements”, 11/08/11

A requirement is a statement of:

1. What a system must do (functional requirement)

2. How well the system must do what it does (quality or performance requirement)

3. A known resource or design limitation (constraint)

More generally,

A requirement is anything that drives a design choice


The purpose of requirements


Clear

All statements are unambiguous, complete,

and concise

Common

All stakeholders share the same understanding

Coherent

All statements are consistent and form a

logical whole

The purpose of requirements is to help establish a clear, common, and

coherent understanding of what the system must accomplish.

So why focus on requirements?

Why focus on requirements?


What are the main software challenges you are facing?

0% 10% 20% 30% 40% 50% 60% 70%

Time/resources in test data compliance (PII)

Defects stemming from ambiguous requirements

Testing innefficiencies leading to higher cost

Lack of test coverage creating defects/rework

Difficulty finding the right data for a particular test

Manual testing leading to project delays

Pre-webinar survey, “Testing Imperatives in the World of Agile Development and Continuous Delivery”, Huw Price & Jonathon Wright, 28/07/15


VAGUENESS

Examples of ambiguity and incompleteness

Poor Scoping

Ambiguous Reference

Omission Acronyms

Aliasing

Sentence Structure

Dangling

Else

Incomplete

Glossary

Implicit Constraints

I.E. or E.g.?

Lexical Ambiguity

Polysemy

Passive Sentences


What makes a good requirement?

NASA, Automated Requirements Measurement Tool, 1999, http://satc.gsfc.nasa.gov/tools/arm (archive.org)

complete

consistent

correct

modifiable

ranked

traceable

unambiguous

understandable

testable

http://satc.gsfc.nasa.gov/tools/arm


What is the problem with requirements?

Though a plethora of techniques exist, most are written in ambiguous natural language

The requirements are “static” - they offer no way to derive tests directly from them…

… no way to update tests when the requirements change

– this has to be done manually

Bender RBT, 2009


What are the challenges with requirements?

1 – Bender RBT, 20092 – IT University, Copenhagen, 20013 - Hyderabad Business School, 20124 – Critical-Logic, 20145 – Standish Group’s Chaos Manifesto 2014

A system can be build around misunderstanding, so that:– At least 56% of defects stem from ambiguity in requirements

1– some

place this as high as 59%2

or even 65%3

– 64% of total defect cost3originate in the requirements analysis and

design phase – some place this as high as 80%1

– 40-50% of project costs are expended in rework4

– On average, only 69% of desired functionality is actually delivered5

How about software engineering?


What is engineering?

Erik Simmons, nuCognitive, 11/09/16 adapted from “a discussion of the method”, Billy Vaughn Koen Oxford University Press 2003

Engineering is the application of heuristics under uncertainty to cause the

best possible change within the available resources (Billy Vaughn Koen*)

A heuristic is anything that provides a plausible aid or direction in the

solution of a problem

Use of heuristics does not guarantee a solution to a problem, and

different heuristics can offer conflicting advice for any given situation

Heuristics work by exploiting the structure of an environment to simply

decision making and narrow the solution search space


How does this apply to software engineering (SDLC)?

Business Analyst Programmer TesterUser

The User Knows what they want

The Analyst specifies what that is

The Programmer writes the code

The Tester tests the program

Clarity & Vision

Simple example of lack of clarity and vision within the software development lifecycle


How about within Lean & Agile?

Dean Leffingwell, Scaled Agile Framework Foundations (v4.0.6), “Embracing Lean-Agile values”, 2016

House of Lean Agile Manifesto

LEADERSHIP

Respect

for

people

and c

ulture

Flo

w

Innovation

Rele

ntless

impro

vem

ent

VALUE

Value in the sustainably shortest lead time

That is, while there is value in the items on the right, we value the items on the left more.

We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

Individuals and interactions over processes and tools

Working software over comprehensive documentation

Customer collaboration over contract negotiation

Responding to change over following a plan

How about requirements engineering?


Definition: Requirement Engineering

ISO/IEC/IEEE 29148:2011

Wikipedia

Interdisciplinary function that mediates between the domains of the acquirer and supplier to establish and

maintain the requirements to be met by the system, software or service of interest. Requirements engineering

is concerned with discovering, eliciting, developing, analysing, determining verification methods, validating,

communicating, documenting, and managing requirements.

Refers to the process of defining, documenting and maintaining requirements to the sub-fields of

systems engineering and software engineering concerned with this process.


What is requirements engineering?

Erik Simmons, nuCognitive, 11/09/16 adapted from “a discussion of the method”, Billy Vaughn Koen, Oxford University Press 2003

Requirements engineering is the use of heuristics for discovering, documenting, and maintaining a set of requirements for a system or service

At the highest level, requirements engineering seeks to:

Define the necessary and sufficient system scope and features to enable system development at an acceptable level of risk

Enable requirements-driven architecture, design, construction, and verification activities

Provide information about system development progress and status

Provide information to organizational learning and continuous improvement efforts


Requirement Engineering Activities


Management

Maintaining the integrity and

accuracy of therequirements

Verification

Assessingrequirements

for quality

Specification

Creating thewritten

requirementsspecification

Analysis & Validation

Assessing, negotiating,and ensuringcorrectness

of requirements

Elicitation

Gatheringrequirements

fromstakeholders

Requirements Engineering (RE) consists of five parallel (not sequential) activities:

Different systems and software lifecycles take different approaches to performing these

activities. For example some lifecycles may front-load requirements work, whilst others

differ it to the time when functionality will be actively developed.


How to maintain integrity and accuracy?

ISO/IEC: 29119-3 – Annex H (informative) Test Design Specification – Test Conditions for Analysis Method

ISO/IEC: 29119-3


How to manage requirements engineering?

To Generate Test Cases

Design and provisionTest Data Virtualisation

To Manage Change in Test Cases

To Generate Automation

Tests

To Estimate Complexity

Populate Story Boards & Backlogs

To Build BetterRequirements

To Improve Existing

Test CasesDesign and provision Service Virtualization

Impact and Differences

built by

Agile Requirements

Designer

Native Support

Introducing the requirement-driven approach


DesignOps is the new delivery paradigm: from ideation to design, development and testing


Model requirements as an “Active” flowchart

A formal model that is accessible to the business who already use VISIO, BPM, etc.

Which is also a mathematically precise model of a system, so that it eliminates ambiguity and incompleteness

It can be used by testers and developers – it brings the end-user, business and IT into close alignment


Testing with the “Active” flowchart

Testers can overlay the flowchart with all the functional logic and data involved in a system

Tests can therefore be automatically derived from it


Exhaustively test a model – extract all possible routes from start -> end.

Each route / path becomes a test case

Number of routes grows exponentially with every added decision.

32 nodes + 62 edges = 1,073,741,824 possible routes

Generating test cases

2145 Paths


Coverage, in most cases, is related to how much functionality is being

covered in a test case.

Traverse the model by satisfying coverage constraints:

Optimization / Coverage

19 PathsEdge Pairs

9 PathsIn Out Edges

5 PathsEdges

3 PathsNodes


Generate test data

Automatically profile data, model it, and accurately measure its coverage

Generate rich synthetic data which provides 100% coverage

Cover every outlier, unexpected result, boundary condition and negative path

Not Ready for Testing!

CA Test Data Manager + Required data characteristics

Provision fit for purpose data anytime and every timeProvision data with or without access to production systems

Ready for Testing!


Provide full traceability with requirements

Know what needs to be re-tested and when the integrity of a system is at risk… “If I change this, what will I break?”

The impact of a change made to an individual component is identified system wide

The impact on test cases and user stories up and down a system can also be identified automatically

LIVE DEMO

https://vts.inxpo.com/Launch/Event.htm?ShowKey=34403

https://urldefense.proofpoint.com/v2/url?u=https-3A__vts.inxpo.com_Launch_Event.htm-3FShowKey-3D34403&d=DQMBaQ&c=_hRq4mqlUmqpqlyQ5hkoDXIVh6I6pxfkkNxQuL0p-Z0&r=Mzo9MjHoKp3F_yBV7BAdLs9cHkX02n3oVfcj_6zCDDk&m=lSLxNfSoRdvzPfpEdE1vNT92LMnUZlweuNkGjaO5d0Y&s=DZXq1eLsW-63YoUhHKXW55FasbFDA7nH0prTn1SvSJY&e=


Fun Example: Pokémon Go

Pokémon Go, Huw Price, 22/07/16, (www.grid-tools-downloads.com/huw/POKEMONGO.zip)

http://www.grid-tools-downloads.com/huw/POKEMONGO.zip

Digital AssuranceExperimental Content Preview


Value-Driven Scaled Agile supported by Actionable Insight

Digital Assurance

LIFECYCLEVIRTUALIZATION

Simulate missing systems, APIs to save up to 640 Hrs.

DIGITAL ENGINEERING & AUTOMATION

Test less, cover more, automate 100% of tests.

DATA VISUALIZATION & VIRTUALIZATION

Reduce time spent waiting on data by 95%.

DESIGNOPS

DIGITAL TESTINGRapid Evolution with DesignOps for Multi-Modal Delivery

(Incubate to Lead, Scale for Growth and Enhance the Core)


Exclusive Preview


idea

DEVELOPMENT OPERATIONS

‘Shift-Up’ Connected Intelligence

‘Shift-Right’ the Digital Ecology

‘Shift-Left’ Rapid Evolution feat. DesignOps

CUSTOMER EXPERIENCE TESTING

DesignDevTestOpsAgility across all 4 increases speed, reliability and efficiency

Continuous Testing

Continuous Improvement

Continuous Delivery

Continuous Intelligence

Continuous Assessment

Continuous Learning

Continuous Innovation

DESIGN

‘Shift-Down’ Data Archaeology (legacy)


RUNDEPLOYTESTBUILDPLAN

INTAKE RequirementsUser StoriesRelease Plan

MODEL Import User Stories to Automatically Create, Visualize and Optimize Test Cases. Initiate Functional Test Automation

FUNCTIONAL / REGRESSION Subset/Mask Test Data Create/Reserve Test Data Test Automation Library Ensure Mobile Experience

PERFORMANCE FEEDBACK

CONFIG/DEPLOY Provision Entire Stack Confirm Configurations Approve Changes Successfully Deploy Internal or External Cloud

MEASURE/FEEDBACK Customer Experience Business Service View Application View Infrastructure View Dynamic Capacity Feedback to PO/PM

INTEGRATIONRemove Constraints with Virtual Services Mobile, Web, App Svr,

Middleware, Backend, MF 3rd Party Systems / API’s

CODEDevelop and Commit Code, Scan Code, Version Control, Continuous Integration. Complete Build and Initiate Release

AUTOMATION AUTOMATION AUTOMATION

Speed/Time-to-Market Quality/Availability

AUTOMATION

P I P E L I N E A S C O D E

Cost/Financial Mix Risk/Compliance

Requirements Environments

Data Automation

CustomerExperience

PERFORMANCE Engineering Simulate Backend Load Test Outlier Conditions Ensure Mobile Experience

Common Drivers “Big Picture” Issues


Rapid Requirements EvolutionContinuous Adaptive Testing (CAT)

Lean & Agile Teams

Developers & Testers

Release Management

ProductOwner

Designer

ProductManager

DailyReviews

Roadmap

Vision

Backlog

SprintBacklogs

ValueDriven

PLAN

ShippableProducts

SHIFTLEFT

DESIGNOPS

SHIFTRIGHT

OPERATE

FeedbackLoops

Requirements ecology?

Ecosystems of Ecosystems

Exclusive Preview


Requirements Ecology

The Economy of Nature Sixth Edition, 2010, W.H.Freeman and Company

Ecology includes the study of interactionsorganisms have with each other, other organisms, and with abiotic components of their environment

Ecology is the scientific analysis and study of interactions amongst organisms and their environments

Exclusive Preview


Data Ecology Journey to Digital

Engineering transformation Lifecycle virtualization unlock

business value-driven insight leveraging data science, statistical , engineering opportunities

Data Visualization of Complex Ecosystems & Ecosystems Visualize the applications

landscape and leverage connected intelligence to overlay business risk and enable both predictive & prescriptive insight into the behaviours of systems (systemic & epistemic)


adaptive IT fluid IT

REQUIREMENTSLEAN ENGINEERINGDISIPLINED AGILEFAIL-FAST EXPERIEMENTS

core IT

DevOpsDesignOps NoOps

Origin of Requirements “Evolution over Revolution”

HARMONISATIONDESIGN OPERATIONS

agile requirements


Requirements from Design to Operations

DesignOps, Jonathon Wright, 2016, “The Digital Manifesto”, (www.DesignOps.net)

Build

Deliver

MonitorMeasure

Learn

Design

Make

Check

Think

Exclusive Preview

http://www.designops.net/



DOMAIN OF DISCOURSE

FASTFEEDBACK

SLOW FEEDBACK

EMERGENT PRACTICE

SYSTEMATIZED(EPISTEMIC & SYSTEMIC

ENTROPY)

BUSINESS INTELLIGENCEREAL TIMEPREDICTIVEPERSCRIPTIVE

core IT

DAD, LeSS & SAFE (v4)LEAN STARTUP / UX WATERFALL / V-MODEL

CONNECTED INTELLIGENCE


Requirements Archology

Archology is the study of the science of governance or the origin of things

Decomposition is the process by which organic substances are broken down into much simpler forms of matter

Exclusive Preview


Data Archaeology Where to start digging?

Lean Engineering techniques to apply the correct mind-set, heuristics, goals, investment and leadership.

Where is the data? By understanding the abiotic

components of the ecosystem of ecosystems and interactions between complex systems (ecology).

When to stop digging? Risk-based approach to data

mining


Design QA/TESTDEV PRODUCTIONPRE-PROD

Specification

CI/Build Functional testing UAT Integration

testing Performance engineering

Deploy to pre-prod

Code commit SCM

Design Spec

Requirements

Business Users

TESTData

TESTStub

TESTData

User BA

Product Manager

CustomerUsers

CustomerExperience

CustomerExperience

CustomerExperience

CustomerExperience

Design

Business Value

Business Value

Business Value

Improvement

Innovation

Intelligence

Assessment

Insight

Learning

Maintenance

Operations

Delivery

Testing

Support

Cu

sto

me

r

Value Insight

Learn Learn

Monitor Monitor TESTStub

REALUsers

TESTData

REALData

TESTStub

USERCase

Bu

sin

ess

Connected Intelligence

idea


Requirements as a Organism

Ecosystems of Ecosystems (EoE)

Requirements

EffectCause

Endpoints

Y2K $600 billion

EUR20 million or 4 percent of total global annual turnover



WHAT SHOULD WE DO?

FASTFEEDBACK

SLOW FEEDBACK

EXPERIMENT TEST

WHAT HAPPENED ?WHAT’S HAPPENING NOW? WHAT WILL HAPPEN ? WHAT COULD HAPPEN ?

core IT

LIBERATIONIMMUNIZATION DECOMPOSITION

WHAT IS HAPPENING NOW?

Ecologist

DevScrumMaster

Therapist Researcher

Designer

DevOpsEngineer

DisciplinedAgilist

Maker

Creative

Infrastructure Engineer

SDET

BA

Tester

Product Owner PM

Ops

Support


Cynefin Framework

Ray Arell, nuCognitive, “Complexity-Informed Organizational Change”, 09/09/16 adapted from the Cynefin framework by Cognitive Edge. CC BY-SA 3.0

Ray Arell

Emma Langman

Erik Simmons

Dave Snowden



INNOVATION (INNOVATION, GROWTH & MATURITY)FAST

INNOVATIONSLOW

INNOVATION

SENSE-ANALYZE-RESPONDPROBE-SENSE-RESPONDACT-SENSE-RESPOND

core IT

HARMONISATIONCHAOS LIBERATION

Lear

nin

g C

ult

ure

s Teachin

g Cu

ltures

High Abstraction

Low Abstraction

Complex

Chaotic

Lear

nin

g C

ult

ure

s Teachin

g Cu

ltures

High Abstraction

Low Abstraction

Complex Complicated

Lear

nin

g C

ult

ure

s Teachin

g Cu

ltures

High Abstraction

Low Abstraction

Obvious

Complicated

SENSE-CATEGORIZE-RESPOND

Applying the Cynefin Framework


Summary: heuristics, risk and requirements engineering

Value is key to delivering software which matters to users:

– a solution must be frequently and iteratively measured for value

Requirements degrade quality but also interrupt the value stream:

– Testers cannot test rigorously or quickly against customer needs

Requirements can also therefore be key to improving the delivery rate of valuable software

Enabling requirements-driven design, construction and verification:

– Model requirements as “active” flowcharts

– Test from the flowchart with auto-generation of tests/scripts

– Generate test data to match the test cases

– Update tests when the requirements change

Director of Digital Assurance

[email protected]

Jonathon Wright

@Jonathon_Wright

slideshare.net/Jonathon_Wright


References #1

Software Requirements (2nd ed.), Karl E. Wiegers, MS Press 2003

More About Software Requirements, Karl. E. Wiegers, MS Press 2005

Competitive Engineering, Tom Gilb, Elsevier 2005

Software & Systems Requirements Engineering: In Practice, Brian Barenbach et al, McGraw Hill 2009

Requirements Engineering: From system goals to UML models to software specifications, Axel van

Lamsweerde, Wiley 2009

Software Requirements – Styles and Techniques (2nd ed.), Søren Lauesen, Addison Wesley 2001

Customer-Centered Products, Creating Successful Products through Smart Requirements Management, Ivy

Hooks and Kristin A. Farry, Amacom, 2001



References #2

Requirements Engineering: Processes and Techniques, Gerald Kotonya and Ian Sommerville, Wiley 1999

Exploring Requirements: Quality Before Design, Donald Gause and Gerald Weinberg, Dorset House 1988

Effective Requirements Practices, Ralph Young, Addison Wesley 2001 Managing Software Requirements: A

Unified Approach (2nd ed.), Dean Leffingwell and Don Widrig, Addison Wesley 2003

Non-Functional Requirements in Software Engineering, Lawrence Chung et al., Kluwer Academic

Publishers 2000

System and Software Requirements Engineering (2nd Ed.), Richard H. Thayer and Merlin Dorfman (ed),

IEEE 1997

Mastering the Requirements Process (2nd Ed.), James and Suzanne Robertson, Addison Wesley 1999



CA Continuous TestingSupporting the total Value Stream


CA Project & Portfolio Management

Unify long term strategy, investment and portfolio planning.

CA Agile Central

Collaboratively plan, prioritize and track work across the enterprise.

Agile Transformation Framework

Connects Strategy & Execution

Executives

BUSINESS INITIATIVE

Project & Portfolio Managers

Application Developers

Product Managers

CUSTOMER

STRATEGY

EXECUTIONDevelop & Test

Synchronize efforts to dramatically speed app development & increase quality

Manage & Monitor

Make a great customer experience a competitive advantage

Release & Deploy

Control the release process, to continuously advance application quality, improve the customer experience and reduce costs.

Strategy

ContinuousDelivery

Customer Experience

AppDev


Requirements & Defects

Dev & Build

Concurrent Testing

Pre-production

Production &

Monitoring

Ideas

CA Release Automation +

Continuous Delivery Edition

CA Service Virtualization

CA Test Data Management

CA Agile Requirements

Designer

CA Test Data Visualization

Total Visibility

Across Entire App Lifecycle

FullyIntegrated Tool Chain

Exceptional Customer

Experience

Automated Testing & 100% Test Coverage

Full Access to Simulated

Environments

Accelerated Development

Synthetic Data Creation & Data

on Demand

Leverage your current investments and tools of choice,while moving forward in your Continuous Delivery journey

Open & Fully Integrated CD Solution Stack

CA BlazeMeter

CA ApplicationTest

63

© 2015 CA. ALL RIGHTS RESERVED.

CO

NTI

NU

OU

S D

ELIV

ERY

MA

TUR

ITY

Manual

Automated

Continuous

Lean Engineering

Connected Intelligence

BUSINESS VALUE

LEVEL 1 LEVEL 2 LEVEL 3 LEVEL 4 LEVEL 5

?

Optimized Organization

for DesignOps

Continuous Insight & Learning

Value StreamMapping

Consistency & Collaboration

Heroes & Heavy Lifting

DIG

ITA

L M

AST

ERY

• DevOps organized • End-to-end orchestration• Microservices & API enabled• Open and complete integrations• Incremental agile/sprint release cadence• Continuous testing at every phase – early and often

Where are you?

Integrated CD Ecosystem

• Silos, manual handovers, waterfall process• One release/year• Monolithic apps• Long term project/resource planning• Error prone dev/test/release processes

Release Virtualization

ReleaseAutomation

Release Insight

ReleaseFrequency

Release Culture

ReleaseContent

ReleaseManagement


CA Continuous DeliverySupporting the total Value Stream

Continuous Delivery

BUILD | TEST | DEPLOY•CA Agile Requirements Designer

•CA Test Data Manager

•CA Release Automation

•CA Service Virtualization

•CA Application Test

•CA Performance Test

•CA Continuous Application Insight

•CA CD Director

•CA API Management

•CA Live API Creator

•CA Mobile App Services

Agile Operations

OPERATE•CA APM

•CA UIM

•CA Mobile App Analytics

Agile Platform

DELIVERY PLAN• CA Agile Central

• CA Agile Services

Chief Data Therapist (CDT)

[email protected]

Jonathon Wright

@Jonathon_Wright

SlideShare.net/Jonathon_Wright

LinkedIn.com/in/Automation

ca.com/us/products/ca-test-data-manager.html

ca - take a value-driven approach to better software

Technology