I've got 99 problems and code

defects are one

Testing seeks to

discover threats to the

value of software

Debugging discovered 1949

I can remember the exact

instant when I realized that a

large part of my life … was

going to be spent finding

mistakes in my own programs

- Maurice Wilkes

Epic failure

• Mars Climate Orbiter doesn't orbit

– $327 million project lost in a minute

• Didn’t approach at correct angle to planet

• Actual thrust was 4.45 times larger than desired

• Therac-25 Medical Accelerator disaster

– Several deaths documented

– Software race condition decided your treatment

• Low power electron beam

• Megavolts of X-Rays

One in a million is next Tuesday

• You have a bug

• You say

– Maybe this is what is happening!

– But the chance of it happening is one in a million!

• At computer speeds you can hit the millionth

execution ‘Next Tuesday’

Welcome to QA

The test role

• Questioning a product in order to evaluate it

– James Bach

• Testing is done to find information

– With that information critical project decisions can be

made

Project headlights

• Imagine the team packed into a 4 x 4 driving off

road at night

• You’re going to want some headlights

• Testers illuminate where you are and the

obstacles they see

• Other team members can decide if we keep

going or turn back

A test mission

• Find important problems fast

• Perform a comprehensive quality assessment

• Maximize testing efficiency

• Assure the test process is accountable

I'm not schooled in the

science of human factors, but

I suspect surprise is not an

element of a robust user

interface

- Chip Rosenthal

Debugging is twice as hard

as writing the code in the first

place

- Brian Kernighan

You’re scaring me. So now what?

Test activities• You find information when you

– Observe

– Record

– Reproduce

– Infer

– Inform

– Collaborate

QA Guiding Value

Observation beats

speculation every time

How do I science?

Writing a bug report

1. Description

– What you expected to happen

– What actually happened

2. Steps to Reproduce

An app’s hierarchy of needs

I've got 99 problems

and code defects are

one

Enough testing

“It works”

• Means that it appears to meet the requirements

to some degree

“Enough testing”

• Means the team has enough information to

make good decisions

So what’s in it for me?

Find the threats to the value of your

work• Be empirical

– Find evidence based on observation

– Make some valid inferences

– Justify your beliefs about the project

– Clarify and banish ambiguity in project

language and results

Sources

• Software Entomology or Where Do Bugs Come From?

– Noah Sussman

– http://www.slideshare.net/noahsussman/software-entomology-or-where-do-bugs-come-from

• http://infiniteundo.com/

– Various posts by Noah Sussman

• Lessons Learned in Software Testing

– Cem Kaner, James Bach, Bret Pettichord

• Foundations of Software Testing

– http://www.testingeducation.org/BBST/foundations

• Epic failures: 11 infamous software bugs

– http://www.computerworld.com/s/article/9183580/Epic_failures_11_infamous_software_bugs

• How to Change Software Testing for New Cloud Configurations

– http://www.cio.com/article/2386122/developer/how-to-change-software-testing-for-new-cloud-configurations.html