matt heusser - keynote - cool new things... and some old ones too
DESCRIPTION
TRANSCRIPT
Cool new ideas... and some old ones, too
MATTHEW HEUSSER – EXCELON DEVELOPMENT [email protected]
QAOrTheHighWay – February, 2014Columbus, OH
Tuesday, February 11, 14
2
A Word of Warning
Tuesday, February 11, 14
System 1, System 2
Tuesday, February 11, 14
4
Your Brain At Work
Tuesday, February 11, 14
Black Swans
Tuesday, February 11, 14
6
Another Example
Tuesday, February 11, 14
Ten Kind of Tests
Let’s Get Beyond Testing the functions
Tuesday, February 11, 14
8
Ten Kind of Tests - II
Dog Fooding, User Scenarios, Failure Modes
A month ago, I took home a build of CityDesk (thinking it was about 3 weeks from shipping) and tried to build a site with it.
Phew! There were a few bugs that literally made it impossible for me to proceed, so I had to fix those before I could even continue. All the testing we did, meticulously pulling down every menu and seeing if it worked right, didn't uncover the showstoppers that made it impossible to do what the product was intended to allow. Trying to use the product, as a customer would, found these showstoppers in a minute.
Tuesday, February 11, 14
96
Ten Kind of Tests - III
Quality Factors, Creative Ideas, States
Tuesday, February 11, 14
1076
Ten Kind of Tests - IV
Data, Environment, White Box, Chartered Missions
76
Tuesday, February 11, 14
11987676
876
Ten Kind of Tests - V
Taxonomies, Previous Failures
76
Tuesday, February 11, 14
12
Ten Kind of Tests - V
Taxonomies, Previous Failures, Quick Attacks
Tuesday, February 11, 14
The Minefield
Tuesday, February 11, 14
Now think about the big failures you've experienced. The Crashes in production, the corruptions of financial transactions, the projects that were cancelled.
Was it ever "gee if only we'd had more time to automation those functional tests?" or is it "huh. We never thought to test that at all?"
Most of the big problems I have experienced in software delivery were Black Swans, the kind of thing that we only could have found by system two thinking -- and excessive focus on functional testing means we don't have time for it.
Actual Risks - Those That Emerge
Tuesday, February 11, 14
Q&A
Tuesday, February 11, 14
Tomorrow
The risks on each build are different - so maybe we don’t want a repeatable process.
Over-focus on automation, or deadlines, or specifications, creates cognitive blinders that cause us to miss black swans.
I know, continuous lean cloudy agile devlops is ‘hot’ - but the real challenge of software testing is cognative. The pendulum may have swung just a little too far. Let’s swing it back.
Thank you, and god bless you.
Ma# Heusser@mheusser -‐ ma#@xndev.com
h#p://www.xndev.com
Tuesday, February 11, 14
1710987676
876
... and where to go for more
James Bach, The Heuristic Test Strategy Model, Specifically SFDPO
http://www.satisfice.com/tools/htsm.pdf
Roubert Sabourin, Ten Sources of Test Ideas
http://www.amibugshare.com/articles/Article_10_Sources_of_Testing_Ideas.pdf
Hans Buwalda, Soap Opera Tests
http://www.logigear.com/logi_media_dir/Documents/Soap_Opera_Testing.pdf
iOs Testing Mindmap, Bernard Shaw Lelunk
http://tuppad.com/blog/2012/09/06/ios-testing-mind-map-how-to-test-an-iphoneipadipod-by-bernard-shai-lelchuk/
Moolya Software Testing Limited’s Blog, Specifically Mobile Tests
http://moolya.com/blog/2012/04/18/test-mobile-applications-with-cop-who-flung-gun/
Elisabeth Henrickson’s Test Heusser-istic Cheat Sheet
http://testobsessed.com/wp-content/uploads/2011/04/testheuristicscheatsheetv1.pdf
For an electronic copy of this with links, email [email protected] 76
Tuesday, February 11, 14
A Famous Ship
Tuesday, February 11, 14