Painting like an engineer

Skills in testing

Alexandra Casapu

@coveredincloth

Heuristic!

Heuristic!

Heuristic!

Heuristic!

Heuristic!

Heuristic!

The engineering method:

Use heuristics…..to cause the best change....in a poorly understood situation....within the available resources

The art and science of testing…

Engineering vs Science

What is science?

Applied engineering

Engineering vs Art

“At last the mountains of Arcadia and the mist-wrapped mountains of the lonely traveler become one as we realize that what we paint, how we paint, that we paint, and, most importantly, why we paint are all art - all; heuristic.

To be alive is to be an artist; to be conscious of being an artist is to be human; to recognize and to be satisfied that you can only be an artist is to be at peace.“

Story structure

How I see skills and skills development

Where can the engineering method help

5 examples from my testing work

Conclusions in terms of skills development

Where are the skills?The Knowledge Dimension

The Cognitive Process Dimension

Remember Understand Apply Analyze Evaluate Create

Facts

Concepts

Procedures

Cognitive Strategies

Models

Skills

Attitudes

Metacognition

How do I know I have acquired a skill?

Second nature

How can I work on my skills?

Cem Kaner :

I think the best way for people to develop skills is to do something,

get feedback on how to do it better, improve it (or do something similar),

get feedback, and keep doing this with problems that are increasingly

difficult or that apply the technique in new ways.

The necessary conditions for developing skills

• Predictability of outcomes• Good feedback

• Attitude • Motivation• Deliberate practice• Amount of experience

“Most judgments and most choices are made intuitively.”

Maps of bounded rationality: Psychology for behavioral economics – Daniel Kahneman

The context of my work

A taxonomy of skills

Signatures of the heuristic:

1. A heuristic does not guarantee a solution,

2. It may contradict other heuristics,

3. It reduces the search time for solving a problem, and

4. Its acceptance depends on the immediate context instead of an absolute standard

• learn from mistakes• generate functionality flows• analyze information• recognize patterns• read actively• create models• use models• read and write code• write effective bug reports• make logical connections• change how testing is done and seen in the team• collect relevant information from different sources• handle complexity well• evaluate the testing work

Attitude determining skills

• asking questions(in a simple, non-offensive manner)• have an empathetic approach• be appreciative, considerate• have the ability to explain my train of thoughts• collect info effectively• offer my availability

Human-Human Interaction skills

• think critically• use counterfactual reasoning• deal with uncertainty and incompleteness• prioritize• work in a timeboxed manner• decide when to stop

Risk controlling skills

• focus• defocus• break a problem into multiple smaller problems• have diverse view points• stick with a consistent view point• use curiosity• fix variables and vary one at a time• exercise stubbornness• don't use stubbornness• create disconfirmatory experiments

Rule of thumb skills

• describe coverage• make maps of features under test• create valuable documentation

Information visualization skills

My personal testing skills state of the art

Examples

Working on new skills (repeatedly) Skills atrophy Skills that work…sometimes

Overlooked skills

Stubborn about using whatever skills needed

1

2

3

4

5

Working on new skills (repeatedly)1

Let’s introduce some automation..

1

On the previous project

Focusing on tasks

Less defocusing

I didn’t use my curiosity

Not so effective at collecting info

Not thinking critically about my testing process

1

On the current project

Defocus from my current taskWork in a timeboxed mannerUse my curiosityWrite codeRead codeChange how I do testingChange how testing is seen in the teamCreate valuable documentation

1

The ‘repeatedly’ part

1

Skills atrophy2

On the previous project

2

Working with APIs

Verifying validity output in the DB

Monitoring http requests

Verifying output in json files

Testing social crawlers algorithms

Skills that work…sometimes3

Not getting invited to meetings

3

Getting invited to meetings

3

Change how testing is done and seen in the team

Collect info effectively

Asking questions

Offer my availability

Overlooked skills4

Logging invalid bugs

4

Logging invalid bugs

4

Logging invalid bugs

4

Re-evaluate the model

Analyze existing data

Stubborn about using whatever skills needed5

Learn by digging the needed info

5

Use curiosityCollect info effectivelyRead code Break a problem into multiple smaller problemsExercise stubbornnessHave diverse viewpointsUse counterfactual reasoning

Conclusions

It’s about skills interaction.An overwhelming accumulation of interactions

Learn by doing

Skills are a procrustean bed

“Each speaks with the accent of his[/her] own sota.”

Each with their own experience

“Since one sota cannot be completely translated into another, each person

remains fundamentally isolated. To this sense of uniqueness, inaccessibility,

alienation, and loneliness some philosophers attribute the absurd nature of the

human condition.”

Nurturing skills

Choose a few areas which I want to improve

Learn the queues that lead to the mistakes I want to eliminate

When I recognize such a situation, slow myself down

And recognize when I need help

Nurturing skills

“Peace comes with the realization that the world in which we live is an acquired taste - one we all, as artists, paint in our own chosen styles.”

References

• Discussion of the method - Billy Vaughn Koen

• How to read a book – Mortimer J. Adler, Charles van Doren

• Maps of Bounded Rationality: Psychology for Behavioral Economics – Daniel Kahneman

• Conditions for intuitive expertise: a failure to disagree – D. Kahneman, G. Klein

• http://blog.utest.com/testing-the-limits-with-cem-kaner-author-of-the-domain-testing-workbook/2014/01/

• The BBST courses

• edX Thinking 101 course

Illustrations:• http://catandgirl.com/ (slide 2)• http://studio-ghibli.wikia.com/wiki/Forest_(Mononoke-Hime) (slide 3)• http://theredlist.com/media/database/muses/icon/cinematic_men/1980/john_malkovitch/022_john_malkovitch_theredlist.jpg (slide 6)• http://thingsorganizedneatly.tumblr.com/post/70411190398/submission-torpedo-typewriter-organized-neatly (slide 34)• http://thingsorganizedneatly.tumblr.com/image/67872166549 (slide 35)• http://thingsorganizedneatly.tumblr.com/image/81360232303 (slide 36)• http://4.bp.blogspot.com/-ZtDBcj1VW1A/TlaUP0xdFhI/AAAAAAAACBs/2DhJ_5UVvaM/s1600/Theseus_Procrustes-600x450.jpg (slide 39)

@coveredincloth