test & evaluation = the knowledge factory
DESCRIPTION
Test & Evaluation (T&E) is presented in the prespective of knowledge creation, distribution, absorbtion and management. Possible "soft tools" are derived from the insight of the sciences of learning and knowledge management.TRANSCRIPT
![Page 1: Test & evaluation = the knowledge factory](https://reader034.vdocuments.site/reader034/viewer/2022051514/5481b581b4af9f500f8b4881/html5/thumbnails/1.jpg)
1
Test & Evaluationas a
“Knowledge factory”
![Page 2: Test & evaluation = the knowledge factory](https://reader034.vdocuments.site/reader034/viewer/2022051514/5481b581b4af9f500f8b4881/html5/thumbnails/2.jpg)
Outline
2
Knowledge-centered perspective on T&E
Shifting focus to the people
Some observations
Glimpse to theory and its applications
![Page 3: Test & evaluation = the knowledge factory](https://reader034.vdocuments.site/reader034/viewer/2022051514/5481b581b4af9f500f8b4881/html5/thumbnails/3.jpg)
System Debugging Loop
3
Discovery
T&E, Reqs
Systems
Knowledge
ActionSE
![Page 4: Test & evaluation = the knowledge factory](https://reader034.vdocuments.site/reader034/viewer/2022051514/5481b581b4af9f500f8b4881/html5/thumbnails/4.jpg)
T&E ~ Empirical Research
4
Make predictions based on theory/design
Verify/Falsify predictions by experimenting
Search for unpredicted behavior
Reporting findings formally
![Page 5: Test & evaluation = the knowledge factory](https://reader034.vdocuments.site/reader034/viewer/2022051514/5481b581b4af9f500f8b4881/html5/thumbnails/5.jpg)
5
Should test engineers
be trained
as empirical scientists?
Should T&E be structured
as scientific discovery
process?
![Page 6: Test & evaluation = the knowledge factory](https://reader034.vdocuments.site/reader034/viewer/2022051514/5481b581b4af9f500f8b4881/html5/thumbnails/6.jpg)
T&E ≠ Empirical ResearchIt requires:
6
Maximal coverage of phenomena
Scheduled delivery of knowledge
Immediate usefulness of knowledge
![Page 7: Test & evaluation = the knowledge factory](https://reader034.vdocuments.site/reader034/viewer/2022051514/5481b581b4af9f500f8b4881/html5/thumbnails/7.jpg)
7
Test & Evaluation asa personal and group quest for knowledge
![Page 8: Test & evaluation = the knowledge factory](https://reader034.vdocuments.site/reader034/viewer/2022051514/5481b581b4af9f500f8b4881/html5/thumbnails/8.jpg)
8
Project / Organization
Test Team
Test Engineer
![Page 9: Test & evaluation = the knowledge factory](https://reader034.vdocuments.site/reader034/viewer/2022051514/5481b581b4af9f500f8b4881/html5/thumbnails/9.jpg)
9
Some observationson how the Test
Engineers think…
Knowledge iscreated by thinking
![Page 10: Test & evaluation = the knowledge factory](https://reader034.vdocuments.site/reader034/viewer/2022051514/5481b581b4af9f500f8b4881/html5/thumbnails/10.jpg)
10
Using all available sources
Creating a mental model
In order to be able
to “tell a story” about the system
Without documentati
on
![Page 11: Test & evaluation = the knowledge factory](https://reader034.vdocuments.site/reader034/viewer/2022051514/5481b581b4af9f500f8b4881/html5/thumbnails/11.jpg)
11
Need to Learn arises when
The production of required
documentsis impeded
The evidence
contradicts the mental
model
![Page 12: Test & evaluation = the knowledge factory](https://reader034.vdocuments.site/reader034/viewer/2022051514/5481b581b4af9f500f8b4881/html5/thumbnails/12.jpg)
12
Test & Evaluation require effective and efficient
knowledge creation and learning
Test & Evaluation is reallya “Knowledge Factory”
![Page 13: Test & evaluation = the knowledge factory](https://reader034.vdocuments.site/reader034/viewer/2022051514/5481b581b4af9f500f8b4881/html5/thumbnails/13.jpg)
13
How to leverage the science and practice of
working with knowledgeto better perform T&E?
![Page 14: Test & evaluation = the knowledge factory](https://reader034.vdocuments.site/reader034/viewer/2022051514/5481b581b4af9f500f8b4881/html5/thumbnails/14.jpg)
14
What is Knowledge?
Knowledge is experience. Everything else is just information.Albert Einstein
![Page 15: Test & evaluation = the knowledge factory](https://reader034.vdocuments.site/reader034/viewer/2022051514/5481b581b4af9f500f8b4881/html5/thumbnails/15.jpg)
15
Planning T&E
Plans
Designing Tests
Test Procedures
Executing Tests Test Protocols
Evaluating Test Reports
Planner Experience
Designer Experience
Tester Experience
Analyst Experience
UndocumentedTacit
Knowledge
DocumentedKnowledge
![Page 16: Test & evaluation = the knowledge factory](https://reader034.vdocuments.site/reader034/viewer/2022051514/5481b581b4af9f500f8b4881/html5/thumbnails/16.jpg)
16
SECI Model (I. Nonaka)
![Page 17: Test & evaluation = the knowledge factory](https://reader034.vdocuments.site/reader034/viewer/2022051514/5481b581b4af9f500f8b4881/html5/thumbnails/17.jpg)
Socialization in T&E
17
i iDevelopers on-site
Customer on-site
Testing in teams
Learn by shared
experience
![Page 18: Test & evaluation = the knowledge factory](https://reader034.vdocuments.site/reader034/viewer/2022051514/5481b581b4af9f500f8b4881/html5/thumbnails/18.jpg)
Externalization in T&E
18
Talking with developers
Talking with customers/users
Meetings and discussions
Writing reports
i
i
ii
i
i
g
Learn by articulating in language
![Page 19: Test & evaluation = the knowledge factory](https://reader034.vdocuments.site/reader034/viewer/2022051514/5481b581b4af9f500f8b4881/html5/thumbnails/19.jpg)
Combination in T&E
Complex Adaptive Integrator19
g
g
g
g oAnalyzing requirements specs
Analyzing design documents
Designing test plans
Designing test procedures
Learn by analysis and synthesis of information
![Page 20: Test & evaluation = the knowledge factory](https://reader034.vdocuments.site/reader034/viewer/2022051514/5481b581b4af9f500f8b4881/html5/thumbnails/20.jpg)
Internalization in T&E
20
i
go Building “mental models” of the
system
“Playing” with the system simulations
Unstructured and exploratory testing
“Playing around” planned tests
Learn by exercise
and practice
![Page 21: Test & evaluation = the knowledge factory](https://reader034.vdocuments.site/reader034/viewer/2022051514/5481b581b4af9f500f8b4881/html5/thumbnails/21.jpg)
“Soft” T&E Practices
21
Actively question the developers, customers and users to externalize their tacit knowledge
Encourage active participation of developers, customers and fellow testers in running tests
Take meetings and non-formal discussions seriously and use them to learn and share knowledge
![Page 22: Test & evaluation = the knowledge factory](https://reader034.vdocuments.site/reader034/viewer/2022051514/5481b581b4af9f500f8b4881/html5/thumbnails/22.jpg)
22
Learning PreferencesEveryone learns better
when the learning matches
their preferences
Teachers should engage multiple “learning muscles” in their audience
Learners should direct their learning to sources that match their preferences
![Page 23: Test & evaluation = the knowledge factory](https://reader034.vdocuments.site/reader034/viewer/2022051514/5481b581b4af9f500f8b4881/html5/thumbnails/23.jpg)
“Soft” T&E Practices
23
Know your learning preferences and use them while studying the SUTs
Augment reports with multimedia
Organize presentations of the results for relevant audiences
![Page 24: Test & evaluation = the knowledge factory](https://reader034.vdocuments.site/reader034/viewer/2022051514/5481b581b4af9f500f8b4881/html5/thumbnails/24.jpg)
Conclusions
24
Test & Evaluation is a knowledge creating and sharing process
Test Engineers are professional learners
T&E should use the insights and tools of knowledge and learning theory and practice
![Page 25: Test & evaluation = the knowledge factory](https://reader034.vdocuments.site/reader034/viewer/2022051514/5481b581b4af9f500f8b4881/html5/thumbnails/25.jpg)
Questions for further research
25
How the knowledge is created and shared by test engineers and test teams?
How do the knowledge processes integrate with testing activities?
What are the knowledge-related factors that contribute to or impede better testing?
What factors facilitate more effective and efficient learning of Test Engineers?
![Page 26: Test & evaluation = the knowledge factory](https://reader034.vdocuments.site/reader034/viewer/2022051514/5481b581b4af9f500f8b4881/html5/thumbnails/26.jpg)
26
Any answers?