step by step in vivo e xperimentation lecture 3 for the iv track of the 2011 pslc summer school
DESCRIPTION
Step by Step In Vivo E xperimentation Lecture 3 for the IV track of the 2011 PSLC Summer School. Philip Pavlik Jr. Carnegie Mellon University. So, you want to run an in vivo experiment?. What is in vitro ? Self-explanation (Chi, Bassok, Lewis, Reimann, & Glaser, 1989) What is in vivo ? - PowerPoint PPT PresentationTRANSCRIPT
July 8, 2008In vivo experimentation: 1
Step by Step In Vivo ExperimentationLecture 3 for the IV track of the 2011
PSLC Summer School
Philip Pavlik Jr.Carnegie Mellon University
July 8, 2008In vivo experimentation: 2
So, you want to run an in vivo experiment?
What is in vitro?– Self-explanation (Chi, Bassok, Lewis,
Reimann, & Glaser, 1989) What is in vivo?
– Example take from: Booth, Siegler, Koedinger & Rittle-Johnson
(2008)
July 8, 2008In vivo experimentation: 3
Step 0
Become an expert in a content area (e.g., electrodynamics).– Find a textbook– Talk to an expert– (Potentially the most time consuming step.)
July 8, 2008In vivo experimentation: 4
Step 1
Hypothesis– Self-explanation: A self-generated
explanation of presented instruction that integrates the presented information with background knowledge and fills in tacit inferences. (from pslc wiki)
– Correct examples Facilitate construction of high-feature validity
knowledge components.– Incorrect examples
Weaken low-feature validity knowledge components.
July 8, 2008In vivo experimentation: 5
Step 2
Select a domain– Algebra, geometry, physics, chemistry,
Chinese, English Select a LearnLab site
– Riverview High School– Central Westmoreland Career & Technology
Center Talk with instructors
– Collaboratively identify “hot spots” in curriculum.
July 8, 2008In vivo experimentation: 6
Step 3
Develop materials– Assessments– Tutor implementation
Get IRB approval.
July 8, 2008In vivo experimentation: 7
Corrective self-explanation
explanation of incorrect worked
example
July 8, 2008In vivo experimentation: 8
Typical self-explanationexplanation of correct worked
example
July 8, 2008In vivo experimentation: 9
Step 4
Self-Explanation of Correct Examples
No Yes
No “As-is” Control Typical
Yes CorrectiveTypical +
Corrective (half of each)S
elf-E
xpla
natio
n of
In
corr
ect E
xam
ples
Design Study
July 8, 2008In vivo experimentation: 10
Step 5
Formulate a Procedure– Pretest
Two forms of the assessment– Intervention
Solve problems with Cognitive Tutor Algebra– Post-test
Talk alternate form of assessment
July 8, 2008In vivo experimentation: 11
Step 6
Run experiment – Collect log files– Collect paper-based assessments
Send Log Data to DataShop– Test Hypotheses
July 8, 2008In vivo experimentation: 12
Step 7
Report your results to the scientific community– Create a wiki page– Give a conference talk– Write a journal paper
July 8, 2008In vivo experimentation: 13
Number of correctly answered problems
July 8, 2008In vivo experimentation: 14
Number of conceptual errors
July 8, 2008In vivo experimentation: 15
Goto Step 1
Using the results from your in vivo experiment:– Replicate with a new population– Replicate with a new domain– Replicate and Extend in a lab experiment
July 8, 2008In vivo experimentation: 16
Summary Step:
0. Become an expert1. Generate a hypothesis (in vitro => in vivo)2. Select domain, site, instructors3. Develop materials4. Design study5. Formulate a Procedure6. Run experiment & log to DataShop7. Report your results
Goto Step 1
July 8, 2008In vivo experimentation: 17
Any Questions?