using peer instruction for active learning
Post on 02-May-2022
2 Views
Preview:
TRANSCRIPT
JMU CFI May Symposium 2016 Dr. Michael S. Kirkpatrick
Teaching, Fast and Slow: Using Peer Instruction for Active Learning
Michael S. Kirkpatrick Department of Computer Science
JMU CFI May Symposium 2016
JMU CFI May Symposium 2016 Dr. Michael S. Kirkpatrick
Agenda
• Introduction • Two systems model of thinking • Heuristics and biases • Why active learning? • Peer Instruction • Formative assessments and ConcepTests • Practice time
JMU CFI May Symposium 2016 Dr. Michael S. Kirkpatrick
Introduction
• Assistant Professor of Computer Science • Primary research focus is CS education
• Misconceptions are central to my courses
• Started at JMU in 2011 • Prior experience teaching mathematics for liberal arts majors at Purdue
JMU CFI May Symposium 2016 Dr. Michael S. Kirkpatrick
Recommended reading
JMU CFI May Symposium 2016 Dr. Michael S. Kirkpatrick
Two systems model
LEFT left
left RIGHT
RIGHT left
RIGHT right
upper lower
LOWER upper UPPER
lower LOWER upper
LEFT left
left RIGHT
RIGHT left
RIGHT right
phlegm NOSE
sneeze FLU FILTHY
muddy clean
HANDS
JMU CFI May Symposium 2016 Dr. Michael S. Kirkpatrick
Two systems model
S O _ P W _ S H IU
S O A P W A S H
W _ S H S O _ P
JMU CFI May Symposium 2016 Dr. Michael S. Kirkpatrick
Two systems model
System 1 • Automatic and quick
• Language processing • Little or no effort • No sense of voluntary
control • Associative machine • Substitutes
easier questions
System 2 • Effortful mental activities
• Calculations and critical thinking
• Depends on system 1 impressions
• Depletes self-control • Lazy controller
• Inhibited by stress, hunger, emotion
What you see is all there is.
JMU CFI May Symposium 2016 Dr. Michael S. Kirkpatrick
Two systems modelRepeated
Experience
Clear Display
Primed Idea
Good Mood
Cognitive Ease
Feels Familiar
Feels True
Feels Good
Feels Effortless
JMU CFI May Symposium 2016 Dr. Michael S. Kirkpatrick
• What is my mood right now?
• Have I caused an accident recently?
• Do I observe some students doing good work?
• How happy are you with your life these days?
• Are you better at driving than the average driver?
• How effective is your style of teaching?
Two systems model
JMU CFI May Symposium 2016 Dr. Michael S. Kirkpatrick
Which group can you make more words from? • XUZONLCJM vs. TAPCERHOB
Which is a more common cause of death? • Strokes or all unintentional accidents/poisonings
combined Which is more common?
• Words that start with “k” or words with “k” as the third letter
Heuristics and biases
Available examples shape
responses
JMU CFI May Symposium 2016 Dr. Michael S. Kirkpatrick
Heuristics and biases
Write this number down on a piece of paper:
25 Are you older or younger than this?
JMU CFI May Symposium 2016 Dr. Michael S. Kirkpatrick
Heuristics and biases
About what percentage of African nations are in the UN? (A) 20% (B) 45% (C) 73% (D)98%
JMU CFI May Symposium 2016 Dr. Michael S. Kirkpatrick
Heuristics and biases
About what percentage of African nations are in the UN? (A) 20% (B) 45% (C) 73% (D)98%
Numbers act as anchors that prime
responses
JMU CFI May Symposium 2016 Dr. Michael S. Kirkpatrick
Heuristics and biases
W A S H
S O A P
phlegm NOSE
sneeze FLU FILTHY
muddy clean
HANDS
JMU CFI May Symposium 2016 Dr. Michael S. Kirkpatrick
Heuristics and biases
Linda is thirty-one years old, single, outspoken, and very bright. She double majored in philosophy and
criminal justice. As a student, she was deeply concerned with issues of discrimination and social
justice, participated in protests against the 2003 invasion of Iraq, and served as a delegate to the
National Black Student Union Conference.
JMU CFI May Symposium 2016 Dr. Michael S. Kirkpatrick
Heuristics and biases
Which of the following is most likely? (A) Linda is a corporate lawyer. (B) Linda is an investment banker for Goldman Sachs. (C) Linda is an elementary school teacher in a wealthy suburban
district. (D) Linda works two jobs as a fast food dishwasher and a
custodian staff manager. (E) Linda is a corporate lawyer who does pro bono work for the
Black Lives Matter movement.
JMU CFI May Symposium 2016 Dr. Michael S. Kirkpatrick
Heuristics and biases
Which of the following is most likely? (A) Linda is a corporate lawyer. (B) Linda is an investment banker for Goldman Sachs. (C) Linda is an elementary school teacher in a wealthy suburban
district. (D) Linda works two jobs as a fast food dishwasher and a
custodian staff manager. (E) Linda is a corporate lawyer who does pro bono work for the
Black Lives Matter movement.
Representative examples suppress base rate statistics knowledge
JMU CFI May Symposium 2016 Dr. Michael S. Kirkpatrick
Heuristics and biases
corporate lawyers
BLM advocates
Linda
Not Linda
JMU CFI May Symposium 2016 Dr. Michael S. Kirkpatrick
Heuristics and biases
Which of these words do you associate more strongly with the passage about Linda?
old vs. liberal
The halo effect creates perceptions that match our emotions
JMU CFI May Symposium 2016 Dr. Michael S. Kirkpatrick
System 1 consequences • Availability heuristic • Priming and anchors • Representative samples & base rate bias • Halo effect: emotionally coherent perceptions • Affect heuristic: preferences influence beliefs • Overconfidence (Dunning-Kruger) and procrastination • Risk assessment and prospect theory
Heuristics and biases
JMU CFI May Symposium 2016 Dr. Michael S. Kirkpatrick
Pedagogical implications
Think-pair-share activity • Think of an instance or a topic that your students
frequently miss on exams. Which biases, heuristics, or system 1/system 2 effects might be contributing to this?
• Pair with someone at your table and describe the instance you thought of.
• Share with everyone at your table, looking for common themes or causes.
JMU CFI May Symposium 2016 Dr. Michael S. Kirkpatrick
Pedagogical implications of system 1 • Computes more than intended (mental shotgun) • Subject to biases and heuristics • Focuses on existing evidence, ignores what’s absent
• “What you see is all there is” • Can be trained by system 2 to detect patterns and execute
skilled intuitions
Pedagogical implications
Active learning and formative assessment train system 1
JMU CFI May Symposium 2016 Dr. Michael S. Kirkpatrick
“Adopting instructional practices that engage students in the learning process is the defining feature of active learning.”
-Michael Prince
ALL
JMU CFI May Symposium 2016 Dr. Michael S. Kirkpatrick
Why active learning?
Pre- vs. post- in physics • Mechanics Diagnostic • Force Concept Inventory • 62 courses (14 trad.) at multiple institutions
• 6542 students (2084) • Worst active learning comparable to best traditional lecture
R.R. Hake, "Interactive-engagement vs traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses," Am. J. Phys. 66, 64- 74 (1998). http://www.physics.indiana.edu/~sdi/ajpv3i.pdf
JMU CFI May Symposium 2016 Dr. Michael S. Kirkpatrick
Why active learning?
Metaanalysis of 225 studies • 158 studies: average 0.47 SDs better on CIs/exams
• 67 studies: average failure rate dropped from 33.8% to 21.8%
Heterogeneity analyses indicated no statistically significantvariation among experiments based on the STEM discipline ofthe course in question, with respect to either examination scores(Fig. 2A; Q = 910.537, df = 7, P = 0.160) or failure rates (Fig. 2B;Q = 11.73, df = 6, P = 0.068). In every discipline with more than10 experiments that met the admission criteria for the meta-analysis, average effect sizes were statistically significant foreither examination scores or failure rates or both (Fig. 2, Figs.S2 and S3, and Tables S1A and S2A). Thus, the data indicatethat active learning increases student performance across theSTEM disciplines.For the data on examinations and other assessments, a het-
erogeneity analysis indicated that average effect sizes were lowerwhen the outcome variable was an instructor-written course ex-amination as opposed to performance on a concept inventory(Fig. 3A and Table S1B; Q = 10.731, df = 1, P << 0.001). Al-though student achievement was higher under active learning forboth types of assessments, we hypothesize that the difference ingains for examinations versus concept inventories may be due tothe two types of assessments testing qualitatively different cogni-tive skills. This explanation is consistent with previous research
indicating that active learning has a greater impact on studentmastery of higher- versus lower-level cognitive skills (6–9), andthe recognition that most concept inventories are designed todiagnose known misconceptions, in contrast to course examinationsthat emphasize content mastery or the ability to solve quantitativeproblems (10). Most concept inventories also undergo testing forvalidity, reliability, and readability.Heterogeneity analyses indicated significant variation in terms
of course size, with active learning having the highest impacton courses with 50 or fewer students (Fig. 3B and Table S1C;Q = 6.726, df = 2, P = 0.035; Fig. S4). Effect sizes were sta-tistically significant for all three categories of class size, how-ever, indicating that active learning benefitted students inmedium (51–110 students) or large (>110 students) class sizesas well.When we metaanalyzed the data by course type and course
level, we found no statistically significant difference in activelearning’s effect size when comparing (i) courses for majorsversus nonmajors (Q = 0.045, df = 1, P = 0.883; Table S1D), or(ii) introductory versus upper-division courses (Q = 0.046, df = 1,P = 0.829; Tables S1E and S2D).
Fig. 1. Changes in failure rate. (A) Data plotted as percent change in failure rate in the same course, under active learning versus lecturing. The mean change(12%) is indicated by the dashed vertical line. (B) Kernel density plots of failure rates under active learning and under lecturing. The mean failure rates undereach classroom type (21.8% and 33.8%) are shown by dashed vertical lines.
Fig. 2. Effect sizes by discipline. (A) Data on examination scores, concept inventories, or other assessments. (B) Data on failure rates. Numbers below datapoints indicate the number of independent studies; horizontal lines are 95% confidence intervals.
Freeman et al. PNAS | June 10, 2014 | vol. 111 | no. 23 | 8411
PSYC
HOLO
GICALAND
COGNITIVESC
IENCE
SSE
ECO
MMEN
TARY
S. Freeman et al., “Active learning increases student performance in science, engineering, and mathematics,” Proceedings of the National Academy of Sciences 111(23), 8410- 8415, 2014. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4060654/pdf/pnas.201319030.pdf
JMU CFI May Symposium 2016 Dr. Michael S. Kirkpatrick
Why active learning?
JMU CFI May Symposium 2016 Dr. Michael S. Kirkpatrick
Peer Instruction
individual problems from traditional exams on the midtermsin the calculus-based course in 1991 !results reported in Ref.4". Finally, in the second semester of the algebra-basedcourse in Spring 2000 !electricity and magnetism", we in-cluded on the final exam one quantitative problem from theprevious year, when a different instructor had taught thecourse traditionally. We found that the students taught withPI !Spring 2000, N!155" significantly outperformed the stu-dents taught traditionally !Spring 1999, N!178", averaging7.4 out of 10 compared to 5.5 out of 10 !standard deviations2.9 and 3.7, respectively". The improvement of the PI stu-dents over the traditional students corresponds to an effectsize of 0.57. All measures indicate that our students’ quanti-tative problem-solving skills are comparable to or better thanthose achieved with traditional instruction, consistent withthe findings of Thacker et al.19
C. ConcepTest performance
Students’ responses to the ConcepTests themselves pro-vide further insight into student learning. We analyzed stu-dent responses to all of the ConcepTests over an entire se-mester, and find that after discussion, the number of studentswho give the correct answer to a ConcepTest increases sub-stantially, as long as the initial percentage of correct answersto a ConcepTest is between 35% and 70%. !We find that theimprovement is largest when the initial percentage of correctanswers is around 50%.4" In addition, the vast majority ofstudents who revise their answers during discussion changefrom an incorrect answer to the correct answer. Figure 4shows how students change their answers upon discussionfor all of the ConcepTests used during the Fall 1997 semes-ter. The answers are categorized as correct both before andafter discussion !‘‘correct twice’’", incorrect before and cor-rect after discussion !‘‘incorrect to correct’’", correct beforeand incorrect after discussion !‘‘correct to incorrect’’", orincorrect both before and after discussion !‘‘incorrecttwice’’". Nearly half of the correct answers given were ar-rived at after discussion, and students changed from correct
Table I. Force Concept Inventory !FCI" and Mechanics Baseline Test !MBT" results.a
Year MethodFCIpre
FCIpost
Absolute gain(post"pre)
Normalizedgain #g$ MBT
MBT quant.questions N
Calculus-based1990 Traditional !70%" 78% 8% 0.25 66% 62% 1211991 PI 71% 85% 14% 0.49 72% 66% 1771993 PI 70% 86% 16% 0.55 71% 68% 1581994 PI 70% 88% 18% 0.59 76% 73% 2161995 PI 67% 88% 21% 0.64 76% 71% 1811996 PI 67% 89% 22% 0.68 74% 66% 1531997 PI 67% 92% 25% 0.74 79% 73% 117
Algebra-based1998 PI 50% 83% 33% 0.65 68% 59% 2461999 Traditional !48%" 69% 21% 0.40 ¯ ¯ 1292000 PI 47% 80% 33% 0.63 66% 69% 126
aThe FCI pretest was administered on the first day of class; in 1990 no pretest was given, so the average of the1991–1994 pretest is listed. In 1995 the 30-question revised version was introduced !Ref. 15". In 1999 nopretest was given so the average of the 1998 and 2000 pretest is listed. The FCI posttest was administered aftertwo months of instruction, except in 1998 and 1999, when it was administered the first week of the followingsemester to all students enrolled in the second-semester course !electricity and magnetism". The MBT wasadministered during the last week of the semester after all mechanics instruction had been completed. For yearsother than 1990 and 1999, scores are reported for matched samples for FCI pre- and posttest and MBT. No dataare available for 1992 !EM was on sabbatical" and no MBT data are available for 1999.
Fig. 3. Mechanics Baseline Test !Ref. 13" scores for introductory calculus-based physics, Harvard University, Fall 1990–Fall 1997. Average score onentire test !circles" and on quantitative questions !Ref. 17" only !squares" vsyear are shown. Open symbols indicate traditionally taught courses andfilled symbols indicate courses taught with PI. The dotted line indicatesperformance on quantitative questions with traditional pedagogy !1990".
Fig. 4. Answers given to all ConcepTests discussed in Fall 1997, catego-rized as described in the text.
972 972Am. J. Phys., Vol. 69, No. 9, September 2001 C. H. Crouch and E. Mazur
These were NOT “worse” students
These were NOT “worse” students (post - pre)
1 - pre
JMU CFI May Symposium 2016 Dr. Michael S. Kirkpatrick
Peer Instruction
Class structure • Daily or weekly reading quizzes to address basic definitions • Minimum of 15 minutes per concept
• 7-10 minutes of lecture • 5 - 8 minutes for ConcepTest
• Vote first, discuss, vote again, debrief • Exams aligned with concepts
• Not multiple choice, emphasize core ideas • 3-2-1-0 grading scheme
Graded but low-stakes
Can be Canvas/JiTT
JMU CFI May Symposium 2016 Dr. Michael S. Kirkpatrick
Peer Instruction
Reading Quiz Question Without air resistance, an object dropped from a plane flying at constant speed in a straight line will 1. quickly lag behind the plane 2. remain vertically under the plane 3. move ahead of the plane 4. not covered in the reading assignment
Exam Question Two stones are released from rest at a certain height, one after the other. (a) Will the difference in their speeds increase, decrease, or stay the same? (b) Will their separation distance increase, decrease, or stay the same? (c) Will the time interval between the instants at which they hit the ground be smaller than, equal to, or larger than the time interval between the instants of their release?
JMU CFI May Symposium 2016 Dr. Michael S. Kirkpatrick
Peer InstructionConcepTest
A person standing at the edge of a cliff throws one ball straight up and another ball straight down at the same initial speed. Neglecting air resistance, the ball to hit the ground below the cliff with the greater speed is the one initially thrown 1. upward. 2. downward. 3. neither—they both hit at the same speed.
Answer: 3. Upon its descent, the velocity of an object thrown straight up with an initial velocity v is exactly -v when it passes the point at which it was first released.
JMU CFI May Symposium 2016 Dr. Michael S. Kirkpatrick
Peer InstructionConcepTest
Consider the following statement: Bill should quit smoking because people who smoke get lung cancer.
What is the best interpretation of the premise? 1. Most people who smoke get lung cancer. 2. Everyone who smokes gets lung cancer. 3. Some people who some sometimes get lung cancer. 4. People who smoke are more likely to get lung cancer than people who
don’t smoke.
Answer: 4. This example illustrates the principle of charity by introducing the claim of likelihood into the speaker’s assertion.
Peer Instruction in the Humanities Project, Monash University http://artsonline.monash.edu.au/peer-instruction-in-the-humanities
JMU CFI May Symposium 2016 Dr. Michael S. Kirkpatrick
Peer Instruction
Examples of Peer Instruction in the humanities • Have students select the main idea in a text, audio, or
video clip. • Engage students in discussion about a key point in an
image, text, audio, or video clip that may have more than one valid answer.
• Solicit views on contentious points and explain their perspectives to peers.
Peer Instruction Blog https://blog.peerinstruction.net/2013/06/10/3-easy-ways-to-use-clickers-and-peer-instruction-in-the-arts-and-humanities/
JMU CFI May Symposium 2016 Dr. Michael S. Kirkpatrick
ConcepTests
Guidelines for writing ConcepTests • Focus on a single concept • Require conceptual understanding, not
solvable with just equations or facts • Use unambiguous language • Use students’ wrong answers from old exams as options • Avoid converging answers • Un-situate concepts with near and far transfer • Be aware of intrinsic, germaine, and extraneous cognitive load
• Provide worked examples, not just solutions
JMU CFI May Symposium 2016 Dr. Michael S. Kirkpatrick
Far transfer
90 ft
90 ft x
Apply the Pythagorean theorem to the above triangle to find the value of x.
In a baseball diamond, the distance between each base is 90 ft. Which of the following is true about the shortest distance between 1st and 3rd bases (the red line shown above)? 1. It is less than 90 ft. 2. It is between 90 and 180 ft. 3. It is greater than 180 ft.
90 ft 90 ft
90 ft 90 ft
JMU CFI May Symposium 2016 Dr. Michael S. Kirkpatrick
Work time
ConcepTest writing • Recall the learning objective that you identified as
challenging for students in the think-pair-share • Construct a stem for that objective • Write a misconception or two that students have about
the topic • Rephrase the misconception as an answer • Test your ConcepTest on a peer from a different
discipline
JMU CFI May Symposium 2016 Dr. Michael S. Kirkpatrick
Backup slides
JMU CFI May Symposium 2016 Dr. Michael S. Kirkpatrick
ConcepTests
The Force Concept Inventory (FCI) is a validated test to measure learning in physics. Students take the FCI on the first and last day of class. In one study, women scored an average of 60% on the pretest; women in traditional lectures scored an average of 70% on the posttest, whereas women in interactive classrooms scored an average of 85% on the posttest. Men scored an average of 70% on the pretest; men in traditional lectures scored an average of 80% on the posttest, compared with 87% in interactive classrooms.
JMU CFI May Symposium 2016 Dr. Michael S. Kirkpatrick
ConcepTests
Which statement best summarizes the pre- vs. post- test gender performance gap in the physics courses studied?
(A) In both traditional and interactive classrooms, the gender performance gap increased on the post-test.
(B) The gender performance gap increased in traditional lectures but stayed constant in interactive classrooms.
(C) The gender performance gap stayed constant in traditional lectures but decreased in interactive classrooms.
(D)The gender performance gap increased in traditional lectures but decreased in interactive classrooms.
JMU CFI May Symposium 2016 Dr. Michael S. Kirkpatrick
ConcepTests
Cooperative learning closes the gender gap
• Pre-test scores were 10% points higher for men
• Gap persisted with lecture alone
• Post-test results for cooperative classes were almost equal
• Requires more than just interactive lecture
T: traditional lectures IE: interactive lectures IE+: interactive assignments, lectures, tutorials
E. Mazur, “The scientific approach to teaching: Research as a basis for course design,” keynote/plenary talk at the International Computing Education Research Conference (ICER), 2011. http://mazur.harvard.edu/search-talks.php?function=display&rowid=1712
JMU CFI May Symposium 2016 Dr. Michael S. Kirkpatrick
ConcepTests
The evidence supporting active learning suggests that passive engagement with information can contribute to reinforcing misconceptions. One study examined how effective in-class physics demonstrations were in helping to understand concepts. All students began with a reading assignment. Some students then took part in an in-class demonstration; the control group did not observe or take part in a demonstration. The students completed a short test to conclude the experiment. Which group did the worst on the post-test, missing the most points?
JMU CFI May Symposium 2016 Dr. Michael S. Kirkpatrick
ConcepTests
Which group did the worst on the post-test, missing the most points?
(A) Students who did not observe a demo (control group) (B) Students who only observed the demo (C) Students who predicted the outcome before it occurred by
writing down a guess (D) Students who discussed the outcome with peers after
observing what occurred
JMU CFI May Symposium 2016 Dr. Michael S. Kirkpatrick
Two systems model
Christopher Chabris and Daniel Simons The Invisible Gorilla http://www.theinvisiblegorilla.com/videos.html
top related