ctd weekly workshops: writing good peer instruction questions

CTD WEEKLY WORKSHOPS:

WRITING GOOD PEER

INSTRUCTION QUESTIONS

Peter Newbury

Center for Teaching Development,

University of California, San Diego

[email protected] @polarisdotca

ctd.ucsd.edu #ctducsd

slides and resources: ctd.ucsd.edu/programs/fall-2013-weekly-workshops/

Wednesday, November 13, 2013

12:00 – 12:50 pm Center Hall, Room 316

2

Writing good peer instruction questions

Clicker Question (Economics)

Writing good peer instruction questions 3

For which of the following professionals is driving an

expensive car a credible signal of their relative abilities

(that is, compared to others in the same profession)?

A) a carpenter

B) a realtor

C) a politician

D) a major league baseball player

(Steve Morris, UCSD)

Typical Episode of Peer Instruction


1. Instructor poses a conceptually-challenging multiple-choice question.

2. Students think about question on their own and vote using clickers, colored ABCD cards, smartphones,…

3. The instructor asks students to turn to their neighbors and “convince them you’re right.”

4. After that “peer instruction”, the students vote again and the instructor leads a class-wide discussion concluding with why the right answer(s) is right and the wrong answers are wrong.

In effective peer instruction


students teach each other while

they may still hold or remember

their novice preconceptions

students discuss the concepts in their

own (novice) language

each student finds out what s/he does(n’t) know

the instructor finds out what the students know (and

don’t know) and reacts, building on their initial

understanding and preconceptions.

students learn

and practice

how to think,

communicate

like experts

Effective peer instruction requires


1. identifying key concepts, misconceptions

2. creating multiple-choice questions that

require deeper thinking and learning

3. facilitating peer instruction episodes that

spark and support student discussion

4. leading a class-wide discussion to clarify

the concept, resolve the misconception

before

class

during

class

What makes a good clicker question?


clarity Students should waste no effort trying to figure

out what’s being asked.

context Is this topic currently being covered

in class?

connection to

learning goals

Does the question make students do the right

thing to demonstrate they grasp the concept.

distractors What do the “wrong” answers tell you about

students’ thinking?

difficulty Is the question too trivial? too hard?

stimulates

thoughtful

discussion

Will the question engage the students and

spark thoughtful discussions?

Is there potential for you to be “agile”?

(Adapted from Stephanie Chasteen, CU Boulder)

t h e l e a r n i n g c y c l e

Peer instruction helps students learn...


BEFORE DURING AFTER

setting up

instruction

developing

knowledge

assessing

learning

Adapted from Rosie Piller, Ian Beatty, Stephanie Chasteen




BEFORE DURING AFTER

setting up

instruction

developing

knowledge

assessing

learning


Clicker question


Melt chocolate over low heat. Remove the chocolate

from the heat. What will happen to the chocolate?

A) It will condense.

B) It will evaporate.

C) It will freeze.

(Question: Sujatha Raghu from Braincandy via LearningCatalytics)

(Image: CIM9926 by number657 on flickr CC)

assess prior knowledge

Clicker question


Which had the most positive impact on the modern world?

A) coffee

B) tea

C) chocolate

D) spice

E) sugar

(Herbst, UCSD)

provoke thinking

clarity

context

learning goals

distractors

difficulty

discussion

Clicker question


In your opinion, which had the most positive impact on

the modern world?

A) coffee

B) tea

C) chocolate

D) spice

E) sugar

(Herbst, UCSD)

provoke thinking

Clicker question


A ball is rolling around

the inside of a circular

track. The ball

leaves the track

at point P.

Which path

does the ball

follow?

P

A

B C

E

D

(adapted from Mazur)

predict




BEFORE DURING AFTER

setting up

instruction

developing

knowledge

assessing

learning


The students have not

(re)solved concept X.

But they’re know X exists

and why X is interesting.




BEFORE DURING AFTER

setting up

instruction

developing

knowledge

assessing

learning


Clicker question


Which of these are reasons for the seasons? i. the height of the Sun in the sky during the day ii. Earth’s distance from the Sun iii. how many hours the Sun is up each day A) ii only B) iii only C) i and ii D) i and iii E) i, ii and iii

clarity

context

learning goals

distractors

difficulty

discussion

probe misconception

Clicker question


How many of these are reasons for the seasons?

height: the height of the Sun in the sky during the day

distance: Earth’s distance from the Sun

hours: how many hours the Sun is up each day

A) none of them

B) one

C) two

D) all three

probe misconception

Clicker question


Select the line that

you feel has the

strongest imagery in

“Fast rode the

knight” by Stephen

Crane (1905).

analysis

Fast rode the knight

With spurs, hot and reeking,

Ever waving an eager sword,

"To save my lady!"

Fast rode the knight,

And leaped from saddle to war.

Men of steel flickered and gleamed

Like riot of silver lights,

And the gold of the knight's good banner

Still waved on a castle wall.

. . . . .

A horse,

Blowing, staggering, bloody thing,

Forgotten at foot of castle wall.

A horse

Dead at foot of castle wall.

A

B

C

D

E

F

G

H

I

J

K

L

M

N

O

P

(David Kurtz, via LearningCatalytics)

Clicker question


Select the line that

you feel has the

strongest imagery in

“Fast rode the

knight” by Stephen

Crane (1905).

analysis

Fast rode the knight

With spurs, hot and reeking,

Ever waving an eager sword,

"To save my lady!"

Fast rode the knight,

And leaped from saddle to war.

Men of steel flickered and gleamed

Like riot of silver lights,

And the gold of the knight's good banner

Still waved on a castle wall.

. . . . .

A horse,

Blowing, staggering, bloody thing,

Forgotten at foot of castle wall.

A horse

Dead at foot of castle wall.

A

B

C

D

E

(David Kurtz, via LearningCatalytics)

Clicker question


Evaluate: A)

B)

(adapted from Bruff (2009))

4

0

32 1 dxxx

23)65(16

9

16

C.

D.

)165(9

2 23

3

1022clarity

context

learning goals

distractors

difficulty

discussion

exercise skill

Clicker question


Which of the following is an incorrect step when using

the substitution method to evaluate the definite integral

A)

B)

4

0

32 1 dxxx

31 xu

dxxdu 2

3

C.

D. none of the above

4

03

1duu

(adapted from Bruff (2009))

evaluation

Clicker question


Susan throws a ball straight up into the air. It goes up

and then falls back into her hand 2 seconds later.

Draw a graph showing the velocity of the ball from the

moment it leaves her hand until she catches it again.

time

velocity

2 sec 0

exercise skill

(CWSEI UBC)

time

velocity

2 sec 0

A time

velocity

2 sec 0

B

time

velocity

2 sec 0

C time

velocity

2 sec 0

D

E) some other graph

Which one is the closest match to your graph? exercise skill

(CWSEI UBC) Writing good peer instruction questions 23




BEFORE DURING AFTER

setting up

instruction

developing

knowledge

assessing

learning


Students have had

opportunities to

try, fail, receive feedback

and try again without facing

a summative evaluation. [3]




BEFORE DURING AFTER

setting up

instruction

developing

knowledge

assessing

learning



Clicker question

Are features X and Y

ridges or valleys?

A) X=ridge, Y=valley

B) X=valley, Y=ridge

C) both are ridges

D) both are valleys

X

Y

(EOSC / CWSEI, UBC)

demonstrate success

Clicker question


For the data given below, which is larger, the mean or

the median?

74, 32, 35, 87, 28, 36, 11, 26, 93, 56, 34, 52, 8

A) mean

B) median

(Peck, mathquest.carroll.edu/resources.html)

review / recap

clarity

context

learning goals

distractors

difficulty

discussion

Clicker question


For the data set displayed in the following histogram,

which would be larger, the mean or the median?

A) mean

B) median

C) can’t tell from the given histogram

(Peck, mathquest.carroll.edu/resources.html)

review / recap

Clicker question


In your opinion, which had the most positive impact on

the modern world?

A) coffee

B) tea

C) chocolate

D) spice

E) sugar

“big picture”

(Herbst, UCSD)

Your turn…


Your turn…


Big Idea/Concept/Skill/ Learning Outcome

Why do you need a peer

instruction question here in

the lesson?

Your turn…


Question: (and choices)

Think about

clarity context

learning outcome

distractors difficulty

discussion

Your turn…


What should students say to explain why this choice is correct/incorrect?

It’s not just about correct

or incorrect.

Direct the conversation!

Your turn…


Are there really five

different, meaningful

conversations?

(Are there even 4?)

Your turn…


Learning outcome:

By the end of this Driving

School lesson, you’ll be

able to judge if you are

following the car ahead

of you at a safe

distance.

Peer instruction helps teachers teach


BEFORE DURING AFTER

setting up

instruction

developing

knowledge

assessing

learning





BEFORE DURING AFTER

setting up

instruction

developing

knowledge

assessing

learning

Do they care about this?

Are they ready for the next topic?

What DO they care about, anyway?

What do they already know?

t h e l e a r n i n g c y c l e Did they notice key idea X?

Where are they in the activity?



BEFORE DURING AFTER

setting up

instruction

developing

knowledge

assessing

learning

Are they getting it?

Do I need to intervene?

t h e l e a r n i n g c y c l e How did I do?

Did they get it?



BEFORE DURING AFTER

setting up

instruction

developing

knowledge

assessing

learning

Can I move to the next topic?

Did that activity work?

Writing good PI questions


It’s critical to have

content knowledge (the concepts)

pedagogical content knowledge (how people learn

the concepts in your discipline and how to teach them)

Running effective PI


It’s critical to

teach the students how to engage in peer instruction

choreograph each episode so students waste no

precious cognitive load wondering what to do

(call us for another workshop!)

You might not write the perfect question the first time so

listen to the students’ conversations

write your self some notes immediately after class

revise and try it again next year

References


1. National Research Council (2000). How People Learn: Brain,

Mind, Experience, and School: Expanded Edition. J.D.

Bransford, A.L Brown & R.R. Cocking (Eds.),Washington, DC:

The National Academies Press.

2. Bruff, D. (2009). Teaching with Classroom Response Systems.

San Francisco: Jossey-Bass.

3. Bain, K. (2004). What the best college teachers do.

Cambridge, MA: Harvard University Press.

What makes a good clicker question?

Writing good peer instruction questions

Center for Teaching Development ctd.ucsd.edu

clarity Students should waste no effort trying to figure

out what’s being asked.

context Is this topic currently being covered

in class?

connection to

learning goals

Does the question make students do the right

thing to demonstrate they grasp the concept.

distractors What do the “wrong” answers tell you about

students’ thinking?

difficulty Is the question too trivial? too hard?

stimulates

thoughtful

discussion

Will the question engage the students and

spark thoughtful discussions?

Is there potential for you to be “agile”?

(Adapted from Stephanie Chasteen, CU Boulder) 43

ctd weekly workshops: writing good peer instruction questions

Education

peer instruction episodes

good clicker question

novice preconceptions

clicker question economics

stephanie chasteen

rosie piller

ian beatty

multiplechoice questions