cu berkeley workshop #1: writing great clicker questions
DESCRIPTION
How does a teacher use questioning effectively? This workshop will focus on writing those questions that engage students, spark their curiosity, help recap material, give you insight into their thinking, or help them learn critical ideas in your discipline. We will focus on the use of clickers with "peer instruction" -- a research-tested method of requiring students to discuss challenging questions with one another. We will discuss how clickers can help facilitate this teaching strategy, investigate the surprising power of multiple-choice questions to achieve critical thinking skills, plus spend time discussing the elements of effective questions and practicing writing and improving questions for our classes.TRANSCRIPT
WRITING GREAT CLICKER QUESTIONS
Make Clickers Work for You
Dr. Stephanie V. Chasteen Physics Department
& Science Education InitiativeUniv. of Colorado at Boulder
http://colorado.edu/sei
Web and blog: http://sciencegeekgirl.com
Email: [email protected]
What do you teach?
A. ScienceB. Engineering or MathC. Social sciencesD. HumanitiesE. Administration / faculty supportF. Other
Show of hands
Have you used response systems (clickers) in your teaching?
A. Not at all, and I haven’t seen them usedB. Not at all, but I’ve observed their use
somewhatC. I’ve used them a littleD. I’ve used them a lotE. I could be (should be?) giving this workshop
Take a clicker & turn it on If the green light flashes,
your vote has been counted
How familiar are you with Mazur’s “Peer Instruction”
A. Fairly familiar, and I like itB. Fairly familiar, but I’m not sure that I like itC. I’ve heard of it but only have a vague idea
what it isD. Not familiar at allE. Not sure
Colored cards
Introducing Me
Applying scientific principles to improve science education – What are students learning, and which instructional approaches improve learning?
Science Education Initiative
http://colorado.edu/SEI
Physics Education Research Group
One of largest PER groups in nation, studying technology, attitudes, classroom practice, & institutional change.
http://PER.colorado.edu
Blogger
http://blog.sciencegeekgirl.com
5
Learning goals
Clic
kers
& p
eer
inst
ruct
ion
Why question?
How many times have you given a lecture and found that students hadn’t followed you?
Can you rely on students to ask questions if they don’t understand something?
Can you rely on students to know if they don’t understand something?
What are the benefits of questioning?
whiteboardCredit: Rosie Piller
8
Agenda
1. When and how we can ask questions2. About clickers as a way to ask questions,
including some common challenges3. Writing good questions. Example
questions, writing our own.4. Action plan
10
Learning goals: Participants will be able to….A. Explain several benefits of questioning and of using clickers to
questionB. Defend the use of best practices in questioning to overcome
common challengesC. Formulate and revise clicker questions to target student
learning goals
Exercise #1: Question brainstorm
What questions could you ask to help students achieve your assigned learning goal -- to test mastery and stimulate learning?
Brainstorm as a group
whiteboard
11
5 minutes
When can we ask questions?
Credit: Rosie Piller and Ian Beatty.
12
BEFORESetting up instruction
MotivateDiscoverPredict outcomeProvoke thinkingAssess prior knowledge
DURINGDeveloping knowledge
Check knowledgeApplicationAnalysisEvaluationSynthesisExercise skillElicit misconception
AFTER Assessing learning
Relate to big pictureDemonstrate successReview or recapExit poll
Some methods of asking questions
Ask rhetoricallyTarget the class (how?)Target someone in particular (in what order?)Wait and then… (call on whom?)Answer your own questionLeave the question unanswered
Or ask out of classBlogsDiscussion boardsHomework…Credit: Rosie Piller
13
Why use clickers to target the class? An outline of Peer Instruction.
14
15
Ask Question
Peer Discussion
Vote
Class Discussion
…Lecture… (Maybe vote)
* See also: Peer Instruction, A User’s Manual. E. Mazur.
Anatomy of Peer Instruction
16
Note: Grading for Formative Assessment
Motivate students to participate, without stressing over the right answer
We recommend extra credit for:•Mostly participation (eg., 2 points)•Some for correctness (eg., 1 point)
A new research study (James & Willoughby, 2011) shows:Giving points for correctness creates less productive classroom conversations! See http://theactiveclass.com
Note: Timing / Groups
◦2-5 questions spaced through an hour
◦Discussion with peers (usually nearest neighbors)
17
18
Ask Question
Peer Discussion
Vote
Class Discussion
…Lecture… (Maybe vote)
* See also: Peer Instruction, A User’s Manual. E. Mazur.
Questions about this process?
19
But not a magic bullet!
Clickers are a tool for questioning
Peer instruction helps students learn
Research shows that:Students can better answer a similar
question after talking to their peersPeer discussion + instructor explanation
works better than either one aloneStudents like peer instruction, from intro to
the junior levelStudents in courses using peer instruction
outperform those in traditional lecture courses on a common test
21
See http://STEMclickers.colorado.edu for various references
Do you want to see the details of some of this
research?A. Yes, cut back on other
stuffB. No, let’s just get to the
question writingC. I’d like to see it in the
Q&A portion afterwards
How is a clicker question the same or different?*
Similar in terms of goalsMultiple choiceAnonymous (to peers)Every student has a voice –
the loud ones and the shy ones
Forced wait timeYou can withhold the answer
until everyone has had time to think (choose when to show the histogram)
26
* From other types of in-class questions
What does this tool help us to do?
U. Colorado clicker resources…27
Videos of effective use of clickers
http://STEMclickers.colorado.edu
Clicker resource page
http://STEMvideos.colorado.edu
2-5 mins long
• Instructor’s Guide• Question banks• Workshops• Literature / Articles
Which of these could be clicker questions?
Credit: Rosie Piller and Ian Beatty.
28
BEFORESetting up instruction
MotivateDiscoverPredict outcomeProvoke thinkingAssess prior knowledge
DURINGDeveloping knowledge
Check knowledgeApplicationAnalysisEvaluationSynthesisExercise skillElicit misconception
AFTER Assessing learning
Relate to big pictureDemonstrate successReview or recapExit poll
Let’s try it.
A. Writing good questionsB. Getting students to engage with the
questionsC. Getting students to share their answers with
the whole class / the same students always share
D. It takes too long for me to learn to do thisE. I have a lot of content to cover, it takes too
much class time
I think that the toughest thing about using clickers and peer instruction must be:
30
A science-related example…
Which superpower would you rather have? The ability to…
A. Change the mass of thingsB. Change the charge of thingsC. Change the magnetization of thingsD. Change the boiling point of things
Question: Ian Beatty, UNC Greensboro Image: Thibault fr on Wikimedia
30
31
Example question: Math
Your sister in law calls to say that she’s having twins. Which of the following is the most likely? (Assume she’s having fraternal, not identical, twins)
A.Twin boysB.Twin girlsC.One girl and one boyD.All are equally likely
Derek Bruff, Vanderbilt
Example Question: Survey
Which of the following are you least comfortable using to solve problems?
A. KinematicsB. Newton’s LawsC. Work-Energy TheoremD. Momentum-Impulse TheoremE. Angular Momentum-Angular Impulse Theorem
32
Ian Beatty, UMass Amherst
Two things to pay attention to in your questions
What is the goal of my question? What am I trying to accomplish?
Is my question at the right level / variety of depth?
Question goals
Credit: Rosie Piller and Ian Beatty.
34
BEFORESetting up instruction
MotivateDiscoverPredict outcomeProvoke thinkingAssess prior knowledge
DURINGDeveloping knowledge
Check knowledgeApplicationAnalysisEvaluationSynthesisExercise skillElicit misconception
AFTER Assessing learning
Relate to big pictureDemonstrate successReview or recapExit poll
COLTT CU 2009
Completely useless
Mostly useless
Somewhat useful
Useful
Very useful
Challenging conceptual
Recalling a previous fact
Recalling a recent fact
Plugging numbers into equation
Types of clicker questions:
0% 10% 20%
30% 40% 50% 60%% of students
91%
35%36%18%
N=4 courses, 66 students
Question Writing Depth
36
But how do we increase the depth of questioning?
Question Writing Depth: Bloom’s Taxonomy
Handout with handy verbs and question stems for different levels, e.g.: UNDERSTAND: match,
paraphrase, restate APPLY: choose, explain, show ANALYZE: compare, classify,
categorize EVALUATE: judge, criticize,
defend SYNTHESIS: combine,
develop, design
37
Handout: Bloom’s Taxonomy
Preparing to Write Questions
Read briefly over the “tips for writing clicker questions” handout.
Which is going to be most challenging for you?Which would you tell a colleague about?
3 minutes
Preparing to Write Questions
In groups of 2-3, choose one of the questions that you brainstormed at the beginning of the workshop. You will write a multiple choice version of this question.
3 minutes
Gallery Walk
As a table, look at the “example questions” trio that I have given you. What’s a common theme(s)?
Write the themes you find down on the sheet so that other groups will be able to read it.
After 5 minutes, circulate to see the themes of questions on other tables.Shop for ideas for your own questions!See handouts for a place to jot your notes.
10 minutes
Gallery Walk: Report Out
What was the theme of your question trio?
When would you use such a type of question?
Exercise #3: Writing Questions42
Using ideas you’ve learned, write a multiple choice version of your question in groups of 2-3.
Show your question to another group (and to me) for suggestions on revising it.
If you have time, write another question from another part of the questioning cycle.
10 minutes
Share-Out about Question Writing
What was challenging?
What worked well for you?
What questions or concerns do you have about writing questions?
How might you write questions that integrate with your lectures?
This workshop can’t do it all
There are great books to readPair up with other instructorsI give free webinars (see iclicker.com)Next workshop, 4:30-6:00, Weds Feb 1st. (4:00-4:30
refresher course for new folks)
44
Making Clickers Work for You: Facilitation
I.e., taking off the rose-colored glasses. What goes wrong? How can this technique work best? BRING YOUR HANDOUTS!
Action Plan
Take a few minutes to write down your action plan to implement ideas you heard about in the workshop
45
Many materials in this workshop (particularly the questioning cycle and the participant
exercises) were adapted from Rosie Piller, Making Students Think: The Art of Questioning. Short papers published in: Computer Training & Support Conference, 1995; ISPI International Conferences, 1991 and 1996; ASTD National Conference on Technical & Skills Training, 1990. Related workshop description at http://www.educationexperts.net/nstworkshop.html. Other materials (particularly
sample clicker questions and goals of clicker questions) adapted from Ian Beatty’s Technology Enhanced Formative Assessment (TEFA) program. http://ianbeatty.com/crs
Resource Page: http://STEMclickers.colorado.eduWeb and blog: http://sciencegeekgirl.comEmail: [email protected]
Thanks!
NOW: Q&A, continued work on questions and revision, individual consultations.
NEXT WEEK, 4:30-6pm – Facilitation Tips & Techniques
Learning Goals
Biology: Recognize the components of a cell and describe why each is necessary for the function of a cell
Physics: Identify the different ways that light can interact with an object (i.e., transmitted, absorbed, reflected).
Chemistry: Explain trends in boiling points in terms of intermolecular interactions
Earth science: Understand the formation of the three major types of rocks (igneous, sedimentary, and metamorphic) and the processes by which they form, relating them by the rock cycle.
Math: Solve a system of linear equations in two variables using algebra or graphing.
What Do I do if…?
What can you do if you ask questions and..There is no responseThe same people keep raising their handsThe answers are called out before everyone
has a chance to thinkThe answers take too longSomeone gives a wrong answerOnly some students are prepared?
48
We’ll discuss in Workshop #2.
For now: Many of these challenges are addressed by clickers