C L I C K E R Q U E S T I O NS I N T H E H U M A N I T I E S A N D S O C I A L

S C I E N C E S : D E S I G N A N D E X A M P L E S

OXNARD COLLEGE INSTRUCTIONAL SUPPORT SERVICES

BOLA KING-RUSHING, INSTRUCTIONAL TECHNOLOGIST

SUMMER 2009, UPDATED JUNE 2013

INTRODUCTION

The first thing to think about when designing clicker questions is what role the

questions will play in your lecture or discussion. What are the key concepts or themes that

you want to emphasize during the course? Is there a particular way you hope to use the

clickers in lecture? Once you get down to the level of individual lectures, what is the goal of

each question therein? There are myriad possibilities, and each one may require different

kinds of clicker questions.

So how do you get started? Although of course there are many ways to proceed, two

main suggestions come up repeatedly. The first is to think about and list the main goals of

the course overall as well as the main learning goals for each lecture; this list may be based

on (and in turn may help you revise) the course student learning outcomes. Use this list as

the starting point for generating questions, keeping in mind your teaching style. The second

approach is to start from your notes or slides (if you use PowerPoint or another

presentation technology). The goal in this case is to identify places in the lecture,

presentation, or discussion that might be good spots for engaging questions. Remember that

Clicker Questions June 2013

2

in general you are advised not to stop your class for questions more than once every ten

minutes or so.

STRATEGIES FOR COMPOSING QUESTIONS

Different researchers and practitioners give, unsurprisingly, different kinds of

advice regarding question design. The University of North Carolina’s Ian Beatty, for

example, admonishes us to avoid “simple factual questions or those that probe memory

rather than understanding” (10). However, in line with the idea of using clickers to “check

in” with students to see whether they are on the same page (with each other or with you),

questions that test recall or factual knowledge can be both valid and useful. This level of

questioning can also be used to inform your lecture or to create discussion. The approach

taken by Beatty et al. is rigorous and exacting, requiring that every question fulfill an

“explicit pedagogic purpose consisting of a content goal, a process goal, and a metacognitive

goal” (abstract 31). This sounds like an ideal way to go, but realistically the development of

a whole set of such questions is probably very time-consuming. Other practitioners are

content to have each question address a smaller goal; here we will introduce several

possibilities.

One overall goal for which we are always striving is to motivate students to do the

assigned reading. Such strategies as written reading responses or quizzes (both announced

and unannounced) have proven effective in both large and small classes. Particularly in

larger classes, though, clicker questions offer an efficient alternative to in-class quizzes,

since time is not lost in handing out or collecting papers. These questions can be used as a

review of assigned work or to highlight aspects of a reading that should be emphasized or

remembered, and they can also serve to preview the types of questions that may appear on

Clicker Questions June 2013

3

an exam so that students get an idea of how they should be thinking about an issue. Former

UCSB instructor Indy Hurt occasionally gave such a question to her class and then let the

students know that if enough of them (usually 90% in her very large geography classes) got

it right, they would see that same question on the next exam. Richard Mayer, another UCSB

professor, similarly promised students that some of his clicker questions would appear on

exams. Hurt sometimes used questions to help students focus on particular sections of their

reading by telling (or warning) them that she would ask a clicker question about a

particular concept during the next lecture (see also Woelk 1404).

The strategy of using clicker questions to promote discussion during lecture is

widespread in the research literature, especially if the question has more than one right

answer or contains ambiguities that might misdirect students. According to Beatty,

ambiguity “sensitizes students to the ambiguous point’s implications, trains them to pay

attention to subtleties in a situation, and motivates a discussion about what aspects of a

question statement are important and how they matter” (10). Hurt and UCSB physics

professor Roger Freedman both espoused the use of ambiguities or distractors in a

question’s answers; successfully executed, this can result in more than one answer that

seems correct on the surface or at first glance. These instructors have used this type of

question as a way to encourage deeper thinking as well as discussion in class: when the

histogram shows that the class is evenly split between two or even three responses, you can

ask for a volunteer to explain or defend his or her answer.

As another option, the students can also spend a few minutes discussing their

answers with their neighbors in an attempt to come to a consensus and then vote again;

Freedman uses this approach as an explicit adaptation of Frank Lyman’s “think-pair-share”

teaching method. Of course, as Freedman points out, even if a consensus is reached, it may

Clicker Questions June 2013

4

not be around the correct response, but such an outcome is still useful as an effective

teachable moment. If there is a marked shift in the responses the second time around, you

can ask the students to discuss why that occurred. These strategies can force students to

think more deeply about why they chose particular responses, opening up avenues to

metacognitive growth. Be aware that this type of question is sometimes difficult to create;

trying to use too many may distract from your other class preparations. It is also possible

and perhaps a bit easier to create other questions that will spark a discussion by focusing on

oppositional readings or other ideas that directly challenge students’ preconceptions.

Clicker questions can be used to provide feedback to the instructor (the anonymity

function available in the i>clicker system can prove particularly valuable here). You can find

out how much confidence students have in their understanding or abilities (how well do

you know the MLA style? How confident are you in your understanding of feminist

criticism?), or ask what material is giving students the most trouble, identifying the

“muddiest point” in the material (see Mosteller as well as Martin). Such feedback can be

helpful as you plan subsequent class sessions, and it can also help you think about future

versions of the course. In addition, this type of feedback can be a key component in a

strategy of “contingent teaching” (Draper and Brown 92), in which you alter your discussion

on the fly in reaction to how well or poorly the students seem to comprehend the material

(this approach is also call “just in time” or “agile teaching”). You can have the students

provide a brief evaluation of the course (is the pace of lecture too fast? are there too many

clicker questions?) at any point during the term as well, rather than waiting until after

grades have been posted to find out how things went. Questions can also provide feedback

to the students: sample exam questions or other review questions can let students know

objectively where their weak spots are.

Clicker Questions June 2013

5

In addition to the classroom-related goals discussed above, you can consider the

cognitive or metacognitive goal(s) of the question you’re designing. One widespread

method for addressing this concern and finding guidance is to consult the Taxonomy of

Educational Objectives, popularly known as Bloom’s Taxonomy. In brief, this is a hierarchy

of cognitive domains (factual knowledge, comprehension, application, analysis, synthesis,

and evaluation), each of which represents a different level of cognitive activity. Thinking

about these different levels can help in designing clicker questions. Identification of terms,

plot points, or characters would fall into the lowest level of recall and factual knowledge;

identifying the perspective a critical writer takes in a particular passage would be a higher-

level problem of analysis or synthesis. (Useful web links regarding the Taxonomy, its

variations, and application to multiple-choice questions are located at the end of this

document.)

Bloom’s Taxonomy is a mainstay in pedagogical research, but many revisions,

rewritings, and complete replacements have been proposed since its appearance in 1956.

Anderson and Krathwohl, for example, oversaw a modest but significant revision in 2001

(Krathwohl was one of the original contributors to Bloom’s volume). On the other hand,

Beatty et al. in their 2006 article ignore the taxonomy altogether in favor of their twelve

“habits of mind.” No system or hierarchy is particularly correct, but any of them can provide

the direction needed to design quality clicker questions.

Moving away from actual assessment, there is another way to use clicker questions:

polling. There are several reasons to ask questions that can have no right answers.

Demographic polling may help break the ice in large lectures; asking students where they

are from, what their standing in school is (freshman, sophomore, etc.), or what age group

they belong to might help them feel less isolated in the large-class setting. Anonymous

Clicker Questions June 2013

6

polling can also be used in conjunction with sensitive questions that might serve, for

instance, to introduce a topic or author.

It’s important to remember that the i>clicker software can do more than just collect

and display responses to questions. Questions can be made anonymous; that is, the

responses are collected and tabulated, but are not associated with individual student IDs.

This function might be useful if you are interested in polling the students for demographic

or sensitive information (what age group are you in? how have you been affected by violent

crime?). The results from two questions can also be displayed side-by-side for comparison,

which would be, for example, an effective way to compare pre- and post-discussion results

on a single question. If you find yourself inspired in the middle of lecture, it is also easy to

create an on-the-fly question right then and there.

Finally, there are a couple of things to be careful of in your clicker preparations. Ken

Masters of the University of Cape Town cautions us to “watch out for cultural bias” in our

question design; this may be a more appropriate caveat in the more “objective” science or

math classes, but is still something to keep in mind in our culture-based and often culturally

sensitive humanities and social-science courses. We are also warned not to rely on the

clicker technology to get the job of teaching done. It cannot be stressed enough that the

technology is merely a tool that can facilitate good teaching; its use alone does not achieve

or guarantee good teaching. Finally, it is important to remember that, if you commit to using

clickers in your classroom, you are no longer simply delivering a lecture; instead, think of it

as an interactive lecture, which can help you make the most of the technology.

Clicker Questions June 2013

7

CLICKER QUESTION EXAMPLES (CORRECT ANSWERS ARE INDICATED IN BOLD WHERE APPROPRIATE)

1. What type of figurative language appears in the following poem?

“Women Are Not Roses”

Women have no

beginning

only continual

flows.

Though rivers flow

women are not

rivers.

Women are not

roses

they are not oceans

or stars.

i would like to tell

her this but

i think she

already knows.

a) symbol b) simile c) personification d) metaphor e) none of the above

(Quotation taken from “Women Are Not Roses,” Ana Castillo)

Clicker Questions June 2013

8

2. What school of criticism does this statement represent?

“A poem is what the reader lives through under the guidance of the text and experiences as

relevant to the text.”

a) New Criticism b) New Historicism c) Psychoanalytic Criticism

d) Reader-Response Criticism e) Deconstructive Criticism

(Quotation taken from “Towards a Transactional Theory of Reading,” Louise Rosenblatt,

1969.)

3. What is the subject of the verb “leaps” in the following passage?

. . .and in some spongy log

Whence headlong leaps the oozy emerald frog.

And a black pupil in the green scum shows.

Her the inhabiter of divers places

Surmising at all doors, I push them all.

a) pupil b) log c) frog d) headlong e) Her

(Quotation from “Still will I harvest beauty where it grows,” Edna St. Vincent Millay.)

4. Gatsby got his wealth from...

a) the railroad industry b) bootlegging c) a family inheritance d) politics

Clicker Questions June 2013

9

5. What do you think about the government’s access to and use of surveillance technologies?

a) I think the government should have access to any and all forms of surveillance in order to

protect our society.

b) I think the government should have access to some surveillance technologies—people

who are not breaking the law have nothing to hide.

c) I think our right to privacy is very important; the government should be held to strict

rules on the use of surveillance.

d) I think privacy outweighs the government’s concerns, and the government should not be

trusted with surveillance technologies at all.

6. Which of these Nobel laureates deserved the prize most?

a) Toni Morrison b) Gabriel Garcia Marquez c) John Steinbeck

d) Ernest Hemingway e) Doris Lessing

7. How confident are you in your ability to analyze poetry?

a) very confident b) somewhat confident c) marginally confident d) not at all confident

8. Read the following passage, then answer the question:

Save that from yonder ivy-mantled tower

The moping owl does to the moon complain

Of such as, wandering near her secret bower,

Molest her ancient solitary reign.

Clicker Questions June 2013

10

Fill in the blanks in this statement: While a new critic might see the “moping owl” as an

example of the ___________________ , a ______________ critic might instead be more concerned with

the ways that the owl reflects the speaker’s own sense of melancholy.

a) patriarchy, new historicist

b) interpretive community, psychoanalytic

c) oedipus complex, reader response

d) interpretive community, feminist

e) pathetic fallacy, psychoanalytic

(Quotation from “Elegy Written in a Country Churchyard,” Thomas Gray)

9. Which of the following characters from The Tempest might a postcolonial critic be most

interested in? Be prepared to justify your answer.

a) Miranda b) Sebastian c) Trinculo d) Caliban e) Antonio

10. Rate the tone of “Montage of a Dream Deferred” and be prepared to explain your

answer:

a) very positive b) somewhat positive c) neutral (neither positive nor negative)

d) somewhat negative e) very negative

Clicker Questions June 2013

11

11. Which of the following best states the New Critics’ stance?

a) The text is a mirror that can reveal information about the society that produced it.

b) The way we respond to a text allows us to learn more about ourselves.

c) Our aim is to dive into the text in order to expose its internal inconsistencies.

d) We are mainly interested in focusing on the text itself as an aesthetic artifact.

e) The text gives us insights into the author’s thoughts and motivations.

Clicker Questions June 2013

12

SOME LINKS TO HELP IN USING BLOOM’S TAXONOMY WITH MULTIPLE-CHOICE QUESTIONS

Multiple Choice Questions and Bloom’s Taxonomy:

<http://web.uct.ac.za/projects/cbe/mcqman/mcqappc.html>.

Bloom’s Taxonomy: <http://www.coun.uvic.ca/learning/exams/blooms-

taxonomy.html>.

Bloom’s Taxonomy of the Cognitive Domain:

<http://www.nccei.org/teaching/taxonomy.html>.

Bloom’s Digital Taxonomy:

<http://edorigami.wikispaces.com/Bloom%27s+Digital+Taxonomy>.

Clicker Questions June 2013

13

WORKS CONSULTED

Anderson, Lorin W., and David R. Krathwohl, eds. A Taxonomy for Learning, Teaching, and

Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives. New York:

Longman, 2001. Print.

Beatty, Ian. “Transforming Student Learning with Classroom Communication Systems.”

Educause Research Bulletin 2004.3 (3 Feb. 2004). Print.

Beatty, Ian D., William J. Gerace, William J. Leonard, and Robert Dufresne. “Designing

Effective Questions for Classroom Response System Teaching.” American Journal of

Physics 74.1 (Jan. 2006): 31-39. Print.

Bloom, Benjamin S., ed. Taxonomy of Educational Objectives, Handbook I: Cognitive Domain.

New York: David McKay, 1956. Print.

Bruff, Derek. Teaching with Classroom Response Systems: Creating Active Learning

Environments. San Francisco: Jossey-Bass, 2009. Print.

---. Teaching with Classroom Response Systems: Resources for Engaging and Assessing

Students with Clickers. 2009. Web. <http://derekbruff.com/teachingwithcrs/>.

Caldwell, Jane E. “Clickers in the Large Classroom: Current Research and Best-Practice Tips.

” CBE--Life Sciences Education 6 (Spring 2007): 9-20. Print.

“Designing Questions for Clickers. ” Clickers. The Ohio State University’s Technology

Enhanced Learning Resources. N.d. Web. <http://telr.ohio-

state.edu/clickers/teaching/guidelines.htm>.

Clicker Questions June 2013

14

Draper, Steve W. Electronic Voting Systems and Interactive Lectures. U of Glasgow. 16 May

2009. Web. <http://www.psy.gla.ac.uk/~steve/ilig/>

Draper, Steve W., and Margaret I. Brown. “Increasing Interactivity in Lectures Using an

Electronic Voting System.” Journal of Computer Assisted Learning 20: 81-94. Print.

Fies, Carmen, and Jill Marshall. “The C3 Framework: Evaluating Classroom Response System

Interactions in University Classrooms. ” Journal of Science Education and Technology

17 (2008): 483-99. Print.

Freedman, Roger. Personal interview. 28 July 2009.

Hurt, Indy. Personal interview. 29 July 2009.

Jenkins, Alice. “Technique and Technology: Electronic Voting Systems in an English

Literature Lecture.” Pedagogy 7.3 (Fall 2007): 526-33. Print.

Kaleta, Robert, and Tanya Joosten. “Student Response Systems: A University of Wisconsin

System Study of Clickers. ” Educause Research Bulletin 2007.10 (8 May 2007). Print.

LaRose, Julie A. “Engage Your Audience: Using Audience Response Systems in SH&E

Training.” Professional Safety 54.6 (June 2009): 58-62. Print.

Lyman, Frank. “The Responsive Classroom Discussion: The Inclusion of All Students.”

Mainstreaming Digest. Ed. A.S. Anderson. College Park: U of Maryland Coll. of Educ.,

1981. 109-13. Print.

Martin, Mary Barone. “Classroom Assessment Techniques Designed for Technology.”

Proceedings of the Mid-South Instructional Technology Conference, Mar. 28-30, 1999.

Web (available via ERIC Online).

Clicker Questions June 2013

15

Masters, Ken, ed. Designing and Managing Multiple Choice Questions. U of Cape Town. N.d.

Web. <http://web.uct.ac.za/projects/cbe/mcqman/mcqman01.html>.

Mayer, Richard E. Personal interview. 23 July 2009.

Mayer, Richard E., Andrew Stull, Krista DeLeeuw, Kevin Almeroth, Bruce Bimber, Dorothy

Chun, Monica Bulger, Julie Campbell, Allan Knight, and Hangjin Zhang. “Clickers in

College Classrooms: Fostering Learning with Questioning Methods in Large Lecture

Classes.” Contemporary Educational Psychology 34.1 (Jan. 2009): 51-57. Print.

Mosteller, Frederick. “The ‘Muddiest Point in the Lecture’ as a Feedback Device.” Derek Bok

Center for Teaching and Learning, Harvard U. 2006. Web.

<http://isites.harvard.edu/fs/html/icb.topic58474/mosteller.html>. On Teaching

and Learning: The Journal of the Harvard-Danforth Center 3 (Apr. 1989): 10-21.

Print.

Robertson, Lorraine. “Twelve Tips for Using a Computerised Interactive Audience Response

System.” Medical Teacher 22.3 (2000): 237-39. Print.

Rosenblatt, Louise. “Towards a Transactional Theory of Reading.” Journal of Literary

Research 1.1: 31-49. Print.

“Student Response System: Faculty: Best Practices.” Student Response Systems (SRS): The

UW-System Clicker Project. U of Wisconsin-Milwaukee Learning Technology Center.

2009. Web. <http://www4.uwm.edu/ltc/srs/faculty/best_practices.cfm>.

“Teaching with Clickers.” Support for Effective Teaching: UNM. U of New Mexico Office of

Support for Effective Teaching. 2006. Web.

<http://www.unm.edu/~oset/teachingwithclickers.html>.

Clicker Questions June 2013

16

Woelk, Klaus. “Optimizing the Use of Personal Response Devices (Clickers) in Large-

Enrollment Introductory Courses.” Journal of Chemical Education 85.10 (Oct. 2008):

1400-05. Print.