Assessment:

Tools and Methods

Gail Matthews-DeNatale

Northeastern University

Dave Consiglio

Bryn Mawr College

The Cycle

Identify

Scope

Interpret

Data

Share &

Act Upon

Findings

Form

Research

Question

Design &

Implement

ResearchYou Are

Here

Turning Gold Into Dimes

Your Ideas?

Turning Gold into Dimes

Open-ended questions on quant

Formative feedback surveys

Focus groups

Observations

Interviews

Qualitative “Usual Suspects”

There’s no such thing as a “perfect” interview

If you ask yes/no questions, that’s what you will get

The better you listen, the more articulate your

interviewees will become

Practicing with recording equipment beforehand is

like carrying an umbrella

Your last interview will be orders of magnitude better

than your first – plan accordingly

Interviewing Truisms

Shadowing

Relational mapping

Usability think-aloud

Participant observation

Photo elicitation (participant-created)

Outside the Box

http://www.flickr.com/photos/artjonak/6250513028

Take pictures of

• Your favorite place to study

• The computer you use

in the library

• One picture of the libraries

to show to a freshman

• All the stuff you take to class

• The place you keep your books

• Your favorite person or people to study with

• A place in the library where you feel lost

• The night before a big assignment is due

Source: Foster and Gibbons, Studying Students (2007)

For Example

Quantitative Surveys

Pre-Plan

Plan

Construct Instrument

Pre-test

Implement

Analyze

Aim for

Exhaustive answer set

Mutually exclusive

One thing at a time

Balanced question wording

Simple wording

Avoiding bias

Question/answer agreement

Other Things to Avoid

Double negatives

Acronyms

Insider language

Extensive recall of the past

Agree/disagree questions

Neutral answers

Open and/or unnecessary questions

Long surveys

Return to Scenarios

Develop research

plan & methodology

20Minutes

Return to Scenarios

Group

Critique

10Minutes

Final Truisms

Quantitative tells you what

Qualitative tells you why

Combining both is great

Final Truisms

Quantitative reaches the

rational mind

Qualitative touches the

emotional mind

To persuade others, use

accordingly