HCI Research ProjectHCI Research Project

Research ParadigmsResearch Paradigms

Theoretical (Theoretical (in the style of mathematicsin the style of mathematics))– Mathematical deductionMathematical deduction– SimulationSimulation– Analysis of algorithmsAnalysis of algorithms

The researcher:The researcher:– Checks many examplesChecks many examples– Conjectures a result in the form of a theoremConjectures a result in the form of a theorem– Constructs the proof of the theorem from known Constructs the proof of the theorem from known

resultsresults

Research Paradigms (2)Research Paradigms (2)

Experimental (Experimental (in the style of psychologyin the style of psychology))– Hypotheses checked with controlled Hypotheses checked with controlled

experimentationexperimentation

The researcher:The researcher:– Poses hypothesesPoses hypotheses

Extensive background readingExtensive background readingAppropriate theoretical modelAppropriate theoretical model

– Conducts controlled experimentsConducts controlled experiments– Performs statistical analysis of dataPerforms statistical analysis of data– Draws conclusionsDraws conclusions

Sample Research QuestionsSample Research Questions

Why don’t typical users invoke Why don’t typical users invoke advanced search strategies?advanced search strategies?

What is the preferred method of What is the preferred method of making a choice from a lengthy list? making a choice from a lengthy list? Note behavior of scroll bar, key indexNote behavior of scroll bar, key index

What is the best semantics for the What is the best semantics for the TabTab key operating on a Web form? key operating on a Web form?

What is the best shopping cart What is the best shopping cart checkout design?checkout design?

Sample Research Questions (2)Sample Research Questions (2)

What features should pie menus What features should pie menus have in a gesture-based interface? have in a gesture-based interface?

How should information scent be How should information scent be shown on search reports?shown on search reports?

What intelligent agents are most What intelligent agents are most helpful in a tiny interface?helpful in a tiny interface?

What ZUI widgets are most helpful What ZUI widgets are most helpful for information retrieval?for information retrieval?

Questions to HypothesesQuestions to Hypotheses

Key is measurabilityKey is measurability IdentifyIdentify

– Independent variable(s): those things Independent variable(s): those things the researcher can controlthe researcher can control

– Dependent variables: those things the Dependent variables: those things the researcher can measureresearcher can measure

Aside: Aside: What What isis measurable? measurable?

Measuring ThingsMeasuring Things

ObjectivelyObjectively– Time: to do, to correct, to learnTime: to do, to correct, to learn– Counts: tasks performed, errors madeCounts: tasks performed, errors made– Distance, or other geometrical attributesDistance, or other geometrical attributes– Text: density, complexity, reading levelText: density, complexity, reading level– Images: size, colorImages: size, color

SubjectivelySubjectively– PreferencePreference– Rating scaleRating scale

Good Experimental DesignGood Experimental Design

Limit number of independent variablesLimit number of independent variables Limit number of values for each Limit number of values for each

independent variableindependent variable Make dependent variables easy to Make dependent variables easy to

measuremeasure Develop good working definitionsDevelop good working definitions Control for external biasControl for external bias

– Guard against learning and practiceGuard against learning and practice

Good Experimental Design (2)Good Experimental Design (2)

Base hypotheses on theoretical Base hypotheses on theoretical modelmodel

Phrase hypotheses in measurable Phrase hypotheses in measurable termsterms

Choose subjects representative of Choose subjects representative of larger grouplarger group

Example 1Example 1

From Goren-Bar, What Do Users PreferFrom Goren-Bar, What Do Users Prefer

Hypothesis 1:Hypothesis 1: …. users in general will …. users in general will find the user-oriented search method find the user-oriented search method as more suitable for searching the as more suitable for searching the web, compared to search methods web, compared to search methods that do not consider the user that do not consider the user perspective.perspective.

Example 1 (continued)Example 1 (continued)

Hypothesis 2:Hypothesis 2: Novice and Novice and knowledgeable users will not have knowledgeable users will not have the same response to both search the same response to both search approaches.approaches.

Example 2Example 2

From St. Amant, … Navigation and From St. Amant, … Navigation and Intelligent AssistanceIntelligent Assistance

Assume that the precision and recall of Assume that the precision and recall of an intelligent assistant can be an intelligent assistant can be controlled.controlled.

Hypothesis 1:Hypothesis 1: Is any assistance better Is any assistance better than no assistance?than no assistance?

Example 2 (continued)Example 2 (continued)

Hypothesis 2:Hypothesis 2: Does task performance Does task performance change with the precision of the change with the precision of the assistance? (Precision = proportion assistance? (Precision = proportion of assistance that is relevant)of assistance that is relevant)

Hypothesis 3:Hypothesis 3: Does task performance Does task performance change with the recall of the change with the recall of the assistance? (Recall = proportion of assistance? (Recall = proportion of relevant information that is provided)relevant information that is provided)

Example 3Example 3

From Basgodan, Touch in Shared From Basgodan, Touch in Shared Virtual EnvironmentsVirtual Environments

Hypothesis:Hypothesis: Touch is an essential Touch is an essential element in performing a task in a element in performing a task in a shared virtual environment.shared virtual environment.

Human SubjectsHuman Subjects

Informed consentInformed consent Minimized risksMinimized risks Institutional Review Board (IRB)Institutional Review Board (IRB) Expedited reviewExpedited review Application form

OrganizationOrganization

Teams of two or threeTeams of two or three– Membership approved by instructorMembership approved by instructor– Members choose leaderMembers choose leader

Items to be SubmittedItems to be Submitted

Initial phaseInitial phase– Team membership preferencesTeam membership preferences– Hypotheses, independent and Hypotheses, independent and

dependent variablesdependent variables– BibliographyBibliography– IRB approval form—includes IRB approval form—includes

experimental design and materialsexperimental design and materials– Oral presentation of projectOral presentation of project

Items to be Submitted (2)Items to be Submitted (2)

Investigative phaseInvestigative phase– Progress reports as requiredProgress reports as required

Final phaseFinal phase– Draft of Introduction, Background and Draft of Introduction, Background and

Theoretical FoundationTheoretical Foundation– Draft of Results, Analysis, ConclusionDraft of Results, Analysis, Conclusion– Final report: oral and writtenFinal report: oral and written– Poster abstract and designPoster abstract and design

Research IdeasResearch Ideas

Turning Issues into ResearchTurning Issues into Research

1.1. Need earlier prompts with some car Need earlier prompts with some car navigation systemsnavigation systems

2.2. Help systems need to be helpfulHelp systems need to be helpful

3.3. Should order of input matter?Should order of input matter?

4.4. Should the system respond after it Should the system respond after it receives input from the user?receives input from the user?

5.5. Speech recognition systems are Speech recognition systems are poor, not very accuratepoor, not very accurate

Issues into Research (2)Issues into Research (2)

6.6. Time delays in responding to input are Time delays in responding to input are annoying, may have high error cost.annoying, may have high error cost.

7.7. Controls should match mental modelsControls should match mental models

8.8. Systems should display current data and Systems should display current data and use a continuous update strategyuse a continuous update strategy

9.9. Error messages should make sense to the Error messages should make sense to the user.user.

Research QuestionsResearch Questions

What is the task analysis of What is the task analysis of purchasing an airplane ticket? How purchasing an airplane ticket? How well is it supported by an airline well is it supported by an airline website as displayed on a smart website as displayed on a smart phone?phone?

How should a PDA support a How should a PDA support a shopping list?shopping list?

How can an email interface support How can an email interface support task management?task management?

Questions (2)Questions (2)

How do people manage their digital How do people manage their digital photographs?photographs?

What is the best interface for a GPS-What is the best interface for a GPS-based navigation system?based navigation system?

What are the features of a good What are the features of a good proximity-aware recommender proximity-aware recommender system?system?

Questions (3)Questions (3)

What design features promote the What design features promote the success of reprogrammable keys? Or success of reprogrammable keys? Or of soft keyboards?of soft keyboards?

How fast should automatic scrolling How fast should automatic scrolling of information on limited size screens of information on limited size screens proceed?proceed?

Sources for More IdeasSources for More Ideas

CHI Conference ProceedingsCHI Conference Proceedings ACM Transactions on HCIACM Transactions on HCI Course web siteCourse web site