Methods for Human-Methods for Human-Computer Computer Interactions (HCI) Interactions (HCI) ResearchResearch

Dr. Xiangyu WangDr. Xiangyu WangDesign ComputingDesign Computingx.wang@arch.usyd.edu.aux.wang@arch.usyd.edu.auAcknowledgement to Acknowledgement to Sasha Giacoppo and Sasha Giacoppo and Kerstin, Severinson Eklundh, and Ann Kerstin, Severinson Eklundh, and Ann Lantz for lecture notesLantz for lecture notes

The purpose of the The purpose of the lecturelecture

To analyze and discuss HCI as a scienceTo analyze and discuss HCI as a science To review different research methods, To review different research methods,

and how they are used in HCIand how they are used in HCI To learn how to plan and perform a To learn how to plan and perform a

scientific investigation in HCI at a basic scientific investigation in HCI at a basic levellevel

Facts about HCIFacts about HCI

HCI is a young discipline (about 30 HCI is a young discipline (about 30 years)years)

Has developed from computer science, Has developed from computer science, with influences from cognitive psychologywith influences from cognitive psychology

Also Influenced by sociology, Also Influenced by sociology, anthropology, communication, design, anthropology, communication, design, ergonomics, etc.ergonomics, etc.

HCI as a ”user science”HCI as a ”user science”

All HCI research includes studies of the All HCI research includes studies of the useuse of technologyof technology Empirical study methods are essentialEmpirical study methods are essential

HCI is also closely related to the HCI is also closely related to the designdesign and and development development of technologyof technology

Important to discuss the relationship Important to discuss the relationship between research and developmentbetween research and development

Inter- v.s. Inter- v.s. multidisciplinary multidisciplinary researchresearch

Interdisciplinary research transcends the Interdisciplinary research transcends the borders between knowledge areas and borders between knowledge areas and organisations … The goal is an integration, organisations … The goal is an integration, gradually developing into a science of its own.gradually developing into a science of its own.

Multidisciplinary research has a clear division Multidisciplinary research has a clear division of labour among the sciences involved – of labour among the sciences involved – Wallén: Vetenskapsteori och forskningsmetodikWallén: Vetenskapsteori och forskningsmetodik

HCI is interdisciplinary researchHCI is interdisciplinary research

HCIHCI

HCI has a wide variety of professional HCI has a wide variety of professional and academic arenas, including:and academic arenas, including: Information Technology & ComputingInformation Technology & Computing Design & DevelopmentDesign & Development Business Business Entertainment Entertainment EducationEducation Library SciencesLibrary Sciences

HCIHCI

More common areas of research related to More common areas of research related to design computing areas:design computing areas: Website design and constructionWebsite design and construction Consumer product design and developmentConsumer product design and development Data Management and Information VisualizationData Management and Information Visualization Ergonomics and Industrial Design Ergonomics and Industrial Design Online Communities and Collaborative Systems Online Communities and Collaborative Systems Entertainment and GamingEntertainment and Gaming

HCI Research MethodsHCI Research Methods

HCI research methods can be grouped into the HCI research methods can be grouped into the following categories:following categories:

Development Methods: Task AnalysesDevelopment Methods: Task Analyses

Survey Methods: Surveys & QuestionnairesSurvey Methods: Surveys & Questionnaires

Controlled ExperimentsControlled Experiments

Ethnographic MethodsEthnographic Methods

Logging & Automated MetricsLogging & Automated Metrics

How to do HCI research? How to do HCI research?

Read, research, and understand the numerous theories Read, research, and understand the numerous theories and domains that investigate HCI related issues. and domains that investigate HCI related issues.

Select the domains of HCI that specifically support your Select the domains of HCI that specifically support your objectives. Read and research the theories and models objectives. Read and research the theories and models specific to those domains you selected.specific to those domains you selected.

Then, review the methodologies available for your Then, review the methodologies available for your investigation.investigation.

Choose the appropriate methodologies, based on the Choose the appropriate methodologies, based on the desired breadth and depth of your investigationdesired breadth and depth of your investigation

Use your investigations to build theories and/or models.Use your investigations to build theories and/or models.

Development Methods: Development Methods: Task AnalysesTask Analyses

Usability engineering (engineering, Usability engineering (engineering, development, and usability development, and usability methodologies) methodologies)

A task analysis is not a single analysis, A task analysis is not a single analysis, but rather a collection of many different but rather a collection of many different analyses.analyses.

Development Methods: Development Methods: Task AnalysesTask Analyses

According to Hackos (1998), a user needs assessment According to Hackos (1998), a user needs assessment and task analysis seeks to understand:and task analysis seeks to understand: What user’s goals are; What they are trying to achieve What user’s goals are; What they are trying to achieve What users actually do to achieve those goals What users actually do to achieve those goals What personal, social, and cultural characteristics the users What personal, social, and cultural characteristics the users

bring to the tasks bring to the tasks How users are influenced by their physical environment How users are influenced by their physical environment How users’ previous knowledge and experience influence how How users’ previous knowledge and experience influence how

they think about their work and the workflow they follow to they think about their work and the workflow they follow to perform their tasks perform their tasks

What users value most that will make a new interface be a What users value most that will make a new interface be a delight for them (i.e. speed? Accuracy? Error recovery? Human delight for them (i.e. speed? Accuracy? Error recovery? Human contact? Fun?) contact? Fun?)

Techniques of Task AnalysisTechniques of Task Analysis

Workflow analysisWorkflow analysis (business process analysis) (business process analysis) Performed to understand how a user, or a group of users, accomplishes a Performed to understand how a user, or a group of users, accomplishes a

particular process. particular process. A horizontal picture of how work moves across and between people. A horizontal picture of how work moves across and between people.

Workflow analysis is important because it allows investigators to identify Workflow analysis is important because it allows investigators to identify which users are doing which tasks in a process. which users are doing which tasks in a process.

Importance to HCI: it does support HCI researchers in their understanding of Importance to HCI: it does support HCI researchers in their understanding of users. More importantly, workflow analysis provides HCI researchers with users. More importantly, workflow analysis provides HCI researchers with important information of user behavior in context.important information of user behavior in context.

Job analysisJob analysis Performed to understand the activities of a single user within a process. Performed to understand the activities of a single user within a process. A vertical picture of all the activities that flow through a single user. A job A vertical picture of all the activities that flow through a single user. A job

analysis should involve direct observation of a person over time, or at the analysis should involve direct observation of a person over time, or at the very least, a logging of daily activities by a person. very least, a logging of daily activities by a person.

Importance to HCI: Job analysis can capture meaningful, quantitative data Importance to HCI: Job analysis can capture meaningful, quantitative data about how and where users spend their time. Information like frequency, about how and where users spend their time. Information like frequency, time to complete, difficulty, etc. can all greatly help researchers designers time to complete, difficulty, etc. can all greatly help researchers designers and developers when it comes to developing, improving, discovering, etc. and developers when it comes to developing, improving, discovering, etc. interfaces and products for users. interfaces and products for users.

Techniques of Task AnalysisTechniques of Task Analysis

Task hierarchiesTask hierarchies A task hierarchy illustrates the tasks and subtasks of a A task hierarchy illustrates the tasks and subtasks of a

process or activity. It seeks to deconstruct and decompose process or activity. It seeks to deconstruct and decompose tasks into its smallest componentstasks into its smallest components

Importance to HCI: It provides researchers with a picture of Importance to HCI: It provides researchers with a picture of how many levels of tasks are involved in a process. The how many levels of tasks are involved in a process. The researcher can then decide which levels and how many researcher can then decide which levels and how many levels of a process need to be investigated.levels of a process need to be investigated.

Procedural analysisProcedural analysis One specific task that is divided into the steps and One specific task that is divided into the steps and

decisions that a user goes through in doing that task. It decisions that a user goes through in doing that task. It illustrates how users carry out their tasks with the tools illustrates how users carry out their tasks with the tools currently available to them.currently available to them.

Importance to HCI: Procedural analysis can create an Importance to HCI: Procedural analysis can create an idealized picture of the process that users follow, or an idealized picture of the process that users follow, or an actual picture of the process that users follow. From this, actual picture of the process that users follow. From this, areas that need to be improved, or where users make a lot areas that need to be improved, or where users make a lot of errors, can be identified, or investigated further.of errors, can be identified, or investigated further.

Advantages/Disadvantages of Advantages/Disadvantages of Task AnalysisTask Analysis

Advantages: Task analysis is a complex battery of Advantages: Task analysis is a complex battery of techniques intended to provide a researcher with a techniques intended to provide a researcher with a complete understanding of users/tasks. complete understanding of users/tasks.

Disadvantages: A task analysis can be an extremely Disadvantages: A task analysis can be an extremely time-and resource-consuming affair. The more time-and resource-consuming affair. The more information you desire to collect, the longer you will information you desire to collect, the longer you will have to burden users with observation and data have to burden users with observation and data collection.collection.

Development Methods: Development Methods: Task AnalysesTask Analyses

User and task analysis is perfectly suited for User and task analysis is perfectly suited for development-oriented researchers and development-oriented researchers and designers. The results of user and task designers. The results of user and task analyses can be used in several phases of the analyses can be used in several phases of the development process (Tucker, 1997):development process (Tucker, 1997): Development of requirements Development of requirements User Interface Design & Evaluation User Interface Design & Evaluation Follow-up After Installation/Use of System/Product Follow-up After Installation/Use of System/Product

Techniques for collecting Techniques for collecting datadata

ObservationsObservations Interviews (visiting, telephone, indirect)Interviews (visiting, telephone, indirect) QuestionnaireQuestionnaire DiaryDiary ……....

QuestionnairesQuestionnaires

Before conducting a study -practicalitiesBefore conducting a study -practicalities TimeTime CostCost RangeRange Questions (type, wording, order)Questions (type, wording, order) Demands reflectionDemands reflection InstructionsInstructions Pictures, anonymous cardsPictures, anonymous cards Language, knowledgeLanguage, knowledge

Checklist for constructing Checklist for constructing a questionnairea questionnaire

Be concreteBe concrete Think about effects due to the order of Think about effects due to the order of

posing your questions (context)posing your questions (context) AmbiguityAmbiguity AssumptionsAssumptions Careful with yes and no questionsCareful with yes and no questions

Question typeQuestion type

Open questionsOpen questions ListList CategoryCategory RankingRanking ScaleScale QuantityQuantity TableTable

Appearance and layoutAppearance and layout

Word processedWord processed Clear instructionsClear instructions Spacing between questionsSpacing between questions Keep response boxes in line (left/right)Keep response boxes in line (left/right) Guide the respondent the right wayGuide the respondent the right way Promise anonymity and confidentialityPromise anonymity and confidentiality

Administering the Administering the questionnairequestionnaire

Electronic, snail mail or face-to-faceElectronic, snail mail or face-to-face Self-addressed envelopeSelf-addressed envelope Instruction, information letterInstruction, information letter Anonymous, confidentialAnonymous, confidential

Piloting the Piloting the questionnairequestionnaire

How long did it take you to complete?How long did it take you to complete? Were the instructions clear?Were the instructions clear? Were any of the questions unclear or ambiguous? If so, Were any of the questions unclear or ambiguous? If so,

will you say which and why?will you say which and why? Did you object to answering any of the questions?Did you object to answering any of the questions? In your opinion, has any major topic been omitted?In your opinion, has any major topic been omitted? Was the layout of the questionnaire clear/attractive?Was the layout of the questionnaire clear/attractive? Any comments?Any comments?

Methods for Data Methods for Data CollectionCollection

MeasurementMeasurement How can variations in a phenomenon be How can variations in a phenomenon be

measured?measured? Subjective measuresSubjective measures

Ratings, question-answer studiesRatings, question-answer studies Objective measuresObjective measures

Time measurements, or process logsTime measurements, or process logs Validity: do you measure what you intended Validity: do you measure what you intended

measure?measure? Reliability: do different measurements give the Reliability: do different measurements give the

same result?same result?

Methods for Data Methods for Data CollectionCollection

Procedure for data collection and Procedure for data collection and analysisanalysis Consider validity and reliabilityConsider validity and reliability

Will the procedure answer the correct question? Will the procedure answer the correct question? Will it do so in a reliable way?Will it do so in a reliable way?

Systematic way of collecting informationSystematic way of collecting information Always be consistent; the same level of detail Always be consistent; the same level of detail

everywhereeverywhere Document the procedure systematicallyDocument the procedure systematically

My own research as an My own research as an exampleexample

Other slidesOther slides