introduction to web usability...

34
b b © Know-Center GmbH, www.know-center.at Usability Evaluation Vedran Sabol (with input from Belgin Mutlu and Cecilia di Sciascio) Web Technologies, 11-11-2019

Upload: others

Post on 27-Jun-2020

4 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Introduction to Web Usability Evaluationkti.tugraz.at/staff/vsabol/courses/wt/wt_slides_usability.pdf · used to compare results with usability requirements efficiency of use Intuitiveness

b

b

© Know-Center GmbH, www.know-center.at

Usability Evaluation

Vedran Sabol (with input from Belgin Mutlu and Cecilia di Sciascio)

Web Technologies, 11-11-2019

Page 2: Introduction to Web Usability Evaluationkti.tugraz.at/staff/vsabol/courses/wt/wt_slides_usability.pdf · used to compare results with usability requirements efficiency of use Intuitiveness

© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics

Overview

▪ Part I: Usability

▪ Definition and Methods

▪ Usability Metrics

▪ Data collection

▪ Part II: Evaluation task annoucement

▪ Optional! ▪ You can earn up to 5 additional points

▪ Visualizer Tool

Page 3: Introduction to Web Usability Evaluationkti.tugraz.at/staff/vsabol/courses/wt/wt_slides_usability.pdf · used to compare results with usability requirements efficiency of use Intuitiveness

© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics

The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a

specified context of use

Usability Definition

Page 4: Introduction to Web Usability Evaluationkti.tugraz.at/staff/vsabol/courses/wt/wt_slides_usability.pdf · used to compare results with usability requirements efficiency of use Intuitiveness

© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics

Why evaluating Usability?

▪ Helps to find the problems when they are easy and cheap to fix (early stage)

▪ Saves time, money and other precious resources

▪ Reduces the risk of building the product in the wrong way

▪ Helps to understand user’s success and time spent to complete tasks

Page 5: Introduction to Web Usability Evaluationkti.tugraz.at/staff/vsabol/courses/wt/wt_slides_usability.pdf · used to compare results with usability requirements efficiency of use Intuitiveness

© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics

Usability Tests

▪ Formative Tests: Finding and fixing usability problems (How?)

▪ forms and shapes a UI design

▪ Summative Tests: Describing the usability of an application using metrics (How much?)

▪ about measuring (e.g. time) and counting (e.g. success rate)

Page 6: Introduction to Web Usability Evaluationkti.tugraz.at/staff/vsabol/courses/wt/wt_slides_usability.pdf · used to compare results with usability requirements efficiency of use Intuitiveness

© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics

Formative Tests

▪ Helps to “form” the design for a product/service

▪ Involve evaluating during development

▪ Iterative

▪ Methods:

Heuristic evaluation: expert evaluates a UI against 10 well-known usability heuristics

Thinking-aloud testing: user comments while performing a task to identify issues

Cognitive walkthrough: evaluates how easy it is to learn using a system for new users

Page 7: Introduction to Web Usability Evaluationkti.tugraz.at/staff/vsabol/courses/wt/wt_slides_usability.pdf · used to compare results with usability requirements efficiency of use Intuitiveness

© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics

Example - Nielsen’s Heuristics

1. Visibility of system status

2. Match between system and the real world

3. User control and freedom

4. Consistency and standards

5. Error prevention

6. Recognition rather than recall

7. Flexibility and efficiency of use

8. Aesthetic and minimalist design

9. Help users recognize, diagnose, and recover from errors

10. Help and documentation

Page 8: Introduction to Web Usability Evaluationkti.tugraz.at/staff/vsabol/courses/wt/wt_slides_usability.pdf · used to compare results with usability requirements efficiency of use Intuitiveness

© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics

Example - Nielsen’s Heuristics

1. Visibility of system status

• system should keep users informed about what is happaning

• provide appropriate feedback, within reasonable time

2. Match between system and the real world

• system should speak the users' language

• use words, phrases and concepts familiar to the user (no system-oriented terms)

• follow real-world conventions

Page 9: Introduction to Web Usability Evaluationkti.tugraz.at/staff/vsabol/courses/wt/wt_slides_usability.pdf · used to compare results with usability requirements efficiency of use Intuitiveness

© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics

Example - Nielsen’s Heuristics

3. User control and freedom

• users choose functions by mistake, need a clear "emergency exit" (no complicated procedures)

• e.g. undo and redo

4. Consistency and standards

• use consistent vocabulary across the tool (no wondering whether different words, situations, or actions mean the same thing)

• follow platform conventions (fonts, date format etc.)

Page 10: Introduction to Web Usability Evaluationkti.tugraz.at/staff/vsabol/courses/wt/wt_slides_usability.pdf · used to compare results with usability requirements efficiency of use Intuitiveness

© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics

Example - Nielsen’s Heuristics

5. Error prevention:

• design to prevent problems from occurring (instead of error providing messages)

• in error-prone conditions check for a confirmation

6. Recognition rather than recall

• minimize the user's memory load

• make objects, actions, and options visible (instead of having to remember them)

Page 11: Introduction to Web Usability Evaluationkti.tugraz.at/staff/vsabol/courses/wt/wt_slides_usability.pdf · used to compare results with usability requirements efficiency of use Intuitiveness

© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics

Example - Nielsen’s Heuristics

7. Flexibility and efficiency of use

• Accelerators may speed up common interactions• not seen by novice users

• allow users to tailor/automate frequent actions

8. Aesthetic and minimalist design

• do not show information which is irrelevant or rarely needed

• useless information competes with the relevant info for attention and diminishes readability

Page 12: Introduction to Web Usability Evaluationkti.tugraz.at/staff/vsabol/courses/wt/wt_slides_usability.pdf · used to compare results with usability requirements efficiency of use Intuitiveness

© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics

Example - Nielsen’s Heuristics

9. Help users recognize, diagnose, and recover from errors:

• error messages in plain language (no codes)

• precisely describe problems, suggest solutions

10. Help and documentation

• design for intuitiveness, but

• make documentation easy to access, search, and read (e.g. lists of concrete steps with screenshots)

• focused on the user's task

Page 13: Introduction to Web Usability Evaluationkti.tugraz.at/staff/vsabol/courses/wt/wt_slides_usability.pdf · used to compare results with usability requirements efficiency of use Intuitiveness

© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics

Summative Tests

Benchmark test

used to compare results with usability requirements efficiency of use

Intuitiveness

low perceived workload etc.

User tasks need to be clearly defined, e.g. identify 5 most relevant documents on web technologies (max.

2 min time)

Find an author with most publications on web technologies (3 min time)

objective measurement of performance of a single design

Page 14: Introduction to Web Usability Evaluationkti.tugraz.at/staff/vsabol/courses/wt/wt_slides_usability.pdf · used to compare results with usability requirements efficiency of use Intuitiveness

© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics

Summative Tests

Comparative test, e.g.

Comparison of a current with a prior version

Comparison of competing products

different sets of users can work with each product (between-subject design)

same users can attempt tasks on all products (within-subjects design) Preferred when small num. of participants perform small

tasks

may not always be possible

Objective comparison of the performance of two or more alternative designs

Page 15: Introduction to Web Usability Evaluationkti.tugraz.at/staff/vsabol/courses/wt/wt_slides_usability.pdf · used to compare results with usability requirements efficiency of use Intuitiveness

© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics

Sample Size

The sample of users should represent the populationabout which we intent to make a statement

To draw conclusions about different types of users, all groups should be represented in your sample

e.g. include novice and experienced users in the sample

Example: 1936 Literary Digest Presidential Poll

2.4 million responses but incorrectly predicted the winner

Problem was not the sample size but of representativeness

Responders tended to have higher incomes and education level-not representative for ultimate voters

Page 16: Introduction to Web Usability Evaluationkti.tugraz.at/staff/vsabol/courses/wt/wt_slides_usability.pdf · used to compare results with usability requirements efficiency of use Intuitiveness

© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics

Data Collection

Lab-based moderated sessions

• Moderator observes and interacts with users when performing the tasks

• Expensive and time consuming

Unmoderated Sessions

• Users perform task on their own computer

• Moderator observes and records their behavior

Crowd-based studies

• No moderator observes, user behavior is recorded

Page 17: Introduction to Web Usability Evaluationkti.tugraz.at/staff/vsabol/courses/wt/wt_slides_usability.pdf · used to compare results with usability requirements efficiency of use Intuitiveness

© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics

Usability Metrics

Effectiveness: The accuracy and completeness with which users achieve specified goals

Efficiency: The resources expended in relation to the accuracy and completeness with which users achieve goals

Satisfaction: The comfort and acceptability of use

[Source: https://usabilitygeek.com/usability-metrics-a-guide-to-quantify-system-usability/ ]

Page 18: Introduction to Web Usability Evaluationkti.tugraz.at/staff/vsabol/courses/wt/wt_slides_usability.pdf · used to compare results with usability requirements efficiency of use Intuitiveness

© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics

Usability Metrics for Effectiveness:Completion Rates

Most fundamental of usability metrics

Collected as a binary measure of task success (coded as 1) or failure (coded as 0)

Number of tasks 𝑐𝑜𝑚𝑝𝑙𝑒𝑡𝑒𝑑 𝑠𝑢𝑐𝑐𝑒𝑠𝑠𝑓𝑢𝑙𝑙𝑦

𝑇𝑜𝑡𝑎𝑙 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑡𝑎𝑠𝑘𝑠 𝑢𝑛𝑑𝑒𝑟𝑡𝑎𝑘𝑒𝑛x 100%

[Source: https://usabilitygeek.com/usability-metrics-a-guide-to-quantify-system-usability/ ]

Page 19: Introduction to Web Usability Evaluationkti.tugraz.at/staff/vsabol/courses/wt/wt_slides_usability.pdf · used to compare results with usability requirements efficiency of use Intuitiveness

© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics

Usability Metrics for Effectiveness:Errors

Unintended action, slip, mistake, omission a user makes (or encounters) while working on a task

Provide diagnostic information on, why users are failing tasks

Can be analyzed as binary measures

1 for user encountered an error

0 for user who did not

Page 20: Introduction to Web Usability Evaluationkti.tugraz.at/staff/vsabol/courses/wt/wt_slides_usability.pdf · used to compare results with usability requirements efficiency of use Intuitiveness

© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics

Usability Metrics for Efficiency:Task Time

How long user spends on an activity

Amount of time it takes to successfully complete a predefined task scenario

𝑗=1𝑅 𝑖=1

𝑁 𝑛𝑖,𝑗

𝑡𝑖,𝑗

𝑁𝑅

[Source: https://usabilitygeek.com/usability-metrics-a-guide-to-quantify-system-usability/ ]

N = Total number of tasks

R = Number of users

𝑛𝑖,𝑗= Result of task i by user j, 1 if successfully completed, 0

otherwise

𝑡𝑖,𝑗 = Time spent by user j to complete the task i: if the task

not successfully completed, the time is measured till user quits

Page 21: Introduction to Web Usability Evaluationkti.tugraz.at/staff/vsabol/courses/wt/wt_slides_usability.pdf · used to compare results with usability requirements efficiency of use Intuitiveness

© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics

Usability Metrics for Efficiency:Overall Relative Efficiency

Ratio of time taken by the users who successfully complete the task in relation to the total time taken by all users

𝑗=1𝑅 𝑖=1

𝑁 𝑛𝑖,𝑗𝑡𝑖,𝑗

𝑗=1𝑅 𝑖=1

𝑁 𝑡𝑖,𝑗x 100%

[Source: https://usabilitygeek.com/usability-metrics-a-guide-to-quantify-system-usability/ ]

Page 22: Introduction to Web Usability Evaluationkti.tugraz.at/staff/vsabol/courses/wt/wt_slides_usability.pdf · used to compare results with usability requirements efficiency of use Intuitiveness

© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics

Usability Metrics: Satisfaction Ratings

Questionnaires that measure the perception of the ease of use of a system

After a task (post-task questionnaires)

At the end of a usability session (post-test questionnaires)

Outside of a usability test

There are standardized questionnaires

Own questions are welcome

might have to be validated by psychometrics tests

Page 23: Introduction to Web Usability Evaluationkti.tugraz.at/staff/vsabol/courses/wt/wt_slides_usability.pdf · used to compare results with usability requirements efficiency of use Intuitiveness

© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics

Post-Task Questionnaires

To provide insight into task difficulty as seen from the participant’s perspective

Typically consisting of a small number of questions

Often take the form of Likert scale ratings, e.g.

worst 1 … 7 best

Examples:

ASQ: After Scenario Questionnaire (3 questions)

SMEQ: Subjective Mental Effort Questionnaire (1 question)

UME: Usability Magnitude Estimation (1 question)

NASA-TLX: NASA’s task load index measures the subjective task workload (6 questions)

Page 24: Introduction to Web Usability Evaluationkti.tugraz.at/staff/vsabol/courses/wt/wt_slides_usability.pdf · used to compare results with usability requirements efficiency of use Intuitiveness

© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics

NASA-TLX

Mental Demand

How mentally demanding was the task?

Physical Demand

How physically demanding was the task?

Temporal Demand

How hurried or rushed was the pace of the task?

Performance

How successful were you in performing what you were asked to do?

Effort

How hard did you have to work to accomplish your level of performance?

Frustration

How insecure, discouraged, irritated, stressed, and annoyed were you?

Page 25: Introduction to Web Usability Evaluationkti.tugraz.at/staff/vsabol/courses/wt/wt_slides_usability.pdf · used to compare results with usability requirements efficiency of use Intuitiveness

© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics

Post-Test Questionnaires

To measure participants’ impression of the overall ease of use of the system being tested

Examples:

SUPR-Q: Standardized User Experience Percentile Rank Questionnaire (13 questions)

CSUQ: Computer System Usability Questionnaire (1 question)

QUIS: Questionnaire For User Interaction Satisfaction (24 question)

SUMI: Software Usability Measurement Inventory (50 question)

SUS: System Usability Scale (10 questions)

Page 26: Introduction to Web Usability Evaluationkti.tugraz.at/staff/vsabol/courses/wt/wt_slides_usability.pdf · used to compare results with usability requirements efficiency of use Intuitiveness

© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics

Positive Version of the Software Usability Scale (SUS)

[Source: https://link.springer.com/chapter/10.1007/978-3-319-20886-2_20 ]

Page 27: Introduction to Web Usability Evaluationkti.tugraz.at/staff/vsabol/courses/wt/wt_slides_usability.pdf · used to compare results with usability requirements efficiency of use Intuitiveness

© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics

Interaction Logging

What can be logged?

Occurrence of interactions with UI components:

e.g. click on tag in tag cloud, click to open a document, drag mouse to move a slider etc.

Record: user id, timestamps, action

Task success

1=success, 0=failure

Task time

in seconds (or milliseconds)

Page 28: Introduction to Web Usability Evaluationkti.tugraz.at/staff/vsabol/courses/wt/wt_slides_usability.pdf · used to compare results with usability requirements efficiency of use Intuitiveness

© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics

Visualizer

Web-application for data visualization URL: https://visualizer.know-center.tugraz.at/

Uses CSV files as datasets Upload from a local file or a URL

Simple data processing methods Cleaning, filtering, aggregation, computing columns

Multiple Visualizations Coordinated multiple views

UI similar to BI industry leaders (Tableau or PowerBI)

Page 29: Introduction to Web Usability Evaluationkti.tugraz.at/staff/vsabol/courses/wt/wt_slides_usability.pdf · used to compare results with usability requirements efficiency of use Intuitiveness

© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics

Visualizer

Page 30: Introduction to Web Usability Evaluationkti.tugraz.at/staff/vsabol/courses/wt/wt_slides_usability.pdf · used to compare results with usability requirements efficiency of use Intuitiveness

© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics

Visualizer - Features

Secure – data management locally in the browser (compressed)

Smart – automatically selects suitable visualizations for selected data

Can be used stand-alone or integrated into existing web-pages

Customizable – by changing the look of the platform

Configurable – by creating and sharing “views” on the data

Collaborative – synchronize user interactions over multiple clients (even mobile)

Extendable – by uploading own visualizations

Page 31: Introduction to Web Usability Evaluationkti.tugraz.at/staff/vsabol/courses/wt/wt_slides_usability.pdf · used to compare results with usability requirements efficiency of use Intuitiveness

© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics

Visualizer – Upcoming Features

Personalized Assistant for Interactive Data Analytics systems collects user interactions (e.g. mouse clicks)

learns how users analyze data (computes models)

Goal: guide users towards useful data insights

Approach: assist users when analyzing data based on previous user behavior e.g. user first selects some data fields

the system recommends (potentially useful) analytical workflows (i.e. a sequence of algorithmic and visual steps)

user can accept a recommendation automatically executes the workflows to create

visualization(s)

Page 32: Introduction to Web Usability Evaluationkti.tugraz.at/staff/vsabol/courses/wt/wt_slides_usability.pdf · used to compare results with usability requirements efficiency of use Intuitiveness

© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics

Visualizer – Upcoming Features

Personalized Assistant for Interactive Data Analytics

Needs click data to learn how users analyze data

Page 33: Introduction to Web Usability Evaluationkti.tugraz.at/staff/vsabol/courses/wt/wt_slides_usability.pdf · used to compare results with usability requirements efficiency of use Intuitiveness

© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics

Visualizer - Evaluation

Student: execute a few analytical tasks for a data set gain an overview

discover trends

identify correlations

find outliers

understand causal relationships

etc.

Fill out a questionnaire

Write a 2-page report (heuristic analysis)

It is all voluntary

Page 34: Introduction to Web Usability Evaluationkti.tugraz.at/staff/vsabol/courses/wt/wt_slides_usability.pdf · used to compare results with usability requirements efficiency of use Intuitiveness

© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics

Visualizer - Evaluation

But, you can earn a few extra points!

Perform tasks and fill out a questionnaire duration 15-20 minutes

1 extra point

Write a 2-page report

Heuristic evaluation

5 extra points

Details will be presented during the last lecture of the course “Visualization in the Web”, to be held on 09.12.2019