Improving user interfaces through a methodological heuristics evaluation frameworks

Progress Report

Supervisors:Dr. Tom GedeonMr. Christopher Chow

Principal Researcher:Manik Mahajan

OverviewDiscount usability evaluations:

Low-cost, low-time, and low-resource intensive techniques of conducting usability evaluations.

A heuristics analysis is a discount usability technique, which typically sets a select number of evaluation criterion (heuristics) against which the usability of an interface is evaluated.

2

Heuristics analysis:

Typically conducted by ‘expert’ usability evaluators. This is to say, individuals with some background or familiarity with usability evaluation theory or techniques.

However, typical heuristics evaluations are drafted uniquely for the interfaces for which the usability is to be evaluated.

Heuristics Analysis Framework:

This research project posits the development and then utilization of a framework that abstracts and encapsulates complex concepts and practices into a standardised platform.

Expert usability evaluators can use this to conduct stand-alone and comparative heuristics analysis, using mixed-methods.

Think Alouds:

Think Alouds are a cheap and easy to implement method of usability testing that alloy rapid generation of user feedback.

Early adoption of feedback has had a demonstrated positive effect in ensuring the success of products and services being developed due to the integration of user-driven insights throughout the development life-cycle.

Heuristics Framework

Help prevent the reinvention of the wheel for conducting discount usability evaluations such as a heuristics analysis.

Encourage expert usability evaluators to perform cheap and quick usability evaluations in early development stages.

Measurement and evaluation the usability of any given interface independent of the platform in which and for which it may have been created. Benchmark usability of a given set of interfaces.

Think Alouds

Think Alouds a low-cost usability testing methods that have a demonstrated history of helping deliver value-driven user feedback in product and service design.

This study helped discover how structured insights can be generated from running Think Alouds for the interface and experience of a product, and uncover interaction and behavioral feedback from users.

Scope for future work in usability testing improvement using more sophisticated practices and different fields such as deep learning was discovered.

Think Alouds with Eye Tracking

Think Alouds are unnatural due to the nature of cognitive workload expected from the experimentees.

This study evaluated how the addition of eye tracking in a Think Aloud would influence the nature of the study.

A multi-dimensional evaluation method for running the Think Aloud can help generate verified and validated user feedback.

Although this is not exactly discount, it is more repeatable in nature.

Benefits of this Research Project

3

Research Processes Background Research

The cumulative methodological repertoire of HCI research is substantially broad, as a result a systematic review of literature has been deemed to be best practice for literature and background analysis in HCI research.

GOMS (Goals, Operators, Methods, and Selection Rules)

Environment Scan

Insights generated through consultation with usability practitioners, and subject matter experts to evaluate the difference between ‘best’ practices and practices employed in the ‘real’ world. (Design Thinking)

Gap Analysis

Instead of creating something for the sake of it, substantial gap analysis of existing literature and practices was undertaken to find the scope of surgical solutions in the problem space. As a consequences of this the Heuristics Analysis Framework was created.

4Source:http://www.sismarketresearch.com/ Source:http://designthinkingforlibraries.com Source:http://technologyadvice.com/blog/sales/finding-filling-gaps-sales-pipeline/

The Design Squiggle, Damien Newman

5

Current Project StateHeuristics Analysis Framework

Usability Heuristics:● Visibility of System● Match Between System and Real

World● User Control and Freedom● Consistency and Standards● Error Prevention● Recognition rather than recall● Flexibility and efficiency of use

Work Done

1 pilot study.

2 dry run studies.

4 Expert Heuristics Evaluations conducted.

8 Think Alouds conducted.

4 Think Alouds with eye tracking conducted.

Running the studies allowed familiarisation with the Ethics Approval process required in human research; design, analysis and running of HCI experiments; and creation of frameworks that alloy structured extrapolation and interpolation of insights from observational data.

Insights Generated

The HCC Workshop has been rated a usability score of 3.85 on average in the studies so far. The ‘feedback’ and ‘experience’ of users of during Think Aloud was generally good.

Think Alouds are immensely valuable in generating feedback, and require relatively low effort in experimental setup. If a large study were to be conducted, Think Alouds could be scaled to generate statistical insights.

Think Alouds with Eye Tracking add cognitive workload for the user.

Think Alouds have a seeming edge over heuristics analysis in helping unravel usability issues.

6

Evaluation Criteria:● Effectiveness● Efficiency● Satisfaction

The Experiments in Action7

Project StateEye Tracking

The Heuristics Analysis Framework helped develop key insights into usability testing and user experience design.

Eye Tracking will help further HCI research skills and knowledge.

Think Alouds with Eye Gaze Tracking allow rapid generation of user testing driven insights that can be coupled with empirical data through eye gaze tracking.

Eye Tracking data gathered can be cleaned and utilised to create heatmaps and opacity maps to for assumption testing.

Studies

The studies were structured as 1-hour sessions with a representative sample of test subjects.

Demographic information and screening performed to ensure experimental integrity.

The experimental surveying framework evaluated real-time feedback during Think Alouds against cognitive functions to map the nature of user ‘experience’.

Exit interviews allowed drawing of feedback to improve both the experimental procedure and the user experience of the study itself.

HCC Workshop

The HCC Workshop is the test object for this experiment.

The experiment will test usability of four tasks: register/login; using eye tracking activity; using the HCI activity; using neural network activity.

These activities were chosen as observation for workshops conducted in the past demonstrated particular interest in the three above mentioned activities.

As a result of this experiment, feedback for the HCC Workshop will be generated that might help bring value to the ANU’s initiatives for science outreach.

8

SummaryThe Heuristics Analysis Framework is an attempt to marry multiple theories and practices in discount usability testing, ergonomics efficiency and standardisation, and human-computer interaction principles, that enables a cheap, easy, and standardised way to test interfaces.

Think Aloud with Eye Gaze Tracking is a relatively discount usability method that can be run as a cheap and easy method by an expert. Eye tracking allows Verification & Validation of verbalisations (that are part of Think Aloud), in addition to allowing users to more deeply identify usability issues.

This research project will generate useful feedback for the HCC Workshop, to help the improvement of user experience and the interface design.

Additionally, the Heuristics Analysis Framework and the comparative analysis of Retrospective Think Alouds help add value to HCI as a discipline by: one, creating a new benchmarked usability testing framework; two, corroborating the goodness of multiple usability testing methods.

9

Core Research ConsiderationsThe World View Model

Questions?

10