random house widget

Feedback from Usability Testing

Semantico/Random House Widget

4th April 2008

Who’s here

• Pete Gale ~ Head of User Experience

• David Harley ~ User Experience Consultant

Overview

• Introduction• Section heading• Section heading• Section heading• Section heading

About Cogapp

• Founded in 1985• Decades delivering projects on demanding timescales• Long-term view, long-term value• Creative methodology• A user-centred approach supported by internal testing

facilities• Growing internet consultancy practice

Our Clients

• Apple UK• Macmillan Cancer Support• BBC• The British Museum• Prudential• Department of Health• Art Institute of Chicago• HSBC• National Gallery, London• Cable & Wireless• MoMA• Design Museum London

• V&A• SONY • Ofcom• Arthritis Research Campaign• Microsoft• National Portrait Gallery• Channel 4• 10 Downing Street• Science Museum London• European CommissionI• CIS• The British Library

Usability & user centred design

• Usability techniques provide the best ROI when used as part of a user centred design process.

• User centred design tests early and often, to validate user needs and design solutions, ensuring designs are based on an understanding of real user needs and behaviours.

“We urge all UK technology companies to put User Centred Design at the heart of their R&D and innovation activities and promote a people-centred culture throughout their organisations. This will help ensure they develop the right products and services for

the right markets, first time and all the time.”

Department of Trade and Industry, Global Watch

Usability Testing within The UCD Life Cycle

• Iterative design process

ROI for user centred design (hidden)

• User centred design provides an opportunity to identify & validate user needs early in the project

• 80% of changes made to a project after launch are because of “unmet or unforeseen user requirements”.

Claire Karat,

Software Engineering in Dollars and Cents

• The cost of change requests increases exponentially as software moves through development and into a live environment.

Roger Pressman, Software Engineering: A Practitioner’s Approach, McGraw-Hill

Cost

of

makin

g c

han

ges

1xDuringDesign

100xAfter Launch

6xDuring

Development

Usability Testing within The UCD Life

Cycle • Iterative Design

Considering different levels of user experience

Proposition What is the value being delivered?

Concept How is that value being delivered?

Structure How is it organised?

Information What information is required?

Interaction How do users interact with the product?

Appearance What does it look like and how is it arranged?

• As proposed by Jesse James Garrett and others, user experience can be considered on a number of interdependent levels

How is Value Derived by the Business ?

• Increased uptake of books via online interactions.

• Development of a better online relationship with Random House book buyers.

How is Value Derived by Users ?

1. There’s something “special” going on.The presence of the “Open the Book” banner denotes a different

treatment for this book perhaps something “special” worth investigating.

2. The ability to read content.The opportunity to engage with the material within the text of the book.

3. The ability to scan meta content to ascertain relevant purchases.

The opportunity to access supporting information about this book to make an effective purchase , e.g. recommendations, table of contents, reviews.

4. A faithful representation of a real artefact (that they might buy).

To get a feel for the real object through a virtual representation of the book.

The Starting Point

The New Functionality:Thumbnail Browser View

User Session 1 “Explore the interface and identify content that is of interest” (Non-Blogging Book

Enthusiasts) 3 users took part in the first user testing session. They were all non-

bloggers and book enthusiasts.

• They did not read the book.

• They focussed on meta content (i.e. information about the content of the book ), particularly the recommendations.


1. Something “special” going on

2. Ability to read content

3. Ability to scan meta content to ascertain relevant purchases

4. Faithful representation of a real artefact (that they might buy)

User Session 2 “Read through the first page of the book” (Blogging Book Enthusiasts)5 users took part in the 2nd user testing session. They were bloggers

and book enthusiasts.

1. They were asked to compare different book browsing approaches looking through a similar novel on each website.– Amazon– Random House US– Random House UK

2. Then they were asked to explore the full functionality of the Random House UK interface.

Amazon

Random House U.S.

Random House U.K.

Ease of Use

After Reading

6.8

7

7.2

7.4

7.6

7.8

8

8.2

8.4

8.6

Amazon Random House UK Random House US

Ease of Use

After Exploring Other Functionality

6.8

7

7.2

7.4

7.6

7.8

8

8.2

8.4

8.6

Amazon Random House UK Random House US

Understanding These Results

• Users had preconceptions of expected use based on Amazon.

• Something about reading through a novel favoured the Amazon interface.

• More learning was required to make the most of the Random House interface.

• Users could see the potential of the Random House

interface when shown more of its functionality.

Different Types of Book: Narrative Versus Information

• Different book content requires different interactions.

• Narrative ( fiction, children’s books and biographies ) are time-bound and sequential. You have to read page by page from beginning to end.

• Amazon and Random House US emphasise sequential interactions. They do this by restricting user options in the interface. The simplest and most immediate interactions move through the book page by page without access to an overview.

Different Types of Book: Narrative Versus Information

• Informational books ( non-fiction, manuals, cookery books, etc ) rely on access at multiple points rather than sequentially.

• Random House UK interface currently emphasises this multiple-point access ( and hence access to informational content). It discourages engagement with narrative.

Different Reading Behaviours

We know from eye tracking research that there are different kinds of reading behaviour. 3 distinct behaviours have been identified:

• Reading• Scanning • Skimming

(Ohno, 2004)

These are dependent on the intent of the reader, the type of book they are reading and the context in which they are reading it, e.g. on a website, in a book or in a newspaper.

Reading Narrative in Print

Reading on the Web

Reading

Scanning

Skimming

What Kind of Reading Did We Observe?

User Session 2

• reading (prompted)• scanning • skimming

User Session 1

• scanning• skimming

Selecting Books Through Different Kinds of Reading Behaviour

Academic research suggests that the most effective way of selecting books with informational content is through the scanning of meta content, in particular tables of contents (Wacholder et al, 2006).


1. There’s something “special” going on

2. The ability to read content.

3. The ability to scan meta content to ascertain relevant purchases.

4. A faithful representation of a real artefact (that they might buy).

Exploring and Reading the Book’s Content

High Priority Issues

ScrollingReading narrative requires a smooth movement of attention from left to right and

down the page. Obstacles to this stop the natural flow of reading.

EvidenceAll 8 users could not initially find any means of scrolling within the browser window.

AnalysisThere is currently no visible scroll control.

RecommendationsScrolling must be visible and accessible from within the viewed page.

Book Entry PointUsers have expectations for what should happen when they ‘Open the Book’.

This is dependent on the type of book being viewed.

EvidenceAll 8 users found it difficult to read the book’s content when first presented with it.

AnalysisExpectations for narrative are reading rather than overview navigation.

RecommendationsThe main browser window should start by displaying pages according to the

appropriate reading style for the book – Narratives should open ‘ready to read’ on the first page of content.– Informational books should open on the table of contents.

Medium Priority Issues

Thumbnail Navigator VisibilityThe significance of the thumbnail navigator is likely to depend on the type of

book.

EvidenceAll the users who discovered that they could hide the thumbnail navigator preferred

reading without it. In terms of navigating through the book all 8 users liked the thumbnail navigator as an innovative feature.

AnalysisThe style of reading (i.e. skimming or scanning for informational content, reading for narrative ) should also be reflected in the availability of the thumbnail versus single page view.

Recommendations1. Narratives should open without the thumbnail navigator.Informational books should

open with it.2. The ability to hide or display the thumbnail navigator should be more prominent,

i.e. the navigator open/close icon should be larger and clearer.


Zoom ControlZooming is closely related to the use of the scroll, both being balanced for

optimum viewing.

EvidenceAll users identified the zoom control and were able to use it to view pages at a

reasonable size. Their inability to find the scroll meant that they struggled to find an optimum size, i.e. one that was visible but did not require scrolling. This meant continual readjustment of the zoom.

AnalysisThe complexity of the zoom control was only necessary because of the invisibility of

the scroll function.

Recommendations1. Pages should appear immediately at a viewable size. 2. Simpler one click controls should be made available to zoom in/out of the viewed

page.

Low Priority Issues

Navigating Around the Main Browser Window


‘Page Forward’ ControlReading, scanning and skimming all follow a similar movement of attention from

top left to bottom right. The virtual turning of pages must accommodate this.

Evidence

All users had to follow a sequence of manoeuvres in order to read through the book page by page. Starting with the mouse pointer in the middle of the page for scrolling they had to interrupt their reading, locate the page forward button, move to it with the mouse pointer, click on it then return their pointer to the middle of the page to resume scrolling.

Analysis

The mouse movements involved in turning the page are excessive and interfere with attention to reading.

Recommendations

The ‘page forward’ button should be made larger and be placed closer to the bottom right of the displayed text (without obscuring the text).


Retaining User Control Whilst Traversing PagesThe thumbnail navigator potentially enables greater control over the scanning of a

book by traversing a number of pages at a time using the top arrow buttons. It is important that the user retains control over this at all times and can interrupt this process.

Evidence2 users got into a situation where they were traversing pages using the top arrow

buttons and decided they wanted to stop on a particular page. This was not possible causing user frustration.

AnalysisImmediate control is an important aspect of scanning

RecommendationsAllow the user to select individual pages whilst traversing pages.

Low Priority Issues

The Differentiation of Navigation ArrowsThe arrow icons at the top and bottom of the main browser window are the same

but offer different functionality. The top ones allow traversal of many pages (useful for scanning and skimming) whilst the bottom ones move one page at a time (and are important for reading).

EvidenceThrough experimentation all users were able to effectively navigate within the book but

none of them were aware of this difference in function until it was pointed out to them.

AnalysisUsers are likely to miss functionality which may be most appropriate to their intended

use of the interface, i.e. reading versus scanning or skimming. RecommendationsThe arrow icons should be changed to reflect the underlying difference of function.

Low Priority Issues

Instant End Point NavigationThe function for going to the ends of the book also shows the travel through the

intervening pages. Those attempting to scan through the book are most interested in the meta content and are unlikely to want to read along the way.

Evidence

3 users found the journey through the pages frustrating because it was slow and did not allow them to interrupt the process.

Analysis

Users engaged in scanning behaviour are most interested in the meta content that is usually on the back cover. They want access to this information as quickly as possible..

RecommendationsThe arrows to the end points of the book should provide instant access to these points and should

not show the travel.

Testing other functionality: “search”


Search ContextThe purpose and context of the search is unclear. The search is not relevant to

narrative content.

EvidenceWhen asked to use the search facility all the users were unsure of its purpose. As one

user explained, “why would I put any word in [the search], I don’t understand that” (User 2). Users’ initial expectations were that the search would be pertinent to book titles rather than book content. This expectation was reinforced by the appearance of the search on other pages, i.e. book info, send to a friend, add widget.

Users could see the potential of search given a book with informational content, e.g. a cook book or a text book.

AnalysisSearch is relevant to informational content and not narrative. The misunderstanding of

the search context could be disastrous for prospective sales is users interpret no results as ‘no book’.

RecommendationsSearch should be removed from narrative content and the context of the search

should be clearly stated, e.g. “ Search this Book”. The results should reiterate this context, e.g. “(search term) “X” appears Y times in this book”


Search ResultsSearch results appear in two categories – one that is part of the available

sample and that can be accessed and another that is contained within the rest of the book and that cannot be accessed. The distinction between these results is not apparent to users.

EvidenceNo user worked out the distinction between the two sets of results. As one user

explained, “I don’t know why I can’t see what I’ve been searching for” (User 6).AnalysisTo be useful the search facility must support effective scanning and skimming. As

such it should be a selective questioning of the content which gives appropriate feedback at all times.

RecommendationsFeedback must be provided when users click on the results to explain why they

cannot access the content, e.g. “If you buy this book you will be able to read the whole paragraph”. The same is true for no results with feedback such as “Sorry, this word does not appear in the book”.

Low Priority Issues

“Search Clear” FunctionWhen search terms are typed into the search box, a “clear search” icon appears

to the right of the text. This is easily misinterpreted as a “search go” button.

Evidence

2 users clicked on the “clear search” icon thinking that it was a “search go” icon.

Analysis

The placing of the “clear search” is contrary to existing conventions for search functionality and hence user expectations.

Recommendations

Replace the “clear search” function with a “search go” function. Rely on the keyboard delete for user control of text.

No Issue

“Search Again” Function

This was understood by all but 2 users. Those that did not understand it found that it did not represent an obstacle for them as they just re-entered their search terms.

Blogging and the Return of the Narrative

• Like narrative, blogs are time-bound and sequential.

.• They differ from narrative in that they start with a sense of ‘now’ and

move backwards in time. Conversely narrative starts in the past and moves forwards towards now.

• Blogs capitalise on web reading behaviour. They make use of the characteristic ‘F shaped’ reading pattern which emphasises the top part of the screen (see earlier slide on reading behaviour).

• Blogs reintroduce aspects of ‘social reading’ which hark back to earlier reading practices, e.g. public readings.

.

How is Value Derived by Bloggers ?





5. Ability to share current or recent reading with others ?

Testing other functionality: “add widget”


The Add Widget Icon “+”The add widget function is accessed by clicking on the “+” icon. This does not

describe the underlying function and is easily misunderstood.

EvidenceThe non-bloggers could not ascertain what this function was about or how the “+”

symbol related to it. Bloggers could understand the concept of a widget but felt that it was inappropriately named and that the “+” symbol did not aid understanding. The “+” symbol was misinterpreted as a zoom by all 3 of the non-bloggers.

AnalysisThe purpose of this function in relation to book browsing remains unclear.RecommendationsThe “+” symbol should be changed to a more suitable one. See www.lastfm/widgets

for an example. The label for this function should also be changed to define its function more clearly, e.g. “Add to My Site”.

http://www.lastfm/widgets



Colour Changing Within The WidgetThe 2 colours used in the widget can both be changed by using the colour

choosing interface. It is easy for users to choose colours that are not harmonious – these changes are not localised within the widget but change the look and feel of the whole browsing experience

EvidenceAll users found changing the colours reasonably easy but once changed they had

great difficulty in choosing colours that they were happy with. The corresponding and simultaneous colour changes in the rest of the browser interface were unexpected and led users to think that they had misunderstood the purpose of the “add widget” function from the start.

AnalysisExtended interactions with the colour changing interface are a distraction from

bloggers’ intended aim of sharing their reading experience.RecommendationsLimit the choice of colours to discrete and harmonious combinations. See

www.lastfm/widgets for an example. Do not change the look and feel of the whole browser interface as a consequence of widget changes.


Testing other functionality: “send to a friend”


Text Input Labels for Send to a FriendCurrently the labels which explain the required user input are already present in

the text boxes and are deleted by the user as they type their own information into the boxes. If users forget what they are typing then they are unsure of how to proceed.

Evidence

2 users assumed what information was required and started typing into the boxes. They then realised that they had not checked what was actually required and could no longer check from the labels.

Analysis

The current set up does not allow for user error.

Recommendations

Move the labels for user input so that they are permanently placed adjacent to the relevant input boxes, either to the left or above the user input boxes.


Content of Sent MessagesCurrently users are not informed of what they are actually sending to a friend by

email and in particular how that email relates to the book they were just reading.

EvidenceAlthough all users were able to go through the motions of sending an email

to a friend, when asked what they had sent none were able to answer the question with any certainty.

AnalysisThe purpose of sending an email to a friend remains unclarified within the

interface..RecommendationsIt should be stated clearly what it is that is being sent and users should be

able to tailor their email content.

How is Value Derived by Bloggers ?





5. Ability to share current reading with others ?

Suggestions

The 2 Browser Views:

• The readable one for narrative

&

• The navigable one for informational content

Suggestions

Increasing Value to Bloggers

• Engagement with and sharing of contentAllow users of Random House book browser to share passages from

the book rather than just the cover shot of the book.

• Encouragement of social aspects of readingIntegration with online book clubs, reading groups.

Suggestions

What Kind of Reading Leads to Book Buying ?

This remains unclear. It would be possible to analyse user interactions once the functionality has been launched online.

From measurements of user clicks and the timings between it would be possible to ascertain whether users are scanning, skimming or reading and which of these are most likely to result in a purchase.

References

Ohno, T. (2004) EyePrint: Support of Document Browsing with Eye Gaze Trace. Proceedings of the 6th International Conference on Multimodal interfaces. 2004, 16-23.

Rayner, K. (1998) Eye Movements in Reading and Information Processing: 20 Years of Research. Psychological Bulletin. 1998, Vol. 124, No. 3, 372-422.

Wacholder, N., Liu, L. and Liu, Y. (2006) Selecting Books: A Performance-based Study. Proceedings of the 6th ACM/IEEE-CS joint conference on Digital libraries. 2006, 337.

Full List of IssuesHigh Priority1. Scrolling2. ‘Page Forward’ Control3. Search Context4. The Add Widget Icon “+”5. Colour Changing Within The Widget6. Text Input Labels for Send to a Friend7. Pop-up Blocker8. Unhelpful Flash Controls

Medium Priority1. Book Entry Point2. Thumbnail Navigator Visibility3. Retaining User Control Whilst Traversing Pages4. Search Results5. Content of Sent Messages

Low Priority1. Zoom Control2. The Differentiation of Navigation Arrows3. Instant End Point Navigation4. “Search Clear” Function

random house widget

Business

ucd life cycle

unhelpful flash controls

main browser window

high priority issues

low priority issues

medium priority issues

ascertain relevant purchases

thumbnail navigator visibility