usability testing - info.4imprint.com · of usability testing, website design is only one of the...
TRANSCRIPT
4imprint.com
Usabi l i ty Test ing
© 2013 4imprint, Inc. All rights reserved
Usabi l i ty test ing: Does your product make the grade? What is usability testing? More importantly, should you care? If you create
products, tools or applications that are used by customers, then usability testing is
something you should not only care about, but something you should consider.
Research shows that successful companies rely on usability testing to create
effective products, tools and websites. Did you know, for example, that at
any given time, Amazon.com® has about 200 usability tests underway?1 Brian
Eisenberg, co-founder and chairman emeritus of the Web Analytics Association,
explained it more plainly: “Alec Baldwin says in the movie Glengarry Glen Ross
to ‘always be closing.’ Online you have to ‘always be testing.’”2 And in the case
of usability testing, website design is only one of the areas where it is valuable; it
can also be used for any product or tool that requires human interaction.
Let’s begin with a basic definition. Usability testing is a method applied
to user-centered designs and products to evaluate how someone actually
uses a product. For example, you could observe a customer trying to
assemble something you designed with printed instructions and see if the
task is completed as intended. Or, you could watch a user navigate your
website to find product information or services. Specifically, it measures
a human-made product’s capacity to meet its intended purpose.3
Usability testing is commonly used for consumer products, websites, Web
applications, computer interfaces and devices. It measures the ease of use
of a specific object or set of objects, and shows a company how people
use its product or application in real life scenarios.
Although the concept of usability testing is fairly clear-cut, there’s a lot that goes
into the process. It needs to be done properly to illicit value and it should follow
a defined process. This Blue Paper® looks at the evolution of usability testing and
reviews the processes and methods that are commonly used. There are many
helpful resources that can help you understand the basics, and these are explored
throughout the paper. If you haven’t conducted usability testing before, consider
this paper your introduction. Rest assured, however, there will not be a test at the
conclusion. Although, once you understand the benefits, you might be tempted
to start testing your customers.
1 Goliger, Sarah. “Why Internet Marketers Must Always Be Testing.” HubSpot, 27 June 2011. Web. 02 July 2013. <http://blog.hubspot.com/blog/tabid/6307/bid/17695/Why-Internet-Marketers-Must-Always-Be-Testing.aspx>.
2 Ibid.3 “Usability Testing.” Wikipedia. Wikimedia Foundation, 20 June 2013. Web. 01 July 2013. <http://en.wikipedia.
org/wiki/Usability_testing>.
© 2013 4imprint, Inc. All rights reserved
The evolut ion of usabi l i ty test ing
The idea of usability testing first appeared during World War II when John
Flanagan, an industrial psychologist, discovered that reducing the number
of buttons, knobs, switches and controls in a fighter aircraft dramatically
improved pilot performance. Soon thereafter, Flanagan developed the
Critical Incident Technique (or CIT) that outlined procedures for collecting
direct observations of human behavior. The observations were tracked as
incidents to solve practical problems.4
The work of Flanagan and others created the early foundation for usability
testing. The concept was simple, whenever a customer interacted with a company,
brand or product through an informational display, it was helpful to observe users
performing tasks in order to identify opportunities for improvement. By viewing
how pilots interacted with fighter control panels in simulated scenarios, the
design of the jet control panel was greatly improved to promote ease of use and
reduce in-flight errors.
Usability research increased during the 60s as computer interfaces and micro-
electronics appeared. During this time, early pioneers such as Doug Engelbart,
Ivan Sutherland and Alan Kay developed many of the user interface hardware
and software designs that are still used today.5 Often called the “pioneers of
cultural software,” these individuals invented concepts that made it possible for
computers to be used for media creation and manipulation.6
In the 1970s, innovation in usability testing was propelled to a new level, largely
due to the efforts of Xerox®. It’s Palo Alto Research Center® (PARC®) created
many of the elements found in corporate usability research. You probably heard
of the acronym “WIMP,” which is the abbreviation for Windows, Icons, Menus,
Pulldowns. These concepts were developed by Xerox PARC, and as the 1980s
began, usability testing was used primarily for academic or corporate research in
companies like Apple®, Sun Microsystems®, AT&T®, Microsoft® and others.
After the dot-com boom in the 90s, usability testing took on a new format. For
the first time, usability testing was used on a wide-scale basis for commercial
purposes. It also was the first time usability, or the “customer experience,” was
used to influence the decision making process at companies like Amazon.com®
and Google®. Amazon.com founder Jeff Bezos, for example, spent a significant
4 “Usability Testing Central.” Usability Testing Central. N.p., 27 Feb. 2007. Web. 01 July 2013. <http://www.usabilitytestingcentral.com/2007/02/the_history_of_.html>.
5 Ibid.6 “1967-69--The FLEX Machine.” TheEarlyHistoryOfSmalltalk. SmallTalk.org, n.d. Web. 03 July 2013. <http://www.
smalltalk.org/smalltalk/TheEarlyHistoryOfSmalltalk_II.html>.
© 2013 4imprint, Inc. All rights reserved
amount of time using the basic principles of usability testing to improve the
Amazon.com experience. He said: “In our first year we didn’t spend a single dollar
on advertising ... the best dollars spent are those we use to improve the customer
experience.”7 It was during this period that usability testing flourished and played
an influential role in helping companies improve website design.
Dr. Jakob Nie lsen
“ It takes only five users to uncover 80 percent of high-level
usability problems.”
—Jakob Nielsen
You can’t discuss usability testing without multiple nods to Jakob Nielsen, PhD.,
who is largely credited for developing many of the principles of modern usability
testing. Not only did he prompt the “discount usability engineering” movement,
but he also developed “Nielsen’s Law of Internet Bandwidth,” which evaluated
the design implications from annual bandwidth growth.8 According to Fortune®,
he is the “reigning guru of Web usability.” He’s also named “the world’s leading
expert on Web usability” by U.S. News & World Report® and the world’s foremost
expert in Web usability by Business Week®.9 As a result of his work, Dr. Nielsen
holds 79 U.S. patents, most of which focus on methods of Web efficacy.
Nielsen holds a Ph.D. in human-computer interaction (HCI) from the Technical
University of Denmark. From 1994-1998, Dr. Nielsen worked at Sun Microsystems
as an engineer. He was hired to improve the user experience in complex
enterprise software. It was during this time that Dr. Nielsen started exploring the
emerging field of Web usability; he later became the lead designer for SunWeb®,
the corporate website and intranet.
This is when Nielsen had a major breakthrough. As he was
testing the website, it occurred to him that small groups of
test subjects were more effective during the development
stages. His rationale was simple: If it was too difficult for two or
three people to figure out a site, there was little to be gained
by watching others suffer through the same flawed design.
Accordingly, he favored small and frequent usability testing at
various stages of development. He found that the best results
7 Ibid.8 “Nielsen Norman Group.” Nielsen’s Law of Internet Bandwidth. N.p., n.d. Web. 02 July 2013. <http://www.
nngroup.com/articles/law-of-bandwidth/>.9 “Nielsen Norman Group.” Jakob Nielsen, Ph.D. and Principal. N.p., n.d. Web. 02 July 2013. <http://www.
nngroup.com/people/jakob-nielsen/>.
© 2013 4imprint, Inc. All rights reserved
came from testing no more than five users at a time and conducting as many
tests as possible. He also defined the five components of usability goals (which
we detail later), which are:
•learnability,
•efficiency,
•memorability,
•errors (as in low error rate), and
•satisfaction.
In 1998, Dr. Nielsen partnered with Don Norman, Ph.D., a well-known academic
and expert on user-centered design to establish the Nielsen Norman Group
(NN/g). The organization provides consulting services and evaluates user
interfaces. Since its creation, NN/g has consulted on projects at well-known firms,
including General Electric®, General Motors®, UPS, Hallmark® and WSJ.com®.
Each year, the company recognizes the “Ten Best Intranets” across industries and
publishes highly respected research on interface design and testing. Nielsen also
publishes an online column, the “Alertbox,” that addresses current issues and
trends in the field. You can access Neilsen’s current research and blogs on his
website. (www.nngroup.com/articles).
Let’s explore the basic components of usability testing in detail and gain a holistic
understanding of current practices and methods.
Usabi l i ty test ing 101
There’s an article that was published in 2012 by the Nielsen Norman Group (NN/g)
on usability testing titled “Usability 101: Introduction to Usability.”10 It claims
to be the perfect introduction for anyone that needs to understand the basics
of usability testing in a short amount of time. It’s a great starting point for any
organization interested in the idea and can point you to other resources that can
help you get started.
If you were to simplify the idea, usability testing begins with creating
a testing scenario, a user and an observer. Scenarios provide realistic
settings where a person performs a list of tasks while observers take notes.
Testing instruments such as scripted instructions, paper prototypes and
questionnaires may be used to gather additional feedback on the product
being tested. It’s a lot like a test you’d get in the classroom, only no
advanced studying is required of testers.
10 “Nielsen Norman Group.” Usability 101: Introduction to Usability. N.p., 4 Jan. 2012. Web. 01 July 2013. <http://www.nngroup.com/articles/usability-101-introduction-to-usability/>.
© 2013 4imprint, Inc. All rights reserved
Let’s assume, for example, that you want to test how a customer would send
a complaint through your website. You would create a real-life scenario that
asks a participant to do this task and provide a space that is equipped with the
necessary tools. You might give a written script to the user that describes the
assignment in detail and ask them to complete the task. The goal is to observe
how users function in a real life scenario so that developers can identify areas
for improvement. In short, it’s an opportunity see exactly how your customers
react to your products in a setting that mirrors real life. Often, usability tests
are videotaped so that actions and reactions can be viewed by others in the
organization to improve product design.
The common perception is that testing is conducted in a quiet room and
observed through a one-sided mirror. In truth, testing can be conducted
anywhere—you don’t need a usability lab or expensive equipment.
Conference rooms or general work spaces are sufficient as long as they are
equipped with portable recording devices and a place to take notes. Testing
can even be conducted remotely when participants are separated by space
and time. Remote testing and other approaches are discussed in detail in
the following section on usability methods.
As previously mentioned the word “usability” refers to methods for improving
ease-of-use and is defined by five key components. The first component is
“learnability.” This refers to how easy it is for users to accomplish basic tasks the
first time they use the design. “Efficiency” is the second component. This looks at
how efficient a user is when performing program tasks once the design is learned.
If a user continues to struggle trying to complete easy tasks, you’d probably get
low marks in this category. The third component evaluates “memorability.” In
other words, how long does it take for a user to reestablish proficiency after a
period of inactivity? Will users remember how to do something quickly or will
they need to relearn basic features? The idea is to have a design that is easily
remembered, so users recall basic functions.
The fourth feature examines errors users make. Do users make a lot of errors
using the product? This component also examines the severity of errors and how
easily a user can recover. As you probably heard before, in life, falling down is
inevitable, what matters is how quickly you get back up. The same concept applies
to errors in testing, users are bound to make (or find) mistakes, it’s important to
see if they can recover.
The final component of usability testing is satisfaction. That is, how pleasant is it
for users to use the program or design? Do users enjoy the program or activity,
or is there a high level of frustration and confusion in the process? Naturally, a
© 2013 4imprint, Inc. All rights reserved
pleasant experience is the goal, but you can learn a lot from users that find your
program or product frustrating during testing. There are a number of different
testing methods that can examine these components.
At a minimum, usability testing is about capturing information on these
components in a formal and defined setting. There are different methods
and approaches that can be used to achieve these goals. Some of these are
summarized in the following section, however, as described later in the paper,
the tools and approaches that are used vary by product, objectives and other
factors. You may choose to use a combination of methods, so understanding the
full range of options is a good starting point. Some of the methods you might use
include Hall Intercept Testing, remote usability testing, and expert reviews . Other
techniques like think-aloud protocols, co-discovery learning and eye tracking
are often used during testing sessions. Each of these methods and techniques is
defined in the following section.
Usabi l i ty test ing methods
In general, there are three widely recognized methods for usability testing. Hall
Intercept Testing, or Hallway Testing, is the method that uses 5-6 participants for
a testing session. Hallway testing includes random users that you might pass
in a hallway, hence the name. It probably is an obvious point, but participants
should not be designers or programmers, but typical users of your product or
service. Hallway testing can be effective in the early stages of a new design
when product managers or designers need to uncover serious challenges.
Remote usability testing is aptly named—it refers to conducting testing
when participants and observers are separated by space and/or time. You might
use remote testing if participants are located in different countries, for example.
Remote testing can be synchronous or asynchronous. Synchronous testing uses
video conferencing or remote accessible tools like WebEx® or GoToMeeting®. Even
though it’s a big name, the idea is simple: users participate at the same time even
if they aren’t in the same place.
Asynchronous testing, on the other hand, applies to websites or mobile device
testing and refers to the systematic collection of a user’s activities while
interacting with an application. For example, asynchronous testing might collect
click streams or user logs of critical incidents. It might also collect task times
and other data associated with how a user navigates a website or mobile site.
Asynchronous tests are carried out in the user’s own environment rather than labs
to further simulate real-life testing. Unlike synchronous testing, it may not involve
two parties interacting together but collect data on the actions by users.
© 2013 4imprint, Inc. All rights reserved
Expert reviews are another method found in usability testing. As the name
suggests, it involves experienced experts in the testing process. Experts can be
leaders in the field or an organization that specializes in usability testing. There
are a number of organizations that offer expert reviews or testing services. An
automated expert review provides usability testing using technical programs
designed to analyze design and heuristics. In plain English, that means either an
expert or an application will conduct testing to provide detailed analytics on your
site or product. Automated reviews may not provide as much detail and insight
as real people, but they are popular because they are cost and time efficient.
ClickTale®, Inspectlet and TecEd are companies that provide automated review.
There are even some services (e.g. UserTesting.com) that will conduct usability
testing for your organization from start to finish.
Not only can you use different methods in usability testing, but you can also
use different techniques. Think-aloud protocols, co-discovery learning and eye
tracking are three of the techniques commonly used during scenario testing to
capture information on the user experience.
Think-aloud protocols, also known as talk-aloud protocols (or TAP), ask
participants to talk as they are performing a set of specified tasks. Users are
asked to share what they are doing, thinking and feeling as they complete an
assignment. Observers take notes as they gain a better understanding of the user
experience. Often, these sessions are recorded so that developers can revisit how
participants reacted to specific tasks or challenges.
Co-discovery learning (CL) relies on two test users to perform a task together
while they are being observed. Specifically, users are encouraged to talk aloud
while they complete a common task. CL is similar to TAP in the sense that both
encourage users to communicate feelings, frustrations and thoughts during the
testing process. However, some believe that CL makes it more natural for users to
verbalize thoughts during a test because they are working in tandem on the same
tasks. It can be difficult to vocalize thoughts and feelings in an isolated room,
but CL sometimes makes it easier to share thoughts because there’s someone else
working on the same assignment.
Eye tracking research uses special equipment to detect where people’s eyes
are focused when they look at a computer screen. This form of usability
research helps organizations understand what users find attractive in a
design and what they tend to overlook. You can download a free copy of
“How to Conduct Eye Tracking Studies” from NN/g that has 65 tips to help
with usability tests. While it may seem like it is a science fiction tool, it’s
actually fairly common in usability testing, and there are a lot of tools that
can help implement this approach.
© 2013 4imprint, Inc. All rights reserved
Any of these methods or techniques can be used to find out how (and why)
customers use your products. The methods you choose largely depend on your
product, website or scenario. Before you get started, it’s important to have a clear
understanding of the objectives you want to achieve. In a sense, you have to act
like a professor and develop a comprehensive plan for testing and review to make
sure that you are testing the right things.
An or ientat ion to usabi l i ty test ing processes
If you plan to conduct usability testing internally, there’s a lot more to it
than sitting a user in front of a product and taking notes. Whether you’re
testing an application, software or product, you need a plan that clearly
articulates what you will test and how it will be tested. You also need a
strategy to analyze and report results that can be implemented to improve
product design. In short, testing requires extensive design and planning in
order to be effective, and it’s a lot more work than expected.
There are generally three phases of usability testing implementation. And, within
each phase, there are a number of tasks and deliverables that must be completed.
These phases are:
1. Planning
2. Testing
3. Analysis and evaluation
This three-step phase should not be mistaken for simplicity, there’s actually a lot
of activity within each phase. In fact, there are entire books written on each of
the phases, or a specific part of the phase. For example, NN/g has a free, 147-page
report that outlines 234 guidelines on how to recruit participants for usability
studies. The Handbook of Usability Testing is a book that gives a broader view
of the processes required in usability testing. The next section summarizes each
phase at a beginner’s level.
To get started, you need to clearly outline the study. This is where you define
what you are testing and why. You will also develop the how in this phase, in
other words, exactly how will you test users on your product? This phase generally
begins with the creation of a scenario. If you recall, scenarios provide realistic
settings where a person performs a list of tasks while observers take notes. Testing
instruments such as scripted instructions, paper prototypes and questionnaires
are used to gather additional feedback on the product being tested. These items
need to be developed in the first phase.
© 2013 4imprint, Inc. All rights reserved
In the planning stage, you also identify the methods and tools you might use
during testing. For example, will you use eye tracking, expert reviews or Hallway
Testing? Testing observers need to be trained on the types of questions to ask
users during testing, if any. Think of the first stage as the opportunity to plan
every detail leading up to testing. Once the plan is developed, it’s a good idea to
share it with stakeholders or project sponsors to make sure the planned testing
meets the expectations of all involved parties. It’s also a good idea to do a dry
run to check for flow, consistency and clarity. A director wouldn’t open a play
without a dress rehearsal; similarly, you shouldn’t unveil a testing scenario until
it’s been rehearsed.
In the next phase, it’s time to recruit participants and administer testing.
Although it might be easier to use someone in house, participants (or volunteers)
should not be designers, developers, account executives, clients or employees.
They should be people that represent the target audience of the product or
service. Experts recommend testing 4-8 users at different times for each testing
scenario. Some believe that recruiting the right participants is the key ingredient
to successful usability testing. As Clara Teoh from the UK-based firm Webcredible
said: “Recruitment is one of the most important things to get right. If your
participants aren’t representative of your target audience then you won’t be able
to trust your findings.”11
In order to identify the right participants you might consider developing
a recruitment profile that clearly articulates who should participate. You
might select people with a range of experiences to make sure there is a
diverse skill set. For instance, if you’re testing how someone might use
a camera, it might be a good idea to test a professional photographer
as well as a recreational user to get a range of perspectives and input.
Don’t avoid testers that may dislike your product, either. Patrick
Neeman, the Director of Product Design at nPario®, said that effective
tests incorporate users that are likely to hate the product. He said: “The
best suggestions I’ve gotten have been from users that provide more
negative feedback.”12
Once you’ve selected users, it’s time to conduct testing. The session should
be observed and recorded so it can be reviewed at later stages by designers,
programmers, or any resources that have a stake in the product. It’s important
to stick to the testing plan that was developed in phase one and follow the
11 Teoh, Clara. “6-step Process for Planning a User Test.” Web log post. Webcredible. N.p., 1 Oct. 2012. Web. 10 July 2013. <http://www.webcredible.co.uk/user-friendly-resources/web-usability/testing-plan.shtml>.
12 Neeman, Patrick. “Four Steps to Great Usability Testing (Without Breaking the Bank).” Usability Counts RSS. N.p., 4 Jan. 2011. Web. 10 July 2013. <http://www.usabilitycounts.com/2011/01/04/four-steps-to-great-usability-testing-without-breaking-the-bank/>.
© 2013 4imprint, Inc. All rights reserved
script and testing scenario exactly as intended. Again, if you were on stage,
you wouldn’t encourage actors to be spontaneous or impulsive, and the same
approach applies to usability testing.
Testing itself should take 15-30 minutes, depending on the scenario. Do not guide
the user during testing; it’s important that the user is given a task and observed
without interference. Although it’s tempting to offer suggestions and advice, the
user needs to figure out the problem independently. It’s okay to ask questions
during testing, as long as they are outside of the test. For example, asking open-
ended questions like “What were you expecting to happen at that phase?” can
help clarify the user’s thought process. You can also ask questions that elicit
feelings or frustrations the user might experience during testing.
The last phase in usability testing is evaluation and testing. Evaluation and
analysis is where you make sense of what users told you during testing, and
translate results into product improvements or process efficiencies. Although this
is the last phase in the testing process, it’s also the hardest. According to testing
expert Michael Summers, “Analysis of usability data is the hardest step in the
process because it is easy to jump to the wrong conclusions.”13 Evaluating the user
experience relies on subjective observations, but the key to success is to translate
agreed upon interpretations to fix problems and improve products.
In this phase, it’s a good idea to assemble the development team
and watch testing videos as a group. Since usability testing is
largely a qualitative process, there’s a lot of room for interpretation
of results. It’s unlikely that two observers will always reach the
same conclusion. For this reason it’s important to review the data
together, so the team can reach a consensus on observations and
brainstorm solutions on how to address problem areas to improve
the product.
During analysis, you might look for patterns in testing. Were there areas that
produced challenges for every tester? It’s also when you review any comments or
concerns that were raised during testing and discuss methods for improvement.
The next step is to develop a plan to make changes to the product.
Usability testing is not a one-time thing, if you recall, experts say companies
should “always be testing.” Once you’ve made the product improvements from
the first round of testing, testing begins again. You might test to see if you fixed
the challenges that were uncovered in previous scenarios. The more you test, the
13 Summers, Michael. “Usability Testing: Critical Success Factor #3, Effective Usability Analysis.” ‘UserResearch.com’ N.p., n.d. Web. 10 July 2013. <http://www.userresearch.com/my-blog/2010/05/usability-testing-critical-success-factor-3-effective-usability-analysis.html>.
© 2013 4imprint, Inc. All rights reserved
more you refine your product. This is why companies like Amazon.com have more
than 200 usability tests underway at any given moment. There is always room for
improvement, and that means there’s always room for testing.14
You may find that usability testing is too difficult to manage internally, but
you’re convinced of the value and importance. Relax, there are a lot of options
available for outsourcing. If you want to use an outside party for testing, there
are a number of companies that provide a full range of usability testing services.
Companies like UserZoom®, Centralis, and TechEd® can help you define, create,
and implement usability testing from start to finish. If you want help with parts
of testing, there are quite a few tools and resources that can be downloaded and
catered to your testing needs.15
The valedictor ian is … usabi l i ty test ing
There’s no question about it, if done correctly, usability testing can yield
significant results and measurable outcomes. Consultant Michael Summers
claims that usability testing led to significant website improvements for his
clients. Summers and his team observed user behaviors in different scenarios
and used the information to improve user interfaces on specific websites. In
one case study, after multiple testing sessions that led to design improvements,
the customer experienced a 54 percent improvement in successful task
completion, a 164 percent reduction in time-on-task, and a 23 percent
improvement in subjective satisfaction.16
In another example, the same firm did a usability study of a retail website.
The website had recently been redesigned and the completion rates
at the checkout process were lower than anticipated. After adopting
recommendations from usability testing, the retailer measured fewer
users abandoning their carts and an increased number of completed
purchases.17 Stories like these make a compelling case for the benefits of
usability testing when it translates observations into action.
There are other benefits usability testing provides. If done correctly, it can
increase sales, improve performance, and decrease costs associated with design
14 Goliger, Sarah. “Why Internet Marketers Must Always Be Testing.” HubSpot, 27 June 2011. Web. 02 July 2013. <http://blog.hubspot.com/blog/tabid/6307/bid/17695/Why-Internet-Marketers-Must-Always-Be-Testing.aspx>.
15 “ Basics of Website Usability Testing.” Web log post. Template Monster Blog. N.p., n.d. Web. 03 July 2013. <http://blog.templatemonster.com/2011/09/14/usability-testing-basics/>.
16 Summers, Michael. “Usability Testing: Critical Success Factor #3, Effective Usability Analysis.” ‘UserResearch.com’ N.p., n.d. Web. 10 July 2013. <http://www.userresearch.com/my-blog/2010/05/usability-testing-critical-success-factor-3-effective-usability-analysis.html>.
17 Summers, Michael. “Usability Testing: Critical Success Factor #3, Effective Usability Analysis.” ‘UserResearch.com’ N.p., n.d. Web. 10 July 2013. <http://www.userresearch.com/my-blog/2010/05/usability-testing-critical-success-factor-3-effective-usability-analysis.html>.
© 2013 4imprint, Inc. All rights reserved
and development. Usability testing can also fast-track design and development
timelines and help influence more accurate designs while reducing errors. Not
to mention, by seeing how customers interact with your product first hand, you
might obtain a competitive edge in the market. If you use testing to improve
a website, you’ll probably see a reduction in your bounce rate (i.e. people that
leave your website immediately) and an increase in sales.
Testing can also provide valuable insight to your internal design team. It provides
a way to capture feedback from a target audience that can be filtered directly to
the project team. That can be immensely beneficial to product designers tasked
with improving the product. It can also help settle internal debates on design
issues; different groups can test options that are being discussed to resolve what
method works best. Finally, it gives an opportunity to identify issues and potential
problems before a product is launched. For these reasons and many others, the
popularity of usability testing has continued to increase.
Jacob Creech from IntuitionHQ published an article on the top 10 reasons
you should initiate usability testing.18 Most of the reasons are similar to those
previously mentioned, like the ability to improve usability, user experiences and
accessibility. Creech also claims it can increase customer satisfaction and improve
the reputation of your brand. He notes that: “Developing a reputation for
producing usable, enjoyable websites will help set you apart from others. Happy
clients who recommend you to others will help you stand out. Even offering a
usability testing service in the first place will help you stand out.”19
Does usabi l i ty test ing real ly make the grade?
Not everyone gives usability testing high marks. Critics claim that usability testing
doesn’t always live up to its full potential, especially if it’s conducted poorly
without a well-planned methodology. Some even challenge Nielsen’s assertion
that five users are adequate to find 80 percent of usability errors in almost any
product. Rolf Molic from .net magazine, for example, claims that this is a myth.
After capturing information from the Comparative Usability Evaluation (CUE)
study with teams of experienced usability professionals, there is some evidence
that suggests that more users uncover greater percentages of errors in a website.
The study found that small groups catch about 30 out of 200 usability problems,
but teams of 15 or more found 60 percent of total errors.20 The CUE also showed
that expert reviews found the largest percentage of errors in a website and were
more effective than randomly selected users.
18 Creech, Jacob. “10 Great Reasons to Usability Test.” IntuitionHQ, 13 May 2011. Web. 11 July 2013. <http://www.intuitionhq.com/blog/2011/05/10-great-reasons-to-usability-test/>.
19 Ibid.20 Molich, Rolf. “Usability Testing Myths.”.net Magazine. N.p., 27 Feb. 2013. Web. 02 July 2013. <http://www.
netmagazine.com/features/usability-testing-myths>.
© 2013 4imprint, Inc. All rights reserved
Another criticism of usability testing is that it is never 100 percent
representative of real life scenarios. Although the goal is to
create scenarios that mirror common use, it’s impossible to fully
replicate how a customer uses your product in a home or a business
setting. For example, one of your testers may be a mother of two,
and unless her children are running around the testing lab, the
scenario is not fully representative of her true life. Likewise, a sales
representative may need to access your site while talking to a client
on the phone, which is unlike a testing situation where he or she is
completely focused on the testing at hand.
Usability testing can be expensive and time consuming, too. As noted, to be
effective, it requires a significant amount of resources for planning and execution.
Another complaint is that test validity depends heavily on identifying the right
target group. You can jeopardize the entire session if you pick the wrong testers.
This happens when people select testers that are friends, family members
or coworkers. While it may seem like these individuals are representative of
potential users, in actuality they are most likely from the same demographic, and
have similar education and economic levels. Suddenly, the study does not have a
wide variety of users.
Dana Chisnell summarized it nicely in a usability-focused blog when she said that:
“for many of us, usability testing is a necessary evil. For others, it’s too much work,
or it’s too disruptive to the development process.” Not every company has the
resources and time available to conduct usability testing, and even when they
do, if it’s not done correctly it won’t produce results. That’s why if you plan to
implement usability testing studies it’s important to get it right, or hire someone
that can do it for you.
The f inal grade
In the final analysis, the pros of usability testing far outweigh the cons. Canoo,
a Swiss firm that claims to specialize in “end-user happiness,” claims that it can
be done on limited budgets and doesn’t necessarily require extensive training
and tools. According to a consultant at Canoo, “The key to getting value from
[usability testing] is to keep investment and costs to a minimum. Train yourself,
don’t commit to fancy tools, resist the allure of pricey consultants, and just get on
with practicing.”21
21 “Canoo RIA Blog.” Canoo RIA Blog. N.p., n.d. Web. 01 July 2013. <http://www.canoo.com/blog/2010/03/19/10-lessons-learned-from-usability-testing/>.
© 2012 4imprint, Inc. All rights reserved
4imprint serves more than 100,000 businesses with innovative promotional items throughout the United States,
Canada, United Kingdom and Ireland. Its product offerings include giveaways, business gifts, personalized gifts,
embroidered apparel, promotional pens, travel mugs, tote bags, water bottles, Post-it Notes, custom calendars,
and many other promotional items. For additional information, log on to www.4imprint.com.
Whatever (if any) testing method you pursue, the key is to develop a plan
and stick to it. Overall, testing can be a valuable tool to make sure your
product receives a passing grade in the market.