usability testing goes mobile

!

!

W8#Session!6/24/2015! !11:30!AM!!!!!

“Usability#Testing#Goes#Mobile”##!

Presented#by:#!

Susan#Brockley#ExxonMobil#

#####

Brought#to#you#by:#####

##

340!Corporate!Way,!Suite!300,!Orange!Park,[email protected]!E!www.sqe.com!

!!!

!!

!!!!! !!!!!

With more than twenty-four years of software engineering experience at ExxonMobil, Susan Brockley is passionate about quality assurance, testing, and requirements management. Although her current title is business analyst in the IT Center of Expertise, Susan spends most of her time consulting on projects about properly scoping work, completing requirements, and engaging the customer. Her credentials―Certified Software Quality Analyst and Certified ScrumMaster―hang on the wall, but Susan prefers to roll up her sleeves and coach teams with whatever they need to begin a successful journey.

!

Susan Brockley

ExxonMobile

Usability Testing Goes Mobile June 2015

Susan Brockley

Introduction What is usability? What makes mobile usability different? What are some process changes should you consider? How can you transition usability into the mobile environment?

Agenda

3

Susan Brockley ExxonMobil Global Services Company • 25 years in software engineering (test and QA)

• Software Engineering Masters from UT Austin (2010)

• Certified Software Quality Analyst (CSQA) • QAI, since 2002

• Certified ScrumMaster

• Scrum Alliance, since 2010

Introduction

What is Usability?

5

[ISO 9241] “…the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use.” • Goals are task-oriented • Effectiveness - the ability to complete a task without errors

• Efficiency - completing the task quickly with minimal wasted effort

• Satisfaction – user’s perception of the product

Usability Basics

6

There are other important usability attributes. • Learnability – the time to accomplish a task without having prior

knowledge of the system

• Memorability – how well a user remembers the system after a period of inactivity

• Simplicity – how easy or difficult it is to complete a task

• Software Ergonomics – the physical workload to complete a task • Traditional input: keystrokes, mouse clicks, mouse movement • Mobile devices: screen gestures, time on device, posture, typing style, usage

Additional Usability Attributes

7

Traditional usability tests include these processes.

• Plan tests • Develop tasks • Decide where to hold the test • Establish a baseline

• Conduct tests • Conduct a pre-test interview • Observe user as they complete tasks • Conduct a post-test interview

• Analyze the data and report results • Compute metrics, summarize comments and survey results, make

recommendations • Share results with development team, management, and stakeholders

The Usability Testing Process

What Makes Mobile Usability Different?

9

Pull out your mobile device. Pair up with somebody else. • Informal poll - raise you hand…

• Who pulled out a phone? • Who pulled out a tablet? • Who pulled out something else? What was it?

• Within your pair, one of you start typing while the other(s) observe. Now switch. • How is viewing a mobile device different? • Were you able to read what the other person was typing? • How much of the person’s facial expression / body language were you able

to observe while you looked at the device?

Activity – Getting in a Mobile Mindset

10

Usability on a mobile device is still a planned activity to determine a product’s efficiency, effectiveness, satisfaction, and ergonomic impact.

• Requires up-front planning

• Involves five or so Testers in a one-hour test session

• Is best communicated to the project team in a summarized form

Mobile Usability – What Stays the Same?

11

Users expect more from mobile applications.

• Users expect that mobile applications will have better performance than their desktop equivalent

• Users expect a friendlier interface, no need to contact help

• Mobile applications will be updated more frequently

• Mobile devices will seamlessly integrate applications and components such as phone, message, camera, calendaring, etc.

So what?

• Communicate heightened user expectations to development team

What Makes Mobility Different? (1 of 8)

12

Mobile devices can be used anywhere and at any time.

• Mobile context – circumstances under which a mobile device is used • Location awareness – delivering information about physical location • Physical environment introduces additional challenges to mobility

• Sunlight/Glare • Water • Heat/cold • Sound

So what?

• Your testing environment just got bigger and more diverse

Why is Mobile Usability Different? (2 of 8)

13

Accessibility means making it easier for seniors or people with disabilities to use mobile devices. • Being able to change text size • Displaying high contrast themes • Enabling voice • Magnifying the screen • Displaying browser captions • Using intelligent agents (example: Siri) • Interfacing with peripherals (example: hearing aids)

So what?

• An application or web site is not usable unless it is accessible


14

Affordance means making it clear what can be selected, tapped, or swiped.

• A button with slightly raised appearance suggests you can push it • An e-book suggests you can swipe a page to turn it

So what?

• Consistent affordance improves usability because the same gesture

means the same thing everywhere


15

Power is now a part of the user experience.

• “…poor battery life contributes to customer dissatisfaction more than any other smart phone feature…” - Survey by J.D. Powers and Associates

• Mobile applications don’t drain the battery directly, they do so by using the devices hardware components

So what?

• Usability testing should measure power consumption to ensure there are no surprises

But wait!

• Consider power loss in context, never by itself


16

User experience will vary with connectivity. • Intermittent connectivity introduces variability in performance and

reliability • …and, when it comes to mobile devices, performance matters

So what?

• Capture comments about performance and connectivity to quantify the user experience

But wait!

• Again, consider connectivity in context, never by itself


17

Protective covers for mobile devices are common and necessary.

• If you loan mobile devices for testing, you may wish to protect them from damage and breakage

…however

• Covers may alter a user’s perception of the product

• Covers may also affect the way the mobile device is used in a normal setting (e.g. using a cover to prop a device is common)

So what?

• Note the use of protective covers and whether they alter user perceptions


18

Caution your Testers about the risk of using devices in field exercises. • Pedestrian cell phone-related injuries have doubled since 2005 • 70% of adults have experienced digital eye strain • There is an increase in the number of deaths and injuries due to

distracted driving

• Encourage safe testing behavior • Discourage mobile use and driving or walking • Ask Testers to observe the 20-20-20 rule

• Take a break way from the mobile device every 20 minutes for 20 second to focus on something 20 feet away

• Remind Testers to take breaks and pay attention to posture • Avoid sitting in the same position for extended periods


What Process Changes Should You Consider?

20

Field tests require additional planning considerations

• Do you loan devices for field tests or have Testers use their own? • What is the right number of Testers? • How long with the test last? • What data will you collect and how will Testers supply it?

But wait! Do you really need a field test?

• A major advantage of conducting field tests is that it takes context into

consideration, which is difficult to simulate in a laboratory

Planning Processes

21

Field test execution is very different from lab-based execution.

• Typically longer in duration

• Less structured and usually does not require tasks

• Collecting results is different • Direct observation • Daily diaries • Software capture • Survey

Execution Processes

22

Test results should always include the following:

• Tester demographics • Test environment (location, device type, connectivity, etc.) • Direct observation (posture, verbal remarks, task completion, timing) • Survey results

Field tests introduce a broader, less structured source of information • Make it clear which results were obtained in a lab setting, which ones

were obtained in the field • Correlate contextual information with Tester’s feedback to better

quantify field results • Example: poor performance observed more often with Wi-Fi than 3/4G

• Follow up with Tester to clarify remarks

Analysis and Reporting Processes

How do You Get Started?

24

• How many of you have performed traditional usability testing?

• How many of you now test or support mobile applications or devices?

• How many of you have a mobile test lab?

• How many of you have done field tests on mobile devices?

• How many of you have a centralized usability or UX/UI team?

• How many of your organizations require usability tests as part of the development process?

Activity – Organizational Usability Maturity

25

Each organization has a different starting point with respect to usability testing – whether traditional or mobile

• The goal is to mature as an organization in usability testing practices

Getting Started

26

Begin performing traditional usability studies on desktop applications. • Practice test planning, execution, and reporting • Find a development team who will work with you • Recruit people to help – Testers, Technical Coordinators, Observers • Conduct a retrospective to improve your process

And when you are ready • Conduct “real” tests and share your results outside your testing group

Getting Started – Step 1

27

• Transition to mobile usability studies in a laboratory setting. • Introduce mobile applications or devices into the usability testing process • Modify what you wish to capture • Practice mobile usability tests • Conduct a retrospective to improve your process

And when you are ready

• Conduct “real” tests and share your results outside your testing group


28

Set up a mobile test environment. • Start simply – just controlling context, connectivity, and power is a big plus • Add loaner devices, protective covers, charging carts, testing software as

time and budget allows

Don’t forget about Mobile Device Management (MDM) • Administers mobile devices • Distributes applications • Sets configurations • Oversees physical device management

• Storage • Power • Check in/out • Protection


29

Conduct mobile field studies. • Begin with a shadowing technique to observe Testers as they go through

their normal activities • Your first studies will provide valuable experience and validate your

assumptions about field tests

If you want to involve more Testers, try a group approach • Communicate objective, schedule, and desired results • Capture results using

• Surveys • Daily diaries • Specialized software • Group feedback sessions

• Thank Testers for their participation


30

Consider creating a User Experience/User Interface (UX/UI) group. • A UX/UI group promotes and encourages software usability as an integral

part of the software development process

UX/UI governance groups:

• Provide projects with tools and processes to conduct usability tests • Stay connected with the industry and share knowledge • Offer training and coaching to develop usability practitioners and experts • (Optionally) steward user interface guidelines


31

Require usability testing in the development process. • Transition from an ad-hoc, grass-roots movement into an organization that

consistently plans for usability in the development process

• If done properly, it touches • Project management methodology • Business analyst processes • Developer guidelines • Testing practices


Questions?

Thank you