oredev 2014 knjohnson: ux & software testing

A VENN DIAGRAM OF UX & SOFTWARE TESTING Karen N. Johnson Oredev 2014

The team needs to build a product. The team readily recognizes the testers need to work

with the developers but the same team often doesn’t consider that the testers also need to

work with the UX staff. Often the UX staff is tucked away in a different part of the office,

working with multiple teams and yet, rarely working directly with the testers. Why? How

can testers review a product without a good understanding of the design? Testers need

closer access to UX and UX would benefit from working directly with the testers. Karen

highlights specific design areas where UX and testing pair well.

About this talk …

Oredev - UX & Software Testing © Karen N. Johnson 2014

•  Software Test Consultant •  Published Author (Beautiful Testing) •  Co-founder of WREST: Workshop on Regulated Software Testing •  Website: www.karennicolejohnson.com or www.karennjohnson.com •  Twitter: @karennjohnson

Karen N. Johnson

So what’s a Venn diagram???

A/B Testing A/B testing refers to a marketing experiment when two options are offered. An example is two versions of a web page to see which page results in more successful progression, successful typically is referring to customers progressing to the next step in a sales progression of an ecommerce site.

A/B Testing from UX perspective

¨  Look and feel ¨  Control test ¨  Funnel optimization ¨  Increased sales

A/B Testing from Software Testing perspective

¨  Don’t break the flow ¨  Test paths ¨  Work with UX to build

accounts to control and test the paths.

UX Page design Market feedback Conversion improvement

Testing

A/B split handled

Both “paths” Don’t break overall flow

Look & Feel Nav Process flow

A/B “Testing” with UX & Testing

Multivariate testing Multivariate testing is a technique for testing changes to many different elements all at the same time on one web page.

Funnel Optimization ¨  Conversion rate is a common

metric for gauging customers taking the next step.

¨  UX may design with funnel optimization in mind but testing does not typically approach a product from this mindset.

Storyboarding

The premise … that UX and Test can benefit by

storyboarding together.

storyboarding

A storyboard visually

tells the story

panel by panel.

UX designers use storyboarding.

Designers focus on the flow

of a feature story.

Testers focus on where and how a story can breakdown.

Let’s try another example. Let’s try this one together.

But if you only learn one question to ask, let the question be …

What if?

•  What if? Is a challenge question.

•  What if? Is the type of question that begs for investigation and

discovery.

•  What if? Does not require technical training or knowledge.

When UX & Test collaborate on storyboarding,

together they can think through a process flow

and also consider how the process can break.

The result: Risks revealed. Testing understood.

Stronger design built. More complete testing.

State Transitions

State transition when defined in

academic terms can be long and

(frankly) dull.

A shorter more practical explanation is:

identify states and the events that

trigger the transition.

Let’s go back to the ecommerce

shopping cart flow and see

where the state transitions

occur.

“State transitions provide a

Bermuda-Triangle-like moment in time

where “things” can go wrong for a user

or system. And once a user or system

is in a bad state, they can be awkwardly

trapped or lost.”

When UX & Test map state transitions in a

process, it becomes visible where points of

vulnerability lie. These potential problem spots

need design and test considerations.

Using personas

The premise … that UX and Test can work these

two concepts together to arrive to a better end.

personas & tours

The concept of using a

more precise model helps

to envision designing a

product for someone, not

just for a “user.”

UX designers use the persona concept

The tour concept represent specific, focused testing advice that guides the tester through paths of the software. The tours help a tester think through inputs, data and environment settings that satisfy the goals of the tour. It's a more methodical and purposeful way to test.

Exploratory testers use the tour concept

Exploratory testing & the tour concept

James Whittaker

At Microsoft a group of us test folk from around the division and around the company are experimenting with tour-guided testing. We have the Supermodel Tour in which all we do is window shop … look at the interface and find superficial (no intended disrespect to any actual supermodel who might be reading this blog) bugs. We have the Money Tour where we watch salespeople demo the product and create variations of those demos as test cases. We have the Intellectual Tour where we ask the software hard questions. We have the Garbage Collector’s Tour, the Competitor’s Tour, the Museum Tour, the After Hours Tour and a host more that we are experimenting with. All of these tours represent specific, focused testing advice that guides the tester through paths of the software. The tours help a tester think through inputs, data and environment settings that satisfy the goals of the tour. It's a more methodical and purposeful way to test.

Source: http://blogs.msdn.com/b/james_whittaker/archive/2009/04/06/tour-of-the-month-the-landmark-tour.aspx

Persona + Context + Exploratory Testing + Tours

Imagine a persona combined with a context (such as a mobile banking) app combined with a “money” exploratory testing tour.

If we design by persona,

why not test by persona?

When UX & Test collaborate on personas and

exploratory tour testing together, they can both

think through not just from a user perspective but

through a model. This combination can help

clarify design needs as well as streamline testing.

Accessibility: Consider an accessibility persona Browsers: Be fluent in browser settings and coach your UX designers Review browser stats on an intermittent basis with your UX designer Mobile: Review stats Watch app feedback and help loop feedback into design

Testers can: •  Give feedback on look & feel •  Share our test lab & devices •  Test with different resolutions, browsers, devices. •  Learn UX language for better collaboration. •  Test by persona. •  Highlight error and boundary conditions where design is needed. •  A/B testing (and multivariate testing) offer new forms of “white box testing.”

UX can: •  Be open to feedback on look & feel •  Share design strategy •  Highlight concerns with different resolutions, browsers, devices. •  Learn testing language for better collaboration. •  Suggest testing by persona. •  Ask testers to explore error and boundary conditions to ensure design “holds up” to expectations. •  Highlight where A/B testing (and multivariate) testing is setup so testers can explore.

Email: karen@karennjohnson.com Twitter: @karennjohnson

Karen N. Johnson

oredev 2014 knjohnson: ux & software testing

ab testing ab testing

testing changes

testing pair

ux designers

ux staff

ux perspective look

karennjohnson karen

software test consultant

Technology

deep dive hands-on in java ee 6 - oredev 2010

knjohnson: the art of asking questions

cross team testing oredev nov 2012 - johan åtting

ux', 'ux design' and 'good ux

oredev 2015 - taming java agents

morris prototyping - oredev - share

oredev: an exploratory tester's lessons on security threat...

participate in the future of java oredev

ux . creative . technology - 미디어포스얼라이언스...

oredev 2011: building social software for the anti-social...

speed is essential for a great web experience (oredev)

from power chord to the power of models - oredev

strategic ux - ux people

eric proegler oredev interpreting performance testing...

retrofitting architecture - oredev 2012

oredev 2008: software + services

how kanban saved a hospital in indoneisa oredev 2016

java 7 whats new(), whats next() from oredev

measuring and improving your app's network performance ...

good ux bad ux