How to build a failsafe mobile usability testing set-up

Fabien Marry - @AlphabUX

1

I started my career working in TV…

Such control rooms have a large number of devices working in a chain, any of which can fail at any moment.

Yet the live show must go on, so they have endless amounts of contingency plans to immediately adapt to most equipment failure.

The lesson I took away: technology fails.

This is the norm, not the exception. This is not an excuse for your service to be interrupted.

Later on in my career, I was in charge of supervising teams of qualified audiovisual professionals with much more experience than me at some high-level events involving heads of state.

Many times, these professionals let me (and my employer) down in a big way.

One VIP speech wasn’t recorded because they forgot to turn the volume back up after turning in down to avoid hearing the noise of delegates settling in their chair before the session. Another key 20 min speech recording was missing a part because the tape had run out and they didn’t anticipate it.

Just like technology, people will fail. You need to anticipate that too.

These experiences taught me it’s unlikely that absolutely everything will work as intended. So now, I always have a plan B (and often a plan C, and D).

Web host Internet Local Network Test device

+ ++

When conducting mobile web usability testing (with a standard setup) you need all four of these elements to work as they should.

Here two elements are completely out of my control: the web host and the location’s internet connection. If they go down, I can’t do anything about it, and it’s game over. This could leave me with confused participants, annoyed clients that wasted their time, an expensive testing facility bill, and a project plan to redo.

We can do better!

Local Network Test deviceYour Mac

+ +

You can use your own laptop as the web server (via the OS built in servers or an application like Mixture).

By doing this, internet and web host issues can be fully prevented.

Now, let’s think about another part of the set-up. How do we record the action?

Some use a document camera, requiring the devices to be kept flat on their back on a table.

But this is not how people use tablet or mobile devices. They are meant to be hand held.

Putting them on a table changes how they are used: - The fingers used will be different, potentially hiding ergonomic issues - The device will be positioned further away from the eyes than it would normally

be held, preventing you from testing legibility properly.

Local Network

Test Device Reflector Your Mac

Silverback

ObservationTV

+++

So here’s my initial setup: testing on an iOS device, using the Airplay mirroring feature to send the screen content to a Mac running @ReflectorApp.

The content of that Mac screen and the face of the participant (captured via a webcam), are then recorded using @SilverbackApp, and mirrored to the TV in the observation room via a cable.

Local Network

Test Device Reflector

Your Mac

Silverback

ObservationTV

+

++

Hue HD camera

When recording hand gestures is important, as it often is for apps, I add another webcam.* The picture from this is displayed on the Mac using Quicktime player (choose “File’ / ‘New Screen Recording’, but don’t actually press record). Silverback grabs everything, including the additional webcam, and displays it on the TV in the observation room.

* I use a @HueHD camera as it’s flexible neck is easy to attach to the participant’s chair.

Local Network

Test Device Reflector Your Mac

Silverback

ObservationTV

+++

Remote client

Skype

When I can’t run a cable to the observation room TV, I use Skype to share the screen and audio with a device in the observation room. You can then connect your device (I use my iPad) to the large observation TV / sound system.

With a group chat (sadly unsupported if one of the devices is an iPad), this setup even allow one or several clients to observe the sessions remotely.

However, this requires the internet, which we don’t control (see slide 7), so I always flag that this as a nice bonus rather that something to rely on.

So this is what the recordings look like (clockwise from top left): - A view of the participant’s hand gestures (via the over-the-shoulder webcam) - A perfect view of the screen content, unrestricted by the hand or the participant moving the

device away from the camera’s view (via Airplay mirroring to Reflection.app) - The participant’s face (via the laptop’s built-in webcam)

Extra tips: - Use a flat colour desktop with no visible icons. This will reduce the output video file size, and

help the viewer focus on the action. - When recording a mobile phone screen, you don’t need the laptop full resolution, so I set my

laptop’s resolution to something low like 800x600, greatly reducing the space taken by the recording files.

15

But remember Sod’s law…

Local Network

Test Device Your Mac Silverback

ObservationTV

You are still relying on the local network. If it fails, a fall back option is to connect the test device straight to the observation room TV via a VGA or HDMI display adapter.

Of course, this means that your recordings won’t have the screen mirroring, but you can somewhat compensate for this with an over-the-shoulder webcam.

Another option: use your test iPhone’s 4G connection and the personal hotspot feature to connect it to the laptop.

You can even use a direct usb connection for stronger interference resilience.

Your own network

Test Device Reflector Your Mac

Silverback

ObservationTV

+++

So here you have it in full!

Reflector is £13, Silverback is £70. Assuming you already own a laptop and your test mobile device, this set-up is basically a sub £100 full mobile usability lab. And it fits in a backpack.

Fabien Marry - Senior UX consultant !fabien.marry@cxpartners.co.uk

Credits !Photos Slide 2: http://tvmedia.hpage.co.in/course_37574641.html Slide 3: Jérémy Dupeu, http://www.camera-forum.fr/index.php?/topic/8376-captation-multicamera/ Slide 4: The World Bank / Fabien Marry, http://web.worldbank.org/WBSITE/EXTERNAL/EXTABOUTUS/ORGANIZATION/EXTPRESIDENT/EXTPASTPRESIDENTS/EXTOFFICEPRESIDENT/0,,contentMDK:20872180~menuPK:51408951~pagePK:51174171~piPK:64258873~theSitePK:1014541,00.html Slide 5: http://lolsnaps.com/news/27654/0/ Slide 6 plan A Slide 9: Harry Brignull, http://www.90percentofeverything.com/2011/12/14/anatomy-of-a-hardware-usability-testing-rig/ Slide 10: huffingtonpost.com Slide 15: Readee, http://s287.photobucket.com/user/Kailexanra/media/SodsLaw.jpg.html !Icons from the Noun Project Wireless & iPad by Edward Boatman Laptop by Olivier Guin Cloud by Dmitry Baranovskiy Television by Andy Fuchs Bluetooth by Muharrem Senyıl