User Insights: Brokerage iPhone AppsA DEMONSTRATION OF THE BENEFITS OF MOBILE USER TESTING

2

INTR

OD

UC

TIO

N

For the past 20 years, Corporate Insight has helped financial services firms improve their customer experience by offering ongoing, objective feedback from the perspective of actual account holders. In that period, the customer website has emerged as arguably the single most important point of contact between financial institutions and their clients. We are now in the midst of a paradigm shift, however, as smartphones and tablets become increasingly popular entry points to the Web. More and more, customers expect to be able to manage their financial accounts via a mobile device, just as they do via the customer website. Financial services firms must meet that expectation if they want to remain competitive in today’s marketplace.

As companies develop and refine their mobile-optimized websites and apps, they face a significant challenge – designing a user-friendly experience for a variety of smaller screens and different operating systems. Unlike traditional websites, where general guidelines for effective design have emerged over the past two decades, mobile platform design does not yet have a similar set of standards. As a result, it’s essential that financial services companies understand what happens when clients and prospects actually interact with their firm via a mobile device so they can design a satisfying experience.

With that in mind, Corporate Insight developed a usability study to shed light on the importance of mobile platform-focused user testing. Our study reviews the iPhone apps offered by four leading brokerage firms – Charles Schwab, E*TRADE, Fidelity and Merrill Lynch. While our findings highlight issues that users encountered with these particular apps, our real goal is to demonstrate the type of information that can be gained when you observe real users interacting with a mobile platform.

For each firm, we met with five participants, all with varying levels of experience with the iPhone and brokerage products. These individuals worked to complete 10 different tasks that are common to the mobile brokerage experience. We focused on identifying aspects of each firm’s design that confused, frustrated or inhibited users from completing the task at hand. Some of the issues raised in this report are particular to one firm, while others represent common problems shared by several firms.

This report provides a summary of the key findings from our usability study. For more information regarding our user testing and user research services, please contact Alan Maginn at 212.832.2002, ext. 116 or amaginn@corporateinsight.com.

Introduction

3

TAB

LE O

F CO

NTE

NTS

Table of Contents

Why Is User Testing Important?

When Should You Test Your Mobile App?

Test Design

• Our Objective

• Tasks

• Device

• Study Participants

• Methodology

Detailed Findings

• Stock Search Functionality

• Stock Quote: Related Actions

• E*TRADE’s Stock Alert Interface

• Trading Interface: Selecting an Order Type

• Trading Interface: Number Entry

• Fidelity’s Trading Interface in iOS 6

• Customer Service Phone Number

Takeaways from This Study

Corporate Insight Is Here to Help

About the Author

4

Why Is User Testing Important?

In an industry as competitive as financial services, user experience plays an important role in the client’s overall satisfaction with their firm. Financial institutions may use products and pricing to gain new customers, but a user-friendly online experience has become critical to both retention and engagement. And with the rise of the smartphone and the tablet, the mobile experience is fast emerging as another key competitive battleground.

With so much riding on the quality of your mobile website or apps, it’s imperative that you understand what happens when real world clients and prospects interact with your designs. By performing user tests throughout the design lifecycle, you can answer the following questions:

Is it easy for your audience to learn to use your interface?

How quickly do users complete common tasks?

Do users make any errors as they work to complete common tasks?

How satisfied are users with your design?

The value of user testing doesn’t stop at improving client satisfaction. The following examples from the Nielsen Norman Group report Usability Return on Investment1 demonstrate the impact an improved user experience can have on your company’s success:

In the early 2000s, MasterCard hoped to redesign its business site to increase its usability and encourage more small business owners to get business cards. The design team brought more content to the homepage, presented direct links to common tasks, and provided targeted content to various groups of users. During user testing, customers were 300% more likely to be able to find the products they were looking for.

Insurance provider New York Life changed the location of product links and brought more content up front on their homepage. Total sale leads per day increased by 213%.

In the 1990s, over 2 million pages of IBM’s website were separately owned, leading to huge design and navigational inconsistencies. When the design and architecture of the site was unified in 1994, online sales increased by 400%.

1 Jakob Nielson, J.M. Berger, Shuli Gilutz, and Kathryn Whitenton. Usability Return on Investment (ROI). 4th Ed. (Freemont, CA: Nielsen Norman Group).

WH

Y IS

USE

R T

ESTI

NG

IM

POR

TAN

T?

5

When Should You Test Your Mobile App?

There isn’t any one particular time when it makes the most sense to test your mobile app. On the contrary, to ensure that your interface is easy to use, you should perform user tests throughout the design process, from conceptual prototypes through the implementation of final designs.

And the testing shouldn’t stop just because you’ve released your app to the public. As new mobile devices come to market and mobile operating systems evolve, it’s important to understand how these changes impact the user experience. Testing your current app is also a great first step when you plan to update your interface or develop a new app for another platform, while testing competitor apps may reveal alternative approaches to some of the design challenges you face.

But won’t all of this testing slow down your design process, while pushing your project over budget? Not necessarily. According to leading Web usability expert Jakob Nielsen, the vast majority of usability issues can be discovered in as few as five tests,2 and as we demonstrate in this report, the tests themselves can be conducted over the course of just a few days.

2 “Why You Only Need to Test with 5 Users,” accessed October 15, 2012. http://www.useit.com/alertbox/20000319.html

WH

EN S

HO

ULD

YO

U T

EST

YOU

R M

OB

ILE

APP

?

Concept

Design & DevelopmentDeployment

Validation & Assessment

User Testing

6

Test Design: Our Objective

The purpose of this study was to demonstrate the value of user testing mobile platforms. We tested some of the investment industry’s most popular apps, focusing on core features like order entry and quote retrieval. While our findings highlight design issues in the apps offered by Charles Schwab, E*TRADE, Fidelity, and Merrill Lynch, our goal was not to critique these particular apps per se. Rather, we wanted to illustrate the kind of insights that can be gleaned from even a small-scale usability test. We hope that the usability problems uncovered in this report inspire you to test your own platforms to understand what happens when real users interact with them.

TEST

DES

IGN

7

Test Design: Tasks

Our tests centered on 10 tasks. By design, nearly all tasks were possible on all four apps, though there were a few exceptions due to the limitations associated with our accounts or because the firm did not offer that particular capability through their iPhone app. These exceptions are noted in parentheses in the list below:

1. Login to an account

2. Retrieve a stock chart for a security without knowing its symbol

3. View the most recent news article for a security

4. Set up a price alert for a security (Charles Schwab and Fidelity)

5. Add a security to a watchlist

6. Set up a limit order for the security

7. View current market value for the account

8. Transfer money from an outside bank account to the brokerage account (Merrill Lynch)

9. Review all transactions within the account for the past 90 days or 3 months

10. Access the firm’s customer service phone number

TEST

DES

IGN

8

Test Design: Device

All of our tests were conducted on an iPhone 4 running iOS 6, which was released shortly before we began our study. Our decision to perform our tests on this particular device was based on a number of factors:

The general trend within the financial services industry has been for firms to launch apps for iOS-based devices before focusing on other devices.

Among the firms we track for our Mobile Monitor service, iPhone apps (30) are more widely available than apps for Android (25) or RIM (12).

Although Android phones account for 68% of the smartphone market as of September, 2012,3 the uniformity of the iPhone’s design makes it a better target for a general user test.

3 “Worldwide market share for smartphones,” Associated Press, accessed October 15, 2012. http://finance.yahoo.com/news/worldwide-market-share-smartphones-220747882--finance.html

TEST

DES

IGN

9

Test Design: Study Participants

In keeping with the concept that most usability issues can be discovered through five tests, we tested five individuals per app for a total of 20 participants. To understand the impact that prior investing experience had on a participant’s ability to complete the assigned tasks, we selected individuals with brokerage accounts as well as those with limited-to-no investing experience.

We were also curious to see how a participant’s familiarity with the iPhone affected their performance. As a result, we met with individuals in the following categories:

Frequent iPhone Users – Individuals who currently own an iPhone, regardless of the model, and have been using the phone for at least six months.

Occasional iPhone Users – Individuals who have some experience with an iOS device, typically gained through the use of a friend or family member’s device. Although they are familiar with iOS, they do not own an iPhone.

Novice iPhone Users – Individuals who have had no exposure to, or nominal experience with, iOS devices.

When designing a usability test, Corporate Insight can work with your firm to recruit study participants based on your target audience or such other factors as:

Age

Experience with financial products

Trading style or frequency

Net worth

Vision issues

TEST

DES

IGN

10

Test Design: Methodology

Participant meetings were conducted in Corporate Insight’s usability lab, located at our offices in Midtown Manhattan. All sessions were recorded using Techsmith’s Morae usability software. Users interacted with the brokerage apps through an iPhone 4 attached to a specialized user testing “sled.” This sled (pictured at right) features a webcam mounted at the top of the device to capture participant’s activity within the apps; a second webcam is used to capture the participant’s facial expression.

Test sessions were moderated by a trained usability professional who guided the participants through the ten tasks, offered clarification when necessary and conducted an interview once the test was complete. A second analyst monitored the session and logged notes from an observation room, separated from the testing room by a one-way mirror.

As participants worked to complete the tasks, they were encouraged to “think aloud” so that we would gain more insight into their decision making process.

Mobile User Testing Sled

Snapshot from Morae Recording

TEST

DES

IGN

11

Stock Search Functionality

When it comes to stock search tools, all four apps tested in this study provide dynamic search results. That means the user can view results as they input a symbol or company name into the search field. Some apps displayed these results more quickly than others, but the availability of this functionality is an industry best practice.

While dynamic search results can make it easier for users to look up a company name or stock symbol, our testing uncovered a related usability issue that was present in all four apps. Rather than select the company name from the list of dynamic results, many study participants clicked the search button once they finished typing the company name. When this occurred, the apps stated that there were no related results, which confused users.

This example drives home an important point. Although your company may intend to create user-friendly interfaces, you can’t know what really happens when users interact with these features unless you test them. By understanding the needs and expectations of your audience, you can design more effective interfaces.

E*TRADE Dynamic Search Results

E*TRADE Stock Search Results

DET

AIL

ED F

IND

ING

S

12

Stock Quote: Related Actions

Many of the tasks we asked users to perform were related to a particular stock – Celgene Corporation (CELG). After asking individuals to find information on the company, we had them perform a number of actions, including set up a limit order, add Celgene to a watchlist and set a price alert for the stock. Although some users relied on the main menu to perform these actions, study participants overwhelmingly expected to find links to related actions from the stock quote page, regardless of their prior experience with brokerage products.

In terms of trading, our users were correct; all four firms provide a link to their trading interface from the stock quote page. On the other hand, links to alerts and watchlists (when available through the app) were not always present, which meant users were forced to navigate through the main menu in order to locate these features. But it’s not enough just to make these links available. Our test demonstrates that the location and appearance of those links is also very important.

Both Schwab and E*TRADE use green buttons to highlight their trading functionality. The color is especially effective on Schwab’s interface because most other content on the page is blue or black. On the other hand, E*TRADE’s Set Alert button features a gray design that was overlooked by some users. In both instances, the buttons appear within the body of the page, which means the buttons are no longer visible if the user scrolls down the page.

Charles Schwab Trade Button

E*TRADE Trade and Alert Buttons

DET

AIL

ED F

IND

ING

S

13

Stock Quote: Related Actions

Merrill Lynch provides three related action buttons – Trade, Alert and Add to Watchlist – from the stock quote page. Unlike Schwab and E*TRADE, though, these buttons are positioned in a static frame at the bottom of the page. With Merrill’s design, the buttons are always visible, regardless of where the user navigates within the page. Although the firm doesn’t use color to draw attention to the buttons, users had the most success with this design.

Fidelity offers a unique solution to the related action issue – Apple’s native “related actions” button. The button, which looks like an email “forward” icon, is featured in other native iPhone apps like Photos. As a result, study participants with significant iOS device experience found the button and its underlying actions very intuitive. On the other hand, iOS novices were either confused by the button or overlooked it entirely.

Merrill Lynch Action Buttons

Fidelity’s Use of Native iPhone Related Action Button

DET

AIL

ED F

IND

ING

S

14

E*TRADE’s Stock Alert Interface

Setting a stock alert on E*TRADE’s iPhone app proved problematic for many participants. Although the feature is accessible from the stock quote page, two of the five participants failed to complete the task because they could not find the set alerts page. These individuals were confused by the Alerts tab in the main menu, which allows users to view alerts but provides no option to modify or set alerts.

The remaining participants successfully located the set alert interface but found the inputs very cumbersome. At first, users tried to set the alert price by using the main slider. Unfortunately, the functionality of this feature was limited; users could move the price a few points but then it would lock in place.

After becoming frustrated with the slider, participants tried to adjust the price with the +/- buttons, only to find that this was either very time consuming or, in the case of those users who held the button down for an extended period, that the interface kept moving well after they’d taken their finger off the phone. In the end, most participants questioned why they couldn’t input a number manually.

DET

AIL

ED F

IND

ING

S

E*TRADE’s Set Alerts Interface

15

Trading Interface: Selecting an Order Type

To test the usability of each app’s stock/ETF trading interface, we asked participants to set up a limit order to buy one share of Celgene stock, good for one day. While all users were eventually successful, some interfaces caused problems due to issues related to design consistency and link affordance, or the visual cue that something can be clicked or interacted with.

Entering the order type proved problematic for several firms. Although it’s the first input in all four apps, participants overlooked the order type buttons on the Schwab, Fidelity and Merrill Lynch apps. When we probed to understand why, users explained that they thought the buttons were labels due to their gray-on-gray color. Schwab’s design proved least problematic of the three, due to the fact that the input fields feature a bold heading and are offset from the other inputs on the page.

E*TRADE, on the other hand, utilizes dropdown menus for its order inputs, which is consistent with the other selection inputs on the page. Users experienced no problems with this interface.

DET

AIL

ED F

IND

ING

S

16

Trading Interface: Number Entry

Another aspect of the trading interface that affected usability were the number input fields. Again, none of the designs kept users from completing the order entry process but it was clear that participants found certain interfaces easier to use than others.

The biggest factor that affected usability was the entry keypad. E*TRADE and Fidelity utilize a 12-button number keypad while Merrill Lynch and Schwab utilize the full 31-button number and symbol keypad. Beyond the irrelevance of symbols in a number entry field, the 12-button keypad is preferable because its numbers are larger targets for the user’s fingers.

Another feature that affected usability was the input fields themselves. E*TRADE and Schwab utilize pop-up windows that focus the user’s attention on the task at hand. Fidelity and Merrill Lynch, on the other hand, have the user input numbers directly on the trading screen. As a result, other input fields remain visible, which can lead to some confusion.

E*TRADE Input Field & Keypad

Merrill Lynch Input Field & Keypad

DET

AIL

ED F

IND

ING

S

17

Fidelity’s Trading Interface in iOS 6

In addition to the basic issues regarding keypads and input fields, we also encountered what appears to be a glitch in Fidelity’s app and we believe it is related to Apple’s recent iOS 6 software update. During testing, users complained that the field for inputting the Limit Price was hidden by the keypad. We tried to replicate the issue on an iPhone 4S running iOS 5 but were unsuccessful. Although it is not a critical issue, this glitch highlights the need for companies to continually test their apps, especially when new hardware comes to market or operating system software is updated.

Fidelity app viewed on iPhone 4S, iOS 5

Fidelity app viewed on iPhone 4, iOS 6

Input for Limit Price shifts to top of page, enabling user to view field as they type.

Input for Limit Price is hidden below keypad.

DET

AIL

ED F

IND

ING

S

18

Customer Service Phone Number

For their final task, participants were asked to locate a customer service phone number. In this instance, usability was related to the intuitiveness of the app’s information architecture, not any functionality. As a result, we didn’t expect there would be much difference between the apps, but our test results suggest otherwise.

Across the board, participants looked to the main menu for a customer service number. Those individuals working with the Schwab app located the number the fastest, as the firm’s menu features an option labeled “Call Schwab.” Merrill Lynch also provides access to a “Contact Merrill Lynch” feature via a fixed frame at the bottom of its main menu but due to the color and position on the page, several participants overlooked the feature at first.

E*TRADE houses its customer service information within a Help tab. Although not as readily accessible as Schwab or Merrill, the location was intuitive to users.

DET

AIL

ED F

IND

ING

S

19

Customer Service Phone Number

Fidelity’s app, on the other hand, houses its customer service number under a tab labeled “Settings.” Although most users eventually located the number, it was often after an exhaustive search through various areas of the app. Participants commented that it was “the last place” they expected to find the information. To make matters worse, the number is positioned below the fold on the Settings page.

So how can you avoid a situation where your users are unable to find important information or features? Aside from testing your interface with users throughout the design process, you can ensure your information architecture is intuitive by involving users in the design process through card sorting exercises. In these exercises, users organize the content of your website either under predetermined headings (closed card sort) or under headings they make themselves (open card sorting).

SCREEN FOLD

Fidelity’s Settings Page

DET

AIL

ED F

IND

ING

S

20

Takeaways from This Study

We hope that this usability study sheds some light on the value of user testing, particularly the importance of testing your mobile platform – and competitor platforms – given this rapidly evolving space. Although our study focused specifically on brokerage mobile apps, there are a number of takeaways that apply to the broader financial services industry:

Dynamic search is a helpful feature but it can confuse users if they do not have access to traditional search results

Action buttons should have the affordance, or look and feel, of a button

Numeric keypads should be used for numeric input fields

Contact information should be easy to find

Apps should be tested to ensure that operating system upgrades do not negatively affect functionality

TAKE

AW

AYS

FRO

M T

HIS

STU

DY

21

Corporate Insight Is Here to Help

User testing is a powerful research tool that should be incorporated into your broader design process. It can help you:

Understand the needs of your clients

Prioritize improvements to your interface

Discover design flaws

Improve customer satisfaction

Corporate Insight is here to help with all of your user testing needs. We can:

Collaborate with you to design a study that tests the key features of your mobile app or website, or the platforms offered by competitors

Recruit participants that represent your target audience

Provide you with a detailed analysis of test sessions

Assist you with your existing usability efforts, moderating tests that you have designed or hosting tests at our facility, conveniently located above Grand Central Station in Midtown Manhattan

In addition to our user testing services, we also offer Expert Reviews, assessing your interface based on established design principles and industry best practices. We also perform in-depth Website Audits, benchmarking your customer website against peer sites in terms of design, functionality, navigation and usability and offering precise, actionable recommendations to improve it.

COR

POR

ATE

INSI

GH

T IS

HER

E TO

HEL

P

22

Upcoming Usability Research from Corporate InsightU

PCO

MIN

G R

ESEA

RC

H

In the coming months, Corporate Insight will be conducting a series of usability studies focusing on the various segments of the financial services industry we track through our Monitor subscription services. Our next study, due to be published in June 2013, will address the usability of banking websites and iPad apps, with a strong focus on transaction interfaces. If you would like to be notified when this new study becomes available or if you are interested to learn more Corporate Insight’s User Research Services, contact Alan Maginn, Director, User Research at 212.832.2002, ext. 116 or amaginn@corporateinsight.com.

23

About the Author

Alan Maginn is the Director, User Research on Corporate Insight’s Consulting Services team and has been with the firm since 2004. He received his Certified Usability Analyst accreditation from Human Factors International in 2011 and has since led the development of Corporate Insight’s new User Research services. In addition to his work regarding usability, Alan has also led CI’s social media research since 2008. Prior to that, he was Senior Analyst for Corporate Insight’s Mutual Fund Monitor-Advisor service.

Alan Maginn, CUASenior Analyst, Consulting Services212.832.2002, ext. 116amaginn@corporateinsight.com @alanmaginn

AB

OU

T TH

E A

UTH

OR