Mobile BankingBenchmark StudyAuthors: Ann Rochanayon, Sneha Kanneganti

www.userzoom.com

Study MethodologyMOBILE BANKING BENCHMARK STUDY

UserZoom’s researchers created a competitive benchmark study to determine which bank offers the best mobile experience for prospective customers and to assess the impact of the mobile banking experience on brand perception, Net Promoter Score and conversion.

200 panelists were recruited by email from SSI, Survey Sampling International during the February 24 – March 2014 timeframe. The participants met the following criteria:

US participants were randomly assigned to evaluate one of the three US Bank sites, while Canadian participants evaluated CIBC. 50 panelists evaluated each site.

Study Methodology

Sites evaluated

Participants

• Ages 26+• Not a current customer of the site evaluated• Have prior experience with mobile banking• Participated in the study via smartphone

5

7

Finding an ATM / branch and researching banking services (two of the three tasks performed) are among top banking activities performed via smartphone.

Selected “Smartphone” when asked “On which of devices, have you conducted the following online banking activities listed below?”

Banking Activities Conducted via Smartphone

Banking Activities Conducted via SmartphoneMOBILE BANKING BENCHMARK STUDY

Banking Activities Conducted via Smartphone

Check Account Balance

View Transactions

Find an ATM / Branch

View / Pay Bills

View Account Statement

Transfer Money

Research Banking Services(e.g., accounts, credit cards, loans)

Deposit Money

90%

87%

76%

74%

72%

71%

60%

43% N=198

9

Participants found it easiest to research accounts on CIBC compared to the other sites and encountered the least number of issues when performing the three tasks.

Key FindingsMOBILE BANKING BENCHMARK STUDY

Although task success was very low for researching an account, ease of use ratings and frustration ratings were not significantly different from the top site (CIBC).

Participants found KeyBank easiest to use when finding customer service information and also rated KeyBank highest when finding a branch.

KeyBank tied with CIBC for the highest overall satisfaction ratingsand the highest Net Promoter Score.

Participants found the site harder to use when researching accounts and finding customer service information and generally found that too many clicks were required to find information.

USAA performed third-best in the study.

CIBC had the highest overall ratings.

10

Key FindingsMOBILE BANKING BENCHMARK STUDY

Although Wells Fargo has a mobile optimized website, participants were forced to navigate to the full site to research an account.

For task #2, locating a customer service phone number, 22% reported frustration in using the full site, even though this information was available on the mobile site.

Clicking on the incorrect link was among the top problems.

Wells Fargo had the lowest overall ratings for ease of use,organization and layout and satisfaction primarily due to difficultiesin navigation and participants having to use the full site to performthe tasks

Participants brand perception improved significantlyfrom pre to post mobile site evaluation

Net Promoter Score for Wells Fargo was lowest overall (-44%), while KeyBank hadthe highest Net Promoter Score (8%)

Wells Fargo had the smallest improvement in brand perception, while the brand perception improvements on other sites were comparable.

BankWells Fargo

KeyBank

USAA

CIBC

Delta+13%

+31%

+32%

+27%

12

Overall ResultsMOBILE BANKING BENCHMARK STUDY

Key Performance MetricsAfter study participants completed the three tasks on one of the banking sites, they answered questions about their overall site experience and questions about brand perception.

CIBC rated highest overall on key performance metrics. Meanwhile, Wells Fargo rated lowest on most metrics and significantly lower than CIBC on ease of use. KeyBank tied CIBC for overall satisfaction and had the second highest overall ratings, followed by USAA.

Ease of use, Organization and Layout, and Satisfaction: Responded 6-7 on a 7-point scale when asked, “Please rate your experience with the following aspects of the mobile site.” where 1 = Negative and 7 = Positive

Likelihood to open an account: Responded 6-7 on a 7-point scale when asked, “Based only on your experience today, how likely would you be to open a bank account with [BANK]?” where 1 = Not at all likely and 7 = Extremely likely

Key Performance Metrics

Ease of Use

Organization and Layout

Satisfaction

Likelihood to open an account

30%

36%

34%

16%

56%

46%

50%

16%

46%

40%

42%

14%

58%

50%

50%

22%

*Significantly lower than site with highest rating at 90% Cl

*

Site with the highest rating Site with the lowest rating

15

Overall ResultsMOBILE BANKING BENCHMARK STUDY

Study participants were asked how they perceived the brand they evaluated prior to interacting with the site and also after interacting with the site. All sites improved in brand perception after the site interaction, however, Wells Fargo experienced the smallest increase in brand perception.

Based on what you know / your experience today, what is your perception of [BANK]?

Responses of 6-7 shown from a 7-point scale where 1 = Negative and 7 = Positive

13%11% 12%

17%

42% 44% 44%

26%

N=48 N=50 N=18 N=50 N=25 N=50 N=46 N=50

Pre Brand Perception Post Brand Perception

Pre vs. Post Brand Perception

BankWells Fargo

KeyBank

USAA

CIBC

Delta+13%

+31%

+32%

+27%

16

Overall ResultsMOBILE BANKING BENCHMARK STUDY

Change in Associated Brand AttributesPre vs. Post Mobile Site EvaluationStudy participants also selected brand attributes they associated with the site pre and post site experience. The chart below shows the change in positive brand association. Many positive brand attributes increased for many of the sites, although there are several decreased for Wells Fargo and KeyBank.

Convenient

Competitive

Comprehensive

Reliable

Trustworthy

Secure

Easy access

Exclusive

+P

osi

tive

Which of the following words or phrases would you use to describe [BANK]?

Wells Fargo KeyBank USAA CIBC

11%25%

20%11%

15%

10%14%

8%

4%

4%2%

1%

3%

5%43%

42%11%

10%-4%

-16%-8%

2%6%

4%2%

-9%

-3%

-5%

-12%-6%

-10%

-17%

17

Overall ResultsMOBILE BANKING BENCHMARK STUDY

Change in Associated Brand AttributesPre vs. Post Mobile Site EvaluationFor Wells Fargo, brand association with “Difficult” increased. Wells Fargo was also perceived as less “Expensive” after site interaction.

Expensive

Untrustworthy

Unreliable

Difficult

--N

egat

ive

Which of the following words or phrases would you use to describe [BANK]?

-13%

-5%

-5%

-5%

2%

2%

-3%

-4%

4%

16%

-2%

-1%

-1%

0%

0%

1%

Wells Fargo KeyBank USAA CIBC

18

Overall ResultsMOBILE BANKING BENCHMARK STUDY

Although fewer participants associated Wells Fargo with the “Expensive” after evaluating the mobile site, 10% still considered Wells Fargo to be “Expensive”.

This Wells Fargo participant talks about her perception of Wells Fargo as “a bank for people with a lot of money.”

00:00

I just fill like it's a bank for people with a lot of money. I don't feel like it's a type of bank that I would go to. I am a stay-at-home mom and I am not sure I can even afford having one of their accounts.

‘‘’’

19

Overall ResultsMOBILE BANKING BENCHMARK STUDY

Net Promoter Score

How likely is it that you would recommend this website to a friend or a colleague?

NPS

-44%

8%

-34%

-22%

60% 24% 16%

26% 40% 34%

52% 30% 18%

38% 46% 16%

Detractor (0-6) Passives (7-8) Promoters (9-10)

KeyBank had the highest overall Net Promoter Score at 8%. Of all the sites evaluated, participants were least familiar with KeyBank and were more likely to be neutral about the brand. Consistent with other overall site ratings, Wells Fargo also had the lowest NPS.

23

Task 1 - Research AccountMOBILE BANKING BENCHMARK STUDY

Problems Encountered

Content

Site Performance

Wells Fargo KeyBank USAA CIBC

18%

38%

32%16%

10%

16%

8%14%

24%

8%

24%6%

2%

2%

12%4%

2%

4%

8%4%

8%

32%

22%28%

Difficult to compare accounts

Not enough account information available

Information was confusing / unclear

Site was slow

Encountered errors / techical issues

I did not encounter any problems or frustations

Problems understanding content were fairly common across all the sites, but difficulty comparing content was reported by about one-third or more of participants on KeyBank and CIBC. While nearly one-quarter of Wells Fargo and KeyBank users found the information confusing or unclear.

Overall, significantly more Wells Fargo participants encountered issues while researching savings account than participants on the other three sites.

Which of the following problems or frustrations, if any, did you encounter while researching savings accounts at [BANK] from your mobile device? [Check all that apply]

26

Task 1 - Research AccountMOBILE BANKING BENCHMARK STUDY

Key Bank First Clicks

Only 4% were successful in finding the minimum balance for a savings account with the highest interest rate on KeyBank. Numerous savings accounts are available and several clicks are required to find the rate information for each account. When users finally reach the rate and account details page, the page is not mobile friendly and additional adjustments are required to be able to view and read the information.

Most efficient path

17%

37%

41%

4%

28

Far Too Much Just Right Too Little

18%

40%

42%

Task 1 - Research AccountMOBILE BANKING BENCHMARK STUDY

Amount of Information Available on the Site

How would you rate the amount of information available on the site?

What information was lacking?

70% of participants thought the CIBC site offered the right amount of information on the mobile site, higher than the other sites in this study. KeyBank was considered most lacking in terms of amount of information provided.

8%

38%

54%

6%

50%

44%

4%

70%

26%

Wells Fargo Key Bank USAA CIBC

Wells Fargo Key Bank USAA CIBC

“Ability to search that would then connect to full site”

“Clearer info about options on mobile site”

“Information comparing each account with the other”

“I would have liked to see more information on how long before you accumulate interest”

“Key details like minimum amounts and interest”

“Details like fees for if your balance drops below the minimum”

“Information about their savings account. There should be a little more information on every title/name of the accounts.”

“A comparison chart for accounts”

Wells Fargo First ClicksOnly 2% of the users took the most efficient path to find a branch.

10% users said that they clicked on the wrong link and that there were too many links to click on before they could find the correct page.

Wells Fargo also had the lowest ease of use ratings at 58%, the most problems encountered and the highest time on task at 1:07 minutes.

39

Task 3 - Find a BranchMOBILE BANKING BENCHMARK STUDY

12%

24%

2%

46%

Most efficient path