track c 1115_slidesharejuhlinmcnally

David Juhlin Peter McNally Bentley University User Experience Center

Best Practices for Bank Web Site Navigation Best Practices and Methods

www.bentley.edu/uxc | [email protected] | 781-891-3142

[email protected] | 781-891-2893

2

• Why We Did This Research

• Marketing Survey

• Tree Testing with a Twist

• Guidelines

• Creation of our own IA

• The Business Case

• Appendices (depending on time)

Presentation Overview



3

• Main issue for clients • “10% of task attempts failed due to problems in IA”

-Jakob Nielsen, April 16, 2009

• Navigation difficulties • Structure/grouping of IA (mental model)

• Labeling (comprehension)

• Graphic layout (visibility)

Why We Did This Research



4

• Survey

• Tree test on three different navigational menus • Blue Hills Bank

• Cambridge Trust

• Rockland Trust

• Used the insight to create our own navigation • User Experience Center Bank (UXC Bank)

• Of these banks, which one do you think performed best?

Our Research

Marketing Survey



6

• Question What information is important to find on a bank’s website?

• Sampling method • Email lists

• Social media

• Online classified ads

• Response rate • 1331 started the survey

• 1132 completed (85%)

• 892 cleaned up responses (79%)

The Survey



7 Most important things on a bank website…

0%

10%

20%

30%

40%

50%

60%

70%

80%

Pe

rce

nt

Percent Respondents Designating Items as "Most Important on a Banking Website When Searching for a New Bank"

(Error bars represent 95% confidence interval)

Tree Testing with a Twist



9

Optimal Workshop (www.optimalworkshop.com)

• Early usability test of organization and labeling of IA

Primary Navigation

• http://bentley.optimalworkshop.com/treejack/bank22

• No visual distraction

• Task success

Tree Testing (basic method)

http://www.optimalworkshop.com/

http://bentley.optimalworkshop.com/treejack/bank22



10 The Three Banks Information Architecture

Top Level IA Options Blue Hills Cambridge Rockland Personal Personal Banking Personal

Small Business Business Banking Small Business

Commercial Wealth Management Commercial

Loans Customer Support Investing

Online & Mobile About Us Planning Tools and Resources

Rates Community Focus

Community About Us



11 The Three Banks Information Architecture

Navigation Within Personal Blue Hills Cambridge Rockland Checking Checking Overview

Savings Savings, CDs, IRAs Online Banking*

CDs &IRAs Online Banking Mobile Banking*

Money Market Mobile Banking mDeposit*

Account Services Mortgages Checking*

Financial Calculators Home Equity Debit Cards

Credit Cards Savings, CDs & IRAs*

Personal Loans Overdraft Options*

More Services Mortgages*

Home Equity

Personal & Auto Loans

Investing

Insurance

Other Products & Services*

*Items had a third level



12 Impact of Information Architecture Size

Number of Levels

Ite

ms

on

Eac

h L

eve

l



13 Task Selection

0%

10%

20%

30%

40%

50%

60%

70%

80%

Pe

rce

nt

Item

Percent Respondents Designating Items as "Most Important on a Banking Website"


- Supported

- Not Supported



14 TreeJack

• Implemented tasks and IAs • Tested the phrasing/wording of the tasks

• Between subjects design (one IA per participant)

• Sampling method • Email lists

• Social media

• Online classified ads

• Participation rates • 700 (213, 232, 255) Started the test

• 595 (178, 209, 208) Completed 85% (84%, 90%, 82%)

• 440 (134, 167, 139) Cleaned up resp. 74% (75%, 80%, 67%)



15 Success Rate

Success rate

Tasks Blue Hills Cambridge Rockland Fees 77% 87% 25%

Online Banking 99% 81% 94%

Interest Rates 58% 98% 68%

Card Options 65% 84% 77%

Mobile Apps 93% 75% 82%

Branch Hours 4% 84% 68%

Open an Account Online 63% 23% 18%

Overdraft Protection 20% 7% 51%

Bank Size 1% 90% 2%

Compare Bank Accounts 61% 80% 77%

Offers Financial Services 43% 57% 73%

Mortgage/Loan Availability 94% 90% 68%

Average Success 57% 71% 59%



16 Survey Result

0%

10%

20%

30%

40%

50%

60%

70%

80%

Pe

rce

nt

Item

Percent Respondents Designating Items as "Most Important on a Banking Website"




17 Average Success Rate Top 5

Success rate

Tasks Blue Hills Cambridge Rockland Fees 77% 87% 25%

Online Banking 99% 81% 94%

Interest Rates 58% 98% 68%

Card Options 65% 84% 77%

Mobile Apps 93% 75% 82%

Average success 78% 85% 69%



18 Weighting

• Weighting formula • 𝑾𝒆𝒊𝒈𝒉𝒕𝒆𝒅 𝒔𝒄𝒐𝒓𝒆 = 𝑰𝒎𝒑𝒐𝒓𝒕𝒂𝒏𝒄𝒆 × 𝑺𝒖𝒄𝒄𝒆𝒔𝒔

• More important tasks are valued higher than less important tasks

• Some tasks are more important than others • Based on customers goals (what we did)

• Alternative approach • Based on business value

• Combination of business value and customer goals



19 Weighted Success Rate

Weighted success rate

Tasks Blue Hills Cambridge Rockland Original average score 57% 71% 59% Top 5 average Score 78% 85% 69% Weighted Average Score 68% 78% 59%

Guidelines



21 Guidelines

1. Replicate some pages/sections (or crosslink) • Retirement account information is beneficial to include

under both ‘Personal’ and ‘Investing’

• Overdraft Options (‘Checking’-> ‘Overview’, ‘Compare Accounts’, ‘Rates & Fees’)

• Replicate ‘Open an a Account Online‘ at multiple places and make sure it’s in close proximity to ‘Enroll in Online Banking’



22 Data from Treejack

Cambridge

Personal Banking 37% 53%

Business Banking 2% 10%

Wealth Management 57% 74%

Customer Support 4% 7%

About Us 1% 3%

Rockland

Personal 34% 38%

Small Business 1%

Commercial 1% 1%

Investing 31% 40%

Planning Tools and Resources 35% 45%

Community Focus About Us

Task 11: Where do you expect to learn whether a bank offers services to help you

manage your retirement accounts? Visited first Visited first

Visited first Visited during

Visited first Visited during



23 Guidelines

2. Don’t use vague terms • ‘Account Services’

• ‘Facts at a Glance’ (under ‘About Us’)

• ‘Other Products and Services’

• ‘Planning Tools & Resources’

3. Be careful of misleading terms • ‘Rates’ Vs. ‘Fees’



24 Guidelines

4. Group items at the right level • Structure ‘Rates’ under ‘Personal’ and ‘Business’ instead of

as an option on the first level (and name it ‘Rates & Fees’)

• ‘Online & Mobile’ should be structured as an option on the first level

• ‘Credit/Debit Cards’ can preferably be structured under ‘personal’ and ‘business’



25 Guidelines

5. Chunking matters • It is better to have an IA with more levels than a shallow IA

with many options on each level. Users will find it as long as it is a clear logic path. (Don’t take it too far)

Creation of Our IA



27 Creation of our IA

• Copied and combined good parts from the other IAs

• Compromises based on importance level

• Implemented our insight and modified accordingly

Top Level IA Options Blue Hills Cambridge Rockland UXC Personal Personal Banking Personal Personal

Small Business Business Banking Small Business Small Business

Commercial Wealth Management Commercial Online & Mobile

Loans Customer Support Investing Investing

Online & Mobile About Us Planning Tools and Resources About Us

Rates Community Focus Customer Service

Community About Us

51 items 53 items 220 items 150 items



28 The Information Architecture

Navigation Within Personal Blue Hills Cambridge Rockland UXC

Checking Checking Overview Checking*

Savings Savings, CDs, IRAs Online Banking* Savings, CDs, IRAs*

CDs &IRAs Online Banking Mobile Banking* Credit Cards*

Money Market Mobile Banking mDeposit* Loans**

Account Services Mortgages Checking* Retirement*

Financial Calculators Home Equity Debit Cards Membership Rewards

Credit Cards Savings, CDs & IRAs*

Personal Loans Overdraft Options*

More Services Mortgages*

Home Equity

Personal & Auto Loans

Investing

Insurance

Other Products & Services*

*Items had a third level **Item had a forth level



29 TreeJack

• Replicated the same test with our IA • Same tasks

• Did not retest the other IAs

• Slightly changed sampling method • Avoid learning bias

• Mechanical Turk

• Craigslist

• Participation Rates • 210 Started the test

• 189 Completed (90%)

• 154 Cleaned up (81%)



30 The result

Success rate

Tasks Blue Hills Cambridge Rockland UXC Fees 77% 87% 25% 90% Online Banking 99% 81% 94% 94% Interest Rates 58% 98% 68% 90% Card Options 65% 84% 77% 85% Mobile Apps 93% 75% 82% 97% Branch Hours 4% 84% 68% 86% Open an Account Online 63% 23% 18% 76% Overdraft Protection 20% 7% 51% 81% Bank Size 1% 90% 2% 88% Compare Bank Accounts 61% 80% 77% 84% Offers Financial Services 43% 57% 73% 94% Mortgage/Loan Availability 94% 90% 68% 91% Average Success 57% 71% 59% 88%



31 Average Success Rate Top 5

Success rate

Tasks Blue Hills Cambridge Rockland UXC Fees 77% 87% 25% 90%

Online Banking 99% 81% 94% 94%

Interest Rates 58% 98% 68% 90%

Card Options 65% 84% 77% 85%

Mobile Apps 93% 75% 82% 97%

Average success 78% 85% 69% 91%



32 Weighted Success Rate

Weighted success rate

Tasks Blue Hills Cambridge Rockland UXC Original average score 57% 71% 59% 88% Top 5 average Score 78% 85% 69% 91% Weighted Average Score 68% 78% 59% 89%



33 The Replication

UXC Bank Visited first Visited during

Personal 59% 61%

Small Business 1% 1%

Online & Mobile 1% 1%

Investing 36% 36%

About Us 1% 1%

Customer Service 2% 2%

Investing

Overview

Rates & Fees Personal Retirement Business Retirement Personal Wealth Management

Institutional Investment Management Fiduciary Services Trust Administration

Newsletters Investment Offices Our Team

Personal

Checking

Savings, CDs, IRAs Credit Cards

Loans

Retirement

Membership Rewards

Task 11: Where do you expect to learn whether a bank offers services to help you

manage your retirement accounts?

The Business Case



35

• Total revenue from the website • life time value/customer × amount of customers from site

$140 × 10,000= $1,400,000

• An improved website provides more customers • Original number of customers + increase of customers

10,000+1,000=11,000

• Total increased revenue over the websites life time • life time value/customer × Increased amount of customers

$140 × 1,000=$140,000

Calculations



36

• Increased amount of customers

Calculations

Bank Website Prospects

Lost prospects

New Customers $

Conversion rate =𝑁𝑒𝑤 𝐶𝑢𝑠𝑡𝑜𝑚𝑒𝑟𝑠

𝑃𝑟𝑜𝑠𝑝𝑒𝑐𝑡𝑠

• New customers from improvement • Prospects × conversion rate × improvement of conversion

500,000 × 0.02 × 0.05 = 500



37

• Total increased value becomes • Prospects×Conversion×Improvement×Customer Value

• Estimates (over the website’s design’s lifetime) • Number of prospects: 500,000

• Old conversion rate: 2%

• Improvement: 5% (new conversion rate: 2.1%)

• Average customer lifetime value: $140

• Total increased value • 500,000 × 0.02 × 0.05× $140 = $70,000

Calculations



38

• Return of investment

• ROI = 𝑮𝒂𝒊𝒏 𝒐𝒇 𝒊𝒏𝒗𝒆𝒔𝒕𝒎𝒆𝒏𝒕−𝑪𝒐𝒔𝒕 𝒐𝒇 𝑰𝒏𝒗𝒆𝒔𝒕𝒎𝒆𝒏𝒕

𝑪𝒐𝒔𝒕 𝒐𝒇 𝑰𝒏𝒗𝒆𝒔𝒕𝒎𝒆𝒏𝒕

• Estimates • Gain of investment: $70,000

• Cost of investment: $30,000

• Return of investment

• ROI = $𝟕𝟎,𝟎𝟎𝟎−$𝟑𝟎,𝟎𝟎𝟎

$𝟑𝟎,𝟎𝟎𝟎 = 133%

Calculations



39

• Market share gains • Competitors loose

• Late discovery of problems • Late development changes - “You can use an eraser on the

drafting table or a sledgehammer on the construction site.” – Frank Lloyd Wright

• Project delay

• Might still need additional research to identify a better IA

• May have to choose • Expensive change vs. under preforming site

Other Considerations



40

• Intranet • Employee productivity

• Training time

• Cost of not finding the information

• Ecommerce site • Conversion rates

• Higher profit/purchase

• Returning customers

• Support site • Decreased support calls

• Increased customer satisfaction

Other Type Of Calculations



41

• Create a business case • Varies for industries and type of site

• Ballpark increase of task success: 5%

• Not the same as increased conversion rate

Your Project

Appendices



43 Demographics – Age Distribution

0%

10%

20%

30%

40%

50%

60%

70%

18-25 26-34 35-54 55-64 65 or over

Pe

rce

nt

Age

Age Distribution of Study Respondents

Survey

Blue Hills

Cambridge

Rockland

UXC



44 Success Rate – Age Groups

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Blue Hills Cambridge Rockland UXC

Succ

ess

Rat

e

Success Rate by Age Group

18-25

26-34

35-54

55-64

65 or over



45 Demographics – Income Distribution

0%

5%

10%

15%

20%

25%

30%

35%

$0-$15,000 $16,000 – $30,000

$31,000 – $50,000

$51,000 – $74,000

$75,000 – $150,000

More than$150,000

Prefer not toanswer

Pe

rce

nt

Income

Income Distribution of Study Respondents

Survey

Blue Hills

Cambridge

Rockland

UXC



46 Success Rate – Income Groups

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Bluehills Cambridge Rockland UXC

Succ

ess

Rat

e

Success Rate by Income

$0-$15,000

$16,000 – $30,000

$31,000 – $50,000

$51,000 – $74,000

$75,000 – $150,000

More than $150,000

Prefer Not to Answer



47 Mechanical Turk vs. Craig’s List

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Craig's List Mechanical Turk

Succ

ess

Rat

e

Participant Source

Success Rate by Participant Source



48 Mechanical Turk vs. Craig’s List

• Number of cheating participants • Mechanical Turk: 18 of 116 (15%)

• Craig’s List: 16 of 67 (24%)

• Cost per completion • Mechanical Turk: $0.5

• Craig’s List: $1

• Issues with Craig’s list

Thank you!

Questions?

David Juhlin Usability Consultant Bentley University User Experience Center [email protected] 781-891-3142

Peter McNally Senior Usability Consultant Bentley University User Experience Center [email protected] 781-891-2893

track c 1115_slidesharejuhlinmcnally

Economy & Finance