8757 miles = 14097 km That's a long way to travel, mobile banking better be really cool down here!
Google&Analy+cs(
Banking((NZ)(
Mobile(banking((NZ)(
Michael D. Dutton
Mobile Banking didn't exist as a search term 2.5 yrs ago in NZ.
Michael D. Dutton
Wish I had kept up with the storm of mobile.
Michael D. Dutton
Michael D. Dutton
Disclaimer, Optimal Usability hasdone 94 user experience projects with the banks in NZ. However, I have not been involved in a single NZ bank project yet.
Even though we are part of the UXalliance,no one had created a solid and objectivemethodology to assess Mobile Banking. Which left us wondering, who's No.1?
Michael D. Dutton
Michael D. Dutton
We're number 1, among US banks.
Michael D. Dutton
Michael D. Dutton
Michael D. Dutton
We were excited to find a NZ bank rated number 2 through a meta-analysis, but then we found out that Mr. Nuciforo worked on the ANZ goMoney app. That's not exactly objective in our book!
Michael D. Dutton
What can users do?
Observed
user
experience?
Mobile design
principles?
Who can use it?
How do
existing
users rate it?
Usability$
InteracGon$design$
Features$User$raGngs$
PlaLorms$
Michael D. Dutton
Michael D. Dutton
Our framework for identifying the best mobile app.
Michael D. Dutton
Michael D. Dutton
Usability$
InteracGon$design$
Features$User$raGngs$
PlaLorms$
Michael D. Dutton
Michael D. Dutton
Six NZ banks were tested along with the three time Keynote winner Chase bank.
Michael D. Dutton
Usability$
InteracGon$design$
Features$User$raGngs$
PlaLorms$
Michael D. Dutton
Michael D. Dutton
Michael D. Dutton
Spider diagrams are pretty simple.Bigger = Better!
ANZ = $7bn revenue. Westpac, BNZ and ASB are roughly the same size at $4bn-ish. KiwiBank is about $1bn. And TSB is “only” $200 million Compare this with Chase bank, which turns over $100 billion +
Michael D. Dutton
Features$User$raGngs$
PlaLorms$
Michael D. Dutton
Michael D. Dutton
Using these three criteria, we created a heatmap showing capability.
Michael D. Dutton
Michael D. Dutton
Michael D. Dutton
Michael D. Dutton
Michael Dutton
iPhone Y Y Y Y Y Y y y
iPad N Y N N N N y y
Android Y Y Y Y Y N y y
Windows Y N N N N N y N
Open3accounts N N Y N N Y N N
No3log9in3balances Y Y Y N N N N N
4/53digit3log9in Y N Y Y Y N N Y
View3account3balances Y Y Y Y Y Y Y Y
View3transaction3lists Y Y Y Y Y Y Y Y
Transfer3between3accounts Y Y Y Y Y Y Y Y
Bill3payment Y Y Y N Y Y Y N
Set3up3payees393From3Mobile N N N N N Y Y N
View3automatic3payments N Y Y N Y Y N Y
Change3upcoming3payments N N Y N Y y N N
View3term3deposits Y Y Y N Y y N N
View3mortgages3&3loans Y Y N Y Y Y Y Y
View3KiwiSaver Y N N N N N N N
Pay3tax N N N N Y Y N N
Alerts:3333333333333333333333333333333333333333Text N N N N N Y Y N
Features come in many different variations. Verve Mobile surveyed 1,200 people and found five core features users want: transfers, bill pay, alerts, person to person payments, and branch/ATM locator. We added a sixth: find a past transaction.
Michael D. Dutton
Michael D. Dutton
0( 5( 10( 15( 20(
Core(Features(
Non(Core(Features(
Michael Dutton
Michael Dutton
Of the six core features we looked for, some had multiple facets. This allowed the banks to score up to nine points in this area.
Michael Dutton
Michael Dutton
Michael Dutton
User$raGngs$
0( 1( 2( 3( 4( 5(
ANZ(
WestPac(
TSB(
Chase(
ASB(
Kiwi(
BNZ(
Google(play(iTunes(All(Versions(iTunes(Latest(
Michael Dutton
Michael Dutton
We found ratings to be more indicative of bugs than actual user experience. Bugs were consistently rated 1 star, but other comments could be rated polar opposites.
Michael Dutton
Michael Dutton
PlaLorms$
Michael Dutton
Michael Dutton
Apps occassionally break, so in addition to the core platforms in NZ, we also added the criteria that they should have a responsive designed website.
Michael Dutton
iPhone(
iPad(
Android(
Windows(
Responsive(Design(`(Main(site(
Features$User$raGngs$
PlaLorms$
Usability$
What to
test? Who:
Millennials
Where?
Method
Michael Dutton
Michael Dutton
Michael Dutton
Michael Dutton
Michael Dutton
We used the six core features that users identified as most desired.
Michael Dutton
Michael Dutton
Traditional lab based testing, counter balanced design, single note taker for consistency, participants couldn't test their own banks app.
Michael Dutton
Michael Dutton
25% of the world's population, heaviest users of mobile banking, 3.4 billion in earnings by 2018.
Michael Dutton
Michael Dutton
We have labs in Auckland and Wellington, so it made sense to test there.
Michael Dutton
Michael Dutton
Michael Dutton
It's a sled system that allows the user have a very natural experience with the smartphone during testing.
Michael Dutton
0(
2(
4(
6(
8(
10(
12(
14(
16(
18(
20(
0(
10(
20(
30(
40(
50(
60(
70(
80(
90(
100(
1( 2( 3( 4( 5( 6( 7(
SUS(score(
Total(#(of(Features(
Usability(Issues(
What(we(found…(
Michael Dutton
Michael Dutton
Compare ASB to TSB. It's a good example that the number of features doesn't impact the overall usability score (SUS). It comes down to having well thought out interaction design.
Michael Dutton
Michael Dutton
Michael Dutton
Michael Dutton
In general, the banks are doing well. Most use SMS for authentication, though BNZ's NetGuard tripped us up. The deployment of custom keyboards like ANZ's goMoney, helped minimize "fat finger" experiences when typing in numeric values.
Michael Dutton
Michael D. Dutton
Michael D. Dutton
Michael D. Dutton
Michael Dutton
Michael Dutton
One of the trickiest areas was account names. Not all banks carried custom account names like "checking" or "savings" through to the mobile app. This made transfering of funds between accounts a little more challenging.
Michael Dutton
SUS(Score(
Uber(Usability(
Average(Issue(Score(
Michael Dutton
Michael Dutton
At the end of the usability testing, we asked participants to stack rank the four apps they just tested. The score of this stack ranking was then added onto the SUS score to create an Uber Usability score.
Michael Dutton
Usability$
InteracGon$design$
Creating an evaluation framework
The evaluation
process
What did we learn?
Michael Dutton
Michael Dutton
We could not find a great interaction design framework, so we created our own. We looked at the context of mobile usage, adherence to platform standards, and efficiency of interactions, just to name a few. We then had our experts double check each other for our own internal consistency.
Michael Dutton
Michael Dutton
Michael Dutton
In total we created ten categories with five criteria per category.
Michael Dutton
Michael Dutton
Michael Dutton
What is an example ofgood interaction design?
Michael Dutton
Michael Dutton
Michael Dutton
An example: On the left Kiwibank provides no affordance that anything is happening during the log in process. Contrast that with ANZ, which has the animated spin allowing the user to visually know something is happening.
Michael Dutton
Michael D. Dutton
Star1ng(&(finishing(
Naviga1on(&(IA(
Errors(
Accessibility(
Comprehension(
Consistency(&(standards(Efficiency(of(use(
Layout(
Visual(Design(
General(
81(
48(
InteracGon$Design$
?
Michael Dutton
Michael Dutton
Raw scores displayed
Michael Dutton
Michael Dutton
Usability$
InteracGon$Design$
Features$User$raGngs$
PlaLorms$
27.5%$
27.5%$
15%$
15%$
15%$
Michael Dutton
Michael Dutton
We realised the raw scored didn't tell an accurate story. An app with only three core features should have an excellent usability score. User ratings aren't consistent between comments, so we decided we should weight things accordingly.
Usability$
InteracGon$design$
Features$User$raGngs$
PlaLorms$
56.01$
Michael Dutton
Michael Dutton
ANZ came out on the bottom. This is in large part because they had three of the core features, and very low user ratings.
Michael Dutton
Michael Dutton
Michael Dutton
Michael Dutton
Usability$
InteracGon$design$
Features$User$raGngs$
PlaLorms$
77.83$
Michael Dutton
Michael Dutton
ASB comes out on top. Their combination of core features and excellent interaction design provided them with a great usability score. This combined with good platform coverage and high user ratings made them the overall winner.
Michael Dutton
Michael Dutton
Michael Dutton
Usability$
InteracGon$design$
Features$User$raGngs$
PlaLorms$
74.86$
Michael Dutton
Michael Dutton
Chase bank did well too. Their user ratings were low due to bugs, and their usability score was hampered by marketing terminology that made tasks more challenging than needed.
Michael Dutton
Michael Dutton
Michael Dutton
Even though Chase came in second overall, we decided this testing was really about Kiwi banks. Therefore, BNZ takes second and WestPac takes third place.