Synopsis of the results of analysis and task-oriented usability tests

April 2015

przyjaznyserwis.pl

THE PROCESS OF OPENING OF BANKING ACCOUNTS AND BASIC INTERNET BANKING SERVICES & FUNCTIONALITIES

Usability Monitor - testing panel for Internet banking systems.

Internet banking is a mass product - it can be stated when talking about services over a dozen or so millions of Polish use. Opening banking accounts through Internet is a common practice. The Usability Monitor Team has set itself a goal of closely monitoring topics related to usability and user experience of Internet banking systems, desktop- and mobile-wise. As part of the testing panel we will be conducting regular usability tests and issue analysis the Users / bank Clients encounter. The testing panel is enclosed in a 12 month cycle:What did we start with? We started with the basic topic: we made a thorough analysis of the banking account opening process through the Internet and a first contact with Internet banking system - we checked how the Users are dealing with both the simplest and most popular activities in an Internet banking system, being checking the balance and the transaction history.

We open a banking account online and...

The subjects of analysis were applications for opening a banking account in 8 Polish banks: PKO Pank Polski, mBank, Bank Zachodni WBK, ING Bank Śląski, Alior Bank, Millenium Bank, Bank Pekao and Getin Bank.

2

Usability Monitor is an innovative testing panel, with it’s aim being in gathering knowledge about user experience in Internet banking systems.

First of all were tested the applications made on desktop computers, and after that the ones on mobile devices, if such a possibility was made by a bank. The banks chosen for the testing, according to the data from Q IV 2014, serviced over 90% of active accounts for

individuals, available for online access. The majority reports a steady increase in the quantity of clients and the degree of utilisation of online and mobile banking services (based on prnews.pl report http://prnews.pl/raporty/raport-prnewspl-rynek-bankowosci-internetowej-iv-kw-2014-6550614.html). This is the main reason for the assumption, that online account applications are an important channel for a lot of potential clients and may (and should) provide a benchmark for the market itself. More so, the online (and mobile) channel is chosen by approximately 30% of the customers for submitting simple loan products applications (global data based on Capgemini Financial Services Analysis, 2015; 2015 Retail Banking Voice of the Customer Survey, Capgemini Global Financial Services).

3

The analysis of usability of submissions for opening a banking account had an aim of identyfying potential issues, which potential bank clients could encounter.

Out of 8 submited account applications on desktop devices all were successful, out of 3 mobile ones 1 failed. In one case the account wasn’t opened.Filling up forms (and additional money transfers, if applied) took from 2 min 43 sec to 5 min 10 sec.The number of field requiered ranged from 18 to 43.

4

The process of opening a banking account consists not only of www form, it’s a complex system, which doesn’t always work...

8 minutes - that’s the time required to open and activate an account, allowing for a login to one of the banks tested (mBank). It’s the best score in the test conducted. In a bank the process took the longest we had to wait 37 hours 49 minutes (not including the one bank we weren’t able to open an account through Internet in). Of course we cannot forget, that opening a banking account can not be simplified to the matter of the application form itself, it’s a number of processes behind this seemingly easy task.

"Unfortunately, we couldn’t proccess your submission..."

The amount of usability issues ranged between 0 and 10, one being a critical error, rendering the submission impossible.The quote form the shown above subtitle comes from one of the tested forms. It’s not a wrong message per se, the problem lies in fact, that it wasn’t accompanied by anything else - a more detailed explanation of the situation and a manual for a potential client would suffice in such a case.

In total 44 errors and issues were identified in the tested forms. Based on those, we prepared several recommendations, general in nature.

5

For 8 desktop and 3 mobile submissions, 44 usability issues were identified.

We suggest: ✓ Paying exceptional attention towards the path required to take to access a desktop and

mobile online form and better construction of messeges leading to them, ✓ An analysis for the fields required for submitting a form by a potential client,✓ Utilisation of a range of possibilities to shorten, simplify and validate data gathering fields

with an input matrix (several personal data, dates, telephone numbers, postal codes, etc.),✓ Exceptional care towards using precise and useful error and help messages.

We do realise, that above mentioned recommendations seem basic and obvious. However, as long as the issues of certain character will keep occuring in the solutions used by tens of thousands of Users, there is a reason to keep repeating those recommendations. For the detailed errors and issues description and identification and the recommendations following, please visit: http://usabilitymonitor.net

What the Users say

In the first wave of testing 6 Users, age 35 to 60, took part, 3 men and 3 women, from big cities, using online banking services regulary, with a varying computer usage fluency.The functionalities of home pages of online banking services have been tested, as well as core functionalities such as balance, available funds, transaction history and blocked funds, of 7 banks in total: PKO Bank Polski, mBank, Bank Zachodni WBK, Alior Bank, Millennium Bank, Bank Pekao and Getin Bank.

6

6 respondents, 12 hours of individual interviews, 164 test tasks.

What’s the most important - how much do I have on my account?

The fundamental information for the majority of bank clients is how much muney do they have: what’s the balance and the amount of the available funds. This diversification is important, as most of the transactions perfomed with debid cards are not accounted for online in real time, which means, that the balance doesn’t take those into consideration. The majority of the banks tested share the information about two of those values, often emphasizing the amount of available funds. Only two banks present on the main site of the online banking services the values of available funds (Getin Bank) or account balance (Bank Pekao) - in the rest of the cases those values are hidden deeper in the site, which was not appraised well by the Users. In majority of the cases this information is clear and instantly visible - apart from one case of invasive advertisement with poorly-visible closing button on Pekao24 service by Bank Pekao (although, there was a capping set on this ad).

7

Checking the balance and browsing the transaction history are the most popular activities done in online banking systems.

So, a dashboard or a wallet then?

Users not always were explicit in interpretation of calling main site of the banking service a „Dashboard” (3 banks called their main site a dashboard), in one case a big issue arose with understanding bank’s intentions (Bank Zachodni WBK) and clarifying, whether the main site is and should be a „Dashboard” or a „Wallet”. It seems, that more intuitive thing to do is connecting the term „Main site/Home page” with a „home” icon - it clarifies the place we are discussing. A site well prepared and constructed allows the User for a quick and effective location of all the fundamental information, increases his general satisfaction and certainty of using its functionalities. As for completeness of the information on the site best score gathered Millennium Bank, as for readability of the layout (and graphical design) - PKO Bank Polski. Worst in the ranking (in both categories) was Getin Bank. So, what should be the general direction? Simplicity, clarity and complexity of information presented.

8

An ambiguous label can hinder usability of a very good online service.

What do I need this graph for - issues with transaction history

It is crutial to deliver the transaction history of his account to the online banking services User. It’s not unusal the information is included on the main site of the system - in a form of a list or a contextual hyperlink. The account transaction history tends to be expanded and complicated - banks often add several additional functionalities to them (for instance, transaction categorisation allowing for an advanced analysis, automatic summing up of the incoming and outgoing transactions, generating a graph) - which not always is positively reviewed by the Users. „What do I need this graph for?” after entering the transaction history site (Bank Zachodni WBK), „I have no idea what a category is” (mBank), „For budgeting I use MS Excell” were the several feedback from respondents.

It is important to focus on the precision of description of labels: „Last transactions” should always mean the same, however, PKO Bank Polski in several cases shows „last transactions” not necessarily in chronological order (it is being veryfied wether or not it was a one time error).

9

An overload of options and functionalities can confuse the Users.

Summary

All audited banks were graded above the neutral score (0 to 10 scale). Out of 164 test task, 17 were failed to complete, the assistance of the moderator was needed in 8 cases.We believe, that the issues identified during the usability tests could be easily fixed, if those simple, ground rule were applied:

✓ Simple communication, clear labels and descriptions,✓ Complete data set, ✓ Clear setup of the site’s layout✓ Uninvasive ads.

Many of those issues are repairable without implementing costly and time consuming IT projects, relying only on modification of presentation layer, improved website copywriting and visualisation of interface elements. We are confident, that only the analysis made by an end-user, being well interpreted, allows for an insight into many usability difficulties, which are often overlooked by the designers due to the fact, that they know the system in-and-out.

10

Out of 164 test tasks, 17 were left unfinished.

Usability Monitor ensures a constant access to the views of such Users, publishing a report to another set of problems connected to usability of online banking systems monthly.

An additional asset of tests run by Usability Monitor is that they are conducted in a natural enviroment, not in a dedicated lab, which allows to raise the reliability of the outcome by eliminating stress.For a detailed description of issues mentioned above and solution recommendations, please visit for a full version of the survey: http://usabilitymonitor.net

11

Usability test conducted in the Users’ natural enviroment allow to identify a larger number of usability issues.

Usability Monitor is a research service, allowing for a constant gathering of Users’ opinions of the online banking systems - desktop and mobile alike, including the forms for opening such accounts.

Every month a select group of respondends conduct usability tests in online banking services of chosen Polish banks or submits test account submissions.Every month an expert analysis of the issues stated in research timetable is conducted.http://usabilitymonitor.net/harmonogram_badan.html

Reseach is conducted by a team lead by Maciej Kostro.

Maciej Kostro since year 2000 involved in online business, focusing on financial institutions. Has a deep knowledge in the fields of designing business processes and services, online solutions, information architecture, usability and user-experience. A creator of online marketing strategies.

Cooperated with a number of financial brands, including, but not limited to: mBank, PKO Bank Polski, FM Bank, Bank BGŻ, Bank Pocztowy, Raiffeisen Bank,

UNIQA TU.

12

We run the Usability Monitor research panel and other usability tests for financial institutions.