RBS Innovate Summary Report

2

Acknowledgements

RBS Innovate was funded by the Royal Bank of Scotland Group and all reports

were written by Helen Aynsley, Evaluation and Policy Manager on RBS Innovate

at Toynbee Hall.

The author would like to thank the four organisations, and in particular the

people, that took part in RBS Innovate and who provided so much important

information: Rupon Miah at Centrepoint, Jane Guy at Plymouth Citizens Advice

Bureau, Lesley Richardson at Prince Bishops Community Bank and Ellen Finlay at

Your Money Garden Financial Education Project.

Thanks also to the people who participated in the focus groups and interviews

that informed this research.

3

Contents

Executive Summary………………………………………………………........................

4

1 Introduction to RBS Innovate………………………………………………………...

8

2 Evaluating impact……………………………………………………………………..

9

Evaluation and financial inclusion………………………………………………….

9

The evaluation process……………………………………………………………….

11

The evaluation approach…………………………………………………...............

12

3 The projects………………………………………………………………………..........

14

Moneywise at Centrepoint…………………………………………………………..

15

The Moneywise model…………………………………………………………..

17

Moneywise evaluation framework……………………………………………

19

Summary of findings……………………………………………………………...

20

Money Active at Plymouth Citizens Advice Bureau………………………........

23

The Money Active model……………………………………………………….

24

Money Active evaluation framework…………………………………………

26

Summary of findings……………………………………………………………...

27

Save and Insure at Prince Bishops Community Bank……………………………

30

The Save and Insure model……………………………………………………..

32

Save and Insure evaluation framework………………………………………

34

Summary of findings……………………………………………………………...

36

Your Money Garden Financial Education Project………………………………

38

The Your Money Garden model……………………………………………….

39

Your Money Garden evaluation framework………………………………...

41

Summary of findings……………………………………………………………...

42

4 Findings…………………………………………………………………………………...

45

The models………………………………………………………………………………

45

Evaluation………………………………………………………………………………..

47

Conclusions……………………………………………………………………………...

49

References………………………………………………………………………………

50

4

Executive Summary

From 2005 to 2009 the RBS Group provided funding for small projects to test out new

approaches to financial inclusion. This programme, called RBS Innovate and run through

Toynbee Hall, helped 13 organisations get new financial inclusion initiatives off the

ground. After three successful years the decision was taken that, in order for projects to

provide maximum learning, it was important to focus on lessons learned and the long-

term impact of interventions. The focus therefore shifted to the evaluation of projects

with emphasis on three aims:

1. Help projects to set up their own internal evaluation structures;

2. Share the resulting evaluation frameworks with the rest of the financial inclusion

sector and apply to other organisations where appropriate;

3. Showcase the work of the projects and offer models of replication.

The role of RBS Innovate was therefore restructured to work with four organisations, each

based in different areas of the UK. The four projects that were selected are:

• Money Wise at Centrepoint (London) • Debt Gateway and Money Active at Plymouth CAB (Plymouth) • Save and Insure at Prince Bishops Credit Union (Consett) • Your Money Garden (Belfast)

This report is a summary of the four separate reports written on these projects.

Evaluating impact

In order to demonstrate the true impact that an organisation is having on their client

group, community or audience, an evaluation needs to stretch beyond outputs.

There are a number of key challenges that the sector faces when it comes to

evaluating work in financial inclusion:

• There is no single accepted definition or method of evaluation; • Financial inclusion is a young sector so does not have one accepted method of evaluation;

• Success in financial capability often means positive results in soft outcomes which can be difficult to measure;

• Defining desired outcomes can often require organisations to make subjective judgments which can be challenging

In order to establish and administer their internal evaluation frameworks, each

organisation that participated in RBS Innovate went through the same process. This

entailed answering the same questions:

• What are the key aims of this service? Establishing the aims of a service or

intervention is the first step in designing an evaluation framework as it enables the

organisation to think about their end goals.

5

• What is the right balance between ideal evaluation goals and actual capacity

to deliver? Creating a robust evaluation framework would guarantee a wealth of

rich information, but if it is not going to be realistic to gather that information it will

not be achievable.

• What information is already being captured? Using existing data as a baseline

from which to start evaluation activity means that no work is replicated and all

available information is utilised. It can also help to shape an evaluation

framework as certain data collection processes can restrict what other activity

takes place (i.e. data that is collected for the purpose of reporting back to

funders).

• What type of evaluation are you interested in conducting? Formative evaluations

ask “how can the programme be improved?’ and are undertaken in order to

provide feedback with the overall goal of progress. Summative evaluations on

the other hand ask “what is the overall merit or worth of the programme? Should

it be modified? Should it be continued?” It is important to determine what type of

evaluation to undertake as it impacts on what kind of information needs to be

collected and when.

• What methods of data collection will be most appropriate? Sometimes

quantitative data such as numbers and figures will provide the most important

information while other times more descriptive, qualitative information will be

most useful; sometimes it will be both.

The evaluation frameworks for all four organisations are most closely aligned with the

decision- and accountability- study framework which is a type of improvement- and

accountability- oriented approach. Decision and accountability oriented studies stress

the need to assess fully a programme’s value and they look for all relevant outcomes,

not just those keyed to programme objectives.

The projects

In total four organisations participated in RBS Innovate over the course of one year and

each worked with the Evaluation and Policy Manager based out of Toynbee Hall in East

London.

Moneywise is a programme within Centrepoint. Its purpose is to support young people

who are homeless or at risk of homelessness in dealing with existing debt, preventing

future money problems and promoting overall financial capability. It is delivered

through a combination of group workshops, one to one and referrals to specialist

advice services.

As well as identifying what specialist advice debt clients need, the Debt Gateway at

Plymouth CAB identifies which clients could benefit from additional financial capability

training. Those that could benefit are referred to a Money Active session – either a

workshop or a one to one. The aim of Money Active is to increase service users’

6

knowledge of budgeting, borrowing, savings, debt and where to go for additional

support and help.

Save and Insure is a scheme operated in partnership with Prince Bishops Community

Bank, Derwentside Homes and RSA insurance group. Through Save and Insure, tenants

of Derwentside Homes sign up to become members of the Prince Bishops Community

Bank. If they save £10 per month for six months they receive a £20 credit to their

account at the end of it. They also get free home contents insurance through RSA (£40

value) for the duration of the six months. The aim of the scheme is to encourage tenants

to join the credit union, develop a sustainable savings habit and take out home

contents insurance. After saving with the credit union for ten weeks members are also

able to access low-cost loans.

Your Money Garden provides financial capability education training to women across

Belfast, Newtownabbey and surrounding areas. The course is delivered across a series of

workshops and aims to address gaps in knowledge around finances. In particular it helps

participants to develop their money management, budgeting and practical skills so

that they may increase their level of financial knowledge and economic activity. There

are three key aims to Your Money Garden:

Common findings

Project models:

• Push, nudge or shove? Service user engagement is often the biggest challenge

when it comes to embedding innovative financial inclusion work and

organisations often find themselves in the position of needing more than just a

good idea to engage their target audience. Both Plymouth Citizens Advice

Bureau and Prince Bishops Community Bank faced and tackled this issue in their

respective organisations.

• Partnership working: Partnerships are important to all four of the organisations that

participated in RBS Innovate but partnership working is particularly highlighted in

the evaluations of Your Money Garden and Prince Bishops Community Bank.

• Make it relevant: Just the word ‘money’ can invoke feelings of guilt at spending

too much of it or fear of not managing it so getting people to open up about

their personal finances or take part in a scheme aimed to improve their money

management can be a challenge. One key way to get people engaged and

keep them engaged is to make money matters relevant to their lives and their

financial priorities as both Prince Bishops Community Bank and Your Money

Garden found.

• Involve service users: When it comes to delivering services to a particular target

audience it is good practice to involve them in the design. Both Your Money

Garden and Centrepoint involved their target audience in the design of their

services to ensure that they were relevant and would make the most impact.

7

• Intensive contact and ongoing support is key: All four evaluations demonstrated

that when it comes to engaging target audiences, it is often necessary to do it

face to face and it often needs to be done multiple times. This is especially true

of the experiences of Prince Bishops Community Bank and Plymouth Citizens

Advice Bureau.

Evaluation best practice:

• The future of evaluation is not paper-based: Anyone who has ever handed out a

questionnaire at the end of a workshop or training session knows that it can not

only break the flow of conversation but can also take valuable time away from

service delivery. Some information simply cannot be captured without the use of

paper-based tools and it is definitely one of the easier methods to employ but

the evaluation of these four projects shows that where it is possible, it is preferable

to explore other methods.

• It is vital to have access to all of the data: Organisations conducting internal

evaluations need to ensure they have access to all necessary data as much as is

possible. Data can be used in a number of different ways and, when some is

missing, it can significantly impact on the overall picture that an evaluation

produces.

• You can test: A lot of evaluations ask subjective questions, such as “do you feel

you are more motivated to take control of your finances as a result of this

training?” While subjective questions provide important information, it is important

not to shy away from asking more knowledge-based, objective questions.

8

Part 1 – Introduction to RBS Innovate

From 2005 to 2009 the RBS Group provided funding for small projects to test out new

approaches to financial inclusion. RBS called this programme RBS Innovate and through

it helped 13 organisations get new financial inclusion initiatives off the ground. The

overall aim of RBS Innovate was to “increase financial inclusion through the

improvement of local financial inclusion services” and the emphasis throughout the

programme has been on innovative interventions, meaningful work with long term

impact, improvement, development and dissemination of learning and best practice.

The 13 projects that RBS Innovate funded were very diverse – some helped individuals

gain access to financial products, some produced guides and resources for trainers,

others supported housing association tenants with specific needs – but all focused on

helping people experiencing financial exclusion.

As the programme progressed RBS decided that, in order for financial inclusion

interventions to provide maximum learning for the sector, it was important to focus on

lessons learned and long-term impact. The focus of RBS Innovate therefore shifted in the

period 2009 – 2011 from funding interventions directly to funding the evaluation of

financial inclusion projects with the emphasis on three aims:

1. Help projects set up their own internal evaluation structures; 2. Share the resulting evaluation frameworks with the rest of the financial inclusion sector and apply to other organisations where appropriate;

3. Showcase the work of the projects and offer models of replication.

Good projects are often under-evaluated so exploring how they can be scaled up and

how this can work across different geographies and with different clients groups is key to

promoting success in financial inclusion. RBS Group and Toynbee Hall therefore

restructured the role of RBS Innovate to work with four organisations based in different

areas of the UK over the course of one year. The four projects were selected using an

open tender process. In December 2009 RBS Innovate announced that evaluation

support was available for any organisation working in financial inclusion or financial

capability. This announcement was disseminated via Transact (the National Forum for

Financial Inclusion), the DWP Champions Initiative and the Citizens Advice National

Financial Capability Forums. The application form was posted on the Transact web-site

and the deadline was set for January 2010. In all, 33 applications were received and 4

projects were selected. Using an open tender process guaranteed that every

organisation that applied was interested in setting up their own internal evaluation

frameworks, having their organisation evaluated and sharing learning with the sector.

Choosing only 4 organisations to work with was difficult as there were many interesting,

innovative organisations interested in RBS Innovate and in the end the decision was

based not only on the projects themselves but also on the combination of the four

projects; it was important to ensure a good mix of size, remit, population served and

geographical location.

9

Part 2 – Evaluating impact

Evaluation and financial inclusion

A lot of organisations are required to evaluate their projects; often for the purpose of

demonstrating to funders that agreed objectives have been met. These objectives are

often demonstrated via ‘outputs’ – the number of clients seen, the number of workshops

held, the number of hits to a web-site, or the number of calls received. This is useful

information to capture but, in order to demonstrate the true impact that an

organisation is having on their client group, community or audience, an evaluation

needs to stretch beyond outputs.

In his original work in 1959, Professor Donald Kirkpatrick made recommendations for

evaluation practice that have laid the basis for a lot of thinking in the subject ever since.

He argues that programme evaluation should concentrate on four levels:

The four levels of programme evaluation:

Level 1: Reaction – evaluating the reaction of participants to the intervention

Level 2: Learning – measuring the knowledge, skills and attitudes gained from the

intervention

Level 3: Behaviour – measuring the changes in behaviour that resulted from the

intervention

Level 4: Results – relating the results of the intervention to wider organisational objectives

Alongside outputs, many evaluations conducted in the financial inclusion and

capability sector only capture the first level of evaluation. This is with good reason:

reaction is the easiest to capture; it helps organisations to improve their programmes;

and it can be easily analysed. Capturing reaction means asking questions such as

“what did you like about the training”, “what did you not like about the training” and

“what suggestions do you have for the training going forward” and is often recorded in

an evaluation form after a course or an intervention has taken place. Just like outputs,

reaction is useful information to capture, but it does not provide the whole picture. In

order to capture real changes and attribute them to an intervention, it is necessary to

go beyond this first level, to look at changes in behaviour and learning and, where

possible, map these changes back to the activities of the organisation. The Charities

Evaluation Services asserts that outcomes are the changes that occur as a result of

activities provided by an organisation. Measurement of these outcomes is what

demonstrates how effective the organisation is, rather than its size, efficiency or

productivityi. It is these changes that occur (both in the short term and the long term)

that really demonstrate the effect that an intervention has had on participants so

capturing outcomes as well as outputs is key to producing a complete evaluation.

10

Confusion about how to capture information such as outcomes is compounded by the

fact that there is no single accepted definition of evaluation, let alone one method.

Some examples of definitions that are widely used include:

“Evaluation is about using monitoring and other information you collect to make

judgments about your project. It is also about using the information to make changes

and improvements” (Charities Evaluation Service)

“Evaluation involves the systematic collection of data about the characteristics of a

programme, product, policy or service. As part of this process, evaluation will often

explore what needs to be changed, the procedures that are most likely to bring about

this change, and whether there is evidence that change has occurred” (Warr et al.,

1970)

“Evaluation is the systematic assessment of the worth or merit of an object” (Joint

Committee on Standards for Educational Evaluation, 1994).

This lack of one accepted definition is in part due to the fact that evaluation is a

relatively new field; real interest in the area can only be dated as far back as the 1970s.

The emphasis in early evaluations was on accuracy and specific measuring techniques

but in recent years this has shiftedii. With the rise of action learning and self-

development programmes, learning is now seen as arising within and through a situation

rather than just through formal evaluations and issues of subjectivity and ethics are of

prime concern in today’s evaluations.

While evaluation is a new field, financial inclusion is an even newer one, so there are not

yet many established examples of evaluation best practice in the sector. Compounding

the problem further is the issue that many outcomes in financial inclusion are ‘soft’

outcomes which can be difficult to capture and, again, sometimes require subjective

judgments to be made. Examples of soft outcomes include changes in knowledge,

confidence, motivation and behaviour; all of which are extremely important and

indicate that a real change has taken place but also challenging to capture.

In summary, there are a number of key challenges that the sector faces when it comes

to evaluating work in financial inclusion:

• There is no single accepted definition or method of evaluation; • Financial inclusion is a young sector so does not have one accepted method of evaluation;

• Success in financial capability often means positive results in soft outcomes which can be difficult to measure;

In addition to the above points, subjectivity is often at the heart of why many

organisations struggle to evaluate their services beyond the reaction level. What

actually is the relationship between inputs and outcomes? Can a change in learning or

behaviour really be attributed to an intervention? What does ‘success’ look like? These

are some of the questions that organisations struggle with when designing their own

evaluation strategies because they often involve making subjective judgments. This

11

brings us to a key question: who is ultimately responsible for making these judgments?

Who has a right to? With few exceptions it is the organisations themselves that know

their client group or target audience best, as well as their organisational objectives. They

also know the resource that has been invested in an intervention and the resulting

benefit to their target group so it is often the organisations themselves that are best

placed to decide the terms of an evaluation. It is for this reason that the Evaluation and

Policy Manager worked closely and in partnership with each of the organisations

involved.

Organisations that differed in geographical location, target group and remit were

deliberately chosen so that the subsequent evaluation frameworks that were produced

would be relevant to as wide an audience as possible. There was no guarantee of how

this would work in practice but the findings show that there are identifiable themes

across all four organisations; each project was evaluated using the same process and

according to one evaluation approach. While the four organisations evaluated through

RBS Innovate all deliver services directly to clients, the key point is that the processes

employed can also apply to other types of organisations working in financial inclusion. It

is the process that is the key to capturing the necessary data.

The evaluation process

In order to establish and administer their internal evaluation frameworks, each

organisation that participated in RBS Innovate went through the same process. This

entailed answering the same questions:

What are the key aims of this service? Establishing the aims of a service or intervention is

the first step in designing an evaluation framework as it enables the organisation to think

about their end goals. In the case of organisations providing services directly to clients,

this stage would entail thinking about where they want their service users to be by the

end of the intervention or process. Do you want them to be able to move out of

supported housing into their own accommodation? To be more confident in managing

their money? Establishing what ‘success’ looks like for the service helps to identify the

components of that success which can then be turned into the indicators that will

measure that success.

What is the right balance between ideal evaluation goals and actual capacity to

deliver? In order to embed evaluation activities successfully within an organisation, it is

necessary to be realistic about what is achievable as regards funding, staff, and

available time. Creating a robust evaluation framework would guarantee a wealth of

rich information, but if it is not going to be realistic to gather that information it will not

be achievable. Therefore there needs to be an optimal balance between ideal

evaluation goals and actual capacity to deliver. One way to do this is to create an

ideal evaluation framework and then, according to available time and resource,

establish what will actually be realistic to deliver. This is often achieved through trial and

error and is an on-going, reflective process.

What information is already being captured? A lot of organisations already capture

information as part of their routine data collection. Organisations working directly with

12

clients, for example, will often conduct needs assessments or capture similar information

when a client first comes to their service. A number of organisations collect monitoring

information to help them improve their services going forward. Funder reports are also

another reason why monitoring and evaluation data is collected; funders will often want

to know the outputs and outcomes of a service that they have provided funding for

requiring organisations to capture key information. Using this existing data as a baseline

from which to start evaluation activity means that no work is replicated and all available

information is utilised. It can also help to shape an evaluation framework as certain

data collection processes can restrict what other activity takes place.

What type of evaluation are you interested in conducting? Formative evaluations ask

“how can the programme be improved?’ and are undertaken in order to provide

feedback with the overall goal of progress. They are relatively informal with the

evaluator working alongside the practitioner in order to identify the strengths and

weaknesses of a programme or interventioniii. Formative evaluations usually take place

during the lifetime of a project as opposed to at the end of it and encourage an on-

going, reflective approach and conversations between the organisation and the

evaluator. Summative evaluations on the other hand ask “what is the overall merit or

worth of the programme? Should it be modified? Should it be continued?” Overall, such

evaluations aim to determine how effective an overall programme or project is and

whether or not it should continue. Summative evaluations are usually more formal in

character with the evaluator operating in a more independent role and, in contrast to

formative evaluations, take place at the end of a project. It is important to determine

what type of evaluation to undertake as it impacts on what kind of information needs to

be collected and when.

What methods of data collection will be most appropriate? Sometimes quantitative

data such as numbers and figures will provide the most important information while

other times more descriptive, qualitative information will be most useful; sometimes it will

be both. Whilst an external evaluator can provide information about the different kinds

of data and their respective benefits, it is up to the organisation to decide which

method will give the richest data to suit their needs and help them to inform their

decisions going forward.

Even though the four projects that took part in the programme were all delivering

different services in different geographical locations, they all went through the above

process to design their internal evaluation frameworks. All four organisations deliver

services directly to clients but this process can work for any organisation, regardless of

remit. This is because, in the majority of cases, it is best to start by establishing end goals

and working out what measures will demonstrate whether or not those end goals have

been reached.

The evaluation approach

The evaluation frameworks for all four organisations are most closely aligned with the

decision- and accountability- study framework which is a type of improvement- and

accountability- oriented approachiv. As the organisations themselves are involved in

their evaluation, the programme does not strictly adhere to any one evaluation

13

approach, but this is the one it most closely resembles in ethos and in practice. Decision

and accountability oriented studies stress the need to assess fully a programme’s value.

They look for all relevant outcomes, not just those keyed to programme objectives and

they generally use multiple qualitative and quantitative assessment methods in order to

provide cross-checks on findings. Following on from the discussion of formative vs.

summative above, this approach emphasizes that programme evaluation should be

used proactively to help improve a programme as well as retroactively to judge its

value. Under this approach an evaluation’s most important purpose is not to prove but

to improve.

A key ethos of the approach is that an evaluation can best effect change in a target

group’s behaviour by involving members in planning, monitoring, and judging the

enterprise. This active participation by stakeholders not only gives them a voice in the

determinations that will affect them but also brings them on board with the enterprise

from the start.

There are some key advantages associated with the decision and accountability

approach:

• It encourages programme personnel to use evaluation continuously and systematically to plan and implement programmes that meet beneficiaries’

targeted needs;

• It aids decision making at all programme levels and stresses improvement; • It balances the use of qualitative and quantitative methods; • It can provide the framework for both internal and external evaluations.

However, there are also some limitations:

• It may overemphasise formative evaluation and give too little time and resource to summative evaluation;

• The collaboration between evaluator and stakeholders may impede the evaluation or bias its results;

• The internal nature of evaluation may mean some of the benefits of an independent, detached perspective are lost.

The fact that all four organisations were evaluated according to the same approach

may seem surprising given that they deliver services to different target groups in

different geographical locations. When decided which approach to go for, however,

we not only had to take into account the desires of the organisations themselves but

also the goals of RBS Innovate. We were clear right from the beginning that the

programme was not about conducting formal, external evaluations, but rather was

concerned with producing evaluation frameworks by working in partnership with the

organisations involved. Taking the aims of each organisation and combining them with

the goals of RBS Innovate the advantages of the improvement- and accountability-

oriented approach outweighed the limitations and it was considered the most

appropriate for this programme of work.

14

Part 3 – The projects

In total four organisations participated in RBS Innovate over the course of one year and

each worked with the Evaluation and Policy Manager based out of Toynbee Hall in East

London. The Evaluation and Policy Manager worked in partnership with each

organisation to produce internal evaluation frameworks, establish what data would be

collected, how it would be collected, analysed the resulting data to produce key

findings and wrote the final reports. As discussed above, this partnership approach

distinguishes RBS Innovate from an external evaluation methodology which proved to

be a strength of the model. Agreeing the terms of the evaluation with the organisations

themselves and showing them how they can be conducted means that the frameworks

are now firmly embedded in the practice of each organisation.

The background to each project, the models for practice, the individual evaluation

frameworks and the findings from each evaluation are discussed in the next section.

15

Moneywise at Centrepoint (London)

Centrepoint is a homelessness charity for young people that was founded in 1969 by

Ken Leech, vicar of St Anne’s in Soho. Concerned about the number of young people

sleeping rough in London’s West End, he and a group of volunteers opened up the

basement of St Anne’s as a temporary night shelter. Over 40 years later, Centrepoint

now has more than 30 services across London and in the North East of England and

through these services they work with over 1,300 young people a yearv. As well as

providing them with a place to live, Centrepoint helps young people with their personal,

social, learning and work needs, helping them to develop a support and development

plan which enables them to identify their needs and achieve their goals. Through this

holistic approach, Centrepoint aims to tackle the underlying issues behind homelessness

and support young people on their path; be it preparing for a future career, taking part

in voluntary work or learning practical skills through the Lifewise programme.

As part of the Lifewise programme, Moneywise supports young people with their money

management. This involves helping them to deal with existing debt and prevent future

money concerns while promoting their overall financial capability. Moneywise has taken

a number of different forms since it first started and has experienced varying levels of

support. When Moneywise was first established in 2005 Centrepoint employed a

Moneywise Support Worker who provided one to one advice on budgeting, but as the

programme progressed Centrepoint decided it needed to expand the programme

further. In 2010 the role of the Moneywise Support worker changed to that of a

coordination role, bringing together activities to complement a broader agenda of

money management and enlisting the support of staff to deliver workshops. The

workshops that are delivered through Moneywise are designed to give homeless young

people the skills they need to move into independent living and are accredited through

AQA’s Unit Award Scheme (UAS)vi. Current Moneywise workshops include ‘Living on a

budget’, ‘Dealing with debt’, ‘Introduction to bill payment’ and ‘Planning and shopping

for a budget meal’. Where further specialist advice is needed beyond the in-house

Advice Surgery, young people are referred to external agencies. The project has grown

and strengthened over the years, receiving renewed vigour at each stage of its

development. The aim going forward is to strengthen the programme in order to further

mainstream Moneywise into the wider work of Centrepoint.

Despite the current holistic package of support, there is concern that the positive

outcomes achieved through the Moneywise programme are not being sustained. Of

the 1,349 young people who left Centrepoint in 2008, 317 (25%) of them needed further

help to reduce their personal debt, rent arrears and service charge arrears. There is also

concern that too much effort is being invested in responding to money problems that

Moneywise is a programme within Centrepoint. Its purpose is to support young

people who are homeless or at risk of homelessness in dealing with existing

debt, preventing future money problems and promoting overall financial

capability. It is delivered through a combination of group workshops, one to

ones and referrals to specialist advice services.

16

have already arisen rather than tackling them early on or even anticipating them

before they start. As Rupon Miah, Moneywise Project Coordinator, said, “We all know

that prevention is better than cure but this is not being reflected in our approach. Too

often we’re fire fighting.” The focus for the past few years at Moneywise has been on

outputs such as the number of workshops delivered or the number of staff trained. With

the appointment of the new Moneywise Coordinator and a renewed desire to further

mainstream Moneywise into the wider work of Centrepoint, they are now looking to

develop a more evidence-based approach to their work. By identifying existing issues

within the current Moneywise model and considering the root causes behind these

issues it is hoped that this evaluation will help Moneywise to grow and change to

achieve its greatest potential and the best outcomes for the young people it supports.

Moneywise has four key aims:

1. Increase the financial capability of young people: While there is no one established definition of financial capability, broadly it refers to the knowledge

and skills necessary to understand personal financial circumstances along with

the motivation and confidence to take actionvii. As effective financial

management is important for a number of areas in life, the first key aim of the

Moneywise programme is to increase the financial capability of young people at

Centrepoint.

2. Effectively train Moneywise staff so that they are able to deliver workshops confidently and support young people in money matters: The support that young

people receive through Moneywise is vital to their on-going development in

financial capability. But if staff are not properly supported or do not have the

necessary knowledge to deliver workshops and one-to-ones then they will not be

able to provide effective support to the young people at Centrepoint. Therefore

providing an effective training and support service for staff at Centrepoint is the

second key aim of Moneywise.

3. Reduce rent arrears/service charge arrears: Rent and service charge arrears are not only a significant cost to Centrepoint; they also indicate that a young person

is not managing their finances as well as could be hoped. This is concerning as,

when young people move on from Centrepoint and are living independently,

rent arrears can be a serious problem; as a priority debt, they can even lead to

evictionviii. Reducing rent and service charge arrears is therefore a key objective

of Moneywise as it will not only benefit Centrepoint as an organisation but help to

set young people up for a successful, independent future.

4. Embed financial capability into the wider work of Centrepoint: Effective money management is an important life skill and, as such, it is important that the

financial capability work taking place through Moneywise is properly embedded

in the wider work of Centrepoint. While Moneywise has been a programme at

Centrepoint since 2007 it has taken different forms and undergone changes so

assessing how to move forward is a key aim for Moneywise.

17

Moneywise has grown and strengthened over the years, receiving renewed vigour at

each stage of its development. This evaluation offered an opportunity to assess

Moneywise in its current form and consider what other steps need to be taken in order

for the above aims to be fully realised.

There are five steps to the Moneywise model:

1. Staff training: Staff at Centrepoint receive training on how to deliver workshops within the Lifewise programme. In the past they have also received external

training but as the funding for this training has come to an end they are currently

reviewing their staff training practices.

2. Assessment: All young people who come to Centrepoint receive a needs and risk assessment. Amongst a number of other areas the assessment captures

information relating to a young person’s ‘economic wellbeing’ including

information on income (employment and benefits), budgeting, and existing

debts. The needs and risk assessment flags up areas where the young person

needs further support and these are built into their support and development

plan under ‘needs’ and ‘risks’. The Support & Development worker assigned to a

young person then works with them to plan the long term desired outcomes

around these needs and risks as well as what actions need to be taken to

achieve key outcomes.

3. Referrals: Most young people at Centrepoint are referred to Moneywise workshops through their Support & Development Worker as a result of their needs

and risk assessment and their support and development plan. If they require

support around a particular issue they are referred to the internal Advice Surgery

at Centrepoint and if they need specialist advice they are referred to an external

agency.

4. Financial capability training: Centrepoint currently runs four workshops through Moneywise – ‘Living on a budget’, ‘Dealing with debt’, ‘Introduction to bill

payment’ and ‘Planning and shopping for a budget meal’. The content for each

of these workshops is developed by first establishing the aim of the workshop and

the intended learning outcomes. Workshops last approximately 2 hours and are

delivered through interactive sessions to groups of young people.

5. Follow up: Moneywise does not currently carry out any follow up activity with young people who have taken part in a workshop. Going forward however they

are planning to introduce two new areas of activity: a review of progress with a

sample of young people who participated in the Moneywise workshops and the

addition of information on anticipated ‘transitions’ in the needs and risk

assessment.

18

Moneywise at Centrepoint

Assessment

• Support and Development worker carries out a needs and risk assessment with the young person • From the needs and risk assessment they develop a support and development plan and identify future actions

• Going forward Centrepoint are considering the addition of a Financial Health Check during the assessment stage

Referrals

• Young people requiring basic skills are referred to a Moneywise workshop • Those that need one to one advice are referred to the Advice Surgery • Young people that need specialist advice are referred to external agencies

Financial capability training

• Delivered in the form of Moneywise workshops that are developed according to key aims and learning outcomes and delivered to young people in an interactive group setting

Follow up

• Identification of future ‘transitions’ that may impact on the plans of the young people during the initial needs and risk assessment

• Review progress with a sample of young people at 3 months and at 6 months

Assessment

Moneywise

Workshop

Advice Surgery Follow up Young Person

External agency Staff Training

19

Moneywise Evaluation Framework When Tool Description

Start of Moneywise workshop Pre-workshop assessment 6 questions capturing confidence,

motivation and current state of rent

arrears and debt in the form of a

questionnaire

End of Moneywise workshop Post-workshop assessment

6 questions capturing confidence,

motivation and attitudes towards

current state of rent arrears and debts

in the form of a questionnaire

Thee months after attending a

Moneywise workshop

Post-workshop follow up (3 months) 5 questions capturing progress and

testing of knowledge

Six months after attending a

Moneywise workshop

Post-workshop follow up (6 months) 5 questions capturing progress and

testing of knowledge

February 2011 Case studies Case studies with young people who

have been through Moneywise

Three weeks after attending the

Moneywise Advice Surgery

In-depth interview – advice surgery In-depth interviews with young people

who attended the Moneywise Advice

Surgery 3 weeks after the session

February 2011 Focus groups Focus group with young people to

discuss the current Moneywise model

and get suggestions for improvement

going forward

Focus group with staff at Centrepoint

to discuss the current Moneywise

model and get suggestions for

improvement going forward

20

Summary of findings

Evaluation of the project

The findings from the evaluation of Moneywise and evaluation of the Moneywise model

are presented under the headings from the original aims and objectives:

• Increase financial capability • Support Moneywise staff • Reduce rent arrears and service charge arrears • Strengthen the Moneywise programme

As Moneywise is currently assessing its practices, the evaluation focused on these four

aims in the context of how well Moneywise is currently able to meet them. The evaluation

period took place over four months (October 2010 – February 2011) and reached 51

young people through the workshops.

Aim: Increase financial capability

Questionnaires were administered both before and after the Moneywise workshops to

capture changes in motivation, confidence and attitudes. The most notable increases in

financial capability through the Moneywise workshops occurred in motivation and

confidence:

• Participants reported a 12% increase in confidence in planning and budgeting • Participants reported a 10% increase in motivation to take control of finances • Participants reported an 8% increase in confidence in managing money

Going forward, the evaluation revealed that there is definitely a lot more information that

can be captured on the impact of the workshops. Actual changes in knowledge, for

example, is an area Centrepoint might want to consider including in their evaluation

framework. As the workshops provide a lot of practical information, it would be useful to

gauge participant knowledge of this information before and after the workshops by

asking specific questions, for example: “what is an example of a priority debt?” The

current evaluation framework captures a lot of subjective data so measuring increases in

knowledge objectively could greatly enhance the evaluation framework.

Aim: Support and training of staff

Aim: Reduction in rent/service charge arrears

These two areas were identified as separate aims in the original Moneywise framework.

However the evaluation revealed that they are actually linked and so the two are

discussed together.

The current relationship between the Support & Development Workers and young people

often leads to a ‘drama triangle’

Staff in the focus group were very aware of the disconnect between the service they

were providing and what young people were expecting and were also fully aware of

what they feel is the root cause. Through Moneywise, Support & Development workers

21

are not only required to support young people with debts to external creditors and their

overall financial capability, they are also responsible for chasing the young people for

any rent arrears. Staff said that because of this they feel that their advocacy role

becomes problematic or ‘tainted’. In order to support a young person to manage their

finances responsibly, it is important for staff to have a relationship of trust with that young

person. When they have to chase that young person for rent arrears that relationship

breaks down.

Staff in the focus group said they felt that this conflict of interest ultimately led to a break

down in communication which created a ‘drama triangle’. In an ideal situation, the

Support and Development worker would be the ‘rescuer’ or ‘hero’ to the young person

who is struggling with debt as the ‘victim’. The source of that debt would be considered

the ‘perpetrator’. As communication breaks down as a result of conflicting priorities the

Support and Development worker goes from being the rescuer to being the perpetrator.

This moves the young person into the role of victim looking for a hero or rescuer. In

addition, changes in legislation and the sometimes specialist nature of the advice given

to young people means that staff do not always feel equipped to provide the young

people at Centrepoint with all of the support they need. By addressing the current

organisational structures and increasing the support that staff currently receive, it is

hoped that Centrepoint will be in a better position to support the financial capability of

young people.

The cost of ‘transitions’ (i.e. moving into work) perpetuates the cycle of debt and rent

arrears

When a young person gets a job or starts a course or training they often find that they

are out of pocket as a result of the up-front costs needed or the break between stopping

welfare benefits and receiving wages. This can not only act as a disincentive to moving

on, but it can also start a cycle of debt that young people can struggle to break. To

combat this, the idea of a ‘transitions fund’ or ‘savings bond’ was discussed in the focus

groups with young people and with staff. This could take a number of forms but the idea

is that young people save whatever amount they can a week and this amount is then

match funded so that young people have a pot of savings whenever they get into

difficulty. This could not only provide some funds for them in times of crisis but also

encourage the formation of a savings habit.

Aim: Embed Moneywise into the wider work of Centrepoint

There is currently low attendance at Moneywise workshops

Staff at Centrepoint are concerned that more young people are not accessing the

Moneywise workshops. The consensus was that it is usually the young people who are

already doing ‘okay’ who attend the workshop and that those most at risk are falling

through the cracks. The young people on the other hand say they feel that the

workshops are not advertised enough so the question going forward is how to ensure all

young people are fully aware of the workshops and are encouraged to access them.

22

Workshops are currently delivered at one level

There are currently four workshops delivered through Moneywise – ‘Dealing with debt’,

‘Living on a budget’, ‘Introduction to bill payment’ and ‘Planning and shopping for a

budget meal’. Both staff and young people said that there are a number of other topics

that would help them to improve their financial situation and that Centrepoint should

potentially look at adding additional topics to the Moneywise programme. As well as

expanding on the topics covered through Moneywise, a popular suggestion was to run

workshops on different levels. Staff pointed out that they have referred a number of

young people to the same workshop as a ‘refresher’ and it would be useful if there was a

workshop structure in place that allowed for increases in knowledge along different

levels.

Moneywise has not been run consistently at Centrepoint

As discussed previously, Moneywise has gone through a number of transitions and this

means that it has not always been run consistently. It was evident from the focus groups

that the legacy of this is that staff and young people are not entirely sure what services

are running through Moneywise. Because of its history, Moneywise has not been

embedded in Centrepoint as firmly as some of the other programmes and this needs to

change as the programme develops going forwards.

A key way to embed Moneywise in Centrepoint is to make it a priority right from the start.

As part of this, Centrepoint are looking into the possibility of developing a Financial

Health Check that would form part of the needs and risk assessment process that a

young person goes through when they first come to Centrepoint. This would hopefully

result in money management becoming a key priority for young people from the start

and make sure that Moneywise is firmly linked up to other services at Centrepoint.

Evaluation of the evaluation

Evaluations can be difficult and time consuming to administer

Paper based evaluation forms need to be administered during the course of the

workshop and this takes time away from actual service delivery. In addition some young

people have difficulty with reading and writing and so require extra support to fill in forms,

making them less than ideal as an evaluation practice. To combat this problem,

Centrepoint is in the process of introducing an electronic evaluation tool through a

company called ResponseWare. Similar to a Nintendo Wii set, operations costs are

reportedly low and real time information is available before, during and after the

workshop. The tool is currently being trialed and Centrepoint is expecting feedback as to

whether or not they can trial them for the Moneywise workshops. This is a very innovative

approach to evaluation and will potentially have a big impact on the evaluation

practices of the organisation going forward.

23

Money Active at Plymouth Citizens Advice Bureau (Plymouth)

As well as identifying what specialist advice debt clients need, the Debt Gateway

at Plymouth Citizens Advice Bureau (CAB) identifies which clients could benefit

from additional financial capability training. Those that could benefit are referred

to a Money Active session in the form of either a workshop or a one to one. The

aim of Money Active is to increase service users’ knowledge of budgeting,

borrowing, savings, debt and where to go for additional help and support. It is

hoped that providing this additional support will increase the financial capability

of service users and therefore reduce future debt problems.

Plymouth Citizens Advice Bureau (CAB) is an independent registered charity that offers

free, impartial, independent and confidential advice to their service users. They are part

of the national network of Citizens Advice Bureaux that can be found across the UK – see

www.citizensadvice.org.uk for your local bureau. Plymouth CAB gives information and

advice to service users on a number of different subjects. Advisors are trained to deal

with queries relating to housing, consumer issues, debt, welfare benefits, employment,

family and personal issues, nationality and immigration. They help people in the

community in a number of different ways; specific issues can be dealt with through face-

to-face advice, on the telephone and by e-mail and additional information is also

available on-line at www.adviceguide.org.uk. Covering such a wide range of issues

means that Plymouth CAB is able to take a rounded view of the problems people face.

This is important as they find that one significant change in circumstance can trigger a

number of problems: for example, losing a job can lead to the loss of a home,

breakdown of a relationship and problems with debt; or falling ill can result in

complicated benefit applications and anxiety about employment and bills. As a result,

Plymouth CAB feels it is important to provide a holistic service to service users so that all of

their needs are met wherever possible. Helping service users with all of their issues can

also help tackle the root causes, thus preventing problems occurring in the future.

Of the 20,000 issues dealt with by Plymouth CAB in 2008/2009, at least 80% were related

to financial inclusionix and they handled over £50 million worth of debt on behalf of their

service users. Over the course of the past 5 years they have built up their Debt Rights &

Remedies Unit which currently employs 12 staff. In order to assess which service users are

in greatest need and ensure that they are seen as quickly as possible, service users who

attend the drop in are given a short diagnostic interview. This is referred to as their Debt

Gateway and it enables the bureau to gather brief details of the client’s enquiry, assess

the complexity and urgency of the issue and determine the best way to deal with it.

Sometimes all a service user needs is some further information on their query, such as a

leaflet or signposting to a web-site, whereas other times they may need a referral to a

specific service within the bureau or even out to an external organisation.

A key role of the Debt Gateway is also to identify who could benefit from further financial

education and capability training; those who could benefit are referred to a Money

Active session. Money Active is a national scheme run through Citizens Advice across the

UK and is funded by Nationwide Building Society. The aim of Money Active is to train and

support staff and volunteers in Citizens Advice Bureaux across the UK to carry out

24

financial education work in their area. It is a three year project, running from 2009 – 2012,

and it supports the recruitment and training of 1,300 volunteers to deliver financial

capability sessions. At Plymouth CAB these sessions take the form of either a two hour

workshop or a one to one session, both of which cover basic skills around budgeting,

banking, credit and saving. Clients are given the option of attending a workshop or a

one to one session in recognition of the fact that they may not be comfortable talking

about money in a group session.

The Debt Gateway/Money Active model at Plymouth CAB has three key aims:

1. Increase financial capability: While there is no one established definition of financial capability, Plymouth CAB focuses on increasing the knowledge, skills and

confidence of service users to manage their money more effectively and take

responsibility for personal financial decisions. Effective financial management is

important for a number of areas in life, so the first key aim of Money Active is to

increase the financial capability of service users at Plymouth CAB.

2. Decrease repeat appointments within the bureau: Debt service users often come back multiple times, sometimes even for the same issue. It is hoped that, by

adding in financial capability training, it may be possible to decrease the amount

of clients who come back in for advice, especially for the same issue.

3. Explore implementation: Compare experiences of one to ones and group sessions to identify which are most successful in tackling debt service users’ money

problems. Explore different methods of working to uncover the best way for

Plymouth CAB to embed financial capability within debt advice without taking

time or resource away from debt service delivery.

There are three steps to the Debt Gateway/Money Active model:

1. Assessment and referral: All debt clients that come to Plymouth CAB are assessed by the Debt Gateway. The advisor at the Debt Gateway assists clients with queries

that can be handled then and there and refers clients that need specialist advice

to the appropriate service. Clients that they feel could benefit from financial

capability training are referred to a Money Active workshop or one to one.

2. Service delivery: Workshops are delivered in a group setting and cover four key subject areas: budgeting, borrowing and saving, dealing with debts and where to

go for help. A4e sessions are delivered to clients facing barriers getting into or

returning to work and the bureau also holds training sessions for advice staff who

work in Plymouth but not in a CAB.

3. Follow up: Plymouth CAB does not usually follow up with clients who attend a Money Active session but they did so as part of this evaluation. The results were

encouraging and the follow up not only provided clients with the opportunity to

reflect on their progress since attending a Money Active session but provided

useful feedback for the staff and volunteers who run the sessions.

25

The Debt Gateway and Money Active at Plymouth Citizens Advice Bureau

Assessment and referral

• All debt clients come through the Debt Gateway • Some clients simply need help filling in a form, information on their query or signposting to a different agency while others are referred for specialist advice

• Approximately 10% of clients are assessed as needing money management training and are referred to a Money Active session

Delivery

• The majority of debt clients who are referred to a Money Active session are referred to a workshop • Clients who are not comfortable with the workshop setting or have specific needs are referred to a Money Active one to one

• Staff are trained to deliver Money Active sessions through group training sessions

Follow up

Going forward Plymouth CAB are planning to continue following up with a sample of clients after they have

attended the workshop to discuss their progress and what knowledge they have retained

Debt Gateway

Money Active

Workshop

Money Active

One to one

Further specialist advice

External agency

Staff training

Follow up (sample)

26

Plymouth Citizens Advice Bureau Evaluation Framework When Tools/Method Details

May 1 2010 – August 1 2010 Existing internal system

Feedback questionnaires

For anyone referred to a Money

Management session (group or one to one)

record:

• Reason for seeking debt advice • Reason for referral • Type of session referred to (group or one to one)

• Did they attend the session?

Existing internal system Monitor those who attended the money

management session to see

• If they come back to the CAB for further money-related advice

• If so, what kind of advice

August 2 2010 – November 1

2010

Phone questionnaire Follow up with a sample of those who

attended a Money Active workshop to find

out:

• Progress since attending the session • If the session influenced their behaviour or prompted them to take

any action

November 2 2010 – December

1 2010

Face to face interviews

Existing internal system

Interview with staff about how they found the

process

• Jane (overall process) • Bill (Debt Gateway) • Sharon (Money Active workshops

Compare captured data to:

• Debt clients who aren’t referred to a money management session

• Data from before the formal referral process was established

27

Summary of findings

Evaluation of the project

The majority of debt clients who were referred to a Money Active session from the Debt

Gateway were referred to a workshop (90%) as opposed to a one to one (10%). The

evaluation revealed that the workshop structure was a key strength of the sessions.

The most common reason a client was referred to a Money Active session (36%) was so

that they could ‘learn to prevent and deal with their debts’ followed closely by ‘learn to

budget their money and complete a financial statement’ (30%).

Level 1: Reaction to Money Active

Reaction captures information on how participants found an intervention; what they

liked, what they did not like and their suggestions for the intervention going forward.

Participants were asked to list what they most enjoyed about the Money Active

workshop and responses were fairly evenly split, with the biggest proportion of

respondents reporting that they enjoyed the session on budgeting (26%).

Some participants were at first unsure of taking part in a group workshop but it actually

ended up being a key strength of the programme; approximately 16% of respondents

listed the group dynamic as the aspect of the training that they found most useful,

without any prompting:

“I liked that everyone was in the same position as me and I didn't feel alone”

“The feedback from other people on the course”

“Looking at all areas of debt and discussing debt openly”

When asked what they would change about the session 99% of respondents said

‘nothing’.

Levels 2 & 3: Learning and behaviour

Three months after they had participated in a Money Active session participants were

contacted by staff and volunteers at Plymouth CAB to discuss what they had learned

and what changes they had made to how they manage their finances. 100% of

respondents said that they felt their attitude towards money had changed since

attending the MoneyActive workshop with 83% saying their attitude towards budgeting

had changed.

Respondents were also asked what positive steps they had taken since attending the

MoneyActive workshop. 57% of respondents said that they had taken steps to settle their

existing debts since the workshop and 23% said that they had set up and were following

a budget.

28

100% of respondents were able to give an example of an affordable source of credit and

79% of respondents were able to give an example of a priority bill after attending a

workshop.

Evaluation of the model

Plymouth CAB found that the main issue in implementation of the model related to

attendance at Money Active workshops. Specifically, while a number of service users

were referred to a workshop, not everyone attended. Staff at Plymouth CAB discussed

the reasons for the lack of attendance and work out some possible solutions:

• Who was running the Gateway service • The information provided to the service user before the session • The time of year, day of the week and time of day • Debt problems resolved in the debt gateway • Those who said they would come but did not turn up • Some information was clearly not needed

There is one final point that is perhaps the most significant in encouraging service users to

attend a Money Active workshop and that is the realisation that attendance at a

workshop means that any follow up debt case work is significantly shortened. If a service

user attends a workshop, any subsequent debt case work takes one third of the time it

takes to see other debt clients. This means that debt case workers are able to slot these

clients into their schedules more easily than other clients and this has had the unintended

result of incentivising attendance at the Money Active sessions. Service users who are

reluctant to attend but keen to get their debts sorted are informed that attending a

workshop will ensure they are seen by a debt advisor much faster.

Evaluation of the evaluation

The type of evaluation conducted through RBS Innovate differs from other types of

evaluation. Halfway between an internal and external evaluation, this method provided

a number of benefits as well as drawbacks:

• Having a balance between internal and external was a positive of the evaluation model. The evaluation framework was developed in partnership with Plymouth

CAB and all major decisions were made jointly which meant that everyone was on

board with the methodology. Balanced alongside this is the fact that the

evaluation findings were evaluated from an external perspective which brought

further insight to the analysis.

• Plymouth CAB had not previously followed up with Money Active clients before this evaluation and in fact there were unanticipated benefits to this evaluation

technique. It not only enabled service users to reflect on the workshop and think

about their progress, it was also a highly motivational exercise for staff. Low turnout

at workshops had left some of them questioning the worth of the service but after

talking to clients and hearing about how Money Active had improved their lives

they were able to realize the benefit much more.

29

• A number of the service users that come to the Money Active sessions have literacy issues as well as other needs. This means that Plymouth CAB will potentially

want to explore other ways of evaluating the workshops that are not reliant on

paper-based tools in the future.

30

Save and Insure at Prince Bishops Community Bank (Consett)

Save and Insure is a scheme operated in partnership with Prince Bishops

Community Bank, Derwentside Homes and RSA insurance group. Through the

scheme, tenants of Derwentside Homes sign up to become members of Prince

Bishops Community Bank. If they save £10 per month for six months they receive a

£20 credit to their account. They also receive free home contents insurance

through RSA (£40 value) for the duration of the six months. The aim of the scheme

is to encourage tenants to join the credit union, develop a sustainable savings

habit and take out home contents insurance. After saving with the credit union

for twelve weeks members are also able to access low-cost loans with a typical

APR of 12.68%.

2.1 Background to the project

The idea for Save and Insure came out of the existing partnership between Prince Bishops

Community Bank and Derwentside Homes. Having worked together before, they met to

discuss what issues tenants were facing and how their partnership could potentially help.

It was just after Christmas in 2009 and some tenants had unfortunately experienced

flooding in their homes while many were also experiencing debt problems from

Christmas. By combining support in the area of savings, loans and home contents

insurance, Save and Insure presented a dynamic solution and an opportunity to support

tenants in a number of ways.

Prince Bishops Community Bank is the result of the merger of East Derwentside Credit

Union in Stanley and West Derwentside Credit Union in Consett – both of which have had

a presence in County Durham for over ten years. Durham County Council covers around

862 square miles and has a strong heritage of agriculture, coal mining and steel making.

Coal reserves became exhausted in the 1980s and the steel works closed in 1980. Since

the decline of these industries there had been a steady decline in the area population

but this has since stabilised. As of 2009, 31.6% of the population in Durham were deemed

to be in the nationally deprived top 20%. The area also had a worklessness level of 11.1%

of the working age populationx.

The merger between East Derwentside Credit Union and West Derwentside Credit Union

took place in August 2009 and the resulting organisation was originally called Prince

Bishops Credit Union. This changed to Prince Bishops Community Bank in February 2011 to

reflect the community-based nature of the organisation. It was felt that, individually, both

East Derwentside Credit Union and West Derwentside Credit Union were too small to

generate the income required to grow the business and increase coverage. Due to their

size and non-charitable status the organisations were not able to attract grant support. It

was decided that “by merging together to form a new credit union to be called Prince

Bishops Credit Union, the new organisation would enjoy the economies of scale to have

the resources to attract new members and provide increased levels of services which

would not be possible for each to do as a standalone organisation”. The vision for this

31

new credit union was that “it will provide an ethical, democratic, good value service for

anyone seeking an alternative to the banks and building societies.xi”

Derwentside Homes is a Registered Social Landlord that was established in December

2006. It currently owns and manages around 6,700 properties across 38 locations that

were formerly owned by Derwentside District Council and it was created specifically for

the purpose of taking on this housing stock. As a charitable, not-for-profit organisation, all

of the income generated by Derwentside Homes goes back into the housing and

services provided to tenantsxii. Its vision is that of “affordable, attractive homes in strong,

safe communities where everyone is valuedxiii” and it is regulated by the Tenant Services

Authority.

Partnerships between credit unions and housing associations, whatever form they take,

are beneficial to both parties. They give credit unions the opportunity to increase their

membership and they provide housing associations with a way to support the personal

finances of their tenants. Save and Insure is also delivered in partnership with RSA. RSA

was formerly known as Royal & Sun Alliance and has worked with a number of housing

associations to deliver affordable home contents insurance to their tenants. Save and

Insure most closely resembles an insurance with rent scheme as administration is handled

by Derwentside Homes. As the Association of British Insurers highlighted in their 2009 Good

Practice Guide, it is good practice to “link incentives to other organisations (i.e. free

cover when opening a credit union account). For example, some landlords arrange

ongoing discounts in local retailers for tenants who abide by the terms of their tenancy

agreement”. Save and Insure incentivises tenants signing up to home contents insurance

by providing it for free for the first six months.

Partnerships between credit unions and housing associations and partnerships between

housing associations and home contents insurance providers are not new phenomena,

but this project is unique in that it combines all three.

The Save and Insure scheme has four key aims:

1. Increase membership of the credit union: As discussed above, both East Derwentside and West Derwentside Credit Unions struggled to increase their

membership. Membership drives take time and resource that was not available

while the two credit unions were separate. Now that the two credit unions have

joined forces to become Prince Bishops Community Bank, resources are better

pooled. Save and Insure provided another opportunity to attract new members

and the original aim was to sign 500 tenants of Derwentside Homes to the scheme.

2. Encourage savings: Many people, especially those living on low incomes, feel that they are not able to save. Even one or two pounds a week can feel like a struggle

when there is rent and bills to pay and other credit commitments taking every

spare penny. Having that pot of money, however, can mean extra security

against unexpected expenses or a way to buy something that previously seemed

unattainable, such as a laptop or a family holiday. Sometimes people just need an

extra incentive to save which is why Save and Insure offered £20 to those that

32

saved £10 in their account every month for six months. £10 per month works out to

roughly £2.50 a week.

3. Increase take up of home contents insurance: As mentioned previously, housing association tenants often perceive barriers when it comes to taking out home

contents insurance. Holding an insurance policy can save both tenants and

landlords money in the long run in the event of flood, fire or theft so a key aim of

Save and Insure is to increase take up of home contents insurance. In order to do

this, home contents insurance from RSA was offered for free for the first six months

of the scheme with the hope that at least 50% of tenants would continue with their

policy after the free period was up.

1 Increase access to affordable credit: The members of Prince Bishops Community Bank are eligible to apply for a loyalty loan through the credit union after saving

with them for twelve weeks. Doorstep lenders and even illegal money lenders are

active on housing estates and so providing tenants with the opportunity to access

lower cost credit is key to ensuring residents are able to avoid the high interest

rates, stress and fear which can arise from engaging with these types of lenders.

There are three steps to the Save and Insure Model:

1. Establishing partnerships: Partnerships between housing associations and credit unions can be positive for both parties; it increases membership and therefore

strengthens the credit union and it provides the housing association with a trusted

source of savings and loans for their tenants. It was the existing relationship

between Prince Bishops Community Bank and Derwentside Homes that provided

the platform for Save and Insure.

2. Engaging the target group: All 6,000 tenants of Derwentside Homes received a leaflet on Save and Insure and around 200 reply cards were received back from

tenants interested in joining the scheme. These were collated and given to Lesley

Richardson, Manager at Prince Bishops Community Bank and Lesley then visited

every tenant in person to sign them up to the scheme. Lesley stresses that the face

to face visit was key to engaging tenants.

3. Continued support: Members who consistently saved at least £10 for six months received a letter informing them that £20 had been credited to their account and

that they were now eligible for a loan from the credit union. As might be

expected, a few members had stopped saving every month so, in order to

encourage them to continue, Prince Bishops Community Bank sent them a letter

to remind them of the benefits. After the free period for the home contents

insurance ran out, Derwentside Homes took on the task of collecting the

payments. Tenants are given the option of paying by swipe card, standing order

or at one of the collection points – three methods designed to make payments as

easy as possible.

33

Save and Insure

Partnerships Established

• Existing partnership between Prince Bishops Credit Union and Derwentside Homes • Existing partnership between Derwentside Homes and Royal & Sun Alliance

Tenant Engagement

• Mailshot sent to all 6,000 tenants of Derwentside Homes inviting them to join Save and Insure by filling in attached form and sending it to Derwentside Homes

• Follow up letter sent to tenants who had expressed interest in joining • 200 forms returned by tenants interested in joining the scheme; these were collated and forwarded to Prince Bishops Credit Union

Incentives

• Tenants able to deposit money at pay points or directly to the credit union. Those that save at least £10 every month for six months receive £20 at the end of the six months

• Tenants who joined the scheme receive six months of free home contents insurance (worth £40, paid by Derwentside Homes)

• Tenants eligible to receive loans from the credit union after saving with them for 13 weeks (savings could be as low as £1 per week)

savings and loans housing association tenants

home contents insurance

+ +

34

Save and Insure Evaluation Framework When Tool Information needed

Profiles of tenants who are currently members of the credit union (DOB,

gender, benefits status, ethnicity)

April 2010 Spreadsheet

Number of tenants who've joined the credit union through the new

scheme so far

Profiles of tenants who joined the credit union as a result of the new

scheme (DOB, gender)

How many new members are still saving £10 per month?

How many new members are not saving £10 per month but are still with

the credit union?

June 2010 Spreadsheet

Have any new members cancelled their credit union account?

How many new members have accessed credit through the credit

union?

How many new members are still saving £10 per month?

How many new members are not saving £10 per month but are still with

the credit union?

July 2010 Spreadsheet

Have any new members cancelled their credit union account?

How many new members are still saving £10 per month?

How many new members are not saving £10 per month but are still with

the credit union?

August 2010 Spreadsheet

Have any new members cancelled their credit union account?

35

How many new members continuously saved £10 per month?

How many new members did not save £10 a month but stayed members

of the credit union?

How many new members cancelled their credit union account?

September 2010 Spreadsheet

How many new members accessed credit through the credit union?

How many new members are still saving £10 or more per month?

How many new members stayed members of the credit union?

October 2010 Spreadsheet

How many new members continued with the home contents insurance?

Focus Group Focus Group: Tenant experience of using their credit union account, if

they think they will continue saving, if they think they will continue with

the home contents insurance

Interview Interview with Lesley to find out how she feels the process has gone, what

she would do differently and recommendations going forward

January 2011

Case Studies In-depth case studies of people who have been through the scheme

Questionnaire Sample size of at least 20%

How many new members are still saving £10 or more per month?

How many new members stayed members of the credit union?

March 2011

Spreadsheet

How many new members continued with the home contents insurance?

36

Summary of findings

Evaluation of the project

By the end of the scheme 195 participants had taken part in Save and Insure; 29%

cancelled their accounts leaving a population of 138 participants.

Save and Insure attracted a slightly different client base to the usual membership –

younger (by 6 years) with a higher proportion of males than previously (39% male as

opposed to 28% male). 68% of participants signed up to both the credit union and the

home contents insurance and 52% of participants in the scheme saved continuously for

the six month period and received a £20 credit to their account at the end of it.

17% of participants took out loans with the credit union and those that did emphasised

how much cheaper their loan through the credit union was compared to other popular

sources. This was evidenced by a large amount of anecdotal evidence and one story

shared in the focus group really stood out:

Pam* needed a new cooker and went to BrightHouse to see how much one might cost.

She found one for £200 which seemed like a good deal but was unsure of how much the

overall charge would end up being. The man helping her gave her a piece of paper with

all of the terms and conditions, but as Pam is not able to read very well, she asked the

man who was helping her if there were any additional charges, like interest fees. The

man told her no. Pam was unable to read the small print and as she was by herself that

day, there was no one else she could ask. To reassure her, the man from BrightHouse said

that she could change her mind within 14 days of purchase. They would send her the

information on her purchase in the post and as soon as that was received she could send

it back if she had changed her mind, as long as it was within 14 days.

It took almost a full 14 days for the information to arrive in the post. Pam’s husband read it

and realised that there were indeed charges that the man at BrightHouse had not told

her about. Unfortunately by then, it was too late to change their minds and return the

paperwork for a refund. Over the course of the loan and because of the high interest

charges, Pam and her husband ended up paying nearly £800 for the £200 cooker. Pam

felt this would not have happened to her if she had not been on a low income: “I could

have gone to the store if I’d had the £200 in cash and paid just the £200”. Because she

did not have this money and because the man in the BrightHouse store did not explain

the terms and conditions to her, she ended up paying nearly four times that amount.

Nearly a quarter (24%) of participants of Save and Insure responded to a follow up

questionnaire that was distributed five months after the scheme had ended. 50% of

respondents said that they had never heard of a credit union prior to joining Save and

Insure and 93% of them had never been a member of a credit union before. This

demonstrates how effective Save and Insure was in introducing people to the benefits of

joining the credit union.

37

Evaluation of the model

Signing tenants up face to face worked very well and is a necessary element for the

success of the model. Tenants had to understand how the scheme was relevant to them

before agreeing to sign up; this involved Lesley explaining how saving £10 a month was

achievable “I said right well, if you do it now, and you save your £10, it’s only £2.50 a

week, it’s only a pint of beer, we relate it to what they think about in their lives”.

Now that the new web-site for Prince Bishops Community Bank is fully functional, this will

be a useful way to advertise Save and Insure in the future and hopefully attract more

participants. Participants also said that letters dropped through the door would be better

than leaflets as they are less likely to throw away mail that is addressed specifically to

them.

In future the leaflet/letter drop will be followed up by a phone call before anyone makes

a home visit so that Lesley has the opportunity to explain the scheme to potential

participants before going to their homes to sign them up. The model is resource-intensive

as it requires Lesley to visit each home so phoning tenants up before making the visits

could be a way to save time.

An original aim of Save and Insure was to sign up 500 new tenants but in the end it

reached 195. This is purely down to resource and Lesley is keen to see more resource

directed at the scheme in future so that more tenants can be reached.

Evaluation of the evaluation

This model of evaluation provides a good balance between internal and external; the

evaluation framework was developed in partnership with the external evaluator and the

organisations and all major decisions were made jointly. Balanced alongside this is the

fact that the evaluator offers an outside view and a different perspective.

Capturing data both before the start of the scheme and after provided a good baseline

of data and therefore a strong basis for comparison. In many cases this approach is

preferable to only conducting ‘retrospective’ evaluation that occurs at the end of a

project or programme as it allows for an on-going, reflective process.

Evaluation frameworks run more smoothly when conducted in house as all data is readily

available. There are some drawbacks associated with conducting an evaluation

remotely such as lack of access to data as well as participants.

38

Your Money Garden (Belfast)

Your Money Garden is a financial education project that provides training to

women across Belfast, Newtownabbey and surrounding areas in Northern Ireland.

The training is delivered across a series of workshops and aims to develop

participants’ money management, budgeting and practical skills so that they

may increase their financial knowledge and economic activity.

The original idea for Your Money Garden came from the project’s founder, Ellen Finlay.

Ellen worked in the community and voluntary sector for 15 years before deciding to

qualify as a mortgage advisor. Once qualified she found that a number of friends came

to her for help with their personal finances and after a while her husband suggested that

there might be others needing the same help. Ellen undertook her own research and was

interested to find that women are often the primary budget managers in their household.

She subsequently decided to focus her work on helping women living on lower incomes

in Belfast, Newtownabbey and surrounding areas in Northern Ireland and so launched

Your Money Garden in September 2009. The name Your Money Garden came out of

Ellen’s desire for participants to feel ownership of the course. Finances can often be

trapped as though by weeds but, through learning how to manage their own financial

garden, participants can free their finances from those weeds and watch their personal

finances grow.

Your Money Garden is a financial education project aimed at developing participants’

money management skills with the goal of increasing their financial knowledge and

economic activity in the long term. Broadly speaking, this area of work is referred to as

financial capability. There is no one standard definition of financial capability; generally

speaking however, definitions all follow along similar lines and broadly refer to the

knowledge, skills, confidence and motivation that are needed to manage personal

finances effectively. An example of a typical definition of financial capability is given by

the Friends Provident Foundation on their web-site: “[Financial capability is] a broad

concept, encompassing people’s knowledge and skills to understand their own financial

circumstances, along with the motivation to take action. Financially capable consumers

plan ahead, find and use information, know when to seek advice and can understand

and act on this advice, leading to greater participation in the financial services

marketxiv”. The important point about this definition of financial capability and others like

it is that they highlight that financial capability is not just about knowledge and skills.

Someone can receive all the financial education in the world and be provided with all of

the skills they could ever possibly need to manage their money effectively; but if they do

not have the motivation or the confidence to use these skills and knowledge, it is not

going to result in positive behaviour changesxv.

Your Money Garden has three key aims:

1. Increase financial capability: According to Transact, the National Forum for Financial Inclusion, financial capability is “having the knowledge, skills and

confidence to manage your money well. This includes understanding financial

products, being able to use them and having the confidence and motivation to

39

do soxvi”. The first aim of Your Money Garden is to improve the confidence,

motivation, knowledge and behaviour of participants in relation to personal

finances.

2. Increase economic activity: A lot of the women who come to Your Money Garden are ‘economically inactive’ for a number of reasons. A key aim of the

course is therefore to increase the economic activity of participants and

encourage them to manage their finances actively. This is achieved through

setting financial goals and reviewing these goals at regular intervals, as well as

providing on-going support.

3. Highlight the need for a financial inclusion strategy in Northern Ireland: While Northern Ireland has a strategy in place for poverty and social inclusion, it does

not currently have a specific financial inclusion or financial capability strategy. The

third aim of Your Money Garden is therefore to demonstrate the impact that

interventions can have on financial exclusion in Northern Ireland and encourage

the development of a formalised strategy.

There are four parts to the Your Money Garden model:

1. Establishing the partnerships: Ellen delivers the Your Money Garden course in women’s centres in the area and this is a key part of the project’s success. As the

centres have an existing relationship with women in the community, the women

already feel comfortable attending the course as it is in an environment that they

know and trust. Delivering the course through the centres also helps the project

gain access to women who might benefit from taking part in the course but who

would not necessarily join otherwise.

2. Designing the tools and developing the course: Ellen developed the course and the tools necessary to complete it through a combination of focus groups with

interested participants and desk based research. She also continuously refined the

tools through trials and on-going conversations with participants until she arrived at

a formula that worked.

3. Engaging learners: As well as delivering the course in local centres, Your Money Garden covers the cost of both travel and childcare for the women who attend

the course. This means that women who would not be able to afford either are

able to attend the course. Ellen also makes sure that the course is relevant to

participants; it is not just a course about money, it is a course about their money

and she relates the course material to issues that impact on their lives.

4. Continued support: Participants receive a copy of the Your Money Garden Participant Guide to keep so that if they need a refresher, they have the

information to hand. Three months after participants finish the course, Ellen phones

them to discuss their progress and what they have learned and achieved since

the course’s completion.

40

Your Money Garden Financial Education Project

Establishing partnerships

• Your Money Garden has a relationship with 15 local women’s centres • These centres already have an existing relationship with women in the community

Designing the course and developing the tools

• Focus groups held to determine what material prospective participants would like included in the course • Desk based research on similar courses both in the UK and abroad • On-going development of the course and its tools through evaluation

Engaging learners

• Pay for travel costs and childcare so that neither pose a barrier to attendance • Make personal finance relevant to participants • On-going adjustments made to the course to ensure engagement

Continued support

• Setting of financial goals • Follow up with participants 3 months after the course to discuss progress and next steps

Your Money Garden

Financial Education Project

Women’s centres Participants

41

Your Money Garden Evaluation Framework

When Tool Description Start of Workshop 1 (Apr and May

2010)

Pre-Course Snapshot 8 questions capturing confidence,

motivation, knowledge and behaviour

before the start of the programme

End of Workshop 1 Post-Workshop Questionnaire Reaction – how participants found the

workshop/what they would improve

End of Workshop 2 Post-Workshop Questionnaire

Reaction – how participants found the

workshop/what they would improve

End of Workshop 3 Post-Workshop Questionnaire

Reaction – how participants found the

workshop/what they would improve

End of Workshop 4 Post-Workshop Questionnaire

Post-Course Snapshot

Reaction – how participants found the

workshop/what they would improve

8 questions capturing confidence,

motivation, knowledge and behaviour

at the end of the programme +

demographic info

3 months on (Jul – Aug 2010) 3 Month Follow Up

Revisit goals, and talk about what

progress has been made in the past 3

months

42

Summary of findings

Evaluation of the project

In total, 73 women participated in Your Money Garden over the course of the evaluation

period. Respondents reported an average of 92% satisfaction with the overall course.

When asked what they liked most about the course, 51% of respondents highlighted the

knowledge gained and 22% of respondents said the social aspect:

“Social aspect”

“Good interaction”

“The craic and exchange of ideas”

“The interaction aspect – [the tutor] kept us engaged and made the lessons really

applicable”

On average, the financial capability of respondents increased by 24%. Financial

capability was assessed according to eight indicators, the scores of which were

captured both before the start of the course and at the end. The eight indicators used

are:

• Motivation to take control • Ability to find information • Confidence in asking questions • Planning ahead • Motivation to budget and save • Ability to construct a budget • Ability to manage a budget • Confidence in managing money

The highest increase was reported by participants as being planning ahead in regards to

their money which increased by 28%. When asked what they had gained as a result of

taking part in Your Money Garden three months later, 71% of respondents highlighted the

skills they learned in budgeting and saving.

Case studies were captured as part of the evaluation to demonstrate some of the ways

that Your Money Garden had impacted on the lives of participants:

Denise works for a women’s centre. She joined the course just to ‘make up numbers’

because a few women pulled out and did not believe she had any problems with

money. During the first and second sessions, participants make a list of everything they

purchase from a pint of milk to their groceries. The second week, Denise came back to

class and looked very pale, so Ellen asked how she got on. She shared that she had not

realised how much money she was spending on unnecessary items such as nail polish

(she already had 10 colours), on false eyelashes (she had a drawer full) and other items.

Denise had not realised that she was wasting so much money. What she realised from

participating in the course was that if she had not started keeping a track of where her

money was going, she would very easily have found herself in debt – and very quickly.

43

Three months after taking part in the course, 71% of respondents said that they most

valued the skills gained in budgeting and saving. 58% of respondents also said they feel

they are doing well:

• “I am still budgeting well and have just returned from a holiday” • “I have stopped spending on things I don’t need”

38% of respondents said they are still struggling:

• “Still can’t say no to the grandson” • “I still struggle with making ends meet”

When asked about their goals going forward, 53% of respondents cited examples of

long-term goals such as putting money away for a rainy day and 37% cited short term

goals such as saving for a holiday or Christmas. This is an interesting finding as saving for

the long term and saving for the short term often involve two very different mind sets.

Short term saving is often associated with incentives – “if I save for the next month I may

be able to buy that new pair of shoes” – whereas longer-term saving is not linked to any

particular incentive other than peace of mind. Lack of long-term savings is of concern;

particularly as less than 25% of people in Northern Ireland would be able to survive

beyond a month if they lost their income. So the fact that the majority of respondents

cited long-term savings goals is very encouraging.

Evaluation of the model

The current model of delivery for Your Money Garden has developed from a

combination of observation, participant evaluation and research. It therefore represents

a good balance between engaging participants and ensuring that the course provides

the necessary learning.

The only suggestion going forward is the re-instatement of the setting of personal

financial goals at the end of the course. This is contingent on what form the evaluation

framework will take going forward; the current framework takes up a lot of time so unless

it is revised there will not be additional time available for the reinstatement of setting

financial goals.

Evaluation of the evaluation

Ellen involved participants in the design of evaluation tools which ensured they were

clear, easy to use and captured the right data.

The semi-internal nature of this evaluation provided a good balance between internal

and external. The evaluation framework was developed in partnership with Ellen and the

external evaluator and all major decisions were made jointly ensuring that everyone was

on board. Balanced alongside this is the fact that the evaluator was independent of Your

Money Garden and so viewed the course from a different perspective.

Literacy issues amongst participants mean that Your Money Garden will potentially want

to explore other ways of evaluating the course in future that are not reliant on paper-

44

based tools. As some participants need help filling in the evaluation forms it means that

the evaluation took up a disproportionate amount of time and this takes away from the

time needed to deliver the course.

Unfortunately names were not recorded on all of the forms which meant that a lot of

valuable data showing a participant’s development through the course of the

workshops was lost. This is in part due to the aforementioned literacy issue but is also often

a product of time constraints in the workshop environment. The project might want to

look at other methods of tracking that can be used going forward.

45

Part 4 – Findings

The models

Push, nudge or shove? Service user engagement is often the biggest challenge when it

comes to embedding innovative financial inclusion work in an organisation. While an

organisation may feel that an intervention will make a significant difference to the

service users they work with this will not always translate into active participation on the

part of the service user or target group. The intended beneficiaries of a service are not

always able to see how a service or intervention will benefit them directly, or they might

think that a service is too good to be true. Whatever the reason, organisations often find

themselves in the position of needing more than just a good idea to engage their target

audience.

Both Plymouth Citizens Advice Bureau and Prince Bishops Community Bank faced and

tackled this issue in their respective organisations. Plymouth CAB were finding that, even

though they were actively marketing their MoneyActive workshops and offering them for

free, take up was lower than they expected and definitely lower than they wanted. This

not only meant that service users were missing out but that staff and volunteers were

becoming discouraged. They considered offering incentives to encourage clients to

attend the workshops but ultimately decided that would send the wrong message; the

workshops were already provided for the benefit of service users so offering an additional

incentive did not seem right. They instead looked at the existing structure of MoneyActive

and considered what else could be done to increase take up. Through the course of the

evaluation staff at Plymouth CAB noticed that certain days of the week and certain

times of day saw higher attendance numbers and so decided to change the schedule

to better suit the availability of service users. They also noted which topics in the

workshops were most relevant to service users and adjusted the workshops curriculum

accordingly. Perhaps the most significant action, however, came from the realisation

that the MoneyActive workshops reduced the amount of time needed for subsequent

debt case work. If a service user attended a Money Active workshop they had already

recorded their income and expenditure in a financial statement and learned about

possible solutions for their particular financial issue. As a result, any subsequent debt case

work took 1/3 of the time it took for other clients who had not attended a Money Active

session and because of this they could be seen much sooner by the debt team as their

case took less time. This reduction in waiting time acts as a much more powerful

incentive than anything else Plymouth CAB have trialled in the past. It can often take

weeks for someone to be seen by a debt case worker so taking part in MoneyActive with

the end result of securing an appointment sooner is a powerful incentive.

Prince Bishops Community Bank also incentivised service user engagement through their

scheme Save and Insure but in a number of different ways. They incentivised savings by

offering tenants of Derwentside Homes £20 for saving £10 in their account every month

for six months; a very clear and straightforward incentive. They incentivised the take up of

home contents insurance by offering it for free for the first six months so anyone who

signed up to Save and Insure automatically got it for free. In this way the incentive acted

as an ‘opt out’ and was based on the assumption that tenants would not only stay opted

46

in but would continue with their home contents insurance scheme even after the free

period. Combining these two incentives into one scheme ensured maximum benefit for

participants and increased the likelihood that they would save and continue with their

insurance policies.

Partnership working: Partnerships are important to all four organisations but were

particularly highlighted in the evaluations of Your Money Garden and Prince Bishops

Community Bank.

As discussed in the point above it is often difficult to engage service users and Ellen found

that, as regards her target audience, existing relationships played a key part. Your Money

Garden is a financial capability project specifically targeted at women and so she

formed partnerships with local community centres that already had a relationship with

women in the geographical areas Ellen works in. This enabled her to attract many more

service users than would have been possible if she was working on her own and having

to form these relationships from scratch. It meant that the women did not have to travel

far to attend the sessions and that they already had a certain amount of trust in Ellen as

she was delivering the course in the centres they were already comfortable in.

Prince Bishops Community Bank has worked with Derwentside Homes for a number of

years and it was actually this relationship that brought about the idea for Save and

Insure. Derwentside Homes also had an existing relationship with RSA which enabled their

tenants to have access to home contents insurance through the scheme. This partnership

model proved to be a real strength of the scheme, particularly as regards generating

new, innovative ideas and maximising the benefit for members of the credit union and

tenants of Derwentside Homes.

Make it relevant: Money is not a term that excites many people and particularly not

those who do not have a lot of it. Just the word ‘money’ can invoke feelings of guilt at

spending too much of it or fear of not managing it so getting people to open up about

their personal finances or take part in a scheme aimed to improve their money

management can be challenging. One key way to get people engaged and keep

them engaged is to make it relevant to their lives and their financial priorities.

When Lesley first started visiting tenants of Derwentside Homes to sign them up to Save

and Insure she found that a number of them did not feel they would be able to afford to

save £10 per month. To encourage them she pointed out that £10 per month is roughly

£2.50 per week – it’s a pint or a few newspapers and it is definitely less than a take away.

Breaking it down in this way enabled tenants to relate that money to what they buy and

what they could potentially do without in order take part in the scheme and start saving

some money.

Ellen did the same with participants of Your Money Garden. Right at the start she

addressed the issue of needs vs. wants and acknowledged that managing your money

can be challenging. By encouraging participants to talk openly about their relationship

to money and what it means to them, she enabled them to see how they could address

current issues and change their negative habits to increase their financial well-being in

the long term.

47

Involve service users: When it comes to delivering services to a particular target

audience it is good practice to involve them in the design. Both Your Money Garden and

Centrepoint involved their target audience in the design of their services to ensure that

they were relevant and would make the most impact.

Ellen involved potential service users right from the start by running discussion groups on

what information would be most relevant and what they specifically hoped the service

would provide. She also piloted the evaluation tools with her service users to make sure

that they were clear and would capture the information they intended to capture.

Rupon involved young people in the design and structure of Moneywise at Centrepoint

going forward. Through focus groups and interviews with young people who had taken

part in the programme he was able to get their views and take their ideas on board

which in turn informed the structure of their new strategy for Moneywise going forward.

Service users will often have a much more in-depth understand of the issues affecting

them and are therefore able to give real insight when it comes to designing services.

They are also able to feedback on what engagement strategies will work best.

Intensive contact and ongoing support is key: These four evaluations show that when it

comes to engaging target audiences, it is often necessary to do it face to face and it

often needs to be done multiple times.

Lesley’s view is that a number of tenants of Derwentside Homes would not have signed

up to Save and Insure if she had not signed them up in person. This was a very labour-

intensive process as it involved Lesley driving to the home of every tenant interested in

the scheme but in the end it paid off and Lesley stresses it is the only way to get people

involved. A number of tenants were not sure what the scheme was or, if they did know,

were not sure if they wanted to be involved. By visiting them face to face Lesley was able

to answer their questions and give them further information.

Plymouth CAB has also started taking a more intense approach when it comes to

keeping service users engaged. They were noticing that a number of service users who

were scheduled to attend MoneyActive sessions were not turning up so decided they

needed to take a more active approach. Recently they have trialled texting service

users to remind them to attend their Money Active session and they have also started

following up with service users who do not attend to find out their reasons. This serves as a

helpful reminder to service users and lets them know that the bureau is taking an active

interest in their attendance. The resulting increase in attendance has shown that this

approach is effective.

Evaluation

The future of evaluation is not paper-based: Anyone who has ever handed out a

questionnaire at the end of a workshop or training session knows that it can not only

break the flow of conversation but can also take valuable time away from service

delivery. Some information simply cannot be captured without the use of paper-based

48

tools and it is definitely one of the easier methods to employ but the evaluation of these

four projects shows that where it is possible, it is preferable to explore other methods.

As discussed previously, Centrepoint is currently in the process of considering some new

technology for their evaluations through a company called ResponseWare. Similar to a

Nintendo Wii, the handsets allow information to be relayed instantly and can also be

used as part of an interactive exercise. With certain service users, such as young people

or those with literacy problems, this type of technology could make a positive change to

the way that evaluation data is captured. By being part of the workshop this method of

evaluation can enhance the experience instead of taking away from it.

It is vital to have access to all of the data: One key drawback to the model of evaluation

employed in RBS Innovate was that the Evaluation and Policy Manager did not always

have access to all of the data. A key reason for this is the data protection policies that

most organisations have in place to protect the identity of their service users but it was

also a result of evaluating organisations based in different geographical locations. While

this issue was unavoidable in this programme, and was navigated as well as possible,

other organisations conducting internal evaluation frameworks should try to ensure they

have access to all necessary data as much as is possible. Data can be used in a number

of different ways and, when some is missing, it can significantly impact on the overall

picture that an evaluation produces.

You can test: A lot of evaluations ask subjective questions, such as “do you feel you are

more motivated to take control of your finances as a result of this training?”, not only

because they are easier questions to answer, but because it seems natural to ask the

opinion of participants. While subjective questions provide important information, it is

important not to shy away from asking more knowledge-based, objective questions. This

can be achieved by first deciding what knowledge is to be gained from an intervention:

will participants be learning about budgeting? Will they be finding out about sources of

affordable credit? Whatever the learning outcomes, it is usually possible to ask direct

questions: “What does APR mean?” “What is an example of a priority bill?” “Where would

you go to get help with your debts?” Asking these questions and recording the answers

both before and after an intervention can demonstrate what knowledge has increased

in a much more objective way.

49

Part 5 – Conclusions

There was a surprising amount of crossover in the results of the four project evaluations.

This was unexpected as all four organisations deliver different services to different client

groups in distinct geographical locations in the UK and was certainly not anticipated at

the start of the programme. The common tie between all four organisations is, obviously,

the remit of financial inclusion or financial capability work and it shows that, even though

these services are delivered in different ways in different organisations, there are certain

key approaches that work across the board.

Some of the findings of the programme have been seen in other areas as well; for

example, some policy initiatives have already moved away from incentivising

participation in schemes to stronger approaches. Reforms to the current pension system

due to come in 2018, for example, will see everyone automatically enrolled in a pensions

scheme which requires them to actively opt out if they do not wish to take partxvii. This

decision was made on the premise that the majority of people will not want to go to the

trouble to opt out so will stay opted into the scheme which will help them to have

adequate pensions provision in the future. The NHS adopted the same approach with

Summary Care Records – electronic records of patient information. Because these

records are useful for NHS medical professionals and beneficial for patients, patients

need to actively opt out if they do not want to take part.

Two other findings that emerged clearly were the benefits of partnership working and the

importance of stakeholder consultation. The strength of partnerships is hard to

overemphasise but unfortunately they do not always exist in practice. Silo working, lack

of time and competition for funding can often mean that organisations are hesitant to

work in partnership. However, these four projects have shows that working together can

help to pool resources and make an impact on a larger scale than could be achieved

working alone. The findings from the four evaluations also highlighted that, as much as is

possible, any new scheme or intervention needs to be at least partially based on the

views of those who will be affected or expected to take part in it. Stakeholder

consultation is timely and often expensive but it can also ensure that it reaches the right

people.

50

References

i Charities Evaluation Services web-site http://www.ces-vol.org.uk

ii Doing Research in the Real World, p. 279

iii Doing Research in the Real World, pp. 285 – 286

iv Stufflebeam, D., and Shinkfield, A. (2007) ‘Evaluation Theory, Models and Applications’ San

Francisco, Jossey-Bass

v Centrepoint web-site ‘Why we help’ www.centrepoint.org

vi Centrepoint press release ‘Innovative project helps young people get ‘Moneywise’’

http://www.centrepoint.org.uk/be-informed/media-centre/centrepoint-news/innovative-project-

helps-homeless-young-people-get-2018moneywise2019

vii Friends Provident Foundation ‘Financial Inclusion’

http://www.friendsprovidentfoundation.org/page.asp?section=106&sectionTitle=Definitions

viii Shelter, ‘Priority and non priority debts’ http://england.shelter.org.uk

ix Plymouth Citizens Advice Bureau: Annual Review 2008/2009

x Ibid ii

xi ibid

xii Derwentside Homes Consultation Document 1

http://www.derwentsidehomes.co.uk/UserFiles/File/Book%201.pdf

xiii Derwentside Homes Annual Report ‘Raising Standards Improving Services: Customer

Commitment 2010 – 2012’

xiv Friends Provident Foundation ‘Financial Inclusion’

http://www.friendsprovidentfoundation.org/page.asp?section=106&sectionTitle=Definitions

xv Aynsley, Helen (2011) ‘What’s In a Name’?

xvi Transact web-site

xvii ‘Pensions Bill 2011: Summary of Impacts’

http://www.dwp.gov.uk/docs/pensions-bill-2011-summary-of-impacts.pdf