_____________________________________________________________________________

1

Submitted December 2014

Minorities are Wales’ Resources II

Final Evaluation Report

_____________________________________________________________________________

2

Contents Abbreviations 3 1. Executive summary 4 2. Introduction 8 2.1 Background 8 2.2 Wider project context 9 2.3 Project logic model 11 2.4 Detailed evaluation objectives and research methods 13 3. Analysis of management information 16 3.1 Project outputs 16 3.2 Profiling individual participants 17 3.3 Profiling employers assisted or supported 31 4. Project structure and processes 33 4.1 Project consortium arrangements 33

4.1.1 Initial set-up: MAWR and AWEMA 33 4.1.2 The MAWRII project: VALREC as lead sponsor 36 4.1.3 Swansea YMCA takes on lead sponsor role 37

4.2 Governance and communication 38 4.3 Monitoring and reporting 45

4.3.1 General Perceptions 45 4.3.2 File visits 47

5. Project delivery: results and impact 51 5.1 Identifying underlying need and recruiting beneficiaries 52 5.2 Needs analysis 60 5.3 Training design 64 5.4 Training delivery 70 5.4 Outcomes 75 6. Conclusions and recommendations 80 6.1 Nature of participants 80 6.2 Project outcomes 82 6.3 Project processes 85 6.3.1 Record keeping and evidence collection 85 6.3.2 Marketing and publicity initiatives 86 6.3.3 Equal opportunity and environmental sustainability 87

6.3.4 Leadership 87 6.4 Recommendations identifying future project direction 89

_____________________________________________________________________________

3

Abbreviations BME Black and Minority Ethnic ECDL European Computer Driving Licence EEM European Economic Migrant ESF European Social Fund ESOL English for Speakers of Other Languages NQF National Qualifications Framework NWREN North Wales Regional Equality Network PIV Project Inspection and Verification SSC Sector Skills Council VALREC Valleys Regional Equality Council WEFO Welsh European Funding Office WG Welsh Government (known as Welsh Assembly Government or

WAG prior to May 2011)

_____________________________________________________________________________

4

1. Executive Summary 20 Degrees Consulting Ltd was commissioned in July 2014 to undertake an evaluation of Minorities are Wales’ Resources II (MAWRII), a project funded under Priority 3 of the European Social Fund (ESF) Convergence Programme for West Wales & the Valleys. At the time of the evaluation, the project was being delivered by a consortium of Swansea YMCA (lead sponsor), VALREC (Valleys Race Equality Council) and NWREN (North Wales Race Equality Network). The project aimed to enable people from Black and Minority Ethnic (BME) and European Economic Migrant (EEM) communities to make a fuller contribution to the economy of Wales through raising their skills levels and improving their employment prospects. The project has had a challenging history. Two of the original six consortium members delivering the project no longer exist, while a third was in administration at the time of the evaluation. Against this backdrop, the project has managed to meet or exceed most of its target outputs. 91% of the 1,452 participants characterised themselves as being from a BME background. By September 2014, targets were substantially exceeded for engagement with females working part-time (354 against a target of 229), older participants (181 against the 150 target and those gaining qualifications (511 compared to the target of 355). The only area where outputs fell below output tolerance thresholds related to enterprises assisted, and this can largely be attributed to the withdrawal of one of the original consortium partners. The evaluation team saw evidence of emerging positive outcomes for beneficiaries. Interim evidence suggested 77% of participants exiting the project perceived that they had maintained or improved their employment through participation in MAWRII, while 18% had secured job interviews. These findings were based on a small sample size – greater confidence in these findings will be possible by the end of the project close down period. The case studies presented within the findings provided more compelling evidence of outcomes having been achieved. A number of key success factors emerged from the findings:

• The project sponsors and their delivery teams (Project Practitioners) were known and trusted within the BME/EEM communities within which they worked. This was critical to engaging participants as word of mouth was the most successful means of promoting the project and securing engagement from BME and EEM communities;

_____________________________________________________________________________

5

• The Project Practitioners tended to work holistically with participants. Practitioners were of a common view that clients’ employment and career needs, ambitions and potential were intrinsically bound up with the challenges of everyday cultural and social issues such as housing, family, health and childcare. It was very rarely possible or desirable to ignore these issues when analysing clients’ training needs which, it was felt, could not be viewed in isolation. Practitioners described a needs analysis process that was person-centred and driven by the client. It was seen to be a highly personalised process; with each individual recognised as having different, specific needs. While mainstream employment support agencies might look to support people with CV writing or interview skills workshops, the Project Practitioners within MAWRII provided mentoring to support clients with cultural, housing, health and childcare issues, for example. Nonetheless, despite the breadth of these interventions, the Practitioners struck an appropriate balance and ensured project outputs were captured as each engagement progressed;

• The Practitioners prepared participants and training providers for any

courses they were to undertake. Evidence presented by training providers highlighted that this approach was very effective in ensuring participants came to courses with the appropriate expectations, and similarly that trainers were properly prepared to work with varying needs or levels of English language proficiency. Evidence from training providers suggested the level of Practitioner preparation provided via MAWRII is uncommon elsewhere;

• While evidence was presented of participants maintaining or securing employment through the qualifications gained from the project, there was equally evidence that employability skills and the development of social capital contributed to participant success. Moreover, gaining a qualification via an examination in the English language, particularly for migrants, typically contributed to confidence gains for participants;

• A thread of evidence running throughout the evaluation was the importance of English language proficiency in securing or maintaining employment. While improved proficiency came through study at ESOL classes commonly, there was also evidence of participants pursuing general qualifications in order to practice English with other course participants, at least in part.

_____________________________________________________________________________

6

A series of recommendations were presented within the conclusions of the evaluation. These may be summarised as: 1. Future pan-Wales projects should recognise the different costs of accessing participants in rural versus urban areas. This factor should be reflected in the allocation of targets and budgets; 2. Future consortium-based projects would benefit from clear targets broken down by project sponsor to ensure that no elements of targeted outputs are without ownership; 3. Future interventions to support BME and EEM participants would benefit from the firm inclusion of English language development components; 4. Future projects should set out clear criteria for both participant target characteristics and broader participant eligibility at the project development stage. Clear tolerances should be established, and agreed by all sponsors, between numbers of target participants and what constitutes an eligible participant; 5. In developing consortium-led projects, careful consideration should be given to the configuration of the consortium, to ensure that all partners have the skills, capacity and organisational resilience to effectively deliver against targets; 6. Future interventions might consider being structured according to 4 strands of support: English language; mentoring support relating to employability skills; short, level 2 ‘threshold’ training interventions and longer, vocationally biased interventions at levels 3+; 7. Future activities could benefit from sponsor staff having experience of engagement with medium to large sized businesses in relation to training and development; 8. Project partners should consider working collaboratively to development a follow on project to MAWRII; 9. The findings, conclusions and recommendations of this evaluation should be considered, to inform and refine future delivery options. While the project can be viewed as largely successful, it is clear that there remains an ongoing need for interventions of this type to support BME and EEM communities. The successes achieved by the project are particularly notable, given the levels of change experienced by the consortium over the course of the

_____________________________________________________________________________

7

project. The resilience and commitment of the existing partners should be credited, and the lessons learned from the project used to inform future interventions.

_____________________________________________________________________________

8

2. Introduction 2.1 Background 20 Degrees Consulting Ltd was commissioned in July 2014 to undertake an evaluation of the Minorities are Wales’ Resources II (MAWRII) project, on behalf of a delivery consortium led by Swansea YMCA. The project has been funded under Priority 3 of the European Social Fund (ESF) Convergence Programme for West Wales & the Valleys. The MAWRII project aims to enable people from Black and Minority Ethnic (BME) and European Economic Migrant (EEM) communities to make a fuller contribution to the economy of Wales through raising their skills levels, improving their employment prospects and living conditions and contributing towards their development so that they become less and less dependent on mentoring and support. Delivery of the MAWRII project has been run through a collaborative consortium, led by Swansea YMCA, and involving a range of joint sponsors with extensive expertise in delivering support services to ethnic minority groups. The membership of this consortium has undergone significant change over the course of the project, for reasons explored in later sections of this report. However, at the time of the evaluation exercise, the Joint Sponsors included the Valleys Race Equality Council (VALREC) and the North Wales Race Equality Network (NWREN). The research specification for this evaluation set out a series of core aims and objectives. These included the need to undertake a detailed assessment of whether the project had achieved its targets and objectives for the period 1st March 2012 to 31st December 2014. Other themes included:

• Assessment of indicators of quality and impact; • Explanations behind the project successes and shortcomings; • Understanding of what would have happened without the intervention; • Develop recommendations to identify possible improvements and

learning points, with a view to informing project continuation and strategic direction post 2014;

• Capture examples of best practice of working with Black and Minority Ethnic (BME) and European Economic Migrant (EEM) people, in order to share with other organisations, especially relevant mainstream service providers.

_____________________________________________________________________________

9

This evaluation report has sought to address these objectives directly, presenting quantitative and qualitative findings over a number of key sections. The report ends with a series of focussed conclusions and recommendations for the consideration of delivery partners, funders, and wider stakeholders. 2.2 Wider Project Context Research carried out for the Joseph Rowntree Foundation1 is amongst the wide body of evidence which indicates that many people in poorly paid jobs would like promotion, but find it difficult to progress for a variety of reasons. Barriers to progression are wide ranging and include workplace cultures, lack of role models and problems in gaining access to the training opportunities which might give them the skills needed in higher-level and better paid posts. Whilst these issues apply across the labour market, they are a particular problem for Black and Minority Ethnic (BME) and European Economic Migrant (EEM) groups who are disproportionately represented in low paid jobs. These concerns are clearly reflected in a range of UK and Welsh Government policy documents, and also feature prominently in the ESF Convergence Programme for West Wales and the Valleys 2007-2013, which argues for

“Ensuring that BME employees are able to access training and promotion opportunities…. A denial of cultural identity in order to fit in should not be a prerequisite for employment or advancement”2.

As well as challenges facing members of BME groups, the Convergence Operational Programme also draws attention to

“migrant workers, those from Central and Eastern Europe and those from non-European countries, (who) are amongst the most vulnerable and poorly paid workers in the UK economy, with many having limited rights if treated unfairly at work, because of fear of losing their jobs”.

It was this backdrop that prompted the development of the Minorities are Wales Resource (MAWR) project whose overall aim was to

“…enable people from Black and Minority Ethnic (BME) and European Economic Migrant (EEM) communities to make a fuller contribution to the economy of Wales through raising their skills levels and improving their

1‘In-work Poverty, Ethnicity and Workplace Cultures’ Maria Hudson and Gina Netto et al 24 September 2013 2 see Annex D to ESF Operational Programme: ‘Equal Opportunities Analysis and Equality Matrix’

_____________________________________________________________________________

10

employment prospects, living conditions and contribute towards their development so that they become less and less dependent on mentoring and support”3.

In more detail, the project was intended to “support people from Black and Minority Ethnic (BME) including European Economic Migrant (EEM) backgrounds to enhance their skills to obtain, maintain and progress in employment. Through engagement with the target group and working simultaneously with employers, the performance of businesses will increase. The target groups’ performance in employment will be enhanced through outreach, confidence building, mentoring and support, language support and referred skills training. It will complement and add value to statutory provision and extend services to groups currently not able to take advantage of the provision. People who are not from BME and EEM backgrounds will also be assisted. However … the great majority of participants are expected to be either BMEs or EEMs.” The MAWRII project has somewhat complex origins, in that it stemmed from an earlier phase of delivery – Minorities are Wales’ Resources (MAWR), which was one of three ESF funded initiatives for which the All Wales Ethnic Minority Alliance (AWEMA) acted as lead sponsor. Joint sponsors for this earlier phase included the Valleys Race Equality Council (VALREC), the North Wales Race Equality Network (NWREN), YMCA Swansea; YMCA Wales and the Minority Ethnic Women’s Network (MEWN). However, in February 2012 the Welsh Government announced publically that an investigation had been initiated to determine whether the public funding provided to AWEMA by the Welsh Government, WEFO and the Big Lottery had been properly safeguarded. Prior to the public announcement, funding to AWEMA had been frozen in September 2011. The investigation findings pointed to significant issues with the management, governance and financial operations at AWEMA. This resulted in AWEMA’s removal from all EU projects in 2012. This situation presented an immediate challenge for the rest of the consortium, who had to enter into detailed discussions with WEFO to agree a way forward and safeguard the project and future delivery. The consortium effectively faced two options: to close the project and develop an entirely new bid or revise the business plan and continue delivery with a new Lead Partner. As part of these

3 source: MAWRII Business Plan, Swansea YMCA

_____________________________________________________________________________

11

discussions it was decided to ‘rebadge’ the project as MAWRII to convey to all stakeholders that the project had entered a new phase. Initially, it was agreed that VALREC would assume the lead sponsor role, but further challenges around speed of ESF grant payments from WEFO meant that by 1st September 2013, this role had passed to Swansea YMCA, with VALREC continuing as a joint sponsor and training provider. Prior to taking on this role, Swansea YMCA worked with VALREC to undertake a due diligence audit of the project. YMCA Wales withdrew from the project altogether during this audit in August 2013. While Swansea YMCA preferred the option of procuring a new partner to replace YMCA Wales, this was challenged by WEFO. As a result, the YMCA Wales’ targets were redistributed to the remaining partners. Finally, in April-May 2014, WEFO froze funding to MEWN which soon led to them being extracted from the project altogether too. This history has had important implications for the delivery of MAWRII. Both the circumstances and impact of these major changes are explored in detail as part of this project evaluation. 2.3 Project Logic Model Figure 2.1 overleaf gives the logic framework established for the project. A number of points can be made at this stage in relation to the operating model and logic framework and the operating model (figure 2.2):

• Although reference is made to skill shortages of employers reported by Sector Skills Councils, there is little evidence of how meeting these shortages is to be assessed and any changes due to MAWRII analysed;

• Previous evaluations of workforce development initiatives (for example, ‘Evaluation of Skills in the Workplace’4 have pointed to some challenges in satisfying objectives to promote the interests of employees (who may well seek qualifications to progress their career with other employers), employers (who may seek limited additional qualifications, perhaps to do little more than meet legal requirements) and wider aspirations to develop a more highly skilled workforce (which is a more strategic and longer term aspiration, potentially of little immediate interest to employers or employees). In both the logical framework and operational model, the focus appears to be principally upon training individuals rather than improving employee performance or contributing towards ‘Improving skill levels and adaptability of the workforce’ (ESF priority 3);

4 CRG Research for Welsh Government, see http://dera.ioe.ac.uk/8827/

_____________________________________________________________________________

12

• Previous projects have, however, found particular difficulty engaging with minority groups, and this project offers good chances of success through the consortium approach which involve a range of organisations with established links to minority groups.

Figure 2.1: MAWRII project logical framework

Aim Objectives Based on evidence

How to be achieved

Indicators monitored Verification

Where specified

Project description

Need and demand for the project

Project management and delivery

Activities and results

Monitoring and evaluation

To enable employed

BME & EEM individuals to

sustain employment & progress in employment

through increased

skills

Contact with 250

employers

Skills shortages of employers reported by

SSCs

Outreach to employers.

Signposting by SSCs Records of

contacts & recruitment

Project records along with

internal and external

evaluations

Contact with 17 community

groups

High levels of informal working known to sponsors

Outreach through

community groups known to sponsors

Production of action plans

for 1500 people

Lack of recognition of qualifications

& lack of access to training

Development Officers

carrying out one to one &

group sessions

Action plans completed

1500 participants to benefit from

project activities

Performance deficits faced by employers

& lack of supervisory

staff

Training referred or

developed at work or with co-sponsors Records of

outcomes from the

interventions 142 gain recognition of prior learning

and experience

Lack of current

qualifications & record of

prior achievement

Accreditation of ‘home’ (i.e.

overseas) qualifications

& prior learning & experience

_____________________________________________________________________________

13

Figure 2.2: MAWRII operating model

Outreach & recruitment through contacts or referrals with employers, other WEFO projects, community groups, TUC & Statutory Agencies

Mentoring, Support and Training

Commitment of participants and production of individual Action Plans following a skills needs analysis with each participant

Engagement of participants through one to one confidence building, group activities, advice, language support and signposting

Training activities through accredited prior learning or referrals (i.e. ESF funded training projects, FEIs, HEIs or private training providers)

Gaining recognised qualifications and skills that are appropriate to the labour market

Improved skill levels along with increased confidence and understanding of the labour market leading to progress in employment

Elements of the logic model, the operational model, project design, governance and delivery are explored in later sections of this report, using analysis of management information and data collected from project sponsors, wider stakeholder and beneficiaries.

2.4 Detailed Evaluation Objectives and Research Method 20 Degrees Consulting Ltd was commissioned in July 2014 to undertake an independent, final evaluation of the MAWRII project. The terms of reference for the evaluation required attention to be given to the following:

Evaluate Participants • The geo-spatial patterns of participants engaging in the project; • The profile of participants engaging in the project; • Patterns of recruitment and engagement relating to Migrants and Non-

Migrants.

Evaluate Project Outcomes • An evaluation of whether the project achieved its objectives; • An evaluation of the efficiency of project activity in meeting objectives;

_____________________________________________________________________________

14

• The impact of the project on beneficiaries and community, particularly in relation to Black Minority Ethnic participants gaining qualifications and the project’s impact on BME progression within employment;

• The impact of any external factors on the project. For example, the context of the Welsh economy and this project’s unique heritage due to the dissolution of AWEMA.

Evaluate Project Processes • To undertake an evaluation of the chain of participant record keeping and

evidence collection processes from frontline delivery to data submission to WEFO;

• To evaluate the effectiveness of project marketing and publicity initiatives;

• To evaluate the effectiveness of addressing Cross Cutting Themes of Equal Opportunity and Environmental Sustainability;

• To evaluate the effectiveness of leadership including quality assurance and communication methods employed by the lead sponsor;

• Provide recommendations to address any issues found. Recommendations identifying future project direction • An examination of the project delivery and outcomes in light of the

present social-economic environment in Wales; • An identification of improvements and or gaps in service delivery.

The evaluation programme proposed consisted of the following elements:

Inception/desk research Meet client, refine understanding of needs, access project management information, draft ethical charter and consent process, draft research instruments, undertake initial research into comparative, published data to act as benchmark. Process evaluation Consider effectiveness of project processes i.e. completeness and consistency of data capture, record keeping, consistency issues across partners, verification of beneficiary eligibility and quality assurance process put in place by Swansea YMCA. Qualitative interviews with key personnel from project partners. Impact evaluation

_____________________________________________________________________________

15

• Quantitative analysis of participants – Profile: geo-spatial characteristics, track ethnicity patterns of engagement/disengagement against demographic data. Analysis of skills development and progression.

• Qualitative - case studies of participants derived from face-to-face interviews

• Focus groups/workshops to capture skills progression from the project. • Employer case studies to highlight the positive impact of skills

development on their workers and the potential impact on improved equality & diversity strategies.

Data analysis and report writing Report evaluating processes, progress against objectives and impact of project

Data was collected for the evaluation between August and November 2014. The evaluation team is sincerely grateful to the many people who gave their time and contributed their insights to this evaluation. Particular gratitude is owed to the project sponsors, wider stakeholders and, of course, the beneficiaries of the project. The following sections of this report present the findings of our research; exploring the Management Information in detail, assessing Project Structures and Processes and finally considering Project Delivery – Results and Impact. The report then moves on to embed Lessons Learned and Options for Future Delivery within the detailed Conclusions and Recommendations.

_____________________________________________________________________________

16

3. Analysis of Management Information 3.1 Project Outputs As part of the evaluation exercise, the research team have examined project Management Information in detail, to assess performance against targets and to consider whether any trends or conclusions can be drawn from the beneficiary data. This data will already have been subject to tracking and examination by WEFO, as part of the project monitoring and reporting process. Table 3.1 summarises progress against target at 30th September 2014 when participant recruitment to the project ceased. This summary position was provided by Swansea YMCA, as lead sponsor, from the central participant database. All beneficiary data is brought together into a central database from project sponsors on a monthly basis. This data is then collated, verified and sent to WEFO as part of the quarterly claim report. It should be noted that the project has been granted an extension to 31st April 2015, consequently the output and outcome positions are likely to be subject to change. Table 3.1 Progress against key targets at 30th September 2014 Target areas Actual total

at 30th September 2014

Total project target

Variance

Total participants 1,452 1424 +28 (+2%) Female participants 810 914 -94 (-10%) Female participants who work part-time (less than 30 hours)

354 229 +125 (+55%)

BME/EEM participants 1,324 1080 +244 (+23%) Older participants (50+) 181 150 +31 (+21%) Participants with a work-limiting health condition or disability

47 49 -2 (-4%)

Participants Gaining Qualifications 511 355 +156 (+44%) Employers assisted or supported 167 220 -53 (-24%) Employers adopting or improving equality & diversity strategies and monitoring systems

39 65 -26 (-40%)

Key: Green – target met or surpassed Amber – Outputs up to 15% below target Red – Outputs more than 15% below target. n.b. WEFO typically works within a 15% tolerance threshold

_____________________________________________________________________________

17

The majority of outputs either exceeded target or were within the typical WEFO tolerance of 15% below target. The only outputs significantly below target were those of Employers assisted or supported and the related Employers adopting or improving equality and diversity strategies and monitoring systems. These targets were originally the sole responsibility of YMCA Wales. When this organisation withdrew from the project, there was little evidence of progress against this target. Consequently, the remaining project sponsors were left with major challenges in relation to this target during 2014, which became an addition to the targets to which they had originally committed. We return to this issue in our conclusions and recommendations chapter.

The project continued to support participants directly until 17th October. It was known that 166 participants were accessing qualifications at 30th September, with up to 115 of these having the potential to contribute to the total number gaining a qualification i.e. it was expected that the ‘Participants Gaining Qualifications’ measure would further improve as additional data was captured during the project closedown period. 3.2 Profiling Individual Participants The participant database holds a significant number of fields of information that are additional to the core reporting requirements for headline targets. This was used during the evaluation to profile the project participants. This profiling was undertaken with data recorded on the participant database on 31st August 2014 when n=1,392 i.e. the number of project participants was 1,392 (96% of the final participant total). The participant database notes whether participants view themselves as a member of a BME group. 1,269 (91%) of participants placed themselves in this category. This is a significant achievement for the project given that BME communities are typically viewed as hard to reach groups5. While not a direct comparison, as other projects would not have targeted BME participants necessarily, it can be noted that the relevant Priority 3 Programme monitoring report identified 5% of participants within this priority characterising themselves as BME at May 20146. The project has recorded activities to engage with a diverse range of participants. The results have been largely successful. Table 3.2 summarises the split of ethnicities declared by participants while table 3.3 provides the split

5Brackertz, N. (2007) ‘Who is hard to reach and why?’, ISR Working Paper accessed at http://www.sisr.net/publications/0701brackertz.pdf 6ESF Convergence 2007-13, West Wales and the Valleys, Programme Monitoring Report, June 2014, p16, Annex 2 -Priority 3: Improving skill levels and adaptability of the workforce

_____________________________________________________________________________

18

of nationalities, again self-declared. The latter table provides an insight into the diversity of backgrounds of participants which remains largely opaque at the ethnic summary level of table 3.2. This is largely due to the scale of the Asian – Other, Black – Other and White – Other categories within the ethnic breakdown used by project partners. An alternative way of classifying project participants can be found in figure 3.1, where participants declare their status as migrants or otherwise. Attempts to engage with a diverse range of participants have not been confined to ethnicity nor nationality. Table 3.4 summarises the local authority in which participants are domiciled. Reasonable coverage of the Convergence area has been achieved, except for Ceredigion and Pembrokeshire. Reference to the Office for National Statistics breakdown of the population by ethnic groups suggests numbers of people from particular ethnic groups in Pembrokeshire and Ceredigion are generally comparable to other counties such as Torfaen and Blaenau Gwent, where there were notably higher levels of engagement7. It might be inferred that engagement levels in Pembrokeshire and Ceredigion may have been influenced by considerations other than need. Reference to the project’s business plan indicates that individual project sponsors took ownership of specific local authority areas. It was noticeable that Ceredigion and Pembrokeshire were the only two counties for which no project sponsor explicitly took responsibility. Interview data pointed to the potential importance of the relative costs (time) of project sponsors engaging with an individual in rural versus urban settings and logistical issues for potential beneficiaries as inhibiting take-up from these very rural areas. 39% of project participants were from the City and County of Swansea. This is clearly a large proportion of the participant population. This may be explained partly by Swansea being the second largest city in Wales and the largest within the Convergence area8. However, when making comparisons with, say, the similar sized population of Rhondda Cynon Taff, interviewees suggested that Swansea’s strong performance is likely to owe more to two of the sponsor organisations (Swansea YMCA and MEWN) being based in that city. There was evidence that other attempts were made to ensure diversity of the participant population. 47 people (3.4% of the total participants) declared themselves to be disabled. Although this is a lower percentage than the number of working age adults in employment who may declare themselves to be disabled people, it represents a reasonable level of engagement with this

7 Office for National Statistics, Population Estimates by Ethnic Group, mid 2009 accessed at http://www.ons.gov.uk/ons/publications/re-reference-tables.html?edition=tcm%3A77-50029 8http://www.citypopulation.de/UK-Wales.html

_____________________________________________________________________________

19

group9. Indeed, there is evidence to suggest proactive attempts were made to engage disabled people to join the project. It was noted that the RNIB was an enterprise assisted via the project.

9 The Disability Prevalence statistics for 2011/12 may be found at https://www.gov.uk/government/statistics/disability-prevalence-estimates-200203-to-201112-apr-to-mar. These statistics suggest 5.7m working age adults in the UK can be classified as disabled. However, 46% of working age disabled people are in employment, reducing the population of UK working aged adults in employment and having a disability to 2.6m (https://www.gov.uk/government/publications/disability-facts-and-figures/disability-facts-and-figures#employment). The number of working age adults (ages 16-64) can be estimated as 41m in 2012 from http://www.ons.gov.uk/ons/rel/lms/labour-market-statistics/april-2012/statistical-bulletin.html. This suggests 6% of working age adults in employment may be disabled.

_____________________________________________________________________________

20

Table 3.2 - Declared ethnicity of participants Ethnicity Number of

participants Sub-total of ethnic grouping

Sub-total of ethnic grouping (%)

Arab 1 Arab - British 1 2 0% Asian – Bangladeshi 11 Asian - British 111 Asian - Chinese 41 Asian - Indian 37 Asian - Pakistani 10 Asian - Welsh 3 Asian - Other 247 460 33% Black – African 48 Black – British 49 Black – Caribbean 4 Black - Other 31 132 9% Gypsy/Traveller/Romany 2 2 0% White – African 1 White – British 75 White - English 3 White – Irish 6 White – Welsh 70 White - Other 470 625 45% Other ethnic background 48 48 3% Ethnicity not declared 112 112 8% Total 1,392 1,392 100%

_____________________________________________________________________________

21

Table 3.3 - Declared nationality of participants African - 48 Albanian - 1 Arab - 11 Argentinian - 1 Armenian - 2 Asian - 1 Australian - 2 Bangladeshi - 83 Bengali - 1 Black African - 1 Bosnian - 1 Botswanan - 2 Brazilian - 11 British - 149 British Italian - 1 Bulgarian - 33 Cameroon - 1 Caribbean - 7 Chilean - 1 Chinese - 66 Columbian - 1 Congolese - 1 Croatian - 1 Czech – 3 Danish - 1 Dutch - 5 Equador - 1 Egyptian - 6 English - 8 Estonian - 1 European - 9 Filipino - 10 French - 5 Gambian - 1 German - 2 Greek – 2 Guinean - 1 Hong Kong - 1 Hungarian - 15 Indian - 91 Indian/Portuguese - 1 Iranian - 7 Iraqi - 3 Irish - 6 Italian - 8 Italian/Brazilian - 1 Jamaican - 1 Jordanian - 2 Kenyan - 2 Korean - 1 Kurdish - 2 Laos - 1 Latin American/Welsh - 1 Latvian - 9 Lebanese - 1 Lithuanian - 13 Malaysian - 1 Maltese - 1 Mixed race - 1 Moroccan - 2 Nepalese - 68 Nigerian - 17 Pakistani - 19 Polish - 275 Polish/French - 1 Portuguese - 31 Romanian - 39 Russian - 3 Saudi Arabian - 1 Scottish - 2 Senegalese - 1 Serbian - 1 Sierra Leonean - 1 Sikh - 1 Slovakian - 4 Spanish - 27 Sri Lankan - 12 Swiss - 1 Syrian - 2 Tanzanian - 1 Thai - 29 Tigrinya - 1 Taiwanese - 1 Traveller - 2 Turkish - 37 Ukrainian - 4 USA - 2 Vietnamese - 4 Welsh - 82 Welsh/Lebanese - 1 White oriental - 1 Zimbabwean - 3 Undeclared - 66

470, 34%

380, 27%

542, 39%

FIGURE 3.1 - DECLARED MIGRANT STATUS OF PARTICIPANTS

Migrant (EU) Migrant (non-EU) Non-migrant

_____________________________________________________________________________

22

Table 3.4 – Domicile of participants by Unitary Authority Unitary Authority Number of participants Blaenau Gwent 39 Bridgend 71 Caerphilly 65 Carmarthenshire 65 Ceredigion 1 Conwy 124 Denbighshire 31 Gwynedd 110 Isle of Anglesey 25 Merthyr Tydfil 68 Neath Port Talbot 36 Pembrokeshire 1 Rhondda Cynon Taff 135 Swansea 539 Torfaen 82 Total 1,392 Figure 3.2 presents a graphical view of the age range of participants. Participants enrolled onto the project represented every age from 16 to 72. Again, the data suggests the project engaged with a diverse age range of participants. It has been noted already in table 3.1 that the project exceeded its target of engaging with 150 participants over the age of 50 by some margin. Overall these findings suggest positive levels of engagements with participants to ensure a diverse participant profile when viewed against 4 of the protected characteristics of the Equality Act 2010 (age, disability, race and sex). These particular characteristics were highlighted in project targets. Data was not captured to enable profiling against the other 5 protected characteristics as they were not the focus of the intervention.

_____________________________________________________________________________

23

Additional data, relevant to the equalities cross cutting theme, was captured in the participant database and is summarised in table 3.5.but it does have to be borne in mind that the project primary focus was on race, disability, age and gender.

Table 3.5 Additional participant characteristics relevant to the equality and diversity cross-cutting theme of ESF Characteristic No. (%) of participants with

characteristic Can speak Welsh (n=1,310) 10 (1) Can read Welsh (n=1,309) 9 (1) Can write Welsh (n=1,294) 16 (1) Lone parent (n=1,387) 60 (4) Work-limiting health condition (n=1,390) 74 (5) Participant profiling of particular relevance to the project’s aims and objectives relates to qualifications and employment, given the projects aim to up-skill the workforce in order to help individuals take advantage of opportunities for achieving more stable or better paid employment. Table 3.6 summarises the highest level of qualification held by participants when they enrolled on the project and the volumes of new qualifications gained or being accessed at 31st August 2014. 268 training courses were also pursued by participants on unaccredited courses.

0

10

20

30

40

50

60

70

16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 52 54 56 58 60 62 64 68 72

Figure 3.2 - Distribution of participant ages

Total

_____________________________________________________________________________

24

Research external to this project suggests that there is typically a relationship between skill levels of individuals and their employment related income:

“…acquiring education and skills is rewarded through both more stable employment and a smaller risk of unemployment, and higher earnings when employed. In particular, such returns are found even for low level skills and qualifications levels, and for basic numeracy and literacy skills. This suggests that raising education levels among the least skilled will increase their employment stability and relative earnings and so reduce income inequality and poverty”10.

The project did not capture participant earnings, therefore a proxy for earnings needed to be used to test the above proposition. Table 3.7 maps the highest qualification level of participants with their positions in their organisations on enrolment to the project. There was no apparent correlation between rising qualification level and rising position within their organisation. Figure 3.3 illustrates this point by presenting the data for staff, manager and director (sample sizes for other roles are too small to be used for trend comparisons). Patterns of qualification levels held are strikingly similar, regardless of the position held by participants in their organisations. On the whole, therefore, it seems likely that highest qualification levels were not the main influence on earnings for participants. The most striking feature of table 3.7 is that 86% of participants classified themselves as staff and there was a broad spread of highest qualifications held across all employment levels. This further supports the conjecture that factors other than highest qualification level held will be significant influences on participant earnings. Returning to the literature, there is a body of evidence to suggest that there is a significant link between migrant status of individuals and their earnings11,12, 13. This largely relates to the number of early years of schooling immersed in the host country’s language. Allied to this issue of years of language development

10Taylor, M., Haux, T. and Pudney, S. (2012) ‘Skills, Employment, Income Inequality and Poverty: Theory, Evidence and an Estimation Framework’, Joseph Rowntree Foundation accessed at http://www.jrf.org.uk/sites/files/jrf/skills-employment-theory-evidence.pdf 11Bleakley, H. and Chin, A. ‘Language Skills and Earnings: Evidence from Childhood Immigrants’, The Review of Economics and Statistics accessed at http://crcw.princeton.edu/migration/files/October%202009/Bleakley%20and%20Chin%20RES.pdf 12Boissiere, M.; Knight, J. B.; Sabot, Richard H. 1985. Earnings, schooling, ability, and cognitive skills. American Economic Review 75(5): 1016-1030. 13‘Earnings differences between immigrants and the Canadian-born – The role of literacy skills’, Statistics Canada. Accessed at http://www.statcan.gc.ca/pub/81-004-x/2008005/article/10798-eng.htm

_____________________________________________________________________________

25

in the host country are factors linked to more general early years development embedded in that country’s culture i.e. the development of social capital. Figure 3.4 clearly demonstrates differences between migrant status and employment levels held by participants. There is a significantly higher likelihood that non-migrants have progressed to the position of manager than migrants. This tends to support the proposition that being embedded in the language and culture of the country from birth (or early years) is likely to lead to higher earnings than if the individual was a migrant. The effect is less pronounced, albeit still visible, for directors/owners. Here, the literature suggests drivers towards self-employment may be influenced by a range of issues ranging from discrimination driving people towards this option through to cultural differences, with some ethnic groups or nationalities having a stronger tradition or regard for self-employment. Table 3.6 – Participant qualifications Qualification level No. held by

participants on joining the project

New qualifications gained

New qualifications still being accessed at 31st August 2014

None 443 - - Below NQF level 2 93 26 60 At NQF level 2 207 389 40 At NQF level 3 280 26 36 At NQF levels 4-6 298 0 11 At NQF levels 7-8 71 5 22 Total 1,392 446 169 Table 3.7 – Breakdown of qualifications held against employment position Highest qualification held None Below

level 2 Level 2 Level 3 Levels

4-6 Levels 7-8

Total

Trainee/ apprentice

2 2 6 1 4 0 15

Staff 374 80 177 240 254 61 1,186 Supervisor 9 0 3 4 4 1 21 Manager 25 3 5 12 14 4 63 Senior Manager

0 0 0 0 0 1 1

Director/Owner 30 8 15 17 21 3 94 Total 440 93 206 274 297 70 1,380

_____________________________________________________________________________

26

Figure 3.3 Normalised view of qualifications held against employment position

Figure 3.4 Normalised view of migrant status against employment position

Looking in more detail at the issue of qualifications, table 3.6 shows, there was a clear emphasis during the project on securing level 2 qualifications. 429 of the potential 615 qualifications accessed during the project were at level 2 i.e. 70% of qualifications accessed. This is despite table 3.6 demonstrating that 46% of participants already held a level 3 qualification or above. This reflects 3 key issues:

0

5

10

15

20

25

30

35

40

45

None Below level 2 Level 2 Level 3 Levels 4-6 Levels 7-8

Staff Manager Director/Owner

0%

10%

20%

30%

40%

50%

60%

70%

80%

Staff Manager Director/Owner

Non-migrant EU migrant Non-EU migrant

_____________________________________________________________________________

27

• 39% of participants held qualifications below level 2 or no qualifications (32%) when they joined the project. Securing level 2 qualifications provided this group with an advance in their qualification position;

• As we will explore through participant interviews later, participants

typically sought short, practical training interventions that would provide them with baseline skills and knowledge for maintaining employment or securing alternative employment. Typical courses included: food hygiene, first aid, fire marshal and ECDL;

• We have already noted that English language skills are key to employment prospects. ESOL was a commonly accessed training intervention. Again, we will note later that there was evidence from some participants that their motivation for accessing training courses was influenced more by their desire to practise their English language skills with non-immigrants or to learn technical English than to gain specific technical skills.

We turn now to the nature of work undertaken by project participants. When the number of hours typically worked by participants was examined, it was apparent that the spread ranged from zero hours through to 70 hours per week. There were peaks in number of hours worked at 16, 30 and 40 hours. If these hours worked are mapped against highest qualification level of participants enrolling on the project we obtain table 3.8. There is no apparent correlation between those working part-time (16 or 30 hours) and level of qualification. Further analysis of the data suggests factors such as gender have a significantly larger effect on whether an individual works part time than their highest qualification level. Females made up 68% of the total working 16 hours per week and 61% working 30 hours per week. This falls significantly to 39% of the total number of participants working 40 hours per week. There was a relatively low prevalence of zero hour contracts reported by participants. These have been added to table 3.8 to explore any relationship between level of highest qualifications held and participants being employed via zero hour contract. However, only 57% of participant records indicate whether they had a zero hours contract or not, making it unwise to draw firm conclusions from extrapolated findings. Nonetheless, for those that did indicate work was undertaken on a zero hour contract, it was apparent that almost two thirds (63%) were qualified to level 2 or below. Two thirds (67%) of participants working a 40 hour week started the project with qualifications at or below level 2. Notably 51% of people working 40 hours had no qualifications.

_____________________________________________________________________________

28

Table 3.8 – Qualifications mapped against most prevalent working hours and zero hour contracts Highest qualification held Typical hours worked

None Below level 2

Level 2 Level 3 Levels 4-6

Levels 7-8

Total

Zero hours contract (n=40)

28% 15% 20% 15% 15% 7% 100%

16 (n=139) 32% 4% 19% 22% 19% 4% 100% 30 (n=112) 26% 11% 16% 14% 28% 5% 100% 40 (n=341) 51% 3% 13% 17% 14% 2% 100% Little can be deduced from participants’ BME status within this section. We have already noted that 91% of participants described themselves as belonging to a BME group. Non-BME participants tended to be drawn from the same employers as their BME counterparts and consequently had similar levels of starting qualifications and employment characteristics. The data does suggest some subtle differences between those participants classifying themselves as migrant workers and those that did not. Figure 3.5 presents these effects. It can be seen that non-EU migrants are more likely to have qualifications below level 2 than other groups but tend to lag behind non-migrants and EU-migrants at higher levels. While it is noted that non-migrant participants had a higher likelihood of holding a level 7/8 qualification than other groups, it would be unwise to deduce too much from that observation due to the small sample size at these levels and the potential for bias: it was noted during the study that a number of university students were participants in the project. While they might well have a level 7 qualification and not have classified themselves as economic migrants, it is common for students to work in lower skilled jobs alongside their studies: they should not be compared directly with economic migrants who seek a salary associated with a full time position.

_____________________________________________________________________________

29

Figure 3.5 Migrant status mapped against highest qualification level

Some insights to the types of sectors in which participants worked when joining the project were gained from analysis of the participant database. Table 3.9 highlights the top 10 sectors in which project participants were working. Data was available for 1,182 participants. Where a particular ethnic group had a statistically significant dominant presence in that sector, it is highlighted.

0%

10%

20%

30%

40%

50%

60%

None Below level 2

Level 2 Level 3 Levels 4-6 Levels 7-8 Total

Non-migrant EU migrant Non-EU migrant

_____________________________________________________________________________

30

Table 3.9 – Top 10 sectors in which participants work Sector No. of

participants % of participants Ethnicity comment

Hospitality/catering 290 25 25% Asian – Other 30% White - Other

Manufacturing/warehousing 216 18 20% Asian – Other 63% White - Other

Nursing/social care 124 10 27% Asian Other 16% White – Other 16% White – British 10% Black - British

Retail/wholesale 105 9 24% Asian – Indian 24% White - Welsh 11% White – Other

Third sector 104 9 40% White – British Domestic/cleaning 100 8 29% Asian – Other

45% White – Other Education/teaching 53 4 30% White – Other Public sector 22 2 32% White – Other

27% - Asian – British Construction 21 2 48% White – Other Hotels/leisure 21 2 - A key question for this evaluation is whether participation in the project led to employment being sustained or career progression. A limitation on answering this question conclusively from the management information was the relatively small proportion of participant exit data on the participant database at 31st August 2014. The status of most participant files at this point was ‘open’. Efforts were being made by the project sponsors to collect exit data from participants during the close down phase of the project. Interim findings can be presented from the available data, but much more robust information will become available at the end of the close down period. Table 3.10 Potential impact of the project on participant employment Question Positive response Has employment been sustained (through participation in the project) (n=107)

82 (77%)

Did the participant secure a job interview during the period when they participated in the project? (n=108)

19 (18%)

Caution needs to be exercised in interpreting table 3.10. The sample size is small in relation to the full participant population, making extrapolation to the full participant cohort unwise. Nonetheless, 77% of respondents maintaining employment (which could mean they held a number of jobs in succession) appears relatively positive, particularly given the wider employment situation in Wales during the lifetime of the project. Similarly 18% securing a job interview suggest positive results arising from periods of CV preparation and interview technique mentoring by Project Practitioners.

_____________________________________________________________________________

31

It is important to note that achieving sustained employment or job interviews may have arisen from a number of factors, not necessarily just from courses accessed and qualifications gained. A key element of the project was the provision of mentoring, confidence building and language support. 99% of project participants were recorded as having accessed this support. Influences on an individual’s employment position are clearly complex: we have already highlighted research evidence external to this project, suggesting that interventions around English language development and cultural support are likely to have benefited participants. We return to this point later in this report. But for now we simply note that it is not possible to infer with confidence which elements of the project interventions will have contributed most towards any improvements in participant employment position - the likelihood being that a combination of factors has been involved. Analysis of career progression in terms of job level and sector at an individual participant level indicates that six participants (6% of the exiting cohort) could evidence career progression (staff to manager; manager to director/owner; staff to director/owner or trainee/apprentice to supervisor). Only two (2% of the exiting cohort) appeared to move backwards in terms of grade (staff to trainee and supervisor to staff). Even for these apparent demotions, it is notable that they involved transitions from generally less to more skilled sectors (hospitality/catering to engineering/technical), therefore the participants may not have viewed the progression as retrograde. Overall, ten participants (9% of the exiting cohort) changed sector during the time they were engaged in the project. However, the crudeness of sector-based classifications means little can be inferred regarding the nature of impact on participants in terms of earnings, working hours or job satisfaction. 3.3 Profiling Employers Assisted or Supported The ‘Employers Assisted or Supported’ or Enterprise Database was analysed with respect to the data held on 31st August 2014. At that stage, data was available for 162 enterprises that satisfied the ESF definition for this project i.e. enterprises that had received support to improve or put in place equalities and diversity strategies or policies. Analysis of the database made clear that the project team had engaged with 568 entities in their efforts to assist employers. This suggests 29% of employers approached took up the offer of support in developing or improving their equalities and diversity strategies/policies. A summary of employer engagement recorded within the Enterprise Database at 31st August 2014 is presented in table 3.11.

_____________________________________________________________________________

32

Table 3.11 Project engagement with employers Characteristic Number At least one female owner 38 At least one male owner 73 At least one BME owner 82 At least one non-BME owner 26 At least one disabled owner 5 At least one owner with a work-limiting health condition

4

Contact understands/speaks/reads/writes Welsh

1

At least one owner in the age range 16-18

0

At least one owner in the age range 19-24

1

At least one owner in the age range 25-49

66

At least one owner in the age range 50+

18

Start-up organisation 4 Social enterprise 1 SME 14 While table 3.11 suggests a diverse range of employers engaged from the perspective of gender, ethnicity and age, plus representation from disabled people, little can be concluded beyond these broad, qualitative findings because of the significant proportion of incomplete records. As an example of the data quality, the database records that 14 SMEs were assisted. A manual check of the employer assisted records was undertaken by the evaluation team. Consideration of business name and sector revealed a substantial number of independent retailers, restaurants, takeaways and specialised small businesses. This led the evaluation team to estimate 122 SMEs were assisted. More will be said of the project’s engagement with employers later. These latter findings lead us naturally to an examination of the evaluation findings around project processes and structure.

_____________________________________________________________________________

33

4. Project Structures and Processes The MAWRII evaluation specification included a requirement to examine project systems and processes, with a view to assessing the effectiveness of governance and implementation structures and to look at how consortium partners have worked together. In order to collect data on these themes, a series of qualitative interviews were conducted with key members of the project Management Team, as well as Key Practitioners and Support Staff from each of the consortium partners. The researchers also spoke with a series of external stakeholders who were identified as having an informed perspective on the work of the project and the wider third sector in Wales. In addition to these qualitative interviews, a File Review Visit was undertaken with each partner, to look at how delivery organisations have been capturing/recording/reporting beneficiary data. The purpose of this strand was not to duplicate any internal audit activity, nor to replicate the compliance testing undertaken at PIV; but instead to assess project processes and the consistency of approach across the consortium. 4.1 Project Consortium Arrangements The MAWRII project was designed around a consortium partnership model, with Joint Sponsors delivering services across the Convergence area, and a Lead Sponsor taking responsibility for overall project management, coordination and reporting to WEFO. It is of course important to recognise the significant changes that have occurred to the consortium since AWEMA originally set it up in 2009. The ensuing period has seen three of the original joint sponsors drop out of the consortium and the Lead Sponsor role changing hands twice. These changes have had a big impact on the delivery of the project. Our interviews sought to explore the circumstances around these events and understand how changes were mitigated and managed. Whilst the focus of this evaluation is on the MAWRII project, the research has considered the earlier MAWR phase as part of the wider contextual work around processes and governance. 4.1.1 Initial Set-up: MAWR and AWEMA The original Minority Are Wales’ Resources (MAWR) project was led by AWEMA and involved a wider delivery partnership that included the North Wales Race Equality Network (NWREN), the Valleys Regional Equality Council (VALREC), YMCA Wales and YMCA Swansea. Most of the partners had worked together previously and had a strong background of working with equality issues and ethnic minority communities across Wales. AWEMA had been actively involved in earlier EU projects, and as such were seen as a natural choice to lead the development and delivery of the MAWR project, as

_____________________________________________________________________________

34

well as two other related bids funded through separate Priorities of the Welsh Convergence Programme. Swansea YMCA and YMCA Wales were slightly different to the other partners in that they did not have an exclusively ‘equalities/ethnic minority’ remit, and had not worked extensively with the other consortium members prior to the project. Swansea YMCA recalled that they were pleased when they were asked to participate:

“The project chimed with our core mission of working to improve opportunities for local communities to access education, training and wider support”.

The make-up of the consortium was therefore designed to provide a good geographical spread across the Convergence Region, and ensure a complementary mix of skills to support the delivery of the project. The strong reputation and track record of the partners in working with BME/EEM communities was also seen as a critical element in the effectiveness of the delivery model. Successful engagement with traditionally hard to reach communities is often reliant upon trust, and it was hoped that this blend of partners could make a big impact. All of the current consortium members recall that they felt a strong commitment to the core project rationale from the outset, and believed that it was built on a strong and evidenced case for support of this kind. Typical comments from interviewees included:

“There was a huge need for a project of this kind; generic training projects cannot reach these communities as the providers don’t have the detailed understanding of the client groups.” “The partners involved were known to the target communities and had good credibility with community leaders. Trust is an important commodity when working with BME communities; without it you will struggle to get people to engage.” “We really believed in the project and were confident that we would be tapping into a massive pool of need.”

The development work around the core project model and bid(s) was reportedly very much driven by AWEMA. Due to their experience of working with EU funded projects, they set the agenda for the project, consulting with prospective partners periodically as the bids came together. Other Joint Sponsors were then signed up to participate and allocated roles, budgets and target profiles as part of the project approval from WEFO.

_____________________________________________________________________________

35

This formative stage of project development is critical in any project, particularly in terms of ensuring a shared, common understanding of project rationale, objectives, delivery targets, budgets, and project processes. When asked about these early stages, current practitioners/consortium members reflected that, with hindsight, there was perhaps a lack of clarity over some aspects of how the project was going to work. Joint Sponsors understandably placed trust in the Lead Partner to ensure that project design and initiation were carried out effectively and that processes were put in place to support the core aims of the project. On reflection, interviewees felt that a more collaborative approach to developing the project would have been helpful, particularly in light of the later developments with AWEMA. Comments on this phase included:

“AWEMA took the lead on developing the business plans for the various strands of the project, and with hindsight, the role of the other partners was probably too passive. It subsequently transpired that this was very much AWEMAs approach to consortium working – they were quite controlling, and as such it would be fair to say that partners were not as aware as they could have been about how the project was set up, and how it was supposed to work”. “It’s clear that there was a lack of clarity and understanding of how the project was supposed to work, the targets, outcomes and the evidence required to demonstrate and verify achievements. That is not to say that good services and outcomes weren’t being delivered, simply that governance and management arrangements were not sufficient”. “On reflection, more work should have been done to ‘strength test’ the Lead Sponsor and scrutinise the delivery model before joining the consortium. However, this is not easy to do, and there was little guidance on how to do this at the time”. “The project had a clear rationale and this was shared by partners from the outset. However, it has become evident over time that individual partners had a variable understanding of the detailed requirements of running an ESF programme of this sort”.

The wider difficulties experienced by AWEMA are well documented and led to WEFO terminating the project agreement with AWEMA following the Welsh Government’s investigation report which identified significant management, financial, and governance shortcomings. This led to a complete overhaul of the

_____________________________________________________________________________

36

project, and the consortium having to work closely with WEFO to safeguard project delivery and ensure continued support for the target beneficiaries. The project subsequently underwent a major re-profile and reconciliation exercise. Moreover, the decision was taken to ‘rebadge’ the project as MAWRII, to emphasise that both the consortium and the project were moving into a new phase. 4.1.2 The MAWRII project – VALREC as Lead Sponsor Following the departure of AWEMA from the project, the consortium entered into detailed discussions with WEFO to look at future delivery options. It was agreed that the Lead Sponsor role would pass to VALREC, with other partners remaining as Joint Sponsors. The first step for this new Lead Sponsor was to undertake a detailed assessment of activities ‘to date’ and consider the appropriateness of governance/management structures. VALREC reportedly attempted to simplify some of the paperwork and also altered the way/frequency that partners fed in their progress against targets. One practitioner noted:

“AWEMA forms were hard to use and asked some irrelevant questions. When VALREC took over, we started from scratch, and scrapped everything except the enrolment and equal opportunities form”.

Partners reported that VALREC made strong attempts to develop collaboration within the consortium, running a number of Steering Group and Practitioner sessions to allow shared learning, discussion of progress, and the identification of challenges. One consortium partner noted:

“VALREC did a good job of bringing the consortium together and ran some excellent Steering Group Meetings”.

Another interviewee noted that:

“Steering Group meetings were very well organised. Practitioners were invited, we often had guest speakers on a particular topic and WEFO attended to provide feedback”.

The project then proceeded with VALREC as the Lead Sponsor until summer 2013. At this point, VALREC began to experience some challenges around the speed of ESF grant payments from WEFO. They then took a decision to notify WEFO that they wanted to stand down as the Lead Sponsor, whilst continuing delivery as a Joint Sponsor. This decision by VALREC was pre-emptive in nature and aimed to prevent cashflow becoming a problem for the wider consortium. One of the other partners described this as:

_____________________________________________________________________________

37

“A bold and mature move”.

Nevertheless, the decision represented a further significant change to the configuration and momentum of the MAWRII project. Following a further period of negotiation with WEFO, it was decided that Swansea YMCA would step in to take on the Lead Sponsor role and take the project forward. 4.1.3 Swansea YMCA takes on Lead Sponsor Role Swansea YMCA took over the leadership of the project in September 2013. Even before this date, during August 2013, their first action as Lead Sponsor was to work with VALREC to initiate a full and detailed audit and reconciliation of the project. This involved close working with both WEFO and consortium partners to collect data, audit files, discuss options and agree next steps. One of the Lead Team commented: “This was the only way we could get a clear perspective on where things stood, and what needed to be done”. To facilitate this work, Swansea YMCA put forward a case to WEFO for a project extension, based on the need for Swansea YMCA to carry out a ‘mobilisation phase’ before pushing ahead with delivery. This extension was agreed in March 2014, along with an increased budget and target profile. Once this was settled, Swansea YMCA proceeded to issue Service Level Agreements to partners and began introducing new guidance, documentation and processes to underpin the ongoing delivery of the project. During the audit of August 2013, it was decided that YMCA Wales would withdraw from the project. They had initially been earmarked to carry out the employer engagement work on the project. However, when Swansea YMCA tested the ability of YMCA Wales to carry out this work and achieve their targets, it was decided that they should withdraw from the consortium. This change led to the project having to alter its approach to meeting employer engagement targets. This caused some initial challenges for partners, in terms of having to take on greater targets. The final change to the consortium came when Swansea YMCA was notified by WEFO that all payments to MEWN were to be frozen (April 2014). This eventually led to MEWN being removed from the project, with the organisation going into liquidation shortly afterwards. Again, this caused significant issues for the project and meant that the team at Swansea YMCA found themselves having to devote many months of effort to manage the situation. This was

_____________________________________________________________________________

38

reportedly an extremely difficult period for both Swansea YMCA and the wider consortium. The MEWN story generated negative publicity for the project and exposed the consortium partners to significant financial risks. Perhaps most importantly, it introduced uncertainty for project participants and potential participants and hit the momentum of the project. Despite the scale of these challenges, Swansea YMCA and partners successfully managed to mitigate the risks and work through the changes with WEFO. The team at Swansea YMCA have won strong praise for the way they handled this work, both from consortium partners and external stakeholders. The consortium has continued delivering services to target beneficiaries and is currently in the process of managing project closure. More detailed comments on the Governance and Leadership provided by Swansea YMCA are detailed in the next section. There are also some important lessons to be drawn from the experiences of partners in terms of how the consortium model has worked. We will reflect upon these in more detail in our Conclusions and Recommendations. 4.2 Governance and Communication The changes described above have had a direct impact on delivery and are a feature of the wider governance of the MAWRII project. Each of the Lead Sponsors has sought to run the project in a slightly different way. Interviewees were able to comment on the strengths and weaknesses of each approach and the issues encountered. It is also worth recognising that all projects (especially those involving consortia) experience change, as partners become used to working with each other, as well as becoming more familiar with the detail and challenges of implementing the project. The initial phase of the project saw AWEMA taking a strong lead on project Governance, which was in keeping with their role as the Lead Sponsor. This period covered the early days of the project, when partners were busy recruiting staff and putting in place the systems needed to deliver the project in their area. Under a consortium model, one would expect to see project partners being heavily reliant on a Lead Sponsor during this set up phase. Partners require clear guidance on how to ensure compliance, maintain records, manage budgets and report expenditure and achievements. Again, whilst it is important to stress that the AWEMA phase (MAWR) sits outside the scope of this MAWRII evaluation; consortium partners were keen to comment on this formative phase. It was felt to provide important context that will help identify lessons to inform the development of future projects.

_____________________________________________________________________________

39

Current consortium members and practitioners were generally critical about the quality of governance and communication under AWEMA. Governance was not seen to be clear nor collaborative. Current consortium members reported a lot of confusion over project targets, budgets, record keeping and reporting requirements. Interviewees were also critical of some of the wider project processes, with project paperwork, reporting and weak communication all coming in for criticism. A current consortium member commented: “There were difficulties in working with AWEMA. There was a lack of clarity on eligibility; indeed, there was very little information from AWEMA, which was very frustrating. While AWEMA required a huge amount of documentation from partners, it provided very little in return. Things have improved massively since those early days”. Comments from interviewees on the governance provided by AWEMA varied, but were in the main critical of the approach adopted. A minority of practitioners did offer some positive comments, suggesting that AWEMA had at least been clear in their expectations and the way they communicated with partners. Example comments are set out below:

“The processes that underpinned the project were clearly patchy and had often been left to the individual partners to interpret and sort out. There was a lack of strong governance from the centre and common understanding of requirements”. “There was a big difference in the data that AWEMA had told partners to collect, and what was really required for an ESF project of this sort…There were difficulties in working with AWEMA…There was a lack of clarity from the lead organisation on eligibility - indeed, there was very little information from AWEMA on all issues…which was very frustrating”. “AWEMA’s approach to consortium working was quite autocratic, and as such it is fair to say that partners were not as aware as they should have been about how the project was set up, and how it was supposed to work. As a result, problems and issues began to emerge as we moved forward with delivery”. “The clearest period of the project was during the AWEMA phase. AWEMA, were all about data and there were lots of forms to record different aspects of delivery. There was some duplication, and it was admin heavy, but you could see the sense of it and it separated out all the data that need to be collected”.

_____________________________________________________________________________

40

In addition, interviewees noted that there were some fairly fundamental misconceptions in play at this time over core aspects of the delivery model. Practitioners reported not knowing that they had dedicated budgets to procure training delivery, which again illustrated inconsistent knowledge and communication across the project. One practitioner summarised this as follows: “Looking back to the beginning of the project, partners weren’t aware that they had a training budget to deliver/pay for training themselves. As a result the project focused on signposting and referring to other training provision. This changed after the transfer to (Swansea) YMCA, which was great, as we were then able to start running courses and directly claiming the outputs/outcomes”. Following the handover from AWEMA to VALREC, the revised project took the opportunity to re-examine its systems and processes, and inevitably VALREC brought a new governance approach to the project. One of the first changes introduced was a rationalisation of the project paperwork, which was felt to have been unwieldy under AWEMA. A number of project forms were altered or decommissioned, with a view to stripping out duplication and better aligning data collection with what the project needed to (a) be compliant, and (b) provide the richness of data to effectively report achievements to WEFO. Whilst interviewees were generally positive about the changes introduced during the time VALREC was Lead Sponsor, most reflected that the consortium had perhaps not appreciated the pressures that running the project would place upon a relatively small third sector organisation such as VALREC. The administrative and financial burden of running ESF projects (especially on behalf of a wider consortium) should not be understated. A number of interviewees commented that VALREC probably did not have the capacity nor direct experience of managing EU projects, to take on this role. One external stakeholder commented.

“I felt that transferring to VALREC could be problematic as it is a small organisation without the necessary infrastructure to cope with the demands of EU funding”.

Other comments included:

“VALREC is a small organisation and so lacked capacity to run such a big scheme - the requirements of WEFO and ESF took their toll. ESF is designed for large organisations but working with hard-to-reach groups generally requires small specialist organisations. So the system is geared against smaller organisations”.

_____________________________________________________________________________

41

“The financial infrastructure of VALREC was tested by the ability to oversee the project on behalf of the consortium”. “We struggled at times to get clear answers from VALREC and as a result, partners seemed to start interpreting things their own way and doing their own thing”.

A big element of the challenge for VALREC of course was of course dealing with the issues inherited from AWEMA. The subsequent exercise to fully reconcile and strength test the project by Swansea YMCA identified some fundamental misunderstandings and issues had been in place since project inception. Whilst VALREC clearly made a conscious effort to improve governance and communication during their time as Lead Sponsor, their own capacity constraints perhaps meant they were not able to really get to the heart of some of the detailed improvements that were needed to transform the health status and performance of the project. The next phase in the governance of the project came when Swansea YMCA took on the Lead Sponsor role. Despite not having had significant experience of running EU projects, the organisation made a strategic decision (with the support of their Board) to invest time and resource into taking on the leadership and tackling the challenges and issues that had frustrated progress in earlier stages of the project. The team were able to draw upon their experiences of taking over the Lead Sponsor role on the P1 project, where they had also undertaken a detailed reconciliation process. The first step for Swansea YMCA was to undertake a full Health Check of the project. This involved examining all aspects of process and implementation, indeed, undertaking a 100% audit of consortium partner files. This work was inevitably very resource intensive and required a lot of coordination between Swansea YMCA and partners. As one practitioner recalled:

“Nothing was taken for granted and the YMCA went through the books with a fine tooth comb”.

This work also tested the institutional capacity, skill set and expertise of Swansea YMCA themselves. Whilst they had cut their teeth on the P1 project, the work on MAWRII required a lot of internal upskilling and consideration around how best to identify and address issues in order to manage delivery in a successful way moving forward. Reflecting on this, interviewees from Swansea YMCA noted that the task was like “having to jump on a moving train”. The

_____________________________________________________________________________

42

project had already started, and there were a range of issues and problems that needed to be understood and tackled quickly to allow the project to push ahead. Despite this, Swansea YMCA feels that that the organisation as a whole have benefitted from the test and learning encountered during this period. The experience is seen to have helped grow the ability and capacity of the organisation to manage complex projects such as this one. New skills have been developed, stronger processes put in place, the right staff recruited and best practice models put in place. Many of the conventions developed for this project (Desk Notes, Risk Registers, Financial Controls, etc.) are now being used across the organisation and on other projects. As one senior manager noted

“The experience has helped the YMCA become a stronger organisation and has been a catalyst for growth and improvement across the board. Staff feel that they can cope with anything having managed the challenges of the past three or so years, and they now have genuine expertise and capacity to run these project successfully –so a lot of positives have come out of the change process. Also, it is important to remember that the projects are doing good and delivering positive benefits for participants”.

The majority of consortium interviewees were extremely positive in their assessment of how this work was handled and the impact it had on the progress of the project. A number of changes were made to the way that the project operated. New information was provided to partners on their budgets, targets and how information was to be collected and reported. Refinements were also made to the database and aspects of the delivery model, especially around employer engagements. As part of the reconciliation process, WEFO insisted that a list of Approved Training Providers be drafted. This generated a lot of work for the Lead and consortium members. Putting this process in place, caused delays and challenges in sourcing training in some parts of the country, especially North Wales, where the availability of specialist training providers was already limited. While the additional targets and aspects of the work around employer engagement did present challenges for some partners, there was a marked difference in the way the project was being led, which was welcomed by the majority of interviewees:

_____________________________________________________________________________

43

“Swansea YMCA took the learning from P1 and put in the correct systems for P3. Staff related that Swansea YMCA put in place good structures, processes and communicated well”. “Since the YMCA took over the lead role, they have brought in a lot of positive changes. There is far better communication, clearer guidance, good budgeting and spending requests are answered immediately with justification for the decision”. “YMCA brought a completely different level of rigour to the management of the project, and this came as a great comfort to Practitioners, who had been working in great uncertainty up to that point”. “They were brave in taking on the project but have done really well. They’ve done themselves a good service, and done an excellent job”.

The team at Swansea YMCA were also faced with having to contend with ongoing changes to the consortium, with first YMCA Wales withdrawing from the project, then MEWN having their funding frozen, leading eventually to them withdrawing from the project and going into liquidation. Coming on the back of the AWEMA situation and subsequent change of Lead Sponsor, this presented another major issue to mitigate and be managed. Staff at Swansea YMCA report this as having been an extremely testing time, with negative publicity and uncertainty impacting negatively on the momentum of the project and morale in general. Nevertheless, extensive work was undertaken with WEFO staff to overcome the challenges, profile the project and ensure continuity of provision (especially for MEWN beneficiaries). Again, consortium members felt that the team at Swansea YMCA handled this period very skilfully and demonstrated their resilience as a Lead Sponsor. That said, all members of the consortium acknowledged that the period caused further delays and uncertainty, and also meant that much of the Lead Sponsor’s time was monopolised for many months. This in turn limited Swansea YMCA’s ability to undertake wider support activity. One area where the MEWN issue was seen to have had an impact on governance and communication was the infrequency of Practitioner Meetings. These were seen as an important mechanism for bringing consortium partners together to share experiences and develop common approaches to delivery. Whilst the Lead Sponsor Team were reportedly very active in terms of travelling out to partners to deliver guidance and training, there was a gradual decline in Practitioner Meetings. One interviewee noted:

_____________________________________________________________________________

44

“An opportunity has been missed to share best practice between partners, and for (Swansea) YMCA to initiate regular meetings between practitioners”.

Staff at Swansea YMCA acknowledge that this was an area where they had planned to do more. They too believed that these meetings reinforced ideas and processes, helped sustain productivity, improved communication and facilitated more effective peer-to-peer learning. In fact, the Lead Team claimed to be able to spot noticeable upturns in performance after each Steering Group/Practitioner meeting. A team member from Swansea YMCA noted:

“The MEWN issue meant that the team were dragged away at a key stage to deal with a potentially disastrous situation for the project. This took up all of our time and means we couldn’t do as much work as we would have liked in bringing consortium partners together”.

One partner was more relaxed on this issue, pointing out:

“I was happy that the Lead Team came out to see us, as practitioner meetings used to take three days out of my month”.

Following the resolution of the MEWN issue and the subsequent re-profile of the project, the consortium has been able to move into a more settled phase of delivery. The new systems and processes that were introduced by Swansea YMCA have become embedded, with partners being generally happy with the way governance and communication on the project has improved. The Lead Partner and Consortium Members highlighted the positive role played by WEFO, and other support organisations such as the WCVA, in helping sustain the project through challenging times. Staff at WEFO had been “accessible” and “understanding” of the pressures that unplanned changes had placed upon delivery partners. One practitioner noted:

“This project has taken WEFO way outside their comfort zone but they worked hard to find solutions and ensure that the project was able to continue delivering services for hard to reach groups”.

In addition, the SET Team at the WCVA were praised for their support to the project:

_____________________________________________________________________________

45

“They have a high level of expertise and are always willing to help and act as a critical friend for the project. I very much hope their role is continued for future Programmes.”

Swansea YMCA are currently coordinating the project closure phase and drawing together the data to evidence what has been delivered and achieved. Quality assurance visits are being undertaken with all partner to go through paperwork in detail, to test compliance and to ensure that all outputs and outcomes are effectively captured and reported. The next section looks in more detail at how the monitoring and reporting aspects of the project have worked and sets out the findings of File Visits undertaken with partners. 4.3 Monitoring and Reporting 4.3.1 General Perceptions As with any project in receipt of ESF support, the MAWRII project has had to comply with monitoring and reporting requirements set out as part of the terms of their grant agreement with WEFO. Applying these rules and requirements to complex projects can be demanding, especially when projects are run across a multi-partner consortium. Success in this area of project management requires a sound understanding of EU rules and regulations, as well as sufficient leadership, resource and experience to effectively build these conventions into the design and implementation of the project. Much of the responsibility for driving this work inevitably falls to the Lead Sponsor. However all partners clearly have a responsibility to buy into the process and ensure they comply with requirements. Clarity is key - unless processes, systems and expectations are understood from the outset, issues will inevitably arise. The preceding section outlines the challenges encountered during the early stages of the project. These issues can be seen to have manifested themselves in the way that project data was captured, recorded and reported over the course of the project. Based on the accounts given by consortium partners, it would appear that many of these deficiencies were not fully unearthed and rectified until Swansea YMCA undertook a full audit and reconciliation exercise. Whilst some improvements were achieved under the leadership of VALREC, it was not until Swansea YMCA unpicked the whole project, that some of the more fundamental issues came to the fore. Practitioners recounted some of the issues identified, which included:

_____________________________________________________________________________

46

• Missing signatures • Poor data sets • Lack of NI numbers, so hard to prove employment • Duplicate participant numbers • Lack of evidence of meaningful participant engagement • Missing dates, addresses, contact details etc • Ethnicity not completed • Poor database management

In response to this, the Lead Sponsor highlighted to partners where remedial action was needed and issued revised guidance around what information had to be collected for beneficiaries at every stage. This was communicated to partners via desk notes and guidance documents. Moreover, the Lead Sponsor visited partners to ensure that they received the message ‘first hand’. Much work was put into revising the Participant (Beneficiary) Database to ensure it covered everything the funder required, as well as including some additional categories to capture wider outcomes. A strong emphasis was placed on making the database user friendly and building in ‘verification features’, so that partners were alerted if they incorrectly/partially entered data. The database was modelled on the requirements of the WEFO online database, to help keep data collection relevant, and to limit the likelihood of verification errors when submitting the data to WEFO. In terms of how the data reporting processes has worked, the Research Officer role at Swansea YMCA has been key to coordinating and collating the collection of data. Partners are asked to complete their beneficiary database on a monthly basis and return it to the Research Officer. This data is then migrated into the Master Database and additional verification checks are carried out. If any issues are identified, then these are communicated to partners and they are given time to rectify, or in extreme circumstances remove the beneficiary from the project return. Partners are also asked to provide beneficiary case studies to add depth to the data collected. Once data has been received, analysis is undertaken and management information is passed to the Project Manager so that progress can be reviewed. Partner returns are also used to complete quarterly claims and facilitate the payment of grant. Consortium members were overwhelmingly positive about the work that had been done around clarifying and strengthening monitoring and reporting processes. Systems were now viewed to be working well. Partners felt generally that their queries were dealt with quickly. There were some issues raised around increases in targets but these were linked more to the project re-

_____________________________________________________________________________

47

profiles/extensions and some of the misunderstandings that were inherent in the earlier stages of the project. Staff at Swansea YMCA were able to reflect that their work on monitoring and reporting on MAWRII, had been challenging, but ultimately effective. One staff member noted:

“We were effectively in a position where we had to explain to partners what they needed to be doing, with only a third of the project remaining. It was important to be thorough, but also pragmatic and conscious of the pressures on delivery staff. Everybody has responded well and there is good compliance now across the project”.

4.3.2 File Visits As an additional feature of the MAWRII evaluation, consortium partners were visited, so that a sample of their beneficiary and employer files could be inspected for quality and consistency. This was not an audit, so the focus of this work was not strictly to look at compliance, though this was considered when looking at files. Greater emphasis was placed at looking at the structure and logic of files, the depth and quality of information recorded, and the level of consistency across the project, over time and between the different partners. No files relating to AWEMA or YMCA Wales were reviewed as these were felt to be beyond the scope of the evaluation. However, MEWN files were assessed, since arrangements were put in place to continue supporting these project participants under the auspices of Swansea YMCA. In advance of each file visit, partners were sent a sample of file numbers, selected at random from the project database. This sample amounted to approximately 10% of files across the project, covering both beneficiaries and employers records over the full timeline of the project. As would be expected of a project that has undergone numerous external and internal file audits, the level of compliance across files and consortium partners was strong. Files have a reasonably consistent structure within organisations, with files divided into clear sections. Each file tends to be built around the key project documents issued by the Lead Partner. It is important to recognise that these documents have changed markedly over the course of the project, so the forms used at the beginning of the scheme are different from those used further into the project. The various iterations of the paperwork all have their respective merits and shortcomings. For example, the Activity Plan introduced by AWEMA was a lengthier, more detailed document, but many felt that it was unwieldy and

_____________________________________________________________________________

48

contained irrelevant or duplicate questions. It was decided that this paperwork needed to be revised, so a ‘leaner’ Enrolment and Consent Form was developed – this was seen to be more efficient, but then at the same time offered less space for detail on Skills and Analysis. Ideally issues around the appropriateness and effectiveness of project paperwork would have been thrashed out early in the project – but clearly this element of the project was caught up in some of the wider collaborative challenges. It was clear from reviewing files from different consortium members that the personal style of individual practitioners had a strong influence on the level of detail collected on beneficiaries. Some files contained the core compliance information, along with certificates of qualifications achieved but minimal narrative on the individual journey of the beneficiary. Other consortium partners were at the other end of the scale, with extensive detail on the beneficiary, the support provided, correspondence and any wider issues encountered. This difference in style was not a problem in itself, as both approaches cover the core compliance requirements. What it may illustrate is the degree of flexibility that partners were given to set their own protocols at different stages of the project. There was also some inconsistent usage of the core forms on the project over time. Part of this appeared to be down to the changes introduced by the different Lead Partners and the fact that there may have been minimal guidance to practitioners on what forms were really important and what forms were simply there as a guide or support tool. Again, it is important to stress that files appeared to be fully compliant, containing the basic information that would be expected. One form that has been used in a variable and inconsistent way is the Beneficiary Evaluation Form. Initially AWEMA requested that practitioners use one of these after every intervention. However, practitioners tended to view this as impractical. As a result partners decided to only use them at the end of a participant’s journey on the project. The only problem with this approach arises because partners have tended to keep beneficiary files open. Consequently, the evaluation exercise is left until the end of the project, by which time beneficiaries may have disengaged or changed contact details. WEFO was content with this approach. Indeed, the logic is understandable. However, in retrospect, an alternative approach perhaps could have been devised to ensure timely capture of feedback. The Lead Sponsor is currently working with partners to collect evaluation forms, as part of the wider project closure. As we have noted earlier, the lack of this management information during the evaluation has only allowed tentative findings in the impact of courses accessed.

_____________________________________________________________________________

49

Employer files were dealt with differently across the consortium. In some cases all employer records were kept together on one file, whereas with other partners, employers were allocated individual file records. In cases where employers had engaged with the equality policy support, or had themselves accessed training, records were more comprehensive. Overall, the quality and consistency of files can be seen to have improved over time and, in the main, have mirrored the changes that have occurred within project governance. There are many features of good practice evident, especially during the latter stages. For example, the use of logs to record interactions with beneficiaries helped clarify what has been done and achieved. Similarly, some partners have made use of summary sheets at the beginning of each file, to enable quick access to the key features of each beneficiary. There are other examples of positive practice that should be captured as part of the project closure exercise, especially where these have come from the ingenuity and creativity of individual practitioners. Where best practice can be identified, this should be shared and adopted for future delivery. In terms of less positive features, many of these can be linked to the wider issues encountered at various stages in the project. With improvements to guidance and leadership, came corresponding changes to the quality of files. Many of the deficiencies evident in early files can be seen to have been resolved as the project has progressed. There were also some significant differences in the volume and depth of information held between different consortium members. Whilst this isn’t a compliance issue per se, it is important that files accurately convey the journey of a beneficiary in a way that is compliant and efficient. Greater consistency could have been achieved and partners might consider how to take this forward in future projects. Some of the relative strengths and weaknesses of the files sampled are set out below. These lists are not designed to be exhaustive. They do, however, provide some learning points for future consideration: Strengths

• Files logically structured; • Strong compliance levels apparent; • Use of Project Logs provide effective way of recording frequency, length

and nature of interactions; • Summary sheets allow reader to quickly access key information; • Guidance and clarity of definitions appear to have improved over time; • Individual practitioners given flexibility to stamp their own style on files,

with signs of innovation in places;

_____________________________________________________________________________

50

• Some of the wider data captured adds value in terms of telling the story of the beneficiary and the impact that the support has had in their lives;

• Files largely correspond to the beneficiary database. Weaknesses

• Some of the issues encountered in the early stages of the project are reflected in the way that files have been kept and the quality of information recorded;

• Many changes to project paperwork over the course of the project, with some forms dropping in and out of usage periodically

• Insufficient and inconsistent use of the Evaluation Form. Future projects could consider better methods of capturing evaluation data, as part of the wider effort to obtain feedback and identify outcomes;

• Wide variances in the amount and depth of information between consortium partners/practitioners. Whilst not a compliance issue, achieving a greater consistency would be preferable and more efficient;

• The level of skills analysis evidenced on some files was minimal; • Following on from above it was evident; in speaking with practitioners,

that they were not aware of how their counterparts in other organisations kept their files. Therefore, providing practitioners with opportunities to share approaches and develop solutions to common challenges should be considered for future projects.

What can be seen through looking at files, and by speaking with project practitioners is the importance of clear guidance and strong governance from the Lead Partner on all aspects of project delivery. The quality of the recording and monitoring of data on a project, are only as good as the guidance and processes that are put in place to support project delivery. Improvements to this aspect of the MAWRII project can be seen to mirror the wider progress and achievements of the project team and consortium.

_____________________________________________________________________________

51

5. Project Delivery – Results and Impact So far, we have primarily considered indications of the results and impact of the project as shown in the management information. Now we move to the perspectives of the project participants and the project practitioners. The structure of this section follows the five stages of project delivery:

• Identifying underlying need and recruiting beneficiaries (individuals and employers) - Locating and approaching potential beneficiaries - Registering the individual or employer and broadly understanding

needs

• Individual needs analysis - Understanding the detailed needs of each individual and employer

and the challenges they face

• Training design - Matching client needs to mentoring, support, tailored or publically

available training

• Training delivery - Delivering training and support appropriate to needs and project

targets

• Outcomes - Demonstrable outcomes: qualifications, increased confidence,

entering job market, promotion, adopting equality and diversity strategies, showing staff they are valued.

The evaluation exercise included a series of 30 individual, semi-structured interviews with staff from delivery partner organisations (practitioners, managers and CEO’s), external stakeholders and a sample of project participants (beneficiaries), plus 8 participant focus groups. In total, 48 individual beneficiaries and 7 employers contributed to the evaluation. The qualitative data and findings collected from these interviews/focus groups has been presented thematically in this section, and used to inform the overall conclusions and recommendations of the evaluation.

_____________________________________________________________________________

52

5.1 Identifying Underlying Need and Recruiting Beneficiaries “The project will support people from Black and Minority Ethnic (BME) including European Economic Migrant (EEM) backgrounds to enhance their skills to obtain, maintain and progress in employment. Through engagement with the target group and working simultaneously with employers, the performance of businesses will increase”. (Business Plan, Minorities are Wales’ Resources 1st March 2012 – 31st December 2014, YMCA Swansea). Practitioners recalled that they drew on their experience and knowledge of their area and community to identify initial need and start individual participant recruitment. Practitioners tended to trade on existing contacts and on their organisation’s reputation within communities. The geographic distance between Project Sponsors (except MEWN and Swansea YMCA) also meant that Practitioners could recruit from independent target areas. This helped to avoid duplicated effort, identification and approaches to the same potential participants, although there was some early competition as the project bedded in. In most cases beneficiaries found the regional bases to be well-located and accessible by public transport, which was considered very important for initial needs analysis sessions and ongoing one-to-one support from Practitioners. Some training was delivered at the regional bases, but in the main delivery took place through outreach within the community. Word-of-mouth was considered by Practitioners to be the single most important tool for identifying need and registering participants. Individual participants spoke of how they recommended the project to acquaintances in their community, at their workplace or in their neighbourhood. Beneficiaries also had a tendency to recommend individual Practitioners to their friends or colleagues, if they had experienced a positive interaction. This acquired trust between beneficiaries and Practitioners became a powerful currency on the project. As a consequence, individuals often contacted Practitioners directly to enquire about training and support. Case Study

In 2013, a couple from Portugal, working in a factory in Wales, decided they were interested in setting up a business of their own and “a friend of a friend who owns a takeaway” told them that (named Practitioner) at (Project partner) had helped him set up his business. They rang the Practitioner in February 2014 year, indicating that they wanted to start a small bakery business. They met and discussed the couple’s ideas, plans and aspirations.

_____________________________________________________________________________

53

“(The Practitioner) has helped us understand what it takes to open a business in the UK. We were helped in filling in forms, attending meetings with the Council, getting our home kitchen authorised to prepare foods to sell and in understanding and registering with HMRC. (The Practitioner) was especially helpful with our first Tax Return in April this year. Our business is small now but it is started and growing and we see opportunities for the future”. Practitioners supplemented their existing knowledge of local communities through research to identify and approach other networks and community groups in their region. Efforts were clearly made to diversify the range of nationalities and ethnic groups engaging with the project. As noted already from the review of the management information, these efforts were largely successful. The project was promoted by Practitioners through visits to community centres, libraries, mosques, colleges, jobs fairs, businesses, shops and other organisations in areas with a known BME and EEM population density. Practitioners gave short project presentations and had conversations with clients at these destinations, leaving their contact details including mobile phone numbers. Practitioners also advertised the project in the early stages by distributing leaflets in public places. A view soon emerged, however, that leaflets were not an effective promotional approach for potential clients with low levels of English language capability. Moreover, there were also high numbers of other projects distributing leaflets, leading to potential participants becoming de-sensitized to this form of promotional effort. One Practitioner recalled an initial leaflet being produced by their Sponsor organisation. However, the leaflet did not meet ESF requirements and had to be withdrawn. The Practitioner suggested this could have been avoided simply by the Lead Sponsor issuing a template for Partners to adapt to local needs. There was a general consensus from Practitioners that a longer lead-in for marketing and recruitment, with some centralised co-ordination of promotional resources would have been valuable. However, Practitioners recognised that their individual experience and contacts were the most critical factor in promotional successes. In addition to approaching individuals to inform them about the project, Practitioners cited the benefits of identifying and approach a “leader”, elder or spokesperson from a given community who could facilitate further contact. A

_____________________________________________________________________________

54

specific example was cited for the Nepalese community in Wales. Practitioners were able to ask this person’s advice on any community-specific issues and how best to approach them. By the same token, the community leader was able to promote the project to the benefit of individuals within that community. The experiences of beneficiaries at one focus group serves to illustrate the different ways they came to the project:

• Two Polish beneficiaries were at a local FE College taking an evening English course - a Practitioner visited the course to explain the project;

• One Bangladeshi client heard about the project partner from the British

Council, having taken a volunteer job there, so was “in the know” when the project launched;

• A Romanian participant was walking past the project partner’s offices

and saw a sign in the window;

• One Turkish beneficiary heard about it from her mother who couldn’t speak English but had been given a leaflet by her employer (for whom she cleaned).

It was common across interviews and focus groups with participants to hear that Practitioners attended English language courses to promote the project or visited a community centre. Word of mouth came across as a strong means of contact.

“If you ask anyone in your community for help, 50% at least will say go to (named Practitioner)”.

Participants recounted that they were keen to seek the Practitioner’s help and ready to take his/her advice because s/he had a reputation among their communities or colleagues as a helpful person and someone to be trusted. Participants were almost unanimous in stating that it would have been difficult to find the training and support they wanted without the support of the Practitioner with whom they worked. The Practitioners’ methods were evidently successful in relation to recruitment of individual beneficiaries, given the numbers successfully engaged. Only one target relating to individuals - female participants - was short of the target (by about 10%). Practitioners took the view that this target would have been achieved without the exceptional situation affecting MEWN.

_____________________________________________________________________________

55

The diversity of ages and inclusion of disabled people within the participant cohort has already been considered within this evaluation. An additional, qualitative observation came through in interviews: Practitioners noted that they encountered clients with mental health issues. They were unsure how many clients were affected by mental health issues and did not record this information:

“People tend to be reticent about declaring this, for personal reasons and perhaps they think it could hurt their career prospects”.

One Practitioner suggested that staff and the project would benefit from further training in Mental Health issues and their impact:

“This is something staff encounter a lot on the ground (anxiety, depression, etc.), and it’s an extremely sensitive and sometimes poorly understood issue within many ethnic minority communities”.

This Practitioner also put forward a view that the use of some of the terms applied to clients, such as “recruitment” and “referral” had negative connotations, implying a transactional relationship, not one that would inspire trust and confidence. It was suggested this could be particularly challenging for a client with mental health issues, but in all cases the relationship between Practitioner and client was critical to the success of the project. Feedback from interviews with individual beneficiaries/focus groups suggested participants found approaches from Practitioners to be helpful, professional, culturally sensitive and appropriate. Moreover, initial engagement with the project was seen as simple, straightforward or easy by participants. They were not deterred by the paperwork, indeed participants typically paid tribute to the enthusiasm and commitment of the Practitioners in helping them sign up to the project and identify which training provision best suited their needs. A female beneficiary from Brazil, who is now running a successful business said:

“In 2010, I had some ideas for the business but had not yet started it. I saw a leaflet about (Project Partner) and support for starting and running a business. I went there and met (Practitioner) who was lively, engaging and positive. This was important as you need other people to be upbeat and encouraging when you are thinking of a big step like going into business”.

_____________________________________________________________________________

56

Practitioners did highlight a lack of clarity in the early stages of the project around the recruitment of non-BME participants. In the first year of the project one Practitioner was steered towards:

“... a 100% focus on BME. But some partners found it difficult to explain to white Welsh people approaching them that they could not be supported. This has never really been a problem for (this project partner). I signpost people to alternatives and I only recall one white Welsh participant wanting to join the project. When VALREC took the lead on the programme, project development officers were told that there is a 20% allocation for non-BME if required. Therefore there should be no discrimination”.

Another Practitioner put forward a view that:

“20% white British participants have been allowed since March 2014 but before this the policy affected employer involvement: Training was declined because the organisation said it could not exclude its white British employees and offer training only to its BME and EEM employees”.

Others said that their role was largely about recruiting BME or EEM participants to the project but that:

“If someone turns up that is white and British, they are not turned away. Anyway there is a 20% allowance. We would look at the needs of the individual and their aspirations”.

While we have observed an emerging picture of individuals recruited to the project through word-of-mouth based upon trust of the Practitioners, there appears to have been difficulty, and some confusion in the early phases of the project, in relation to identifying and recruiting employer beneficiaries. Generally, Practitioners felt there was a lack of central guidance in this area and most felt that they were not appropriately qualified in the area of Employment Equality legislation. Some spoke of having avoided approaching employers. Others tried to approach employers but received little or no response. There was no consistent approach to identifying employer need and encouraging their involvement across the project partners. The majority view was:

“Equality Policy is a specialised area, not one (we) can easily involve employers in”

_____________________________________________________________________________

57

“A different approach is needed – Equality Policy is irrelevant or at least not a priority for the employers I meet, who are mostly sole traders. Also someone more senior should take responsibility for this area of the project”.

In one case, a Practitioner described twice approaching a large organisation about training for their employees but receiving no response. By chance that Practitioner met some employees from that same organisation while promoting MAWRII at a community centre. Six employees from the company subsequently took MAWRII training that was relevant to their work, having heard about it directly from the Practitioner, with the company not involved at all. The Practitioner expressed a view:

“It is about helping the individuals rather than necessarily their employer. (I have not) got anywhere with employers. My approach is about building relationships within communities. As I help participants, I write up their issues in such a way to demonstrate how their employers are being helped with equality and diversity issues”.

This Practitioner was also approached by a client whose employer was providing basic Food Hygiene training which did not lead to a qualification. The company was not interested when approached about accredited training (possibly, the Practitioner thought, “because they are worried about qualified employees moving on to other jobs”). The Practitioner arranged for delivery of an accredited Food Hygiene qualification on a Saturday morning and asked the client who had originally expressed interest to ask interested colleagues in his workplace to contact him/her. The Practitioner was contacted by 20 employees from the company as a result. Another Practitioner had some links with smaller employers, especially local hotels

“...but the large employers, for whom diversity training would be very relevant such as construction companies and leisure parks, are not interested”.

While that Practitioner felt confident in writing a Diversity and Equality Policy for a company it was felt to be a pointless task:

“...if it is a tick-box exercise and sits on a shelf and does not reflect company practice”.

_____________________________________________________________________________

58

By contrast, one Practitioner focused on employer engagement rather than recruitment of individual participants, going out to businesses to explain the project and “selling it to them as a benefit”:

“The main pitch has been to help train their staff in areas that will strengthen their business compliance, as well as develop skills of individuals. (You have to) tailor the training provision offer to the sector/industry by identifying which licenses or training certificates a business might need, and then match provision to this business need. Recruitment can’t be based on a ‘one size fits all’ approach. Some of the employers that the project works with (are not) accessible to more mainstream business support programmes, so MAWRII provides a bridge into these sectors/communities”.

This Practitioner took the view that the training opportunities are very attractive to certain employers (for example, those requiring food hygiene certificates for staff) and had worked hard to “hit the equal opportunities policies target”. The Practitioner described going to a restaurant where the project had already supported training for the staff and presenting the owner with a draft equal opportunities policy:

“You have to sell the benefits of adopting an equal opportunities policy and get them to sign another piece of paper and the target will be secured. You explain to the employer how they can be protecting themselves by having such a policy i.e. selling the benefits”.

It may be helpful to codify this successful process for future reference: 1. Research training needs of employers in a given sector e.g. need for staff to hold basic food hygiene certificates in restaurants/takeaways; 2. Approach employers directly with a compelling training offer e.g. face-to-face offer of free or subsidised training in food hygiene for all staff; 3. Approach employer again after having met training need (and therefore having built trust and confidence). Explain how the employer can protect him/herself and the business by holding and following the correct policies; 4. Provide a draft Equalities Policy and talk the employer through the process of adapting to the specific requirements of their business; 5. Employer signs completed policy.

Case Study The interviewee manages one of five homes run by the organisation. This home specializes in working with young females between the ages of 11

_____________________________________________________________________________

59

to 19 who have been sexually exploited and typically display self-harm. Consequently, staff require basic training in areas such as fire safety, food hygiene and first aid. The interviewee related, “The company is quite small and cash is scarce. Although they want to train staff, funding is often a barrier. The baseline position is that team leaders are trained in first aid. That means in a house supporting four residents, only one displaying self-harm tendencies can be accommodated. The project enabled all of the staff to be trained in first aid. The impact is massive. All three current residents display self-harm and they are to be joined soon by a fourth. The girls are very vulnerable and display a heightened state of behaviour during the first month of their residence as they settle in. First aid is incredibly useful in that type of environment”. “The whole experience of working with (the Practitioner) was positive. It was all very professional. He also provided us with a generic equality and diversity policy. We were able to tailor to make company-specific. It was sent to the HR person in the company, who reviewed and then adopted as policy”

In summary, operational approaches to identifying need varied across the partners and practitioners felt they would have benefitted from a central project recruitment strategy. As well as the “grey areas” early on in the project in relation to targeting employers and the inclusion of non-BME beneficiaries, there were differences in the level of independence and responsibility given to Practitioners: In some cases, a manager might decide to run a course based on discussions with contacts, then ask a Practitioner to organise and recruit to these sessions, without the Practitioner having involvement in diagnosing need or determining what courses might work. In other cases the Practitioner might be given full responsibility for diagnosing need and determining appropriate support. Practitioner interviews suggested the process of identifying and evaluating need could have been improved by central consultative planning and sharing of effective guidelines and templates to be adapted locally. Some Practitioners expressed a view that the project would have benefited from more guidance on training priorities, with training budget allocations flowing naturally from these priorities. Recruitment also involved registering beneficiaries and recording their details. Again, in the early stages of the project, client registration would have benefited

_____________________________________________________________________________

60

from a central model and system. In terms of current project management, monitoring and reporting requirements, practitioners felt the project was now in good order and were clear about what information and evidence they needed to collect and record, but a bespoke central database from the start would have helped the project run more smoothly, saved time and allowed more targeted reports to be run against participant data. Practitioners spent significant time ensuring that files are kept up to date and would have all appreciated more administrative support to help deal with monitoring requirements, to free practitioners to focus on recruitment and delivery. There was a general consensus that since YMCA Swansea took over the lead role, business co-ordination and communications for all stages of the project improved significantly and there was a much improved hub and spoke model in place. One Practitioner said “YMCA brought a different level of rigour to the management of the project, and this reassured us as practitioners”. Despite the complexity and challenges of the project in the early stages, all of the practitioners were highly motivated and committed to the project and its beneficiaries. 5.2 Needs Analysis “An assessment of each participant will be carried out through a skills need analysis which identifies the present and future skills required to progress in employment including the identification of training”. “The project will overcome the barriers that this target group face in progressing in employment by engaging with them through outreach, mentoring, support, group activities, referral to training and involvement of employers. The key essential will be to develop signposting mechanisms and then support participants to other convergence programme projects”. (Business Plan, Minorities are Wales’ Resources 1st March 2012 – 31st December 2014, YMCA Swansea). All of the Practitioners articulated an approach to working with participants that took a holistic view of the person. Therefore, this was a notable feature of the needs analyses carried out by Practitioners i.e. it was wider than Training Needs Analyses typically observed within ESF projects. Practitioners were of a common view that clients’ employment and career needs, ambitions and potential were intrinsically bound up with the challenges of everyday cultural and social issues such as housing, family, benefits, health and childcare. It was very rarely possible or desirable to ignore these issues when analysing clients’ training needs which, it was felt, could not be viewed in isolation. Practitioners described a needs analysis process that was person-centred and driven by the client. It was seen to be a highly personalised process; with each individual

_____________________________________________________________________________

61

recognised as having different, specific needs. Emphasis on meaningful engagement was very important to Practitioners, so that the project was not simply transactional in nature. We noted in the preceding sub-section that Practitioners were trusted by participants and their communities. Practitioners recounted their perceived need to build trust and invest in relationships with clients before enrolling them on courses. This is exemplified by the case of one beneficiary:

Case Study The project participant ran a Newsagent and Off-Licence and took training arranged by one of the project partners, including a course in Prevention of Under-Age Sales. During the time of his interaction with the project, he suffered an assault, which caused serious injury. He turned to the Practitioner at the project partner with whom he was working at that point for help and advice. The Practitioner helped him with the consequences of the assault – advising on solicitors and the police and acting as an advocate for him at court. The beneficiary is still recovering from the physical effects of the assault. His daughter’s view was that he could not have dealt with any of this process without the Practitioner’s support, which she said was “extensive”. The Practitioner explained that she could not involve clients in training and courses while ignoring circumstances that affect the rest of their lives. “These things could prevent them from achieving in life as much as, if not more than, lack of training”.

This case study is an extreme example of the lengths to which Practitioners would go to provide mentoring and support to participants. Nonetheless, it illustrates the holistic approach to supporting project participants and furthering their journey to reaching their potential. Feedback from beneficiaries in relation to the Practitioners’ approach to needs analysis was unanimously positive. Practitioners were perceived to be genuinely committed to helping participants, took time to understand their situations and the challenges they faced and went out of their way to ensure that the solutions they proposed were based on the individual’s needs. In some cases, clients credited practitioners with having a life-changing impact. One beneficiary said:

“The most important aspect of (the Practitioner’s) work has been in building my self-confidence. (Another scheme) I was involved in was good but wanted to push me into an HR job straight away. I lacked the confidence and did not understand UK paperwork. Also I had to counter people’s expectations of me: I am white, I look British. People were

_____________________________________________________________________________

62

surprised when they hear my English, which they expect to be better. There is a definite need for this project. (They) understand people who are different. Receiving help of the kind I am receiving through this project is out of my experience in Hungary. Such interventions simply don’t exist”.

Some beneficiaries were very highly qualified but failed to gain commensurate employment. In one case, the Practitioner had to introduce the notion of “over-qualified” during the needs analysis session and how to combat this, exploring the idea of transferable skills:

“The beneficiary had a PhD in Theology. At the point of his initial involvement with MAWRII he did not fully appreciate that this might not be a sought-after qualification by employers. With needs analysis he grew to understand the UK job culture and to recognise his own transferable skills. He is now working as an academic researcher”.

Practitioners generally felt that the type of training the clients needed in order to improve their employment prospects did not always meet the ESF accreditation requirements. Clients often lacked self-confidence. They may not have understood the UK job culture, or realised how best to apply for jobs or approach interviews. There was sometimes a need to prepare clients before they could enroll on recognized courses, which Practitioners felt was not fully recognized within the project. Practitioners expressed a view that to ignore this situation could carry a danger of obtaining qualifications for qualifications’ sake. Practitioners spoke of how they had to be flexible to accommodate some of the vital softer skills support, as well as the more formal accredited training. A recurring perception emerged from participant interviews to underline this view: People from their community found it very difficult to find work, especially if their English was poor and they did not have a car. A good level of English language skills and mobility – to be able to get to a job that might involve changing shift patterns and different locations - were seen as basic prerequisites to get onto the bottom rung of the job ladder. One practitioner worked with participants to promote and explain the importance of networking to initially gain and then progress in employment. This Practitioner championed the use of LinkedIn as a tool and helped clients to build a suitable on-line profile so that they could benefit from their existing contacts and build new ones over time. Nonetheless, one focus group of 6 EEM beneficiaries expressed a consensus that it was important that training carried some sort of certificate that they could keep for future promotion or future jobs. They did not think it was difficult to find

_____________________________________________________________________________

63

training courses in their area but it was useful to have the courses that met their needs delivered here (in the location of the project partner), because it was local and accessible and the project partner understood their needs. Evidence provided by participants suggests that Practitioners were highly-effective mentors and efficient at signposting clients with all levels of education and qualifications to relevant information, support and training. Practitioners voiced a concern on behalf of some clients who tried to access public services:

“Referrals are not getting picked up. People simply don’t have the skills to cope with people from very different cultures and backgrounds not knowing the UK systems. It is really important to spend time with people in order to get anywhere with them”.

By and large, needs analysis did not lead to beneficiaries being referred between the project partners, mainly because they were working in different regions and, perhaps, partly due to a residual element of competition to meet targets where they were co-located in a city. However, there was evidence of cross-referral of clients and a close working relationship with the so-called P1 and P2 projects which had been initiated by AWEMA at the same time as MAWRII (the P3 project). There was also evidence of close working with a wide range oftraining providers and with organisations that support or provide volunteering opportunities, immigration advice and legal advice. In general, needs analyses did not identify training for participants that could not be supported via the project. Only one example of highly specialised nurse-training requested by beneficiaries could be identified as the exception.

Case Study The beneficiary came to Wales from Poland in 2006 via a work agency. He spoke very little English and had various low-skilled jobs in Poland. The agency job was at a meat factory, working initially as a cleaner, then a packer, then a despatch worker. The project participant viewed it to be a good employer but there was no real opportunity to move up to a well-paid job. In 2012, he started a college course in English as a Second Language. The MAWRII Practitioner visited the college and introduced the project. The beneficiary was immediately interested and met with the Practitioner for a needs analysis session. He decided to train as a lorry driver. The Practitioner found details of an Instructor who provided training for the LGV Driving Licence Class 2. The participant trained and got the licence. He has since taken his CPC Drivers Licence (to carry goods up to 18 tons)

_____________________________________________________________________________

64

and is continuing at the meat factory and driving for an agency while he does his Class 1 CE Licence (long-distance continental lorry driving). Once he has this, he will apply for that sort of job. Now he speaks English he thinks he will be well-qualified for this type of job, which he views as very well-paid. He said that the Practitioner’s approach came at the right time and the support with his CV, applications and other administration throughout had been essential: “I couldn’t have done it without them”.

5.3 Training Design “Participants gain access to fully accredited qualifications as a result of the participation in this funded project. A qualification would be defined as being within the Credit and Qualifications Framework for Wales.” (Priority 3, Minorities Are Wales Resource II –Key Indicators, Outputs and Outcomes) Training design followed naturally from the needs analysis approach of being highly person-specific. It included support and mentoring, commercial “off-the-shelf” courses and bespoke training covering:

• Individualised one-to-one, face-to-face “pre-employment” support and ongoing mentoring in areas such as confidence-building, skills analysis, interest analysis, assertiveness training, CV building, business advice (mentoring, finding information), NARIC qualifications conversions, creating awareness of networking opportunities, teaching about job expectations and work culture and identifying volunteer work placements;

• Accredited short courses or workshops (2 hours, half a day or one or

more days) resulting in a basic certificate in areas such as First Aid, Fire Safety & Marshalling, English Language, Health & Safety, Food Hygiene, Food Allergy Legislation, Anti-Trafficking Legislation, Lifting & Handling for Carers, Setting up a Business, Holistic & Beauty Therapy, Cake-Making, European Computer Driving Licence, Excel Basic, introductory courses such as Introduction to Teaching for Classroom Assistants and Introduction for Nursery Assistants. The levels of such courses were typically basic skills or levels 1 and 2;

• Accredited longer courses, usually part-time, towards qualifications in

areas such as Accounting, Book-keeping, Childcare, Construction, Electrical, Security, Transport and Logistics, Translation & Interpretation and Education & Teaching. These interventions were typically at levels 3 and 4.

_____________________________________________________________________________

65

The training to which beneficiaries enrolled across the project varied according to their needs - ranging from mandatory workplace training relevant to low-skilled, low-paid jobs, through vocational training such as NVQ’s for Nursing, Teaching Assistants and Childcare Assistants, to professional qualifications. At the highest level, 15 graduates were supported through a 2-year part-time Post-graduate Certificate in Education. Training design was expected to build on the approach taken by Practitioners: requiring a strong knowledge of cultural sensitivities and a commitment to providing 1-to-1 support and solutions in each case. The complexity of need was evident from the participant database and from the sample of participants interviewed. Practitioners commonly expressed a view that the training required a depth of intervention and support that generic training provision (public courses) would not easily replicate. One practitioner gave the example that it is possible to secure qualifications but still lose a job through lack of cultural understanding (your own or others), racism or inequality. It was not enough to refer a client to training provision or another project and assume that was sufficient. Instead, this Practitioner took the view that there was a need to maintain a mentoring relationship. All Practitioners suggested it is essential that training reflects clients’ circumstances and needs. All training for the project had to be delivered by an accredited provider and result in specific qualifications. Practitioners understood the rationale behind this requirement but, as one Practitioner pointed out, the clients most in need of training tended not to qualify for the accredited training. Some training interventions such as some of the pre-employment training to help clients prepare for interviews and understand the UK workplace or the Food Allergy Legislation training which is useful for clients in restaurants, catering and hospitality, was not accredited and therefore not recognized as training under ESF rules. It therefore had to be recorded as “mentoring” not as training. Also, it was suggested by some Practitioners that clients could not attend courses run by accredited providers on occasion due to work or family commitments. This led one Practitioner to suggest that if clients could be registered for an (non-accredited) on-line training course, for example in English language, they could work on this at one of the Partner offices assisted by a Practitioner at times that suited them, such as lunchtimes. Practitioners put forward the view that this sort of specialised provision is vital if BME and EEM communities are to be effectively supported to progress in employment. It was suggested the approach has to be completely tailored to the needs, characteristics and sensitivities of the client group. This requires a

_____________________________________________________________________________

66

detailed understanding and knowledge of the people Practitioners are working with – right down to food preferences. The great majority of the training described by Practitioners and Participants alike fell into the categories of 1 to 1 support and mentoring, and level 1 or 2 accredited courses. This accords with the findings within the management information analysis. While we have just explored the Practitioner view that provision needed to be tailored, a large number of beneficiaries took 2- or 3-hour off-the-shelf courses in Food Hygiene and First Aid. Reconciling these two positions suggests the importance of the 1 to 1 mentoring/support. Moreover, given the spread of starting qualifications held by participants prior to joining the project, it may be the case that not all of the beneficiaries needed these specific courses as a priority to gain, keep or progress in a specific job. However, the courses provided a relatively quick, accessible route into training for participants and one that was straightforward for project partners to commission. In this sense, these courses may have acted as an introduction to training, helping build confidence, focus participants on their personal development and employability and providing them with a qualification recognisable to UK employers. Such qualifications may also have been logical choices given the project’s stated focus on the following sectors, where there is evidence of a continuing need for relatively low level skills development14,15,16,17.

• Property, facilities management, housing and cleaning • Hospitality, leisure, travel and tourism • Retail • Social care • Food & drinks manufacturing & processing

The project’s business plan notes that these sectors have a high proportion of lower skilled jobs, which are associated with high levels of turnover and low productivity. The business plan noted also that businesses in these sectors are subject to shortages of supervisory staff and skills deficits in supervision, 14 Hospitality, Tourism and Sport Sector: Sector Skills Assessment 2012 accessed at https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/306403/briefing-paper-ssa12-hospitality-sport-tourism.pdf 15 Real Estate and Facilities Management Sector: Sector Skills Assessment 2012 accessed at https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/306429/briefing-paper-ssa12-real-estate-facilities.pdf 16 Sector Skills Insights: Retail, UKCES, 2012 accessed at https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/304319/Sector_Skills_Insights_Retail_evidence_report_53.pdf 17 Sector Skills Insights: Health and Social Care, UKCES, 2012 accessed at https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/304330/Sector_Skills_Insights_Health_and_Social_Care_evidence_report_52.pdf

_____________________________________________________________________________

67

communication, customer handling, team working and IT. However, there was no evidence of a material number of training courses developed in these areas (except IT). This may have been due to the relatively low level of meaningful engagement with employers during the project. We can only speculate that more courses may have been designed to address these recorded skills shortages if the project had been able to engage with significant numbers of medium-sized or large employers and involve them in training design. A comment from a participant from the Nepalese community resonated with this latter point:

“My community, and others in my view, need more long-term training towards recognised national certificates in areas such as plumbing, carpentry, electrician, marketing and customer services”.

Nonetheless, the beneficial effects of short, level 1 or level 2 training interventions cannot be overlooked. The beneficiaries interviewed frequently cited these qualifications as having been key to them gaining a job interview, a job, a promotion or had acted as a stepping-stone to career change or setting up their own business. The beneficial effects of mentoring and short courses are difficult to unpick, with each strand of activity reinforcing the efficacy of the other strand.

Snapshot from a Focus Group The participant ages ranged from early 20’s to mid-40’s. They have been in the UK for between 2 and 10 years and during that time involved in a wide range of courses and support. All of the participants are now employed or self-employed in some capacity: A Polish man took an English language course, then a Food Hygiene course to get a job as a Kitchen Assistant. He wanted his own business. The Practitioner gave him lots of mentoring, ideas and advice and he then bought a burger-van which he now runs. A Polish man considered himself “unskilled” when he came to the UK. He took an English course and a fork-lift qualification and now works as a fork-lift driver. He now plans to get his “scaffolding card” to increase his earnings. Another Polish man had taken various courses (First Aid, Food Hygiene and ESOL). He had a high level of school education from his home country and good English. The Practitioner arranged for his home qualifications to be translated via NARIC and he has now applied to study Radiology at a university in Wales.

_____________________________________________________________________________

68

A Thai woman had some basic computing skills. She has taken a Business Start-up course and some IT courses, including Excel. She now works from home for a (Welsh) company that provides IT solutions around the UK. She aimed to deliver some IT training on a freelance basis as the starting point for her own business. Another Thai woman had no qualifications from home. She was helped by the Practitioner in job-searching, interview techniques and an ESOL course. She applied for and got a job as a Cleaner with the local authority. She attended the Business Start-Up course and hopes at some point to set up a cleaning business. A third Thai woman began a catering business from home, initially parties for Thai people in her area. To help her start, the Practitioner suggested the Food Hygiene Certificate and made arrangements for the Local Council to inspect her kitchen and catering processes at home. This resulted in her being granted a licence to sell home-made food. She has since taken a Business Start-Up course and wants to learn more about Book-keeping and to take a course in Specialist Cake-Making to develop her business.

It was equally challenging to assess the stand-alone value of accredited longer courses, such as the Post-graduate Certificate in Education, against the intended impact on clients’ employment and career prospects. The clients already had degree level qualifications and were generally, though not always, more familiar with UK job culture. They were also often in a position of greater choice and less financial pressure than most of the beneficiaries taking Level 2 training. Participants seldom described a need to take the first job available and tended to be more vocal about their ideals and preferences. Of the clients who took post-graduate level training sponsored by MAWRII, one subsequently gained employment as a Human Resources Assistant at a university. This participant suggested s/he would not have got this job without the PGCE. Another participant was looking for a teaching job, but

“It’s very difficult to find teaching jobs in (this town) – they don’t come up much at post-16 level which is my area”.

The participant had applied for a job as a Domestic Abuse Educator for schools and was waiting to hear the outcome at the time of our interview. Three other beneficiaries were interested in training in Management or Leadership, one was

_____________________________________________________________________________

69

seeking funding for a PhD and one wanted to take further training relevant to Psychology.

Case Study A Nigerian man had moved to Wales 3 years earlier. He held a BSc in Engineering and wanted to work in the construction industry. On arriving, he found that his degree was not recognised and he needed a CSCS card to gain entry into the industry. He also had additional language needs that made it harder for him to access jobs. He was in a junior warehouse position at a large distribution company when he spotted one of the MAWRII flyers. In terms of the support provided and the specific nature of the training accessed, he spoke enthusiastically about the quality of the provision. He had benefitted from some of the softer support on offer, including help re-drafting his CV and identifying wider skills needs, and he said that in particular a NARIC translation of his degree certificate was invaluable. This broader support had been really useful and a good relationship had developed with the Practitioner and the team. The actual training provision was also commended. The client had sat his CSCS exam and obtained the certificate – thereby opening up access to job opportunities in the construction sector.

A further participant was a 3rd year Law student at a university in Wales. This participant was fluent in English and the Practitioner explained that s/he was initially unsure whether s/he could provide the client with any help or, indeed, whether the individual was a suitable candidate for the project, given s/he was clearly a high-achiever already. It is worth reporting the involvement however, as the engagement was felt to be extremely productive for the participant:

Case Study The participant was looking for a provisional career path – ideally work experience, to get on the right career track. S/he texted the Practitioner fortnightly to chase this and “within 3 months s/he had e-mailed most if not all the solicitors in South Wales on my behalf”. Through this the participant got work experience at X Solicitors in (town in Wales) and then in their (another town) office. S/he said, “This will be a big advantage for me as competition is very tough”.

_____________________________________________________________________________

70

The Practitioner also advised the client to contact the Citizens’ Advice Bureau. S/he did so and is now an Advisor there. S/he also put the participant in touch with a Welsh Government project called ‘Youth On Board’, which connects young people with company Boards. The participant has subsequently observed several Board meetings and also participated once in the Board of an organisation. This has given him/her good contacts in (a global organisation) and the Welsh Government. The Practitioner also introduced him/her to a Public Appointments Scheme. S/he took a course, over 2 weekends, on how to prepare for a job in the Public Sector and found this very useful. The participant has also applied for a charity programme called ‘Pedal Power in Wales’ and is waiting to hear if s/he has been successful. S/he thinks it would not be too difficult to get a job in the UK – s/he is fluent in English, having visited his/her family here regularly over the years – but knows it is difficult for most people from outside the UK who do not have these advantages.

5.4 Training Delivery There is little in the Business Plan to govern how, when, where or how the quality of training is assured, other than mentioning that it should be delivered by recognised providers. We have already heard that the support and mentoring provided by Practitioners (“Pre-employment training”) was needs-based and appropriate. All of the project participants interviewed or involved in focus groups expressed a view that the quality of the 1 to 1 support and mentoring received was excellent. In general, this support included an initial interview to understand the client’s background, interests, education and skills, their employment aspirations, how mobile they could be for work and whether they could work full- or part-time. This initial 1- to-1 interview gave the Practitioner and participant an opportunity to tailor a training plan to meet the client’s specific needs. Such plans might include significant mentoring to build confidence and assertiveness, and to identify life skills such as the ability to manage household finances (as opposed to qualifications) and gaps in essential skills and experience, such as English language or work experience in the UK. This was typically followed by the Practitioner and participant working together to build or re-design their CV, agreeing initial training options and, in some cases, mock job interviews. Several interviewees suggested that they found this initial mentoring transformational. Others talked of the Practitioner and the project as a ‘lifeline’. Some interviewees certainly attributed their success in securing interviews and jobs to their newly developed CV’s or interview techniques as they had not previously even received acknowledgements of their

_____________________________________________________________________________

71

applications. The overwhelming majority of participants highlighted the close working relationship with their Practitioner as a key success factor. One beneficiary commented:

“S/he worked with me to identify which course I needed to go on, and kept in regular contact throughout the project”

Another stated:

“Having a strong relationship with (the Practitioner) made everything work well – s/he would always remind me about dates, and where to go...nothing was too much trouble”.

Short generic courses up to level 2 were also almost always deemed by individual participants to be useful, practical and professionally delivered. In the rare exceptions to the commendations given for this off-the-shelf training, one participant felt it to be too low-level for his/her level of education, while another other found it

“Satisfactory, but not really for me.” Flexibility of provision was important to the vast majority of clients. Beneficiaries of the project needed to be working full-time or part-time. Many also had family responsibilities. Some cited child-care as an issue when it came to balancing training and employment. They appreciated being asked when they could be available for training, rather than being told to attend at a fixed date or time. Practitioners noted that despite the positive feedback, attendance at courses was still a challenging issue. They felt that it was inevitable that, regardless of how much preparatory work they put in, and despite phone calls or text reminders, some participants would fail to attend. One Practitioner suggested that a contract of commitment with participants could be introduced to help their focus, attendance and commitment.

_____________________________________________________________________________

72

Case Study A Polish woman, in her early 30s with 2 young children had been a shop-assistant in Poland. When she came to the UK 10 years ago she had very little English. She found work as a cleaner. About 5 years ago, she met with (the Practitioner) who was recommended by a Polish friend. She explained that (the Practitioner) has since helped with all sorts of challenges in her life: finding a solicitor when she was going through a divorce, training courses, references, applying for and interviewing for a job. She described the support as “holistic”. She initially undertook training arranged in First Aid, as well as an Introduction to Becoming a School Learning Assistant. She secured a job but found that she did not enjoy the Learning Assistant work. She decided to change direction. The Practitioner then helped her get her first job as a Carer, which she now pursues full-time and enjoys, seeing it as having career potential. In the future she would like to become a Support Worker for people with mental health problems. She has been doing some shadowing in this area, also arranged by the Practitioner. She would like to train in the future in specialised care for brain-injured patients and mental health nursing. She could not speak highly enough of the support she had received from the Practitioner and the perceived importance to her in getting trained and securing her job.

The majority of employers that became involved with MAWRII training were small or medium-sized businesses, albeit there were examples of a small number of large organisations, including a national charity and a large hotel chain. In relation to the training they accessed, the evidence from interviewees is divided between those individuals who were planning to start or had just started running their own businesses, small businesses with 3 or 4 (often part-time) staff and larger employers who took up training for their employees. The individual entrepreneurs and sole traders typically took advantage of mentoring support, networking meetings and short courses in Starting and Running a Business. They commonly took advantage of short courses specific to their business needs also, such as Food Hygiene, Book-keeping and Beauty Therapy. This group of participants provided a view that support was especially valuable in helping them navigate their way through the legal and tax implications of running a business in the UK.

_____________________________________________________________________________

73

Small businesses also sent their staff on short courses to gain certificates in skills that tended to be mandatory in nature including Food Hygiene, Manual Handling, First Aid and Health and Safety.

Case Study The business is a medium-sized off-licence, grocery and newsagents with a Manager and 4 part-time employees. The Manager heard about (the project partner) from another small business in the community and s/he has taken the Food & Hygiene Course and put all of the staff through the Prevention of Under-Age Sales Course. All participants from the business successfully passed all of the training they undertook. The employer found the training “really useful and informative, full of good facts. I thought I knew everything about the business but the courses were a real eye-opener. In the Prevention of Under-Age Sales Course we were given photos of people whose ages we had to guess and good training on the law and on how to deal with under-age customers”.

Employers were generally very positive about their experience of training secured via the project. Typical feedback was of employees achieving the qualifications being studied following needs analyses that were tailored and specific. Training was generally focused in relation to their business needs, but in at least one case, where the need was more complex, the training suffered from insufficient tailoring of materials by the training provider:

Case Study The employer is a national charity working with a specific disability group. There are 8 staff and approximately 40 volunteers in total based in Wales. Initial engagement with the project was very positive – the (project partner) worked with them to scope needs and identify training. It was agreed that staff would attend one generic course (Food Hygiene), and that a second bespoke (First Aid) course would be run specifically for the charity. The MAWRII project was very supportive of the needs of the group, but there were mixed results in terms of how the trainers coped with the specific needs. The charity fed this back to the project team. “It’s important that training is done properly, as it damages people’s perceptions of the project if it goes wrong. Also, as I organised the training for my staff, my reputation is also damaged by any mistakes”. Despite this feedback, the charity was grateful for the training and felt it had been positive for those taking part.

_____________________________________________________________________________

74

The relationship between project partners and training providers was critical to the training delivery and outcomes. Those who deliver the training are the public face of the project as much as the Practitioners and other staff within the project partner teams. Training providers were impressed by the way in which training was contracted and managed:

“(The project partner) communicates well and knows what it wants. They obviously do their homework as they identify what employers and employees want/need. They also put in the time to understand what the service users can achieve and prepare the service users for the course: expectations are set and it is noticeable that (the project partner’s) delegates give 100% effort to their learning. We have worked with similar organisations with similar clientele and the same cannot be said of all organisations. (The project partner) do a better job than most at preparing their delegates”.

The evidence presented suggests project partners took steps to ensure the quality of the training delivery and manage its impact. As we have seen, these measures were successful, in the main. Practitioners were probed about the constraints of the project. They commonly indicated that they were reluctant to send participants on more than one or two courses (depending on cost). This was partly due to the availability of funds and partly due to the ESF constraint that the beneficiary can only be counted for one qualification. As a result, progression was perceived to be an issue for some participants. As a result, participants starting at lower levels of training were sometimes not supported through to the higher level courses that potentially could have had a more significant and meaningful impact on their lives. In this vein, two Practitioners suggested the project would have benefitted from better access to longer-term, vocational training courses at FE colleges (for example in subjects such as welding, plumbing, construction or Electrician training). Differences were apparent in the level of independence and responsibility given to Practitioners in the contracting of training, as was the case for training solution design. Some Managers were reputed to have made decisions without Practitioner involvement while others gave the Practitioner responsibility for the training portfolio. Several Practitioners raised the issue of project partners not being permitted, under ESF (procurement) regulations, to deliver training themselves. One Practitioner was of the opinion that given appropriate lead-in time and resources the Project Sponsor could have acquires the appropriate status to deliver most of the short course training itself (e.g. Health & Safety and Food Hygiene). This

_____________________________________________________________________________

75

Practitioner suggested this approach would have saved time and money in the long run and guaranteed accessibility to clients. An alternative view was presented by a training provider: delivering training on this project helped them to “raise their game”. The provider suggested the project built the capacity of training providers to work with under-represented groups and thereby improved their capability as trainers. This issue of ESF constraints on recognised training was also pertinent to Practitioner feedback on the need to provide a high level of “pre-training” support to some clients, particularly those with low levels of education, qualifications or English language. This type of support did not carry accreditation and had to be categorised as “mentoring”. 5.5 Outcomes The outcomes expected from this project were improved career prospects for participants – improved job security or career progression – and improved business practices for employers. There was evidence that significant effort went into mapping participant needs and what outcomes they wanted to achieve in terms of progressing in employment and improving their prospects. As a consequence, participants often received more than one training intervention, despite the project only being able to count an individual once against targets (and associated budget restrictions). This was driven by Practitioner focus on outcomes rather than outputs, in the main. Of the fourteen individual beneficiaries interviewed singly, almost all identified what they considered to be better jobs or improved job prospects as a result of their interaction with the project. Evidence was presented to demonstrate a range of positions secured, ranging from a part-time dinner supervisor in a school through to people starting their own businesses in sectors ranging from catering to complementary therapies. Case studies throughout this report evidence these types of outcomes. The interviewees came to the project with a range of skills from school-leaving qualifications to post-school vocational training and degrees. Those with lower skills gained employment or started businesses, which they attributed to the project support and training, particularly around Running Your Own Business Skills, CV development, making job applications and interview practice. Those who joined the project with some existing vocational qualifications appeared to see most benefit in the accredited training, taking advantage of specialist vocational training that they credited with helping them to access the next level of their career. An example was a participant who was a qualified electrician

_____________________________________________________________________________

76

and CCTV engineer on arrival in the UK from Brazil. Following courses in English language he studied for and gained his CAA Licence, then with support from the project in CV preparation and interview techniques, won a job as an Engineer. At the time of the interview he was taking further training (to use fork-lift and cherry-picker equipment) which he anticipated would gain him further promotion. Those who had started a business also perceived the training or mentoring they had received as instrumental in helping them start these businesses. Almost all interviewed were sole traders (with one husband and wife team), with one growing the business to a small enterprise employing three staff. All of the active sole traders were female, with businesses including freelance IT support, catering, beauty therapy and jewellery-making, although there were male participants with plans to launch businesses in the near future. All of the sole traders worked from home and the small enterprise referred to above recently re-located to rented external premises. Two graduate participants were also interviewed. Both had gained professional employment related to their degree in the UK and both viewed the support and training they received from MAWRII had helped them obtain these posts. Furthermore, all of the Practitioners had secured the translation of degree transcripts or other academic qualifications via NARIC. This enabled higher qualified individuals to make better use of their prior qualifications in this country and avoid “skills traps”: A graduate from outside the EU may find that his or her qualifications are considered lower than a “comparable” UK qualification. Additionally his/her English language skills may be too low. Consequently, they can end up working in low-paid, unskilled jobs, while they try to find the time and money to upgrade their qualification. The causal link between qualifications gained or mentoring support and positive career outcomes was very clear for individuals interviewed. The note of caution needs to be borne in mind given the small sample size interviewed and the fact that all were volunteers rather than selected at random. Indeed, there was a more mixed perception among the participants within the eight focus groups of the nature of any causal link, if it existed, between the MAWRII intervention and gaining or progressing in a job:

• Almost all of the participants in the focus groups considered the training and mentoring had or would have an important impact on their employability and career development;

_____________________________________________________________________________

77

• Around one-fifth (generally women with young children, for whom family priorities and child-care issues had an impact on their availability for work) viewed this as a potential longer-term outcome;

• About one-third had started or had plans under way to start their own businesses. These participants were clear that the Practitioner support and the courses they had followed were a critical enabling factor in helping them realise these aspirations;

• The remaining focus group participants suggested the variety of courses and support they had experienced was very important to their current employment but no evidence could be elicited to substantiate these views.

The majority of participants provided a view that they valued the short course training as providing quick benefits. They perceived that vocational, level 2 courses, such as Food Hygiene, helped them to get or keep jobs, which in turn gave them work experience and income. A picture emerged from the participant interviews and focus groups that these short, level 2 courses act as a catalyst to genuine change, particularly for those interested in starting their own businesses. The impact often appeared to be twofold:

• Acquisition of a recognised qualification which evidences a new or improved skill; and

• Gaining confidence, self-esteem and contacts. This second category of social capital benefits are less tangible but, as we explored in the wider literature in section 3, may be equally important in terms of employability and career development. Practitioners, while recognising these effects, questioned their long-term impact, suggesting that longer vocational courses such as Holistic Therapy and teaching qualifications such as An Introduction to Teaching for Classroom Assistants really change lives. This view was supported by some of the feedback, in particular from participants who had been in Wales for 10 years or more and felt able to speak on behalf of their community. Such participants suggested more long-term training towards recognised national qualifications was required if there was to be a step-change in benefits to their community and the contribution they could make. They went on to suggest that this would not necessarily be a single qualification gained over the long-term, such as a PGCE; indeed it might be a series of successive, related qualifications that develop an individual through a profession.

_____________________________________________________________________________

78

A further challenge is that courses above level 2 tended to be accessed by higher qualified individuals, who in some cases were effectively taking a backward step to re-qualify. For example, a person might be qualified as a teacher in Poland and need 200 hours of teaching practice to achieve qualified status in the UK. As a result, such a person might train initially as a teaching assistant or enrol on a PGCE here.

Case Study A beneficiary came to Wales in October 2010 with Bachelors and Masters level degrees, plus 2 years teaching experience in Romania. She took various casual jobs until 2013 when she met (the Practitioner) at a job fair and heard about a 2-year PGCE, part-time, and enrolled. She has completed year one. Although she is already a qualified teacher she finds it useful as the education system in the UK is different from home – and she thinks schools will want a UK PGCE, not a Romanian qualification. She has recently registered with a Teaching Agency for supply work. They immediately responded to her CV and called her for an interview. She suggests this is due to the PGCE and the work the Practitioner provided helping with her CV.

Participants also highlighted the importance of mastering English language. They expressed a clear view that communication is key to settling into a new culture and country. This is perceived to be particularly true with regards to getting a job and progressing. Interviewees who had not been able to do so explained that they suffered from lack of confidence and were restricted in their choice of jobs. One participant described some of the women from her community as “ghettoised” by a combination of lack of English language and low self-confidence. Case Study The beneficiaries, a husband and wife, left Portugal because of desperate circumstances. They could not afford to buy food. They described the Practitioner as “a lifeline, showing that someone cared”. The Practitioner gave them moral support and helped build their confidence. From this position, the Practitioner helped the husband to get a job at a fast food takeaway. The family’s son followed them to Wales. All three members of the family are now working shifts at the same fast food chain. The employer had been impressed by the man’s hard work and, we were told, was pleased to find work for the other family members.

_____________________________________________________________________________

79

The husband said that he has “found a new side of me” through working there and being able to talk to customers. He has found that he is good at relating to older customers, children and those with learning difficulties. This has given him the idea of training as a Home Careworker. The challenge is to find the time, energy and money to do this around his shifts. His wife had very little English – she did not speak during the focus group other than occasionally in Portuguese which her husband translated – and appeared less happy with her life in the UK. She was working without any customer interaction because of her lack of English. Her husband said she was a trained fishmonger at home in Portugal and would like to do this in a factory or supermarket here (but she appeared to lack the confidence in her ability to master English and seemed to hold out little hope of her career developing). This case study introduces a cycle that became familiar in the stories of participants: there is a need to earn money and yet there is a need to study in order to improve lives. One Practitioner reflected that in his/her experience of working with migrant participants, it is typically in years 3 to 5 of their journey that people achieve real changes to their lives. Outcomes for employer beneficiaries were less clear than those for individuals. Nonetheless, all employers interviewed were very positive about the training provided for their staff. This comes as little surprise, given the bulk of qualifications achieved by their employees reflected legislation-driven training that is mandatory for employers to provide, for example health and safety or first aid training. Nonetheless, evidence was presented to suggest employers supported more employees in MAWRII supported training than they would have provided without the project’s intervention. Employers also expressed gratitude for support with Equalities policies. Employer focus was predictably on protection of their business through compliance and the ability to evidence appropriate policies. Other employers referred to the Equalities policies being useful when bidding for new contracts. In summary, it is clear that the majority of individuals who participated in the project believe that they gained benefits that may be viewed as positive project outcomes. While employers were positive about the support gained from the project, outcomes were less tangible. In addition, Practitioners were positive and proud of what had been achieved for the target communities, despite the implementation challenges that had arisen over the course of the project. Practitioners were also able to highlight a number of valuable learning points

_____________________________________________________________________________

80

from the experience of running the project, and express clear views on future delivery options – both of which are covered in the next section of this report.

_____________________________________________________________________________

81

6. Conclusions and Recommendations Here we present our conclusions and recommendations arising from the data collected over the course of the evaluation. The structure reflects the terms of reference provided in the evaluation specification, which was reproduced in section 2.4. Headline Conclusion. The project largely delivered the intended outputs. Evidence was found to suggest intended outcomes were achieved for the majority of participants who achieved greater job stability or progressed in their careers. There is evidence linking 1-to-1 mentoring support by project Practitioners and accredited qualifications achieved during the project to some of these outcomes. English language skills stand out as a key area where development leads to dividends for participants. While the project can be viewed as largely successful, it is clear that the need for interventions of this type to support BME and EEM communities remains. The successes achieved by the project are particularly notable, given the levels of change to the consortium over the course of the project. The resilience and commitment of the existing partners should be credited, and the lessons learned from the project used to inform future interventions. 6.1 Nature of Participants The findings revealed a very diverse range of participants, drawn from at least 92 nationalities and embracing a broad range of ethnic backgrounds ranging from White European (the largest group) through to Traveller and Arab (the smallest groups within the participant cohort). 39% of the participants did not classify themselves as migrants, with the EU and non-EU migrants respectively making up 34% and 27% respectively of the 1,392 BME participants in the project. Participants were domiciled across the Convergence area, providing clear signs that the project sponsors had engaged with potential beneficiaries across the geographical area supported by the project. This suggests the consortium approach of sponsors distributed across Wales was, effective overall. Nonetheless, there was a high proportion of participants from the City and County of Swansea (39% of the total cohort) where two of the original project sponsors were based and negligible engagement from people based within Ceredigion and Pembrokeshire, which can be traced back to a combination of higher costs of engagement per participant in the more rural parts of Wales (for the sponsors and the participants) and also a lack of ownership of these territories by any specific project sponsor.

_____________________________________________________________________________

82

Recommendation: Future pan-Wales projects should recognise the different costs of accessing participants in rural versus urban areas. This factor should be reflected in the allocation of targets and budgets. Recommendation: Future consortium-based projects would benefit from clear targets broken down by project sponsor to ensure that no elements of targeted outputs are without ownership. Diversity of participants was not confined to race nor to ethnicity. The evidence suggested that positive attempts to engage with women, particularly those working part time, and disabled people were largely successful. Similarly, the ages of participants ranged from 16 to 72, with the main body of participants being in the age range 20 to 45. Consequently it is possible to conclude that the project was largely successful in engaging with a very diverse range of participants with the protected characteristics of gender, age, race and disability. That 91% of participants classified themselves as belonging to a BME or EEM group is a powerful indicator of the success of the project at engaging with people that are traditionally categorised as hard-to-reach groups. Participants held a broad spread of educational attainment prior to joining the project. 39% of participants held no qualifications or were qualified at or below level 2 but the range of highest qualifications held by participants extended to 5% holding level 7/8 qualifications (Postgraduate degrees and Doctorates). No particular correlation could be identified between qualification levels and factors such as migrant status, nationality or ethnicity. Specifically, no correlation could be identified between participant level of qualification on joining the project and the position they held within their employment. Migrant status, however, appeared to have a strong influence on level of employment position. A significantly higher proportion of non-migrants held higher positions of employment than their migrant counterparts. This supported qualitative evidence presented by participants and Practitioners, and conclusions drawn from wider studies, that proficiency in English language is a major differentiator in employment progression. Recommendation: Future interventions to support BME and EEM participants would benefit from the firm inclusion of English language development components. The findings largely mirrored the occupational sectors expected for participants

_____________________________________________________________________________

83

in the business plan, including: Hospitality/catering (25% of participants), manufacturing/warehousing (18%), nursing/social care (10%), retail (9%) and third sector (9%). The inclusion of the third sector had not been anticipated in the original business plan. However, little weight should be placed on this finding, given 40% of participants in this sector were ethnically white-British i.e. not part of the original target group for this project. The inclusion of non-BME participants in the project arose primarily from engagement with employers. Training interventions with employees who were largely from BME or EEM groups were undertaken on a non-discriminatory basis i.e. non-BME participants were included. There was evidence of a lack of clarity in the early stages of the project regarding the ability of the project to support non-BME participants. Recommendation: Future projects should set out clear criteria for both participant target characteristics and broader participant eligibility at the project development stage. Clear tolerances should be established, and agreed by all sponsors, between numbers of target participants and non-target yet eligible participants. 6.2 Project Outputs and Outcomes The majority of outputs either exceeded target or were within the typical WEFO tolerance of 15% below target. Indeed, there were significantly higher than anticipated levels of attainment in reaching females working part-time, participants from BME or EEM communities, or over the age of 50. There was a similarly higher level of attainment in the number of participants gaining qualifications. These are clearly major successes for the project. The only outputs significantly below target were those of Employers assisted or supported and Employers adopting or improving equality and diversity strategies and monitoring systems. The evidence of the findings suggest that the under performance against the two enterprise targets is largely related to the withdrawal of YMCA Wales from the project. Activity associated with these targets was primarily the responsibility of that sponsor. Consequently, the level of progress against that target by the remaining sponsor organisations might be viewed as an achievement rather a failure, given that most of the progress was delivered in the final 9 months of the project. Nonetheless, performance against these two targets does lead to a number of conclusions. Progress against this target only came to the fore when Swansea YMCA took on the Project Lead role. This relates to project governance issues that are best explored in the next sub-section. The issue also raises questions

_____________________________________________________________________________

84

about the structure of the original consortium. At one level, with project sponsors largely focussed on working with minority groups (of differing characteristics), there was a perceived lack of expertise in dealing with employers and their needs. That was the rationale put forward for YMCA Wales’ involvement as a project sponsor – they were to bring such expertise. While this approach may have appeared logical in South Wales, there was the potential of duplication of effort with more than one sponsor working with a single organisation – one with the employer (and a cohort of employees) and another with individual employees from that same organisation. Clearly, this is not an efficient model if the work could be undertaken by a single sponsor. In rural areas and, it was suggested by some sponsors, in areas such as North Wales, such duplication was a likely possibility. Recommendation: In developing consortium-led projects, careful consideration should be given to the configuration of the consortium, to ensure that all partners have the skills, capacity and organisational resilience to effectively deliver against targets. While project outputs are easily measured, demonstration of project outcomes tends to be less clear cut in most evaluations. In this case, individual participants expressed clear views that the project produced positive outcomes i.e. enabled participants to maintain employment or progress their employment position. These views were supported by a firm body of participant testimony about the nature of their progress, summarised in the outcome findings of the previous section. More objective measures of these outcomes such as the number of participants gaining job interviews following project Practitioner mentoring or the number of participants maintaining employment throughout the period of their engagement with the project were evidenced (18% and 77% respectively). These figures would have held more weight had more exit data been gathered as the project progressed rather than being left until the project close down phase. Our conclusions can only be tentative as they were based on a sample size of less than 10% of the total participant population. Evidence relating to causes of positive outcomes appears to point towards more than one factor, indeed there may be a hierarchy of causes:

• English language proficiency appears to be a threshold skill that determines the ease for people to enter the employment market. Non-migrant groups are at an advantage compared to migrants from countries where English is not the primary language of communication. Allied with English as the primary means of communication in employment, is the

_____________________________________________________________________________

85

confidence an employee has as a result of being able to communicate fully;

• Mentoring provided by Practitioners was important in building social capital, particularly for migrant workers. This ranged from the provision of cultural understanding, for example how CV’s are presented and interviews conducted in the UK, through to leveraging Practitioner contacts for participants to build their own networks or to secure first interviews;

• Short, relatively low level qualifications in areas such as Health and

Safety or First Aid are perceived to provide threshold qualifications that are recognisable to UK employers and also provide valuable confidence building opportunities to participants that they can achieve positive milestones en route to their intended outcomes;

• Longer, possibly higher level occupation-specific qualifications or a series of related qualifications as a career develops are suggested as a key route to ultimate aspirations for non-migrants or migrant workers who may have been in the UK for 3-5 years and have satisfied the previous stages in the needs hierarchy.

Recommendation: Future interventions might consider being structured according to 4 strands of support: English language; mentoring support relating to employability skills; short, level 2 ‘threshold’ training interventions and longer, vocationally biased interventions at levels 3+. Outcomes for employers were less clear. There was evidence that outputs were attained for those employers participating in the project. Overall, however, employer engagement was the area where the project performed least well. The causes of this were rehearsed in the findings, stemming from the structure of the original consortium but also embracing communications issues around common understanding of targets and evidence requirements for WEFO, and finally governance of the project prior to Swansea YMCA adopting the Lead Partner role. The context surrounding the project clearly had an impact. The wider economic situation following the banking crisis was referenced by participants as challenging but did not form a major theme within the study. The implication of there being higher unemployment generally in Wales was that issues of job insecurity experienced by BME or migrant populations became more pronounced.

_____________________________________________________________________________

86

The heritage of the project had a more significant effect on the project’s ability to promote intended outcomes. We shall rehearse some of the key issues from the project’s history shortly. Here we shall simply note that the lack of effective collaborative project design led to inconsistent understanding of intended project outputs between partners. Moreover, the repercussions from the loss of AWEMA, YMCA Wales and MEWN from the consortium and dealing with the consequences of their departure took up significant management time and effort. Resolution of these two fundamental issues diverted time and energy away from more productive activity. It was the perception of several interviewees that outputs relating to numbers of female participants and employers were lower than would have been the case without these issues. These perceptions are consistent with the evidence presented and represent reasonable conclusions. 6.3 Project Processes 6.3.1 Record Keeping and Evidence Collection As is the case with most projects, the MAWRII project adapted and made steady improvements to the way that it collected and reported data on beneficiaries over the course of the project. At the time of evaluation, consortium partners were able to report strong levels of satisfaction with the way that systems and processes were working, and paying tribute to the work that the Lead Partner had carried out when taking over the leadership of the project. The detailed audit work (including 100% file testing), issuing of revised guidance, and regular monitoring/support visits, had allowed the consortium to identify a range of issues that needed attention, and gave the consortium and WEFO confidence that the project could progress through to closure. Given some of the significant challenges and changes encountered over the course of the project, the achievement of these improvements (and stability) should not be understated. The file testing that took place as part of the evaluation confirmed that whilst there were differences in the way that different consortium partners collected and recorded data, compliance levels were high and that data was stored in an orderly and logical manner. The differences in the way that records had been kept over the course of the project were evident, and this can be seen to be a feature of the changing approach adopted by the different Lead Sponsors. It is also important to acknowledge that there were reportedly some fundamental problems and deficiencies caused by the period when AWEMA led the project. Some of these deficiencies continued to frustrate aspects of delivery and record keeping for some time after AWEMA left the project. Indeed some issues were only addressed when Swansea YMCA took the opportunity to undertake a full

_____________________________________________________________________________

87

‘health check’ on the project as part of the mobilisation phase negotiated with WEFO. The beneficiary database and the process for submitting data to WEFO reportedly worked smoothly and had benefitted from efforts made by the Lead Partner to make the database more ‘user friendly’ and work in a way that was aligned closely with the WEFO online database. The appointment of a dedicated Research Officer as part of the Lead Partner team provided strong guidance, expertise and support to consortium partners, on this aspect of the project process. Again there were changes to the methods of data collection and reporting over the different phases of the project, with improvements observed over time. In terms of current project management, monitoring and reporting requirements, practitioners felt the project was now in good order and were clear about what information and evidence they needed to collect and record. That said, some interviewees noted that having a bespoke central database from the start would have helped the project run more smoothly, have saved time and allowed more targeted reports to be run against participant data. Practitioners also noted the significant administrative burdens involved with ensuring that files were kept up to date and would have all appreciated more resource and/or administrative support to help deal with these monitoring requirements. This would have given practitioners more time to focus on recruitment and delivery. The evaluation also identified a range of specific lessons and suggested improvements, for consideration by the consortium partners, especially in the context of thinking about future interventions of this sort. For example, the process of client registration would have benefited from a central model and system, applied consistently across the project. Similarly, a more proactive approach to issuing and collecting beneficiary evaluation forms would have helped ensure a stronger evidence base of the experiences of participants. Specific learning points aside, it is fair to conclude that since Swansea YMCA took over the lead role, business co-ordination and communications for all stages of the project strengthened significantly and there was a much improved hub and spoke model in place. 6.3.2 Marketing and Publicity Initiatives The most successful recruitment route for the project was via word of mouth. Engagement largely arose from Practitioners, from all sponsor organisations, being/becoming known and trusted figures within the communities within which

_____________________________________________________________________________

88

they worked. Trust was crucial to their success. This was largely built on efforts to forge deep relationships with participants and community leaders. The evidence presented suggests the ability of the Practitioners to forge such links was a critical success factor for the project. Promotional activities typically took the form of Practitioners visiting community centres, libraries, mosques, colleges, jobs fairs, businesses, shops and specific employers. Face-to-face contact or referrals through community leaders were important. By contrast, there was less evidence that promotional leaflets were as effective in attracting participants, albeit evidence certainly exists that they did lead participants to project sponsors. There was little evidence of engagement between the project and medium-sized to large employers. This suggests promotion to such employers was not as effective as that to individuals and small, typically retail/service businesses. Recommendation: Future activities could benefit from prior staff experience of engagement with medium to large sized businesses in relation to training and development. 6.3.3 Equal Opportunity and Environmental Sustainability The project can be seen to have delivered strongly against its commitments to the cross cutting themes of the ESF Convergence Programme. The equality achievements are particularly strong, primarily at the level of the beneficiary, but also in terms of the guidance and support offered to employers around adapting and improving equal opportunities policies, as part of their engagement with the project. The project has also enjoyed success meeting gender targets and targets for older participants, which is a target that many generic ESF projects have struggled to meet. The MAWRII consortium were also aware of their environmental commitments and ensured they had the necessary policies and practices in place to ensure they were compliant in this regard. The Lead Sponsor noted that the consortium could have done more on promoting environmental good practice, had they not needed to attend to the significant changes brought about by the withdrawal of MEWN, quite late in the project. 6.3.4 Leadership The MAWRII project has experienced a number of significant changes to the configuration and leadership of the consortium. The impact of these changes

_____________________________________________________________________________

89

on the momentum and effectiveness of the project cannot be understated. It is a credit to the resilience of the remaining consortium partners that the project has been able to overcome some of the serious challenges encountered, and continue delivering services to beneficiaries as intended. The quality of governance has reportedly improved markedly since Swansea YMCA took over the Lead Role. Moreover this organisation has also introduced a raft of quality assurance measures to support delivery and ensure strong compliance moving forward. The quality of communication across the consortium can also be seen to have improved (and fluctuated) over the course of the project, with different Lead Sponsors adopting different approaches and style. Whilst communication had improved by the time of evaluation, there remained regret that more peer-to peer learning had not happened through Practitioner Meetings. The frequency of these meetings reportedly suffered as a result of the Lead Sponsor having to focus on resolving the challenge presented by the withdrawal of MEWN. Again, this can be seen to be a feature of the significant amount of ‘change’ with which the project has had to contend. The evaluation points to a number of key learning points relating to how consortium projects such as this can best be set up and governed. Whilst these findings are not unique to this project, the diverse challenges encountered by the MAWRII consortium, bring into sharp focus the importance of good collaboration, governance, leadership and communication. Some key learning points include:

• The importance of partners achieving a common understanding of the project rationale, delivery model and reporting requirements before the project begins. Leaving one partner to set the agenda exposes consortium members to risks. It is clear that the progress of the MAWRII project has repeatedly been hit by issues that stemmed from some of the misunderstandings and information gaps that were in place from the outset;

• Partners need to apply 'stress tests' to themselves and each other, as part of the consortium building process. This will ensure that all partners have the skills and the capacity to lead and/or participate, before the project gets underway;

• All partners need input and shared ownership of project design, data

capture and reporting processes. This helps ensure common understanding of what is required and allows partners to work together

_____________________________________________________________________________

90

to highlight improvements and ‘iron out’ any issues as the project progresses;

• Project consortia should strive to build in regular opportunities for

practitioners to meet, share their experiences, and develop common approaches to challenges and opportunities that occur in the project;

• The Lead Partner (working in collaboration with other Joint Sponsors)

should issue clear guidance around eligibility, indicator definitions, use of key paperwork, data capture and reporting protocols. Whilst building in some flexibility might be useful, the main focus should be on producing clear, effective and compliant processes that both meet the needs of the funder and help the project team deliver the project;

• The importance of maintaining open and constructive communication

with the funder, to allow challenges to be identified, discussed and resolved in way that safeguards delivery to beneficiaries.

There were also a number of operational learning points that can be used to inform future projects:

• Operational approaches to identifying need varied across the partners, and would have benefitted from a central project recruitment strategy;

• The process of identifying and evaluating need could have been improved by central consultative planning and sharing of effective guidelines and templates, to be adapted locally;

• Client registration would have benefited from a central model and

system: A bespoke central database from the start would have helped the project run more smoothly. Use of mandatory fields and ‘drop down boxes’ help to enforce data collection, which in turn reduces beneficiaries rejected by WEFO and improves final evaluation datasets.

6.4 Recommendations identifying future project direction The MAWRII project has been on a journey that has involved significant challenges and periods of change. The consortium has seen partners withdraw, sometimes under contentious circumstances, and the Lead Sponsor role change hands on two separate occasions. Many of these issues can be seen to have their roots in the origins of the project. It is undoubtedly the case that these changes have frustrated delivery, damaged momentum and led to lower achievements than might have been possible, had things run more smoothly. The experience demonstrates the challenges and risks associated with running

_____________________________________________________________________________

91

complex ESF projects under a consortium model, especially for third sector organisations with capacity and resource constraints. More positively, the project has managed to achieve considerable success in engaging with hard to reach communities, and delivering support that has been of benefit to participants, both individuals and employers. The project has also developed the capacity and expertise of those remaining consortium members, so that they will be better placed to deliver projects along these lines in future. The frustration felt at the way external events hampered project progress, should be tempered by a celebration of the resilience of remaining consortium members and their achievements in supporting hard to reach beneficiaries and delivering successfully against the indicator profiles agreed with WEFO. The project has successfully steadied itself after the changes in Lead Sponsor, and the subsequent withdrawal of YMCA Wales and MEWN. For the latter period of the project, the consortium has enjoyed a period of stability and a matching up-turn in achievements. The question that is inevitably difficult to answer is “what might the project have achieved, had it not had to contend with such major challenges and changes?” More could certainly have been achieved by the project if it had enjoyed a different history. Rather than speculate on the scale of what might have been, we turn to the future. The need for a project such as MAWRII remains. All beneficiaries were clear on this point – individuals and employers. Recommendation: Project partners should consider working collaboratively to develop a follow on project to MAWRII. In considering this recommendation, project partners might wish to explore the merits of focussing on the area where they have been particularly successful – engaging and supporting participants from BME and EEM groups – or whether to expand their reach across a more general project embracing a wider range of protected characteristics. Conclusions from these discussions should inform the consortium formed to take forward the initiative. We put forward a series of points that should be considered by project partners in framing any future project to promote employment prospects for people from BME and EEM communities:

• There is clear need for non-accredited support such as low-level English language, confidence-building, and understanding the UK job culture “to prepare trainees for accredited training”;

_____________________________________________________________________________

92

• MAWRII demonstrated that participant employment potential was

intrinsically bound up with everyday cultural and social issues. Solving some of these social issues typically involved accessing public or professional services, yet effective access appeared to represent major challenges for many participants – a feature of hard to reach groups. A recurring theme of the evaluation was the engagement of project Practitioners in helping to resolve these wider issues. A future project could incorporate this aspect in its intended outcomes;

• Consideration should be given to the optimal role of employers in the

delivery model and whether the ‘point of entry’ for an individual has any impact on the nature and depth of support provided. It was not clear from the evidence presented that individuals referred by their employers as a cohort for training had the same project experience as those that were recruited directly into the project. The work with employers has supported an increase in beneficiary numbers, and there were clear examples of where this interaction has also delivered good, relevant support and training for individuals. There were however some comments from practitioners that raised the question of whether some employers tended to direct their employees to undertake training that primarily benefitted the company, rather than best met the needs of the individual. This should not be a surprise, given company directors hold a fiduciary duty towards their company. Some further thought around how to fine tune the role of employers on the project, could help drive participant numbers and ensure that support is tailored to the needs of beneficiaries and employers;

• Provision of more long-term training at levels 3+ is required if there is to be a step-change in benefits to BME/EEM communities and the contribution of participants to society. Due consideration should be given to the costs of accessing training, including issues such as child care.

The final recommendation relates to the opportunities for shared learning that are offered by the findings of this evaluation. The data, testimonies and viewpoints collected over the course of this evaluation, provide a rich body of information for consideration by project partners, key stakeholders and funders. As a study of the organisational challenges (and the means of overcoming them) that can occur within a consortium project, the MAWRII experience may well be unique – and the lessons that stem from these events have the potential to directly inform the robust design of future projects of this kind. In addition, the project has successfully engaged with a diverse client group that has long

_____________________________________________________________________________

93

been considered ‘hard to reach’, providing access to a range of mentoring, support and training services that have helped people stay and progress in their job. The ‘currency of trust’ is something that all partners highlighted as having been a vital enabling factor in their work, and this is something that those planning similar projects in future should reflect upon. Given the ongoing need for support of this kind, and the equality, skills and employment targets that exist in the new EU Programmes for Wales, the success and learning factors of this project require closer examination to inform future delivery. Recommendation: The findings, conclusions and recommendations of this evaluation should be considered, to inform and refine future delivery options.