2nd external evaluation report of seecel activities

84
Confidential 1 Final Report for External Evaluation Comprehensive external evaluation of the SEECEL activities under Grant Contracts 2013/316- 501 and 2013/334-013 conducted at the SEE level (in SEECEL member states). 25 May, 2016 prepared by CARDET www.cardet.org Dr. Charalambos Vrasidas, Demetris Hadjisofoclis, Višnja Novosel & Sotiris Themistokleous July 2016 This project is funded by the European Union and co-funded by the Ministry of Entrepreneurship and Crafts of the Republic of Croatia This publication has been funded with support from the European Commission and he Ministry of Entrepreneurship and Crafts of the Republic of Croatia (MINPO). The content of this publication reflect the views only of the author, and can under no circumstances be regarded as reflecting the position of the European Commission and MINPO.

Upload: ngotuyen

Post on 14-Feb-2017

224 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: 2nd External Evaluation Report of SEECEL activities

Confidential 1

Final Report for External Evaluation Comprehensive external evaluation of the SEECEL activities under Grant Contracts 2013/316-

501 and 2013/334-013 conducted at the SEE level (in SEECEL member states).

25 May, 2016

prepared by CARDET

www.cardet.org

Dr. Charalambos Vrasidas, Demetris Hadjisofoclis,

Višnja Novosel & Sotiris Themistokleous

July 2016

This project is funded by the European Union and co-funded by the Ministry of Entrepreneurship and

Crafts of the Republic of Croatia

This publication has been funded with support from the European Commission and he Ministry of Entrepreneurship and Crafts of the Republic of Croatia (MINPO). The content of this publication reflect the views only of the author, and can under no circumstances be regarded as reflecting the position of the European Commission and MINPO.

Page 2: 2nd External Evaluation Report of SEECEL activities

Confidential 2

Contents Executive Summary ........................................................................................................................ 4

Overall Recognition .................................................................................................................... 4

Successes..................................................................................................................................... 5

Challenges ................................................................................................................................... 6

Recommendations ....................................................................................................................... 7

Introduction ..................................................................................................................................... 9

Entrepreneurial Learning in Europe.............................................................................................. 10

The EU Context ........................................................................................................................ 10

Entrepreneurship Education ...................................................................................................... 11

School Curriculum .................................................................................................................... 13

Teacher Training and Support .................................................................................................. 13

Project Overview .......................................................................................................................... 14

Organizational Structure ........................................................................................................... 14

Mission, Values and Strategic Goals ........................................................................................ 15

SEECEL Achievements ............................................................................................................ 16

Project Contracts under External Evaluation ............................................................................ 17

Evaluation Method ........................................................................................................................ 19

Aims .......................................................................................................................................... 19

Data Collection and Analysis.................................................................................................... 20

Overview ............................................................................................................................... 20

Qualitative Data: Interviews, Focus Groups, and Document Review .................................. 20

Quantitative Data: Online Survey Design............................................................................. 20

Data Analysis ........................................................................................................................ 21

Findings from the Educational Institutions Survey ...................................................................... 22

Findings from the Steering Committee and Stakeholders Survey ................................................ 36

Findings from Study Visits, Interviews, and Focus groups .......................................................... 47

Successes................................................................................................................................... 47

Recognition ........................................................................................................................... 47

Meeting its Objectives and Strategic Goals .......................................................................... 48

Page 3: 2nd External Evaluation Report of SEECEL activities

Confidential 3

Policy and Regional Cooperation ......................................................................................... 49

Governance ........................................................................................................................... 50

Schools .................................................................................................................................. 50

ISCED 3 ................................................................................................................................ 51

SBA and TNA ....................................................................................................................... 52

Challenges ................................................................................................................................. 53

Funding ................................................................................................................................. 53

Policy - Governance .............................................................................................................. 54

Schools and Teacher Training .............................................................................................. 55

Recommendations ......................................................................................................................... 57

Strategic Objectives & Policy ................................................................................................... 57

Communication ......................................................................................................................... 57

Governance ............................................................................................................................... 58

Program Implementation .......................................................................................................... 58

School Related .......................................................................................................................... 59

Community of Practice ............................................................................................................. 59

Teacher and Stakeholder Training ............................................................................................ 60

Monitoring and Evaluation ....................................................................................................... 60

References ..................................................................................................................................... 61

Annexes......................................................................................................................................... 63

List of Interviews ...................................................................................................................... 64

Tables from Educational Institutions Survey Results ............................................................... 67

Tables from Steering Committee and Stakeholder Survey Results .......................................... 72

Executive Summary for Grant Contract 2013/316-501 ................................................................ 77

Executive Summary for Grant Contract 2013/334-013 ................................................................ 82

Page 4: 2nd External Evaluation Report of SEECEL activities

Confidential 4

Executive Summary This report presents the findings and recommendations regarding the external evaluation of the

SEECEL activities under Grant Contracts 2013/316-501 and 2013/334-013 conducted at the SEE

level (in SEECEL member states). The aims of the evaluation were:

To document SEECEL’s progress against SEECEL’s mission and vision, its key policies

and the strategic targets set out in the 2013-2016 strategic plan.

To review and assess the relevance and appropriateness of the 2013-2016 strategic plan to

SEECEL’s mission, and the specific objectives of the current project.

To recommend changes of strategy and emphasis that should be incorporated into the

SEECEL’s 2017-2020 strategic plan.

Data were collected and analyzed using both quantitative and qualitative methods. In summary,

the data collected were from the following sources:

Review of the literature, documents, policies, reports, past evaluations, and EU documents.

Study visits in 4 countries (e.g. Croatia, Bosnia and Herzegovina, Montenegro, and the

Former Yugoslav Republic of Macedonia) and interviews with local stakeholders and

implementing institutions.

Skype interviews, phone interviews and/or questionnaires from, Albania, Serbia, and other

relevant stakeholders.

Two online surveys distributed to education institutions, Steering Committee members and

other stakeholders.

Interviews and focus group discussions with SEECEL team

Overall Recognition

SEECEL’s work is aligned with a number of European policy documents, strategies, and

initiatives. The EU2020 strategy places big emphasis on developing the Human Capital of Europe.

Recognizing that the large majority of enterprises in Europe are SMEs, building their capacity and

promoting entrepreneurship is of paramount importance. In 2006, the European Commission had

identified the ‘sense of initiative and entrepreneurship’ as one of the eight key competences for

lifelong learning. The 2008 Small Business Act (SBA) for Europe, the 2012 Communication on

Rethinking Education, the Entrepreneurship Action Plan 2020, the New Skills Agenda for Europe,

and the very recent EntreComp: The Entrepreneurship Competence Framework 2016, all place

entrepreneurship education and entrepreneurial learning at the center of the debate for improving

competitiveness, combating unemployment and supporting growth and sustainable development.

Furthermore, entrepreneurship is a key driver in the European Commission’s Investment Plan for

Europe (Juncker Plan) and is also integrated in the United Nation’s 2030 Agenda for Sustainable

Development.

After detailed data collection and analysis, it became apparent that SEECEL has achieved its

objectives and that its strategic goals. In some occasions success exceeded the expectations. Data

Page 5: 2nd External Evaluation Report of SEECEL activities

Confidential 5

analysis identified some issues that when addressed will constitute SEECEL as one of the most

respected, effective, and professional regional organizations in the field of Entrepreneurial

Learning. Below a summary of the most important findings are presented. For the full findings

and recommendations, please refer to the full report.

Successes

Detailed data collection and analysis demonstrated that SEECEL is successful in creating

the foundation for a developed entrepreneurship culture across the region. Already there is

expression of interest from countries beyond the region and the EU to participate and

collaborate with SEECEL. Furthermore, SEECEL fosters alignment with the EU policies,

strategies and actions in the field of entrepreneurial learning.

SEECEL has been extremely successful. As a result, the European Commission has

recognized SEECEL as a good practice in strategic regional cooperation, good practice for

conceptual solutions in the field of entrepreneurial learning as a key competence and good

practice in line with the first principle of the Small Business Act for Europe. SEECEL has

become a reference point both for EU Member States and the pre-accession countries and

was awarded with the following awards:

o Creators for Centuries reward and recognition for the year 2014 for contribution

to the development of entrepreneurship in South East Europe, awarded to SEECEL

by an independent international committee during the Regional summit of

entrepreneurs of the South East Europe the best 300;

o European Enterprise Promotion Award (EEPA) as National Winners in 2014

for Promoting the Entrepreneurial Spirit, awarded to SEECEL by the European

Commission;

o The Champion of Regional Cooperation of 2013, awarded to SEECEL by the

Regional Cooperation Council for its active and dynamic approach to regional

cooperation and contribution to the SEE 2020 preparation process;

o The European Projects Awards IPA 2013 - for the project entitled ''Developing

the Entrepreneurial Society in Western Balkans and Turkey'', in the category

“ongoing” projects and best actors in Regional and Local Development (2013);

o The Knowledge Economy Network (KEN), an international Best Practice Award

for "original and successful practice in any domain of knowledge society and

economy development" and especially for "good practice in successful regional

cooperation in training and education”.

The work of SEECEL is based on a successful model. Implementing the policy and

providing the field training and school implementations, is what makes SEECEL different

from other similar projects.

Page 6: 2nd External Evaluation Report of SEECEL activities

Confidential 6

SEECEL strongly supports and encourages networking and regional cooperation. SEECEL

had a big impact in member states at the policy level. Without SEECEL some of these

countries could not be at the same level when it comes to entrepreneurship learning.

Since 2009, SEECEL has developed in-depth knowledge and core expertise in the area of

entrepreneurship learning. As a result, SEECEL is a member of many expert groups at the

European Commission level working in the area of entrepreneurial learning.

All stakeholders are very satisfied with the staff and management team of SEECEL. The

staff and management were very supportive to countries in implementing programs and

reaching their targets.

Teachers, principals, and school staff interviewed, seemed to be very satisfied with the

program and the benefits of entrepreneurial learning. Teachers clearly indicated that their

greatest motivation and reward in participating in the program was the impact the program

had on learners.

SEECEL contributed substantially to the progress towards the Small Business Act for

Europe which SEECEL member states demonstrate. Particularly, with the human capital

dimensions (1&8) which are the focus of SEECEL.

As it was clearly stated in the OECD report “SME Policy Index: Western Balkans and

Turkey 2016”, SEECEL member states have made substantial progress towards the

implementation of the SBA, which is a good indication of the hard and dedicated work of

SEECEL and its team.

The Training Needs Assessment (TNA) is an important tool that contributes substantially

towards the evidence-based policy making. TNA supports SEECEL member states in

developing the training and support mechanisms their SMEs need to succeed and grow.

SEECEL participates in several EC expert and advisory groups such as the groups for

Transversal skills (DG EaC), Indicators (DG EaC), EnreComp (DG JRC), and WEGATE

– EU WE platform (DG GROW).

Challenges

Given the fact that SEECEL has already made substantial contribution in the region and

beyond, the European Commission should provide long-term support to SEECEL to

continue this very important work.

One of the challenges is due to the absence of a mandate to the Steering Committee

Members from some SEECEL member states. National governments need to provide such

a mandate because that will be the vehicle that will facilitate getting SEECEL related

activities, budgets, and other resources into the national budgets and that can support

SEECEL and member states strategic objectives.

The evidence-based model, tools and resources on Entrepreneurial Learning developed by

SEECEL, can only be implemented successfully with the support of teacher training

institutes in SEECEL member states.

Page 7: 2nd External Evaluation Report of SEECEL activities

Confidential 7

All activities and communication is conducted in English. As the project expands it needs

to have a plan for allowing local stakeholders to operate in their local language.

During program implementation, it is important to include local experts from the countries

in which the program is implemented.

None of the schools had prior experience with entrepreneurial learning and how to

implement it in their teaching practices. Schools would have been more efficient and

effective if more training was provided in advance of implementation activities.

Participating in teacher training is not required for schools that join the program which

leads to discrepancies in program implementation. The training offered is insufficient and

the actual implementation of the training (delivering to the teachers) needs to be improved.

In order to increase its reach and represent multiple stakeholders, and have more impact on

the local, national, and regional level the Steering Committee needs to include active

representatives and perhaps mentors from the business community.

The history of the region and the political complexities related to this history, present

certain challenges, particularly when there is a need to get stakeholders from various

countries to collaborate.

Recommendations

Since 2009, SEECEL has developed in-depth knowledge and capacity in the area of

entrepreneurship learning. SEECEL makes an important contribution towards

entrepreneurial learning, and it is important to continue and expand its work. Furthermore,

SEECEL could also play a more active role in the Mediterranean region and Eastern

Partnership Countries (EaP) since there was already strong expression of interest from

other non-EU countries to participate in SEECEL activities.

SEECEL has strong potential and the capacity to become part of the infrastructure solution

for EU, and it has developed an evidence-based model, tools and processes to implement

SME and entrepreneurship policies and practices.

The communication strategy of SEECEL needs to be adjusted so that it improves the

awareness and branding for SEECEL’s vision and role.

Many resources are already available on the SEECEL website and the Community of

Practice. As the project grows, it is important to build mechanisms to allow for more users

to engage with the online tools and benefit from SEECEL activities and interaction with

the SEECEL community. Also, the TASK BOX on the Community of Practice can be

further developed to allow for teachers to receive prompt feedback on their work.

Examples of integrating entrepreneurial learning in SEECEL member states and good case

studies and success stories need to be better documented and shared. Good success stories

have the potential to increase awareness among teachers, parents, policy-makers and the

general public about the many benefits of Entrepreneurial Learning.

Page 8: 2nd External Evaluation Report of SEECEL activities

Confidential 8

Business representatives, chambers of commerce, and SMEs, should be given a more active

role. Some actions that will strengthen the cooperation between the business sector, the

education sector and the broader community need to be designed and implemented.

Education plays an important role in the entrepreneurship learning ecosystem. As SEECEL

member states progress towards integrating entrepreneurial learning in their schools, they

need to begin drafting solid plans on how to scale up from a few schools to the whole

education system.

As the program grows, SEECEL should use elearning technologies to develop the training

of teachers, mentors and stakeholders to allow for flexibility of delivery and possibility to

reach out to as many countries as possible.

It is extremely important that teacher training is required for all schools that decide to join

the program. Teacher training needs to be well structured, and offered at school level in

parallel with ongoing teacher support.

SEECEL member states and their respective ministries of education and teacher training

authorities can develop a certification system which will certify teachers who complete the

training and implement the program and also act as a reward system for teachers and

schools.

In order to properly monitor the long term impact of entrepreneurial learning, it is important

to establish a benchmarking process including metrics and other tools to collect long term

data.

The full implementation of the Small Business Act for Europe should be continued. The

very important work of the Training Needs Assessment should be expanded and the

findings from the assessment need to be used by education authorities to develop custom

programs that will target each country’s specific needs.

Page 9: 2nd External Evaluation Report of SEECEL activities

Confidential 9

Introduction This document presents the methods, findings and recommendations from the external evaluation

of the SEECEL activities under Grant Contracts 2013/316-501 and 2013/334-013 conducted at the

SEE level (in SEECEL member states). The evaluation was conducted by the International

Research Center CARDET. The overall objectives of SEECEL are:

To create the foundation that could foster further development of the entrepreneurship

culture across the region and foster alignment with the EU in the field of entrepreneurial

learning.

To further work on systematic approach for the development of entrepreneurially literate

societies across the region and to support alignment of national policies with EU

recommendations and policy essentials related to lifelong entrepreneurial learning.

The specific objective was to build on existing activities and results from the 2009 – 2012

implementation period and developments achieved, to further develop the lifelong entrepreneurial

learning system through identified set of strategic goals in line with Human Capital Dimension

of the Small Business Act (SBA) for Europe, with special focus on ISCED 3 level general part

of secondary education.

The aims of the evaluation were:

1) To document SEECEL’s progress against SEECEL’s mission and vision, its key policies

and the strategic targets set out in the 2013-2016 strategic plan.

2) To review and assess the relevance and appropriateness of the 2013-2016 strategic plan to

SEECEL’s mission, and the specific objectives of the current project.

3) To recommend changes of strategy and emphasis that should be incorporated into the

SEECEL’s 2017-2020 strategic plan.

The evaluation was assigned to the International center CARDET (www.cardet.org). The

evaluation team consisted of the following:

Dr. Charalambos Vrasidas. Team Leader. More than 20 years of global expertise in

evaluation, education reform, project development.

Višnja Novosel, Local Expert. More than 10 years of professional experience.

Demetris Hadjisofoclis, Entrepreneurship Global Expert with more than 25 years of

professional experience.

Sotiris Themistokleous. Evaluation Expert with more than 10 years of professional

experience.

Page 10: 2nd External Evaluation Report of SEECEL activities

Confidential 10

Entrepreneurial Learning in Europe

The EU Context

The framework for developing the entrepreneurship capacity in Europe, is provided by several key

documents and policies which relate to employment, competitiveness, education and sustainable

development. The assumption is that entrepreneurial skills can be learned and therefore there is a

strong interest in integrating such skills and competencies in formal and informal learning

environments and curricula. In 2006, the European Commission had identified the ‘sense of

initiative and entrepreneurship’ as one of the eight key competences for lifelong learning.

One of the key policy documents that relates to the building of the human capital of SMES, is the

2008 Small Business Act (SBA) for Europe. The SBA is the policy framework for SMEs for EU

member states and pre-accession countries. As part of the accession process, pre-accession

countries are implementing the SBA and reporting on their progress to the European Commission.

The SBA consists of ten principles with specific indicators. Principle 1 focuses on entrepreneurial

learning and women entrepreneurs and principle 8 focuses on enterprise skills and innovation, with

one objective the better planning and fit between supply and demand for training of SMEs.

Conducting systematically Training Needs Analysis (TNA), is one of the key activities of

SEECEL within the framework of SBA. Recently, the OECD report “SE Policy Index: Western

Balkans and Turkey 2016” was released which focused on assessing the implementation of the

SBA for Europe. SEECEL member states have made substantial progress towards the

implementation of the SBA, which is a good indication of the hard and dedicated work that

SEECEL is doing in the region.

The EU2020 strategy places big emphasis on developing the Human Capital of Europe.

Recognizing that the large majority of enterprises in Europe are SMEs, building their capacity and

promoting entrepreneurship is of paramount importance. The 2008 Small Business Act for Europe

described earlier, the 2012 Communication on Rethinking Education, the 2013 Entrepreneurship

Action Plan 20201, the New Skills Agenda for Europe, and the very recent EntreComp: The

Entrepreneurship Competence Framework 2016, all place entrepreneurship education and

entrepreneurial learning at the center of the debate for improving competitiveness, combating

unemployment and supporting growth and sustainable development.

Within the “New skills agenda for Europe”, one of the key focuses is the development of key

competences and higher, more complex skills. As stated in the report, “Special attention will be

paid to promoting entrepreneurial and innovation-oriented mindsets, including encouraging

practical entrepreneurial experiences (EC 2016, A New Skills Agenda for Europe, p. 4-5)2.

1 http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=celex%3A52012DC0795 2 http://ec.europa.eu/education/news/2016/0610-education-skills-factsheet_en.htm

Page 11: 2nd External Evaluation Report of SEECEL activities

Confidential 11

Furthermore, entrepreneurship is a key driver in the European Commission’s Investment Plan for

Europe (Juncker Plan)t3 and is also integrated in the UN’s 2030 Agenda for Sustainable

Development. Specifically, In the United Nations Agenda 20304 article 4.4 states that one of the

aims is that “by 2030, substantially increase the number of youth and adults who have relevant

skills, including technical and vocational skills, for employment, decent jobs and

entrepreneurship.”

The development and promotion of entrepreneurial learning in schools constitutes a main objective

of policy makers across the European Union for several years (Eurydice, 2016). More people

constantly become aware of the young people’s ability to set up and develop a business, therefore

they create innovations in the region where they work. Entrepreneurship education is pivotal in

order to cultivate a relevant culture and mind-set, providing all the necessary attitudes, skills and

knowledge to young people in order to become creative, active citizens and successful in their

entrepreneurial endeavours (Martinez et al., 2008).

Entrepreneurship Education

One of the eight most important competences for lifelong learning was identified as “a sense of

initiative and entrepreneurship” (European Union, 2006). Entrepreneurship education has been

further studied and is perceived as a main competence by the “European Commission Thematic

Working Group on Entrepreneurship Education”, which formed the following definition:

“Entrepreneurship education is about learners developing the skills and mind-set to be able

to turn creative ideas into entrepreneurial action. This is a key competence for all learners,

supporting personal development, active citizenship, social inclusion and employability. It

is relevant across the lifelong learning process, in all disciplines of learning and to all forms

of education and training (formal, non-formal and informal) which contribute to an

entrepreneurial spirit or behaviour, with or without a commercial objective” (Thematic

Working Group on Entrepreneurship Education, 2014, p.9).

Research on entrepreneurial learning within the school context indicates that the implementation

practices in a national and European context may vary significantly. This resulted in developing a

diverse understanding and perceptions of entrepreneurship education across Europe (McCoshan,

2010). Following this, there is an imperative need to understand the definitions of entrepreneurship

education in all the different European countries, which will reflect on the future learning

outcomes. Notably, approximately half of the European countries use the abovementioned

definition, while around one third of them employ their own developed definition of

entrepreneurship education. Another noteworthy fact is that almost ten (10) European countries

have no common definition agreed in their national context for entrepreneurship education

(Eurydice, 2016). The Eurydice Report 2016 indicated that the majority of the national definitions

3 http://ec.europa.eu/priorities/jobs-growth-and-investment/investment-plan_en 4 http://www.un.org/sustainabledevelopment/development-agenda/

Page 12: 2nd External Evaluation Report of SEECEL activities

Confidential 12

for entrepreneurship educations share the perspective of the European definition that

entrepreneurship also reflects on the individual’s life.

The Eurobarometer survey published in 2012 (Entrepreneurship in the EU and beyond) revealed

that a 23% of the respondents across the EU participated in a school activity related to

entrepreneurial learning. Younger participants of the survey were almost 50% more likely to have

participated in a course relating to entrepreneurship (Eurobarometer, 2012). However, students

age 15 do not possess significant skills, such as problem-solving, which are important

entrepreneurial skills. The problem-solving skills are proved to be enhanced by the teachers and

the school curriculum (PISA, 2012).

In terms of strategies, the Eurydice Report 2016 sustained that specific strategies regarding

entrepreneurship education are more “coherent” and offer a comprehensive approach. A greater

range of actions will derive related to entrepreneurship education. In addition, countries that have

chosen to develop specific strategies are now moving towards incorporating these in innovation

strategies, which are broader. The OECD (2010) sustained that broader strategies related to

innovation could significantly contribute in coordinating education policies to increase innovation

of students. Interestingly, countries of northern Europe and West Balkans implement more narrow

and specific education strategies as far as entrepreneurship is concerned. This could be a factor

leading Denmark, Sweden and Finland to the top ranking of the European Innovation Scorecard

2015 (European Commission, 2015) and on the top ten (10) places worldwide according to the

Global Innovation Index (Johnson Cornell University, 2015).

The establishment of learning outcomes does not seem to constitute an important action for

strategies related to entrepreneurship education in Europe. One part of SEECEL’s success is that

three out of eight countries (Estonia, Poland, Denmark, Austria, Wales, Montenegro, the

former Yugoslav Republic of Macedonia and Bosnia and Herzegovina) which have developed

a strategic action regarding learning outcomes, are SEECEL member states (Eurydice Report,

2016). The lack of specific learning outcomes in relation to the entrepreneurship education may

be characterised as a main obstacle towards the development of a good quality and working

entrepreneurship education. It is important to note that SEECEL’s innovative approach in the

development of learning outcomes includes financial literacy and environment friendly insights

and suggestions. Furthermore, solely Estonia and Denmark have introduced the entrepreneurial

learning outcomes in the evaluation procedure, which is thought to be a main feature of an effective

entrepreneurship education. The development of Entrepreneurial Learning outcomes, led by

SEECEL is one main contribution to the field. The Eurydice Report (2016) also highlighted the

need of developing a solid monitoring framework across EU countries, as far as entrepreneurial

education is concerned. Only a few countries monitor the effectiveness of a strategy, while some

other broader strategies are not directly linked to learning outcomes or the impact they might have

on entrepreneurship education.

Page 13: 2nd External Evaluation Report of SEECEL activities

Confidential 13

School Curriculum

The “sense of initiative and entrepreneurship” has been identified as one of the eight most

important key competences for lifelong learning. Entrepreneurship education is mostly seen as an

objective of a cross-curricular form in the primary level of education across the EU.

Entrepreneurship education is more dominant during later stages of education, such as the

secondary level. It could take the form of a separate course or be a part of another subject, most

common of them being the economics, business studies and social sciences. It is also taught as a

part of an optional subject. Moreover, there is a lack of teaching guidelines, since guidelines are

most commonly provided to upper secondary schools, rather than lower education levels.

SEECEL’s work and the recent JRC “EntreComp: The Entrepreneurship Competence Framework”

report provide a good basis for countries to adopt and integrate entrepreneurial learning in their

school curricula. It is important to note that SEECEL’s work places a big emphasis on the whole

entrepreneurial eco system and this is based on the belief that schools can be the agents of change

which can contribute towards the development of entrepreneurial literate societies. In this sense

schools can be the key factors which contribute into developing and expanding the entrepreneurial

eco system.

Teacher Training and Support

Given the great importance of entrepreneurship education, there is a strong need to prepare pre-

service and in-service teachers to be able to integrate entrepreneurial competences in their

teaching. According to Eurydice (2016) thirteen (13) countries have already stated that there is a

need to train teachers in order to gain the necessary skills to teach entrepreneurship lessons.

Interestingly, “continuing professional development” courses related to entrepreneurship

education are provided to some teachers in twenty-eight (28) countries. Teachers are mainly

supported by the relevant authorities with the allocation of funding or with the development of

materials to guide their lessons. Only twelve (12) countries across the EU are provided with

guidelines for entrepreneurial learning, developed by the central authorities of the countries. No

country in Europe has been found so far to have fully incorporated entrepreneurship education in

the education system (Eurydice, 2016).

Page 14: 2nd External Evaluation Report of SEECEL activities

Confidential 14

Project Overview

Organizational Structure

According to the Strategic Plan 2013 – 2016, the South East European Centre for Entrepreneurial

Learning (SEECEL) is the first expertise centre and think tank that arose from South East European

countries’ interest to establish structured strategic cooperation in development of life-long

entrepreneurial learning (LLEL), in line with EU policies. SEECEL was established in 2009 in

Zagreb on the initiative and support of the Government of the Republic of Croatia with two co-

founders: Ministry of Economy, Labour and Entrepreneurship (nowadays Ministry of

Entrepreneurship and Crafts) and the Croatian Chamber of Economy. SEECEL also received full

written support from all its member states and the European Commission. SEECEL member states

are as follows: Albania, Bosnia and Herzegovina, Croatia, Kosovo*, the former Yugoslav

Republic of Macedonia, Montenegro, Serbia and Turkey.

All SEECEL member states share ownership of the process and equally participate in the

governance of SEECEL, in content development and implementation. As an institution, SEECEL

is governed by an international steering committee composed of two representative members of

each SEECEL member state – one from the Ministry of Education and one from the ministry in

charge for the Small Business Act for Europe (SBA). This ensures the policy dialogue between

the world of education and economy.

A Management Support Team (MST) provides advice and support to the SEECEL Director on

management and technical issues. The MST is composed of 3 national co-ordinators of the Small

Business Act for Europe (the Croatian co-ordinator as permanent MST member and two other on

a six-month rotation applying the "EU troika" rotation principle ). Current state members of the

MST are as follows (since April2016): Kosovo*, Montenegro and Croatia.

SEECEL has an international governing board composed of appointed representatives from eight

SEECEL member states (one from the ministry of education and one from the ministry responsible

for the implementation of the Small Business Act for Europe) and representatives of the European

Commission's DG NEAR and DG GROW, European Training Foundation (ETF), Croatian

Chamber of Economy (CCE), Regional Cooperation Council (RCC) and the Organization for

Economic Cooperation and Development (OECD). Because of the core expertise developed,

SEECEL participates in several EC expert and advisory groups such as the groups for Transversal

skills (DG EaC), Indicators (DG EaC), EnreComp (DG JRC), and WEGATE – EU WE platform

(DG GROW).

From its establishment SEECEL operates in the regional environment by involving the national

authorities and experts in its activities. From 2013, SEECEL is also applying the principle of

secondment and based on this experience further encourages the equal opportunities of

employment of experts coming from the region. SEECEL is also an active member in various

Page 15: 2nd External Evaluation Report of SEECEL activities

Confidential 15

working groups and thematic forums established under the EU and different macro-regional

strategies, such as Transversal Skills, Danube Strategy, Adriatic – Ionian Strategy and South East

European 2020 Strategy, as well as reference point for various EU bodies and their expert working

groups. SEECEL, with the full support of all participating countries, is the sole approved regional

infrastructure project for social infrastructure, through the Western Balkans Investment

Framework (WBIF).

Mission, Values and Strategic Goals

According to SEECEL’s Strategic Plan 2013 – 2016 SEECEL’s mission is to work on systematic

development of lifelong entrepreneurial learning as a key competence and on alignment of policies

and practice with those of the European Union by strengthening the structured regional

cooperation. SEECEL’s vision is to strengthen entrepreneur-friendly environments and mind-sets

for building entrepreneurial literate societies that lead to sustainable economic growth and

development.

According to the Strategic Plan 2013 – 2016, since 2009 SEECEL has undertaken an intensive

work programme in line with a defined strategy and SEECEL member states’ recommendations.

The first years were dedicated to establishing links with relevant institutions across the region and

raising awareness of upcoming SEECEL activities to be further followed by direct and concrete

activities in each of the participating countries. SEECEL’s work and decision-making is fully

based on EU Open Method of Coordination (OMC), which continually supports and strengthens

cooperation and collective decision-making in the field of education and entrepreneurship – all

with the goal of fostering competitiveness and supporting sustainable growth.

According to the Strategic Plan 2013 – 2016, SEECEL’s strategic goals are the following:

1. To continue to support participating countries in their efforts to accommodate EU

recommendations for promotion of entrepreneurship as a key competence by specifically

addressing entrepreneurship in early education (ISCED 1 and ISCED 2 level).

2. To support participating countries in their efforts to accommodate EU recommendations

for promotion of entrepreneurship as a key competence by specifically addressing

entrepreneurship at general part of secondary education (ISCED 3 level).

3. To continue to enhance the contribution of third-level education to the competitiveness

drive by way of increased awareness and understanding among higher education

establishments of entrepreneurial learning particularly in non-business disciplines (ISCED

5&6).

4. To continue to reinforce more sustainable development of enterprise- driven training needs

analysis frameworks in participating countries, both general ones (Training Needs

Assessment) and those focused on women (WETNAS).

Page 16: 2nd External Evaluation Report of SEECEL activities

Confidential 16

5. To support participating countries in SBA assessment by establishing a peer-learning

methodology for SBA assessment (for principles 1 and 8) and by running the first regional

SBA peer-learning exercise.

6. To actively involve national teacher training authorities in development of lifelong

entrepreneurial learning as a key competence and its introduction to the educational

systems at national levels.

7. To support development of policies and to promote best policy practices in women

entrepreneurship in line with the Small Business Act for Europe and capacity building of

national and regional women entrepreneur’s networks & associations.

8. To further work on development and improvement of Information Gateway, called

Community of Practice (CoP) that proved to be a relevant tool for exchange of information

and good practices, as well as a useful platform for expert work.

SEECEL Achievements

SEECEL has successfully implemented the Grant Contract signed with the European Commission

from IPA Multi-beneficiary program (85% EC funds, 15% Croatian government) in total value of

2 million euro. Area of action: 8 countries. Implementation period: 2009 – 2013. In addition,

SEECEL successfully implemented the Grant Contract for the project “Women Entrepreneurship

– a job creation engine for South East Europe” signed with RCC funded by SIDA (100% financing)

in total value of 1 million euro. Area of action: 9 countries (with addition of Moldova).

Implementation period: 2011 – 2015.

In 2012, Croatia hosted the "Entrepreneurship-Education Regional Summit” where the respected

Ministers (or their appointed envoys) from the fields of economy and education from eight (8)

SEECEL Member States signed "A Charter for Entrepreneurial Learning: the Keystone for Growth

and Jobs" thus reconfirming their strong commitment to support the regional cooperation in the

field of entrepreneurial learning and human capital development. The summit was organized

jointly by the Ministry of Entrepreneurship and Crafts of the Republic of Croatia and SEECEL,

with the support of the Ministry of Science, Education and Sports of the Republic of Croatia and

the European Commission.

The European Commission has recognized SEECEL as best practice in strategic regional

cooperation, best practice for good conceptual solutions in the field of entrepreneurial

learning as a key competence and best practice in line with the first principle of the Small

Business Act for Europe.

Page 17: 2nd External Evaluation Report of SEECEL activities

Confidential 17

Project Contracts under External Evaluation

Currently SEECEL is implementing two Grant Contracts signed with the European Commission

from IPA Multi-beneficiary program (the same ratio 85%-15% of financing) in total value of 4

million euro. Area of action: 8 countries. Implementation period: 2013 – 2016. These two contracts

are the focus of this evaluation.

Grant Contract 2013/316-501: Developing the entrepreneurial society in Western Balkans

and Turkey - support to the South East European Centre for Entrepreneurial Learning

(SEECEL)

The overall objective of the project is to further work on systematic approach to develop

entrepreneurially literate societies across the region and to support alignment of national policies

with EU recommendations and policy essentials related to lifelong entrepreneurial learning. The

specific objective of the project is to further develop the lifelong entrepreneurial learning system

through identified set of strategic goals in line with Human Capital Dimension of the SBA for

Europe. The estimated results are:

1) Existing entrepreneurial learning instrument further developed, strategically piloted and

disseminated for ISCED 1, 2 and 5/6 level institutions (focusing on curriculum through learning

outcomes, teacher training and school management).

2) In-service teacher training modules further developed, strategically piloted and disseminated to

agencies and institutions for teacher training.

3) Second generation of the Training Needs Assessment (TNA) questionnaire deployed at regional

level; first set of criteria for quality assurance system for training, according to TNA results defined

at regional level – both actions which support development of a Training Needs Analysis System.

4) Countries actively participate in the SBA assessment using peer learning/peer review

methodology and show progress in relation to implementation of Principles 1 and 8.

5) Knowledge sharing platform (Community of Practice) expanded and developed as a reference

source for all entrepreneurial learning developments for increased participation of experts and

institutions.

Grant Contract 2013/334-013: Developing the entrepreneurial society in Western Balkans

and Turkey at ISCED 3 level general part of secondary education - support to the South East

European Centre for Entrepreneurial Learning (SEECEL)

The specific objective of the project is, Building on existing activities and results from the 2009 –

2012 implementation period and developments achieved, to further develop the lifelong

entrepreneurial learning system with special focus on ISCED 3 level general part of secondary

education through identified set of strategic goals in line with Human Capital Dimension of the

SBA for Europe. The estimated results are:

Page 18: 2nd External Evaluation Report of SEECEL activities

Confidential 18

1) Based on the existing entrepreneurial learning instrument, further developed, strategically

piloted and disseminated Entrepreneurial Learning (EL) instrument for ISCED 3 level general part

of secondary education focusing on curriculum through development of learning outcomes and

development of entrepreneurial school with special focus on teacher and school management

training.

2) The developed entrepreneurial learning instrument for ISCED 3 level general part of secondary

education strategically piloted in 8 participating countries.

3) In collaboration with SEECEL, peer visits/peer learning exercises by representatives of

SEECEL entrepreneurial schools.

4) Knowledge sharing platform (Community of Practice) further improved and recognized as a

reference source for all entrepreneurial learning developments for increased participation of

experts and institutions, with special focus on introducing the CoP to the ISCED 3 level.

Page 19: 2nd External Evaluation Report of SEECEL activities

Confidential 19

Evaluation Method

Aims

The aims of the evaluation, as presented in the Terms of Reference (ToR) and as discussed during

the kickoff meeting in Zagreb on May 9, 2016, are:

To document SEECEL’s progress against SEECEL’s mission and vision, its key policies

and the strategic targets set out in the 2013-2016 strategic plan.

To review and assess the relevance and appropriateness of the 2013-2016 strategic plan to

SEECEL’s mission, and the specific objectives of the current project.

To recommend changes of strategy and emphasis that should be incorporated into the

SEECEL’s 2017-2020 strategic plan.

For the purpose of assessing the above aims, we developed some key questions to guide the process

of data collection and analysis:

To what extent are the strategic goals and approaches in the 2013-2016 plan and key

policies, relevant to the mission of SEECEL?

To what extent are the SEECEL activities relevant to its goals?

To what extent have the approaches identified in key policies, the 2013-2016 strategic plan

and SEECEL’s rolling work plan addressed the conditions, needs and interests of key

stakeholders and enabled SEECEL to pursue its mission?

Were the “approaches” and the targets set out in the strategic plan relevant and realistic?

How effective were the different approaches in the key policies, the 2013-2016 strategic

plan and SEECEL’s rolling work plan in pursuing the mission and vision of SEECEL?

Did SEECEL have the necessary resources and capacity to undertake the activities

required to achieve its strategic goals?

Are SEECEL’s key stakeholders satisfied with the approaches and activities used to date

in achieving its mission?

Page 20: 2nd External Evaluation Report of SEECEL activities

Confidential 20

Data Collection and Analysis

Overview

Data were collected and analyzed using both quantitative and qualitative methods. First, the

evaluation team reviewed carefully the ToR and the project proposals for the two contracts under

evaluation, the SEECEL website, the Community of Practice (CoP) and key documents relating to

SEECEL. Based on this first stage review, an interview and a focus group guide were developed

and used during the kick off visit of the evaluation team in Zagreb.

Qualitative Data: Interviews, Focus Groups, and Document Review

Data were collected from the following sources:

1. Study visits in 4 countries (e.g. Croatia, Bosnia and Herzegovina, Montenegro, the former

Yugoslav Republic of Macedonia) and interview with local stakeholders and implementing

institutions. Details on the interviews conducted during study visits are presented in the

ANNEX.

2. One focus group per country was conducted in each of the four countries visited (Croatia,

Bosnia and Herzegovina, Montenegro, the former Yugoslav Republic of Macedonia).

Participants were school staff and key stakeholders and the aim was to examine the

successes and challenges of SEECEL.

3. A face-to-face focus group discussion was facilitated with the SEECEL team to better

understand the project, its successes and challenges.

4. Skype Interviews and/or questionnaires from, Albania, Serbia, and other relevant

stakeholders (It was not possible to get any response for an interview from Turkey and

Kosovo*, despite several attempts from the evaluation team).

5. Skype and phone interviews with key Steering Committee (SC) members and stakeholders

(full list is presented in ANNEX).

Additional qualitative data came from the sources below:

Minutes from meetings and researcher memos

Policy and other EU documents

School documents (action plans, lesson plans, peer learning and piloting reports)

SEECEL proposals and deliverables from each contract

Quantitative Data: Online Survey Design

In order to develop the two survey instruments used in this evaluation, the evaluation team first

conducted in-depth qualitative research and a study visit in Croatia during which they interviewed

several key stakeholders, including Ministry of Education and Ministry of Entrepreneurship and

Crafts representatives, Chamber of Commerce representatives, school staff, and members of the

SEECEL team. Based on the findings of the interviews, along with a detailed review of key

documents, the two survey instruments were developed.

Page 21: 2nd External Evaluation Report of SEECEL activities

Confidential 21

Two survey instruments were prepared and distributed: one for SC members and key stakeholders

and one for education institutions. The evaluation team analyzed in detail the qualitative data (e.g.

interviews and observations) and identified the main categories regarding the aspects of the

program to evaluate. Then, items for the questionnaires were designed to collect data for each of

the key categories (e.g. project features, communication, frequency of resource use, etc.).

In order to validate the tools, a first version of the instruments was distributed to research design

experts, members of the SEECEL team, and independent reviewers with expertise in EL, with the

aim to pilot the instruments and make the necessary revisions. Once finalized, the tools were

developed into an online format using the online tool Survey Monkey. They were tested again, and

once finalized they were distributed to the list of key stakeholders and education institutions which

participated in SEECEL activities.

Data Analysis

Once data were collected, they were analyzed by following both the deductive and inductive stages

(Stake, 2011). The survey data were analyzed by calculating descriptive statistics. Concept maps,

tables and data displays were prepared to better understand the success and challenges of the

program. Reading the data multiple times, allowed the evaluation team to gradually identify key

themes and assertions trying to address the evaluation questions posed for the study. Data were

examined and carefully sorted “according to topics, themes, and issues important to the study”

(Stake, 2011, p. 151). In order to triangulate the findings presented in this report and strengthen

the validity of findings claims, evidence is used from both the qualitative and quantitative sources.

Each finding was reviewed and audited by two members of the evaluation team to ensure the

validity of findings. For each of the scales on the surveys the Cronbach alpha was calculated,

which indicated high internal validity of the tools used.

Page 22: 2nd External Evaluation Report of SEECEL activities

Confidential 22

Findings from the Educational Institutions Survey The number of respondents who answered the survey varied among certain survey questions. The

initial questions were answered by 78 respondents. Below we present some of the key findings.

Detailed tables regarding Median scores and Standard Deviations are presented in the ANNEX.

SEECEL Features

The respondents from educational institutions were asked to indicate their level of

satisfaction/dissatisfaction with the 18 distinct project features, on the five point Likert scale (1-

very dissatisfied – 5 very satisfied). The Cronbach's alpha for the 18-item scale was 0.940,

indicating the excellent scale reliability. According to the findings presented in Figure 1, the

respondents were the most satisfied with the: Opportunity to implement lesson plans in teaching

(44.90% very satisfied; 46.40% satisfied); Opportunity to implement certain activities from lesson

plans in teaching (56.50% very satisfied; 33.30% satisfied); Experiences from piloting the action

plan (58.00 % very satisfied; 31.90% satisfied); Experiences from peer learning activities

(73.90% very satisfied; 15.00% satisfied); Communication and cooperation with other people in

your school involved in the project (49.30% very satisfied; 37.70% satisfied); Student

participation in activities (75.40% very satisfied; 15.90% satisfied); Overall benefits for students

(65.20% very satisfied; 26.10% satisfied); and Overall benefits for the local community (39.10%

very satisfied; 37.70% satisfied).

It is important to emphasize that 75.4% of respondents were very satisfied with the SEECEL’s

response to problems that came up during implementation. It is apparent that the respondents were

satisfied with program features which were directly linked to the school and student involvement

and SEECEL’s role in the programme. The respondents were the least satisfied with the: Feedback

we received on lesson plans; Orientation program for school management; and Overall benefits

for the local community. Nevertheless, even in these categories more than 60% of the respondents

were satisfied or very satisfied with the above mentioned project feature.

Page 23: 2nd External Evaluation Report of SEECEL activities

Confidential 23

Figure 1 Level of the respondents’ satisfaction/dissatisfaction with the distinct project features.

Participation in SEECEL Activities

The respondents were asked to indicate their level of agreement/disagreement with the 8

statements describing the impact of their participation in SEECEL activities, on the five point

Likert scale (1- strongly disagree – 5 strongly agree). The Cronbach's alpha for the 8-item scale

was 0.887, indicating the good scale reliability. According to the findings presented in Figure 2,

the respondents recognized the positive impact of the activities on all listed aspect of school work

and organizational culture. The large majority of the respondents agreed or strongly agreed that

the project had positive impact on the: Improvement of creativity among students (60.87% strongly

agreed; 34.78% agreed); Improvement of the entrepreneurship culture in their school/organization

42,00

37,70

26,10

44,90

56,50

36,20

50,70

58,00

73,90

75,40

49,30

52,20

75,40

46,40

65,20

39,10

68,10

66,70

43,50

49,30

36,20

46,40

33,30

39,10

36,20

31,90

15,90

13,00

37,70

24,60

15,90

39,10

26,10

37,70

23,20

26,10

5,8

05

,80

15

,90

2,9

04

,30

14

,50

7,2

02

,90

4,3

04

,30

4,3

01

5,9

0

4,3

07

,20

2,9

0

14

,50

4,3

02

,90

1,4

04

,30

17

,40

2,9

02

,90

7,2

02

,90

2,9

02

,90

4,3

04

,30

2,9

01

,40

2,9

02

,90

5,8

01

,40

1,4

07

,20

2,9

04

,30

2,9

02

,90

2,9

02

,90

4,3

02

,90

2,9

04

,30

4,3

02

,90

4,3

02

,90

2,9

02

,90

2,9

0

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

1.     The training I received from SEECEL and its partners

2.     Quality of the lesson plans developed as part ofinvolvement in SEECEL activities

3.     Feedback we received on lesson plans

4.     Opportunity to implement lesson plans in teaching

5.     Opportunity to implement certain activities fromlesson plans in teaching

6.     The orientation program for school management

7. Experience from developing your organization’s action plans

8.     Experiences from piloting the action plan

9.     Experiences from peer learning activities

10. SEECEL’s response to problems that came up during implementation

11.   Communication and cooperation with other peoplein your school involved in the project

12.   Collaboration with Mentor Schools

13.   Student participation in activities

14.   Overall benefits for teachers

15.   Overall benefits for students

16.   Overall benefits for the local community

17. The program’s contribution to the school’s entrepreneurial culture

18.   Overall participation in SEECEL activities

Very Satisfied Satisfied Neutral Dissatisfied Very Dissatisfied

Page 24: 2nd External Evaluation Report of SEECEL activities

Confidential 24

(46.38% strongly agreed; 49.28% agreed); Improvement of school management team (52.17%

strongly agreed; 36.23% agreed); and Engagement of students in the classroom (43.48% strongly

agreed; 47.83% agreed).

The respondents recognized the overall positive impact of the participation in SEECEL activities,

and listed some additional benefits of their participation in SEECEL activities such as: new

contacts, improved cooperation between teachers and with parents, new tools and new school

curriculum, benefits for students (preparation for lifelong learning, work opportunities with local

businesses, improved motivation and initiative).

Figure 2 Level of the respondents’ agreement with the statements on positive impact of the

participation in SEECEL activities.

CoP Platform

The Community of Practice Task Box is divided into 5 main sections. The first four sections, are

divided in accordance to the ISCED levels and allow educational institutions to submit project

documentation. The fifth section provides for teacher training authorities (TTA) project

documentation submission. The documents under Task Box were visible just to the SEECEL

experts and institutions/authorities who authored/submitted the documents. As part of the project,

the educational institutions were required to submit 18 different documents, while TTA’s were

required to submit 14 different documents.

46,38

20,29

27,54

60,87

52,17

37,68

43,48

43,48

49,28

65,22

57,97

34,78

36,23

50,72

47,83

44,93

2,9

0

11,59

13,04

2,9

0

8,70

8,70

5,80

8,702,

901,

451,

452,

901,

451,

451

,45

1,4

51,

451,

451

,45

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

1.     The entrepreneurship culture in myschool/organization has improved

2.     Teaching has improved

3.     Learning has improved

4.     Creativity among students has improved

5.     School management team has improved

6.     Creativity among teachers has improved

7.     Students are more engaged in the classroom

8.     The overall entrepreneurship eco-system culturehas improved (Classroom, school, and local community

culture)

Strongly Agree Agree Neither Agree nor Disagree Disagree Strongly Disagree

Page 25: 2nd External Evaluation Report of SEECEL activities

Confidential 25

The survey respondents evaluated the frequency and purpose of the CoP platform use through 6

statements, on the five point Likert scale (1-never, 2- 1 to 2 times per semester, 3- 1 to 2 times per

month, 4- 1 to 2 times a week, 5 – daily). The Cronbach's alpha for the 6-item scale was 0.829,

indicating the good scale reliability. The findings are presented in Table 1 and Figure 3, were it is

visible that the respondents on average used the CoP platform up to 2 times per month and that

they recognize it as a tool for Learning and Downloading resources. According to the respondents’

answers, the frequency of the CoP platform use differed in various stages of the project. The

respondents are rarely using the platform for Discussion with other stakeholders, but they

emphasised that besides CoP platform, they used popular social networks (e.g. Facebook,

WhatsApp, regular email) to discuss and share project ideas which helped them in many school

activities.

Table 1 Frequency and purpose of CoP platform use.

Daily 1-2 times a

week

1-2 times

per month

1-2 times per

semester

Never

1. Sharing resources and ideas

16 27 17 1

2. Learning 2 15 30 14

3. Downloading resources 4 12 26 17 2

4. Viewing other school’s lesson plans 1 14 15 29 2

5. Discussions with other teachers 5 8 17 22 9

6. Discussion with other stakeholders 1 6 14 31 9

Page 26: 2nd External Evaluation Report of SEECEL activities

Confidential 26

Figure 3 Frequency and purpose of CoP platform use.

The respondents who did not use the CoP platform were asked to specify reasons for not using it.

Their answers included reasons like the use of other means of communication (emails, social

networks, and phone conversations), lack of understanding on how to use platform and what to

talk about under the CoP discussion forums, lack of computers and internet at school, the setup of

the CoP platform which is not aligned to teachers’ needs. As one respondent wrote: ‘The CoP

might have been more and better utilized if there had been spaces for subject teachers (not just the

PI's managers and coordinators) to collaborate, revise Lesson Plans and other activities, and

exchange best practices’.

The main advantage of the available document submission structure are straightforward guidelines

on type of required documents and clear deadlines for document submission. Also, it seems like

the project participants appreciate the possibility to communicate with the SEECEL’s experts

under the different sections of the site (which are directly linked to the particular type of

document). The document templates are clearly visible. Also, after receiving feedback on the

initial submission from the SEECL’s expert team, the participants have the opportunity to resubmit

the documentation. One of the challenges, based on the reviewed documents, is the fact that all

participating educational institutions did not have sufficient human resources to deal with the

preparation and write-up of project documentation in English language. Hence, some of the

documents that required more complex descriptions and reflections on project implementation,

were written in poor English and/or did not include well elaborated answers. While the deadlines

for document submission are clearly stated, the participants frequently commented that the

document exchange should occur more frequently. This comment could be linked with the

participants’ need for more frequent feedback on the project materials and implementation. The

1,5

4,6

7,7

3,1

9,2

3,1

24,6

23,1

18,5

21,5

12,3

9,2

41,5

46,2

40,0

23,1

26,2

21,5

26,2

21,5

26,2

44,6

33,8

47,7

6,2

4,6

7,7

7,7

18,5

18,5

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

1.     Sharing resources and ideas

2.     Learning

3.     Downloading resources

4. Viewing other school’s lesson plans

5.     Discussions with other teachers

6.     Discussion with other stakeholders

Daily 1-2 times a week 1-2 times per month 1-2 times per semester Never

Page 27: 2nd External Evaluation Report of SEECEL activities

Confidential 27

main challenge with the TTA section of the task box are poor responses from at least two national

teacher training authorities. While the cooperation with TTA’s is crucial for the project success, in

order to have insight into the work of national TTA’s, the alternative ways of document collection

and communication should be considered.

According to the lesson plan evaluation data, it is possible to conclude that the reviewed lesson

plans focus on EL objectives in the large number of different subject areas. Another positive

element, linked to the lesson plans is that once lesson plans are approved by the SEECEL editorial

board, they are published under the Teacher knowledge base (TKB) and can be freely reached by

interested parties. The final report structure is clear and the project teams attempted to answer all

the required sections of the report. Nevertheless, the analysis of the final reports indicate large

variations in the challenges or problems which project teams were facing during the project

implementation. In conclusion, the final reports differ by the quality which commonly resembles

the quality of the project implementation.

The respondents were asked to evaluate the CoP’s ease of use through 6 statements, on the five

point Likert scale (1- strongly disagree – 5 strongly agree). The Cronbach's alpha for the 6-item

Level of the respondents’ agreement with the statements on CoP’s ease of use scale was 0.892,

indicating the good scale reliability. According to the findings presented in Figure 4, the

respondents appreciate the User friendly features (49.92% strongly agree; 43.1% agree) and Ease

of use (43.1% strongly agree; 49.2% agree).

Figure 4 Level of the respondents’ agreement with the statements on CoP’s ease of use.

SEECEL Website

The survey respondents evaluated the frequency and purpose of the SEECEL Website use through

3 statements, on the five point Likert scale (1-never, 2- 1 to 2 times per semester, 3- 1 to 2 times

43,1

40,0

41,5

49,2

40,0

35,4

49,2

50,8

46,2

43,1

50,8

41,5

3,1

3,1

6,2

3,1

4,6

9,2

3,1

4,6

4,6

1,5

3,1

10,

8

1,5

1,5

1,5

3,1

1,5

3,1

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

1.     It is easy to use

2.     It is easy to understand

3.     It is fast

4.     It is user friendly

5.     It is flexible

6.     I can use it without instructions

Strongly Agree Agree Neither Agree nor Disagree Disagree Strongly Disagree

Page 28: 2nd External Evaluation Report of SEECEL activities

Confidential 28

per month, 4- 1 to 2 times a week, 5 – daily). The Cronbach's alpha for the 3-item scale was 0.884,

indicating the good scale reliability. The findings are presented in Table 2 and Figure 5, were it is

visible that the respondents on average, used the SEECEL Website up to 2 times per month, and

primarily recognized SEECEL Website as a tool for Learning. In addition, the respondents

emphasized that they would benefit from the website in their native language, because ‘many

teachers do not have sufficient English language skills’.

Table 2 Frequency and purpose of SEECEL Website use.

Daily

1-2 times a

week

1-2 times per

month

1-2 times per

semester Never

1. Sharing resources and ideas 1 9 22 27 1

2. Learning 2 12 26 20 0

3. Downloading resources 2 7 25 24 2

Figure 5 Frequency and purpose of SEECEL Website use.

Respondents were asked to evaluate the SEECEL website ease of use through 6 statements, on the

five point Likert scale (1- strongly disagree – 5 strongly agree). The Cronbach's alpha for the 6-

item scale was 0.903, indicating the excellent scale reliability. According to the findings presented

in Figure 6, the respondents appreciate the User friendly features (50% strongly agree; 42.2%

agree) and Flexibility (42.2% strongly agree; 50% agree) of the SEECEL website.

1,6

3,1

3,1

14,1

18,8

12,5

34,4

40,6

39,1

43,8

34,4

39,1

6,3

3,1

6,3

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

1.     Sharing resources and ideas

2.     Learning

3.     Downloading resources

Daily 1-2 times a week 1-2 times per month 1-2 times per semester Never

Page 29: 2nd External Evaluation Report of SEECEL activities

Confidential 29

Figure 6 Level of the respondents’ agreement with the statements on SEECEL website ease of

use.

Organization, Collaboration and Support

On the five point Likert scale (1- very dissatisfied – 5 very satisfied), the respondents were asked

to indicate their level of satisfaction/dissatisfaction with the program related communication and

collaboration through 4 statements. The Cronbach's alpha for the 6-item Level of the respondents’

satisfaction/dissatisfaction with the program related collaboration scale was 0.727, indicating the

acceptable scale reliability. According to the findings presented in Figure 7, the respondents were

very satisfied with collaboration with SEECEL staff (71.4% very satisfied; 19% satisfied).

Figure 7 Level of the respondents’ satisfaction/dissatisfaction with the program related

collaboration.

43,8

43,8

45,3

50,0

42,2

34,4

45,3

46,9

40,6

42,2

50,0

50,0

4,7

3,1

6,3

1,6

3,1

6,3

3,1

4,7

6,3

3,1

3,1

4,7

3,1

1,6

1,6

3,1

1,6

4,7

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

1.     It is easy to use

2.     It is easy to understand

3.     It is fast

4.     It is user friendly

5.     It is flexible

6.     I can use it without instructions

Strongly Agree Agree Neither Agree nor Disagree Disagree Strongly Disagree

71,4

36,5

44,4

19,0

39,7

36,5

3,2

12

,71

2,7

0,0

3,2

1,6

6,3

7,9

4,8

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

1.     Collaboration with SEECEL staff

2.     Collaboration with the teacher training agency

3.     Collaboration with other schools involved in theproject

Very Satisfied Satisfied Neutral Dissatisfied Very Dissatisfied

Page 30: 2nd External Evaluation Report of SEECEL activities

Confidential 30

On the five point Likert scale (1 - never, 5-very often), the respondents were asked to indicate the

extent to which they received support from different stakeholders through 3 statements. The

Cronbach's alpha for the 3 item scale was 0.578, indicating the questionable scale reliability.

Nevertheless, the measures of correlation indicate high correlation among the scale items (the

correlation is significant on the 0.01 level). According to the findings presented in Table 3 and

Figure 8, the respondents were most frequently supported by SEECEL staff. It is surprising that

they received least support from the Teacher training agency in their country. As reported by the

respondents, the schools received support from: governmental and local educational institutions,

entrepreneurship associations, mentor schools, non-profit organisations, local authorities and

parents.

Table 3 Extent to which the respondents received support from different stakeholders.

Very Often Often Sometimes Rarely Never

1. SEECEL staff 26 30 3 0 0

2. Teacher training agency in your country 11 16 17 9 6

3. Other schools involved in the project 12 21 19 3 4

Figure 8 Extent to which the respondents received support from different stakeholders.

When asked about mentoring approach which is an important feature of the project, as shown in

Figure 9, almost 30% of respondents answered that they did not use mentoring approach as a part

of this programme, while the respondents who used the mentoring approach were satisfied

(39.7%) or very satisfied (28.6%) with it.

41,3

17,5

19,0

47,6

25,4

33,3

6,3

28,6

31,7

3,2

17,5

7,9

1,6

11,1

7,9

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

1.     SEECEL staff

2.     Teacher training agency in your country

3.     Other schools involved in the project

Very Often Often Sometimes Rarely Never

Page 31: 2nd External Evaluation Report of SEECEL activities

Confidential 31

Figure 9 Respondents satisfaction/dissatisfaction with the mentoring approach.

On the five point Likert scale (1-never, 2- 1 to 2 times per semester, 3- 1 to 2 times per month, 4-

1 to 2 times a week, 5 – daily), the survey respondents evaluated the frequency of different types

of support they received from SEECEL staff, teacher training agency, and other schools. Three

scales with total of 10 items dealing with the different types of support received through the project

have Cronbach's alpha 0.897, indicating a good reliability of the scales. The findings for each of

the three scales are expected as they are aligned with the incidence of actual administrative

deadlines. The findings for support received from SEECEL staff are presented in Table 4 and

Figure 10, were it is visible that the largest percentage of respondents received a support from the

SEECEL staff up to 2 times per month and that they most commonly received Guidance about

administrative issues.

Table 4 Frequency of different types of support from SEECEL staff.

Daily

1-2 times

a week

1-2 times

per month

1-2 times

per

semester

Never

1. Guidance about administrative issues 3 7 28 20 0

2. Guidance about the education aspect of the

program

2 6 25 22 3

3. Workshops on how to integrate

entrepreneurship in the classroom

1 5 14 26 12

4. Additional resources 2 9 17 20 10

28,6

39,7

28,6

1,6 1,6

Very Satisfied Satisfied I have not used this approach Very Dissatisfied Dissatisfied

Page 32: 2nd External Evaluation Report of SEECEL activities

Confidential 32

Figure 10 Frequency of different types of support from SEECEL staff.

Furthermore, the survey respondents evaluated the frequency of different types of support from

teacher training agencies. The findings are presented in Table 5 and Figure 11, were it is visible

that the largest percentage of respondents (approximately 50% in each category) received support

from the teacher training agency up to 2 times per semester. In additional comments, one of the

respondents mentioned that teacher training agency in one of the participating countries organized

only one workshop during the year.

Table 5 Frequency of different types of support from teacher training agency.

Daily 1-2 times

a week 1-2 times

per month

1-2 times per

semester

Never

1. Guidance about the education aspect of program 1 2 11 34 11

2. Workshops on how to integrate entrepreneurship in the

classroom

1 1 11 34 12

3. Additional resources 0 2 10 31 16

Figure 11 Frequency of different types of support from teacher training agency.

4,9

3,3

1,6

3,3

11,5

9,8

8,2

14,8

45,9

41,0

23,0

27,9

32,8

36,1

42,6

32,8

3,3

8,2

23,0

19,7

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

1.     Guidance about administrative issues

2.     Guidance about the education aspect of theprogram

3.     Workshops on how to integrate entrepreneurshipin the classroom

4.     Additional resources

Daily 1-3 Times a Week 1-2 times per Month 1-2 times per Semester Never

1,6

1,6

0,0

3,3

1,6

3,3

18,0

18,0

16,4

55,7

55,7

50,8

21,3

23,0

29,5

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

1.     Guidance about the education aspect of program

2.     Workshops on how to integrate entrepreneurshipin the classroom

3.     Additional resources

Daily 1-3 Times a Week 1-2 times per Month 1-2 times per Semester Never

Page 33: 2nd External Evaluation Report of SEECEL activities

Confidential 33

Also, the survey respondents evaluated the frequency of different types of support from other

schools. The findings are presented in Table 6 and Figure 12, were it is visible that the largest

percentage of respondents (approximately 50% in each category) received support from other

schools up to 2 times per semester. The respondents’ additionally commented on help and benefits

gained through cooperation with mentor schools.

Table 6 Frequency of different types of support from other schools.

Daily 1-2 times a week

1-2 times per

month

1-2 times per

semester

Never

1. Guidance about the education aspect of program 0 5 12 30 11

2. Workshops on how to integrate entrepreneurship in the

classroom

0 3 13 32 11

3. Additional resources 0 3 11 31 14

Figure 12 Frequency of different types of support from other schools.

The respondents were asked to indicate their level of agreement/disagreement with the 6

statements describing different types of additional support for project participation. The

Cronbach's alpha for the 6-item Level of respondents’ agreement/disagreement on need for

different types of support in this project scale was 0.700, indicating the acceptable scale reliability.

According to the findings presented in Figure 13, the respondents recognized need for support in

a variety of areas. In addition, these findings are showing a need for future work of SEECEL. The

most commonly recognized areas that require support are: Peer learning (50% strongly agree;

38.3% agree) and Training (48.4% strongly agree; 40.3% agree). Educational institutions received

additional support for the project participation through teacher training, exchange of good practices

face-to-face and through social networks, teachers' manuals for lesson plans delivered by school

subject counsellors. Furthermore, the respondents commented that they would benefit from the

online EL teacher training course under the CoP platform.

1,6

1,6

1,6

8,1

4,8

4,8

19,4

20,6

17,5

48,4

50,8

49,2

22,6

22,2

27,0

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

1.     Guidance about the education aspect of program

2.     Workshops on how to integrate entrepreneurshipin the classroom

3.     Additional resources

Daily 1-3 Times a Week 1-2 times per Month 1-2 times per Semester Never

Page 34: 2nd External Evaluation Report of SEECEL activities

Confidential 34

Figure 13 Level of respondents’ agreement/disagreement on need for different types of support

in this project.

Finally, the respondents were asked to indicate their level of agreement/disagreement with the 5

statements dealing with project challenges. The Cronbach's alpha for the 5-item Level of the

respondents’ agreement with the listed project challenges scale was 0.817, indicating the good

scale reliability. According to the findings presented in Figure 16, most of the respondents did not

perceive listed challenges as problematic. Still, the respondents believed that Lack of funding for

project activities (12.9% strongly disagree; 41.9% disagree); Lack of funding for student and

teacher participation in peer learning activities (12.9% strongly disagree; 30.6% disagree) and

Lack of time for teachers to integrate entrepreneurship learning in the classroom (16.4% strongly

disagree; 31.1% disagree) may be somewhat more problematic than other listed challenges. Also,

the respondents mentioned that participation was difficult due to everyday obligations which they

have in the classroom and some other administrative issues which they are facing in their schools.

One of the complaints focused on the amount of administrative work involved with the project

participation.

31,1

48,4

33,9

53,2

45,9

50,0

59,0

40,3

46,8

30,6

36,1

38,3

4,9

3,2

1,6

6,5

6,6

5,0

3,3

4,8

12,9

4,8

9,8

5,0

1,6

3,2

4,8

4,8

1,6

1,7

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

1.     Resources

2.     Training

3.     Support

4.     Funding

5.     Time

6.     Peer learning

Strongly Agree Agree Neither Agree nor Disagree Disagree Strongly Disagree

Page 35: 2nd External Evaluation Report of SEECEL activities

Confidential 35

Figure 14 Level of the respondents’ agreement with the listed project challenges.

When asked to provide concrete recommendations for improving the implementation of SEECEL

activities, in an open ended question, the majority of the respondents mentioned the following:

more teacher training/workshops and assistance for teachers through the year;

more peer visits and sharing of experiences with other schools (school partnerships);

additional funding for peer visits;

transparent calendar with task deadlines and peer visits;

detailed and timely instructions on the materials that need to be submitted;

certificates and prizes for teachers and students involved with the project; and

Improved cooperation and collaboration with teacher training institutions, local

government and business sector.

Finally, the project participants from the educational institutions were asked to name additional

roles/services which SEECEL should be offering. The respondents suggested the following:

educational services

o provision of online training/workshops;

o facilitation of peer visits for the teachers in European schools and organizations;

o better cooperation with educational bodies (agencies, ministries);

o more involvement in coordinators’, advisors’, teacher trainers’ training and work.

organizational services

o organization of annual conferences and competitions among schools;

o organisation of students' international training in entrepreneurship;

o provision of larger school network;

o provision of funds and funding opportunities for a variety of projects.

14,5

29

,0

12

,9

12,9

16,4

43,5

48,4

41,9

30,6

31,1

8,1

6,5

6,5

12,9

4,9

24,2

14,5

37,1

37,1

37,7

9,7

1,6

1,6

6,5

9,8

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

1.     The project is not aligned with the schoolcurriculum

2.     We did not receive adequate support

3.     Lack of funding for project activities

4.     Lack of funding for student and teacherparticipation in peer learning activities

5.     Lack of time for teachers to integrateentrepreneurship learning in the classroom

Strongly Disagree Disagree Neither Agree nor Disagree Agree Strongly Agree

Page 36: 2nd External Evaluation Report of SEECEL activities

Confidential 36

Findings from the Steering Committee and Stakeholders Survey A total of 21 respondents from the key stakeholders and steering committee answered the survey.

There were 15 survey respondents that are steering committee members: Albania (Institute for

Education Development; Ministry of Economic Development, Tourism, Trade and

Entrepreneurship); Bosnia and Herzegovina (Ministry of Civil Affairs; Ministry of Foreign Trade

and Economic Relation; Regional Cooperation Council - RCC); Croatia (Ministry of

Entrepreneurship and Crafts); the former Yugoslav Republic of Macedonia (Bureau for

Development of the Education); Montenegro (Directorate for SMEs Development); Serbia

(Ministry of Education; Ministry of Economy); and Turkey (Board of Education). Finally, two

respondents were from EU agencies (OECD and ETF). The stakeholders (6 survey respondents)

were mainly from education related institutions: Croatia (Education and Teacher Training

Agency); the former Yugoslav Republic of Macedonia (Foundation for Management and Industrial

Research; Ministry of Economy Ministry of Education and Science); and Montenegro (Bureau for

Education Services).

The survey respondents were asked to indicate how strongly they agree or disagree with the 7

statements related to SEECEL, on the five point Likert scale (1- strongly disagree – 5 strongly

agree). The Cronbach's alpha for the 7-item Level of the steering committee members’ agreement

with the statements related to SEECEL scale was 0.890, indicating the good scale reliability.

According to the findings presented in Figure 15 and Figure 16, the respondents in average agreed

or strongly agreed with all of the statements. They most strongly agreed with the statements that

SEECEL’s activities are contributing to the development of entrepreneurship culture in the region

and that SEECEL supports the alignment of national policies with EU recommendations and

policies related to entrepreneurial learning.

Figure 15 Level of the Steering Committee members’ agreement with the statements related to

SEECEL.

70

80

80

60

60

40

40

30

20

20

40

40

60

50 10

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

1.        The results which have been achieved by SEECELin the period 2013-2016 are satisfactory

2. SEECEL’s activities are contributing to the development of entrepreneurship culture in the region

3.        SEECEL supports the alignment of national policieswith EU recommendations and policies related to…

4. The strategic goals and approaches in SEECEL’s 2013-2016 strategic plan and key policies are relevant …

5.        The SEECEL activities are relevant to its goals

6. The approaches identified in SEECEL’s 2013-2016 strategic plan, addressed the conditions, needs and …

7. The approaches identified in SEECEL’s rolling/ongoing work plan addressed the conditions, …

Strongly Agree Agree Neither Agree nor Disagree Disagree Strongly Disagree

Page 37: 2nd External Evaluation Report of SEECEL activities

Confidential 37

Figure 16 Level of the stakeholders’ agreement with the statements related to SEECEL.

Furthermore, on the five point Likert scale (1- strongly disagree – 5 strongly agree) the respondents

were asked to indicate how strongly they agree or disagree with the 20 statements about

appropriateness, effectiveness, and relevance of SEECEL. The Cronbach's alpha for the 20-item

Level of the steering committee members’ and stakeholders’ agreement with the statements about

appropriateness, effectiveness, and relevance of SEECEL scale was 0.934, indicating the excellent

scale reliability. According to the findings presented in Figure 17 and Figure 18, the respondents

in average agreed or strongly agreed with all of the statements. They most strongly agreed with

the statements that SEECEL’s work supports structured regional cooperation and that SEECEL’s

work is in line with EU policies. Although the participation in SEECEL activities was high, the

respondents indicate that in the future they would benefits from more participation in SEECEL

activities.

60

60

60

60

60

60

60

40

40

40

20

20

40

40

20

20

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

1.        The results which have been achieved by SEECELin the period 2013-2016 are satisfactory

2. SEECEL’s activities are contributing to the development of entrepreneurship culture in the region

3.        SEECEL supports the alignment of national policieswith EU recommendations and policies related to

entrepreneurial learning

4. The strategic goals and approaches in SEECEL’s 2013-2016 strategic plan and key policies are relevant to

the mission of SEECEL

5.        The SEECEL activities are relevant to its goals

6. The approaches identified in SEECEL’s 2013-2016 strategic plan, addressed the conditions, needs and

interests of key stakeholders

7. The approaches identified in SEECEL’s rolling/ongoing work plan addressed the conditions,

needs and interests of key stakeholders

Strongly Agree Agree Neither Agree nor Disagree Disagree Strongly Disagree

Page 38: 2nd External Evaluation Report of SEECEL activities

Confidential 38

Figure 17 Level of the Steering Committee members’ agreement with the statements about

appropriateness, effectiveness, and relevance of SEECEL.

60

60

60

60

70

60

40

20

30

90

60

30

50

80

80

70

80

50

40

50

40

40

40

30

20

10

60

70

70

10

40

60

50

20

20

30

20

30

20

10

10

20

30

30

10

10

20

10

10

10

10

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

1.        The SEECEL activities in the 2013-2016 strategicplan were effective in pursuing its mission.

2. SEECEL’s rolling work plan was effective in pursuing its mission.

3.        The peer-learning activities were implemented ata satisfactory level.

4.        The peer-learning activities contributed to thedevelopment of a culture of entrepreneurship

5.        SEECEL activities facilitated the exchange of goodpractice among stakeholders

6.        SEECEL had the necessary resources and capacityto undertake the activities required to achieve its…

7. SEECEL’s key stakeholders were satisfied with the approaches and activities used to achieve its mission.

8.        There was a fair participation among stakeholdersin SEECEL activities

9.        The targets and measures of success set out in thestrategic plan were realistic.

10. SEECEL’s work is in line with EU policies.

11. SEECEL’s work is of benefit to my country’s national development.

12. SEECEL’s work is of benefit to my organization’s development.

13. SEECEL’s work is of benefit to regional development.

14. SEECEL’s work supports structured regional cooperation.

15.     The structured regional cooperation run by SEECELpromotes exchange of knowledge and good practices…

16. SEECEL’s work contributes substantially to the development of Human Capital in the region (SBA)

17.     SEECEL supports the institutionalization of regionaldialogue and cooperation in the area of lifelong…

18.     The Training Needs Analysis (TNA) is an importanttool for understanding the needs of SMEs in the region

19.     The Training Needs Analysis (TNA) supportsnational developments in the area of entrepreneurship

20.     The Training Needs Analysis (TNA) contributes tothe development of entrepreneurship in the region

Strongly Agree Agree Neither Agree nor Disagree Disagree Strongly Disagree

Page 39: 2nd External Evaluation Report of SEECEL activities

Confidential 39

Figure 18 Level of the stakeholders’ agreement with the statements about appropriateness,

effectiveness, and relevance of SEECEL.

80

80

60

60

40

40

40

40

20

60

60

40

60

80

60

60

60

20

40

40

20

20

40

40

60

40

60

20

80

40

40

40

40

20

40

40

40

80

60

40

20

20

20

20

20

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

1.        The SEECEL activities in the 2013-2016 strategicplan were effective in pursuing its mission.

2. SEECEL’s rolling work plan was effective in pursuing its mission.

3.        The peer-learning activities were implemented ata satisfactory level.

4.        The peer-learning activities contributed to thedevelopment of a culture of entrepreneurship

5.        SEECEL activities facilitated the exchange of goodpractice among stakeholders

6.        SEECEL had the necessary resources and capacityto undertake the activities required to achieve its…

7. SEECEL’s key stakeholders were satisfied with the approaches and activities used to achieve its mission.

8.        There was a fair participation among stakeholdersin SEECEL activities

9.        The targets and measures of success set out in thestrategic plan were realistic.

10. SEECEL’s work is in line with EU policies.

11. SEECEL’s work is of benefit to my country’s national development.

12. SEECEL’s work is of benefit to my organization’s development.

13. SEECEL’s work is of benefit to regional development.

14. SEECEL’s work supports structured regional cooperation.

15.     The structured regional cooperation run by SEECELpromotes exchange of knowledge and good practices…

16. SEECEL’s work contributes substantially to the development of Human Capital in the region (SBA)

17.     SEECEL supports the institutionalization of regionaldialogue and cooperation in the area of lifelong…

18.     The Training Needs Analysis (TNA) is an importanttool for understanding the needs of SMEs in the region

19.     The Training Needs Analysis (TNA) supportsnational developments in the area of entrepreneurship

20.     The Training Needs Analysis (TNA) contributes tothe development of entrepreneurship in the region

Strongly Agree Agree Neither Agree nor Disagree Disagree Strongly Disagree

Page 40: 2nd External Evaluation Report of SEECEL activities

Confidential 40

The respondents were asked to indicate level of satisfaction/dissatisfaction with the 6 distinct

features related to SEECEL work, on the five point Likert scale (1- very dissatisfied – 5 very

satisfied). The Cronbach's alpha for the 6-item Level of the steering committee members’ and

stakeholders’ satisfaction/dissatisfaction with the listed features scale was 0.754, indicating the

acceptable scale reliability. According to the findings presented in Figure 19 and Figure 20, the

respondents in average were satisfied or very satisfied with all of the statements. The respondents

were the most satisfied with the Collaboration with SEECEL management team and Collaboration

with SEECEL staff.

Figure 19 Level of the Steering Committee members’ satisfaction/dissatisfaction with the listed

features.

70

70

60

40

50

50

30

30

40

40

40

40

20

10

10

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

1.        Collaboration with SEECEL management team

2.        Collaboration with SEECEL staff

3.        Collaboration with SEECEL Steering CommitteeMembers

4.        Collaboration with other stakeholders in thecountry and region within the framework of SEECEL

activities

5.        SEECEL adopted feedback from SteeringCommittee members and integrated it in its activities

6.        The Steering Committee meetings were engagingand productive

Very Satisfied Satisfied Neutral Dissatisfied Very Dissatisfied

Page 41: 2nd External Evaluation Report of SEECEL activities

Confidential 41

Figure 20 Level of the stakeholders’ satisfaction/dissatisfaction with the listed features.

The respondents were asked to indicate how strongly they agree or disagree with the 5 statements

related to challenges that SEECEL faces, on the five point Likert scale (1- strongly disagree – 5

strongly agree). The Cronbach's alpha for the 5-item Level of the steering committee members’

and stakeholders’ agreement with the statements about challenges that SEECEL faces scale was

0.704, indicating the acceptable scale reliability. According to the findings presented in Figure 21

and Figure 22, the respondents on average were neutral or agreed that SEECEL lacks sufficient

funding and disagreed that the project is not aligned with the national life-long learning strategy

of their country. In addition, the respondents listed some additional challenges that SEECEL is

facing. The listed challenges are: need for increase of the pilot institutions network, need for more

concrete results (e.g. SEECEL should develop model of entrepreneurial education), more

promotion activities in partner countries.

80

60

40

20

20

20

20

40

20

80

40

20

40

40

60

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

1.        Collaboration with SEECEL management team

2.        Collaboration with SEECEL staff

3.        Collaboration with SEECEL Steering CommitteeMembers

4.        Collaboration with other stakeholders in thecountry and region within the framework of SEECEL

activities

5.        SEECEL adopted feedback from SteeringCommittee members and integrated it in its activities

6.        The Steering Committee meetings were engagingand productive

Very Satisfied Satisfied Neutral Dissatisfied Very Dissatisfied

Page 42: 2nd External Evaluation Report of SEECEL activities

Confidential 42

Figure 21 Level of the steering committee members’ agreement with the statements about

challenges that SEECEL faces.

10

10

20

40

50

40

20

30

10

30

20

50

50

70

30

10

10

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

1.        Lack of sufficient funding

2.        The project is not aligned with the nationalentrepreneurial learning strategy in my country

3.        The project is not aligned with the national life-long learning strategy of my country

4.        Lack of sufficient communication amongstakeholders

5.        Lack of adequate support

Strongly Agree Agree Neither Agree nor Disagree Disagree Strongly Disagree

Page 43: 2nd External Evaluation Report of SEECEL activities

Confidential 43

Figure 22 Level of the stakeholders’ agreement with the statements about challenges that

SEECEL faces.

The survey respondents were asked to indicate how strongly they agree or disagree with the 7

statements related to changes that are needed in order to achieve better results, on the five point

Likert scale (1- strongly disagree – 5 strongly agree). According to the findings presented in

Figures 23 and Figure 24, the respondents recognized that following changes are needed: Training

for all stakeholders and need for Additional resources for SEECEL activities. The respondents

disagreed that there is a need for Changes in organization and functionality of SEECEL’S

management and governance. In addition, the respondents commented that it would be necessary

to establish coordination body in each partner country and assure that steering committee activities

are more transparent to all stakeholders.

20

40

20

60

40

40 20

80

40

20

20

20

40

20

20

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

1.        Lack of sufficient funding

2.        The project is not aligned with the nationalentrepreneurial learning strategy in my country

3.        The project is not aligned with the national life-long learning strategy of my country

4.        Lack of sufficient communication amongstakeholders

5.        Lack of adequate support

Strongly Agree Agree Neither Agree nor Disagree Disagree Strongly Disagree

Page 44: 2nd External Evaluation Report of SEECEL activities

Confidential 44

Figure 23 Level of the steering committee members’ agreement with the statements about

changes that are needed in order to achieve better results.

Figure 24 Level of the stakeholders’ agreement with the statements about changes that are

needed in order to achieve better results.

10

10

50

50

30

20

20

30

40

50

50

50

30

10

10

10

10

20

50

40

10

30

30

20

10

10

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

1. Changes in organization and functionality of SEECEL’S management and governance

2.        Change in the role of the Steering committee

3.        Additional resources for SEECEL activities

4.        Training for all stakeholders

5.        Additional funding

6.        Better communication

7.        More time for implementing activities

Strongly Agree Agree Neither Agree nor Disagree Disagree Strongly Disagree

20

40

60

60

20

40

20

20

40

40

40

40

40

60

40

20

20

20

20

20

20

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

1. Changes in organization and functionality of SEECEL’S management and governance

2.        Change in the role of the Steering committee

3.        Additional resources for SEECEL activities

4.        Training for all stakeholders

5.        Additional funding

6.        Better communication

7.        More time for implementing activities

Strongly Agree Agree Neither Agree nor Disagree Disagree Strongly Disagree

Page 45: 2nd External Evaluation Report of SEECEL activities

Confidential 45

Respondents were asked to indicate how strongly they agree or disagree with the 3 statements

related to SEECEL’s future, on the five point Likert scale (1- strongly disagree – 5 strongly agree).

According to the findings presented in Figure 25 and Figure 26, the respondents on average

strongly agree that SEECEL should become the European centre and/or agency for

entrepreneurial learning.

Figure 25 Level of the steering committee members’ agreement with the statements about

SEECEL’s future.

Figure 26 Level of the stakeholders’ agreement with the statements about SEECEL’s future.

50

30

40

40

40

40

10

30

10 10

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

1.        SEECEL should become the European centreand/or agency for entrepreneurial learning

2.        SEECEL should focus more on the strategic level ofentrepreneurial learning

3.        SEECEL should work closely with training institutesfor implementing the program in schools and the

education system in general

Strongly Agree Agree Neither Agree nor Disagree Disagree Strongly Disagree

40

40

20

20

20

40

20

20

20

20

20

20

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

1.        SEECEL should become the European centreand/or agency for entrepreneurial learning

2.        SEECEL should focus more on the strategic level ofentrepreneurial learning

3.        SEECEL should work closely with training institutesfor implementing the program in schools and the

education system in general

Strongly Agree Agree Neither Agree nor Disagree Disagree Strongly Disagree

Page 46: 2nd External Evaluation Report of SEECEL activities

Confidential 46

The survey respondents were asked to provide 3 concrete recommendations for improving

SEECEL and its impact. It is possible to group the responses in the following general

recommendations:

increase funding (both to SEECEL and pilot institutions);

extend network of schools and higher education institutions to build their capacities for

entrepreneurial learning as a key competence;

improve stakeholder mapping and interaction;

more peer learning activities between countries;

on-line course/training for youth.

Finally, respondents were asked to name 3 roles or services SEECEL should be performing in the

future. According to the survey responses SEECEL should

become central point for entrepreneurial learning activities for neighbouring regions in

Europe and beyond. In particular, in the future SEECEL should focus on: EL curriculum

model development;

assure better coordination for pilot institutions on the country level;

advocate EL on the policy level;

support linkages between business and higher education institutions;

organize regional trainings for SBA experts;

assure provision of more interactive activities between countries of the region.

Page 47: 2nd External Evaluation Report of SEECEL activities

Confidential 47

Findings from Study Visits, Interviews, and Focus groups After detailed data collection and analysis, it became apparent that overall SEECEL has achieved

its objectives and that its strategic goals were met. In some occasions success exceeded the

expectations. Below we present the main findings from the qualitative data divided in two main

categories: Successes of the program and the challenges faced by SEECEL and its stakeholders.

In some of the occasions, we integrate some of the findings from the quantitative section to

highlight the strong success of the program. Building on the successes, and in an effort to address

the challenges, we present recommendations in the last section of this report.

Successes

Recognition

The European Commission has recognized SEECEL as a good practice in strategic regional

cooperation, good practice for conceptual solutions in the field of entrepreneurial learning as a

key competence and good practice in line with the first principle of the Small Business Act for

Europe. SEECEL has become a reference point both for EU Member States and the pre-

accession countries and was awarded with the following awards:

- "Creators for Centuries" reward and recognition for the year 2014 for contribution to the

development of entrepreneurship in South East Europe, awarded to SEECEL by an

independent international committee during the Regional summit of entrepreneurs of the

South East Europe the best 300;

- European Enterprise Promotion Award (EEPA) as National Winners in 2014 for

Promoting the Entrepreneurial Spirit, awarded to SEECEL by the European Commission;

- The Champion of Regional Cooperation of 2013, awarded to SEECEL by the Regional

Cooperation Council for its active and dynamic approach to regional cooperation and

contribution to the SEE 2020 preparation process;

- The European Projects Awards IPA 2013 - for the project entitled ''Developing the

Entrepreneurial Society in Western Balkans and Turkey'', in the category “ongoing”

projects and best actors in Regional and Local Development (2013);

- The Knowledge Economy Network (KEN), an international Best Practice Award for

"original and successful practice in any domain of knowledge society and economy

development" and especially for "good practice in successful regional cooperation in

training and education”.

The enthusiasm, efforts and dedication of all SEECEL member states and in conjunction with EU

support have resulted in a robust exchange of experience and knowledge, bearing fruit to a tested

model for implementation of entrepreneurial learning as a key competence in various levels of

education and society. These achievements represent, not only the vision of individual countries,

but also a regional vision to fully engage resources and efforts in the promotion of knowledge-

based economies that will fully support entrepreneurship and innovation. This regional

Page 48: 2nd External Evaluation Report of SEECEL activities

Confidential 48

cooperation and consensus is an achievement, given not only the complexity of the subject matter,

but also the great variety of perspectives from the region as a whole. SEECEL also offers a very

unique solution to structured strategic cooperation between education and the business world

across local, national, regional and international levels.

SEECEL has a strong potential to continue with these developments, to spread the acceptance and

acquisition of developed tools on national levels and to continue to build momentum across all

levels of society and governments. As one of the key stakeholders stated: “The region as a whole

has a wealth of untapped human capital potential that can serve as a competitive advantage in

promoting the sustainable, smart, inclusive and integrated growth and development”.

Meeting its Objectives and Strategic Goals

From all the data collected, it was evident that SEECEL is successful in creating the foundation

for a developed entrepreneurship culture across the region with a spillover effect beyond the

SEE. Already there is expression of interest from countries beyond the region and the EU.

Furthermore, because of its inclusive approach, SEECEL fosters alignment with the EU in

the field of entrepreneurial learning.

Visiting schools, discussing with teachers, and reviewing lesson plans and reports, it is evident

that SEECEL contributes, through a systematic approach, to the development of

entrepreneurially literate societies across the region and it supports alignment of national

policies with EU recommendations and policy essentials related to lifelong entrepreneurial

learning. However, as stated by one of the key stakeholders, “Cultural change is something

that takes a long time.” SEECEL is only one part of a chain support instrument. SEECEL

cannot change all education systems in the pre-accessing region. As one key stakeholder

indicated during the interview, “SEECEL can provide the social infrastructure to facilitate

people’s thinking and mindset towards entrepreneurship learning in the region and Europe”.

The SEECEL model has proven successful so far and it is one of its most important strengths.

This double dimension, implementing the policies and providing the field training, is what

makes SEECEL different from other similar and like-minded projects. Most of the countries

interviewed reported that SEECEL provided a framework and a model for them to work on

integrating EL. Stakeholders understand that the impact will be visible within the next 5 to 10.

SEECEL has demonstrated that when practitioners work together, and policy makers are

setting the agenda in the region, it makes things move forward.

For the majority of the stakeholders, funds allocated to SEECEL, are European money well

spent. SEECEL is doing something meaningful that makes a significant difference in the

countries where they are involved. SEECEL goals and objectives are aligned with the

European Commission objectives. Entrepreneurship education and entrepreneurship in

general are EU’s political priorities. The premise is that focusing on entrepreneurship

education will help improve competitiveness and long-term employment for young people.

Page 49: 2nd External Evaluation Report of SEECEL activities

Confidential 49

Policy and Regional Cooperation

One of the core strengths of SEECEL is the support for networking and regional cooperation.

SEECEL made a big difference in member countries at the policy level. Without SEECEL

some of the member countries could not be at the same level when it comes to Entrepreneurship

Learning. There is a general trend in all the SEECEL Member countries to develop faster in

entrepreneurship education/learning. Through the combined efforts of SEECEL and the

European Training foundation the region is developing faster than the rest of Europe in the

area of Entrepreneurial Learning.

One of the biggest success stories is that countries come together as a club, understanding that

the future development of economy of this region has to be with the rest of the Europe, and

that they need to become more competitive in this region. They begin to understand how

education can build up a more entrepreneurial generation. Different countries work together,

and in terms of the policy linkages which happen in the Small Business Act, policy tracking,

and policy assessments, SEECEL provides the link with the policy and implementation.

The success of SEECEL in the region, led some stakeholders to advocate that SEECEL can

play a role in the broader region not excluding Asia, Africa, and the Middle East.

Compatibility between the national members is not an issue because everyone agrees that

entrepreneurial education/learning is important and there is a comprehensive agreement on the

overall SEECEL policies that promote this issue. The National Policies can be and are

independent but they include the core competencies promoted through SEECEL’s work. To

emphasize the importance of this, in Bosnia, that is considered to be the most challenging area

due to their overall administrative structure that makes it difficult to have national cohesion on

decision making and policy setting, Entrepreneurial Learning policy is one of the few common

strategies that they have for the whole country.

The SEECEL’s Steering Committee comprises of representatives from the Ministries of

Economy, the Ministries of Education and other relevant organizations. As a result of the

structure of SEECEL activities, there is evidence that various member state ministries (e.g.

Ministry of Education, Ministry of Economy, Ministry of Entrepreneurship, etc.) begin to

cooperate and collaborate, not just about EL and SEECEL activities, but also about other issues

facing their countries. The fact that these stakeholders align their actions with a common

vision, it is a success of its own. In Serbia they are including EL in their strategic documents

and there is a coalition between the Ministries.

Last year during the Turkish presidency of G20, as a result of SEECEL activities, supporting

women entrepreneurs and entrepreneurial education were recognized as essential for

sustainable development.

One of the major roles and impact of SEECEL, as indicated by one stakeholder is “re-

connecting this region. SEECEL’s approach is about connecting people, and that’s very

important.” As another stakeholder stated: “What do we do to ensure a brighter future for our

children in the region? It doesn’t matter which country’s model was, they brought all people

together for entrepreneurial learning. This is a good example of SEECEL’s success.”

Page 50: 2nd External Evaluation Report of SEECEL activities

Confidential 50

Because of the in-depth knowledge and core expertise developed in the area of

entrepreneurship learning, SEECEL is member and participates in many expert groups at the

European Commission level working in the area of entrepreneurial learning.

Governance

Several stakeholders believe that all members get the same value out of SEECEL activities.

National members attach a lot of importance on what SEECEL is doing. The interactions

among the members are professional and there were no identifiable controversial issues.

Regarding the governance, some stakeholders suggested that it might be useful to have a

rotating chair of the board. However, this issue has been addressed in SEECEL’s statues

through the Management Support Team (MST). The MST is chartered to provide advice and

support to the SEECEL Director on management and technical issues. The MST is composed

of 3 national coordinators of the Small Business Act for Europe (the Croatian coordinator as

permanent MST member and two other on a six-month rotation applying the "EU troika"

rotation principle).

The majority of stakeholders are very satisfied with the management team and the staff of

SEECEL. They demonstrated exceptional commitment and showed support and great team

work to create the environment and to facilitate countries to meet their objectives and

implement the SEECEL programs.

Schools

The most important SEECEL differentiator is their mission/objective/goal to implement their

policies and programs on the ground through the school programs, and the teacher training.

There are a lot of policy development Think Tanks but almost all of them stay at the policy

development level. SEECEL implements on the ground and that is unique.

As a result of the success of the program, some countries like Albania, have a priority to

expand the SEECEL program to as many schools as possible. Albania changed their formal

curriculum to include the SEECEL program. The SEECEL experience and learning process

will be included in the education system as a whole and that includes subjects other than

entrepreneurship education and learning.

SEECEL is a great organization, program and process that can be applied in areas beyond

entrepreneurship education/learning. This program benefited tremendously the schools that

participated. In addition to EL, as a byproduct, the program taught the schools some basic

financial management skills through the small budget management they have to cater to.

School leaders and teacher interviewed, seemed to be very satisfied with the program, and the

idea of entrepreneurial learning as a whole and they stressed that teachers are willing to

participate because they see the impact on children. One of the teachers said that pupils engage

more in school and that these project activities brought together whole schools and

municipalities to work together. In Herzegovina schools managed to involve many local

entrepreneurs not just for financial support but also for welcoming pupils in their firms to learn

Page 51: 2nd External Evaluation Report of SEECEL activities

Confidential 51

more about the real world business sector, which is an indication of the impact of SEECEL

activities on the whole entrepreneurship ecosystem.

SEECEL activities led to broader impact on the level of national development. For example, a

school from Montenegro became the best EU practice example. Montenegro started to have

the SEECEL programme as a nationally accredited official programme in their teacher training

curriculum.

SEECEL serves as a platform for the dissemination of different kind of best practices for

entrepreneurial learning and all other education fields which is necessary for the business

communities and schools.

SEECEL served as a spark for countries in the region to consider integrating entrepreneurial

learning in cross-curricular subjects and the program is compulsory at elementary level in

Montenegro. All ISCED 1, 2 and 3 teachers in Montenegro were involved in entrepreneurial

learning education (more than 50% directly) cooperation of external experts (from different

countries and from different stakeholders)

In Montenegro they succeeded to develop special learning model for teachers and almost every

teacher in Montenegro embraced this model. As indicated by a key stakeholder, at least 50%

of teachers in Montenegro participated directly, and they had the obligation to become

ambassadors and present what they have learned to their schools, so we can say that every

teacher in one way or another had some exposure in entrepreneurial learning. Entrepreneurial

learning is a cross-curricular subject, so they have to implement it in their primary subjects.

Before SEECEL, the majority of schools in the region, had no experience with international

cooperation and no experience working with projects. As part of SEECEL activities, more and

more schools are involved in other projects and making significant connections with schools

from the region and the EU. Team work in schools also improved since the teachers embraced

collaboration and this had a positive impact on teachers and schools (collective mind-set is

changing). An added value for schools is that they learned how to manage projects from the

beginning to the end including financial management.

Another success has to do with the connection of EL and employment. As an example, in a

SEECEL member state one school established a career center in an effort to gather alumni

students to work on projects and collaborate and network with the business sector in

conjunction with the students and the school. Teachers felt that this kind of community

collaboration and projects can help shift mindsets.

ISCED 3

The majority of the successes that apply to schools and discussed in the previous section, also

apply to ISCED 3 schools. In addition, a few that stick out for ISCED 3 are discussed below.

As a result of SEECEL activities, local companies opened their doors to students from ISCED

3 schools and students, having a positive impact on the entrepreneurship ecosystem: the school,

the community and SMEs. Students’ visits and interaction with local companies, facilitated

their decision making with regards to future employment, professions to choose, and studies

to follow.

Page 52: 2nd External Evaluation Report of SEECEL activities

Confidential 52

When SEECEL activities started, several schools decided to have their own fundraising

activities and that was very well received in their local communities. SEECEL funds were just

partly enough for some of the activities. Organizing fundraising, bazaars selling crafts and food

to parents and to the broader community (neighborhood) was very successful and brought the

community together. At the beginning, teachers from other schools were skeptical and they did

not like to talk about entrepreneurship as they were connecting it strictly to businesses, but

progressively more teachers expressed interest to be involved in this type of projects.

Teachers reported that, as a result of participation in SEECEL activities, and developing and

implementing EL lesson plans, students were more engaged with the subject matter, and their

innovative thinking and creativity have improved. The principle of thinking out of the box,

creativity, and risk taking, are important characteristics of a mindset that EL supports,

encourages and develops through their programs. As identified by the European key

competencies framework entrepreneurship and initiative taking are critical in creating an

entrepreneurial culture.

SBA and TNA

One of the key successes of SEECEL is the progress towards the SBA which SEECEL member

states demonstrate. Particularly, with the human capital dimensions (1&8) which are the focus

of SEECEL.

The OECD report “SE Policy Index: Western Balkans and Turkey 2016” clearly indicates that

SEECEL member states have made substantial progress towards the implementation of the

SBA, which is a good indication of the hard and dedicated work that SEECEL is doing in the

region. More specifically, the report indicates: “overall the 2016 report finds that the SEE

region is indeed making further progress towards EU SME policy standards and EU good

practices as defined in the SBA” (p. 17).

The peer reviews, through the SBA policy assessment work, is an activity that is valuable and

contributes to the building of the EL culture in the region. The model works and has added

value to the partner countries because, when engaging an expert from the former Yugoslav

Republic of Macedonia to do a peer review in Montenegro, this encourages critical appraisal,

and changing of the mind-set of people, organizations and communities; changing mindsets,

is what EL is about. Being constructively critical to other people’s work is easy, but being

constructively critical about one’s work and self-reflect on what you can improve is not easy.

The competency of sharing their work with other people is important. Engaging people into

multi-country peer reviews boosts confidence, and entrepreneurial confidence is one of the

ingredients missing from the region, according to an EU policy officer.

Conducting systematically Training Needs Analysis (TNA), is one of the key activities of

SEECEL within the framework of SBA. The TNA is a strong tool that links the SMEs with the

world of education. Without the TNA it is difficult to build curricula and competence

frameworks to integrate entrepreneurship learning across all levels of education.

Page 53: 2nd External Evaluation Report of SEECEL activities

Confidential 53

The TNA contributes substantially towards the evidence-based policy making and to facilitate

SEECEL member states in developing the training and support mechanisms SMEs need in

order to succeed and grow.

SEECEL had significant impact in the former Yugoslav Republic of Macedonia in supporting

and networking in order to develop entrepreneurial learning. Macedonians had access to

experts in the region and collaborated with them to develop strategies and policies. The TNA

and SBA implementation via the support of SEECEL, is a way to bring all the relevant

stakeholders together to address the national and regional needs collectively.

Montenegro is recognized as regional leader in developing entrepreneurial learning (according

also to SBA results) so this is good direction where they have to systematically work further

to improve on their successes and enjoy long term results. The Directorate for development of

SMEs developed special EL programs based on TNA findings which is obligatory for start-

ups, newcomers, and the business sector.

SEECEL ensures education for people involved not just in project implementation, but also by

involving experts with the SBA assessment. By engaging the whole entrepreneurship

ecosystem of a country the benefits are many and long term. SMEs, Chambers of Commerce,

Ministries and government agencies, schools, and universities, all can benefit from

participating in such initiatives.

Challenges

Funding

Given the fact that SEECEL has already made substantial contribution in the region and

beyond, the European Commission should provide SEECEL with the support needed to

continue its very important work.

Financing SEECEL is a challenge that needs to be addressed. States should be contributing

more and make a financial commitment. Since every SEECEL member state is satisfied and

enjoys benefits from the SEECEL program, each country should be contributing to the budget

according to its size and resources available.

Members of the SC and some of the stakeholders, although they feel that the SEECEL

operating budget is balanced and sufficient, for the sustainability of its activities and impact,

and to increase the potential for future successes even more, there is a need to increase the

resources available. It will foster further development for SEECEL and it is also necessary for

scaling up the success of SEECEL.

Schools reported that they could use additional funding if they were to implement the actions

they undertook better, broadly, more effective and more comfortable. They would also like to

be able to take some of their students to peer learning activities and visits and exchanges, since

peer learning was rated as a valuable activity by most of the schools and stakeholders

participating in the program.

All SEECEL member states are treated the same way. However, some of them pay full their

fee and some not. This, in conjunction with the fact that countries have their own regulations

Page 54: 2nd External Evaluation Report of SEECEL activities

Confidential 54

and processes for managing external funds, leads to some discrepancies with regards to the

speed and process in which funds are reaching implementing organizations.

Policy - Governance

There is an issue of mandate absence from governments to some SC Members. National

governments have different level of commitments to the project. The absence of an official

mandate precludes participants from asking for official support from their governments of any

type; money, development, policy, agenda and the like. SEECEL member states need to

provide strong support to SC Members so that they can participate and contribute more in

SEECEL activities.

Although SEECEL activities are important, there needs to be a process where the context of

countries will be accounted for and more country-specific activities can be implemented

which could address specific problems which each country has, which is clear now according

to SBA report. The SBA and TNA are contributing to this end, but project activities need to

engage the local businesses.

Given the fact that SEECEL is based in Croatia, supported by the EC and the Croatian

government, some partners feel it is a bit too Croatian focused. The balance of being physically

based in Croatia and serving the region, needs to be revisited. That will give SEECEL the

status to command funding from other governments on the network.

Stakeholders believe that SEECEL organizes and presents its work in a professional way and

the information flow from SEECEL to the Board members is sufficient and satisfactory. Some

members of the SC feel they would like to contribute more towards the management and

implementation of SEECEL. Although the agenda is set by SEECEL, when circulated to the

members, very few SC members provide feedback.

In order to increase its reach out, representation of multiple stakeholders, and impact, the

Steering Committee needs to include active representatives from the business community.

Stakeholders feel that SEECEL and its partners need to involve the business sector more. In

addition to building the mindset, improving competencies at school level, it is equally

important to make the connection with real world business sector in each country in the region.

As SEECEL grows, office space will be a challenge. With the new building under construction,

which will be able to house more than 100 staff, SEECEL will be in a position to run its

operations and scale up in more countries, provided sufficient funding is provided by the

European Commission and partner countries.

Because of the history of the region and the political complexities, it is challenging to get

stakeholders from various countries to collaborate. Collaboration requires patience and good

will, as well as the development of a common vision.

Page 55: 2nd External Evaluation Report of SEECEL activities

Confidential 55

Schools and Teacher Training

One of the biggest challenges schools faced was that they had limited to no knowledge of

entrepreneurial learning and how to implement it in their teaching practices. Teachers who

wanted to engage had to invest time in order to educate themselves and develop an action plan

and find some examples on the internet. There were several theoretical discussions about EL

in all countries, but SEECEL’s practical approach was welcomed by all.

There is a gap between developing and implementing training for the teachers. The training is

insufficient and the actual implementation of the training (delivering to the teachers) is

problematic. The only countries that implement some sort of teacher training is Montenegro

(the best practice example) and Croatia (with mixed reviews to date).

SEECEL developed an evidence-based model and package of material, but does not have the

capacity on its own to implement at the school level. SEECEL needs to develop solid

agreements with implementing organizations and agencies in each country that will ensure

teachers receive the necessary training and support for sustainable impact.

There are still weaknesses in the region, for example the education experts, curriculum experts,

teacher training experts, they need time in order to be so critical of their own work before they

become critical of someone else’s work and be able to participate effectively in peer-learning

activities. There is a need for more experts and trainers from both education and business

community. Developing and supporting the train the trainer model will facilitate future scale

up in more schools and communities.

In the beginning there was resistance at institutional level since some schools and education

institutions already had several projects going on, but at the meetings we observed, principals

and teachers were convinced to be a part of this project and they were happy with that decision.

SEECEL policies and tools such as curriculum and lesson plans are adopted by the various

schools at various levels. Some countries adopt them more than others. SEECEL should

continue to support member states to align national policies, processes, and curricula to meet

the jointly agreed vision and objectives.

Teachers expressed the need to have an interactive platform (like the CoP and maybe

something new) so they could have better networking with colleagues. As it stands now, each

school has limited access. It would be good if all teachers could have an account on the

platform. They also suggested the development of an activities calendar for all schools

involved so they can decide easier when to peer visit someone or just to see what other schools

are doing; as well as SEECEL members visiting schools.

Teachers and stakeholders suggested that what is missing is some kind of tracking of the

quality of all activities, not as monitoring, but some kind of methodology which could note the

quality level as it is difficult to see the real measurable impact of entrepreneurial learning itself.

During peer review, activity planning, and implementation it is important to include local

experts from the countries in which the program is implemented, as they know better the

situation and are more aware of the needs of the local environment. By engaging local experts,

we also build capacity at the local and national level with potential long term impact.

Page 56: 2nd External Evaluation Report of SEECEL activities

Confidential 56

The fact that all deliverable, activities, documents, and communication is conducted in English,

it becomes a challenge. Not all partners and schools in the region can communicate effectively,

develop lesson plans and prepare reports in English. As the project grows it needs to have a

plan for allowing local stakeholders to work in their local language.

Page 57: 2nd External Evaluation Report of SEECEL activities

Confidential 57

Recommendations There is a large number of recommendations stemming from the data analysis. These

recommendations are considered critical to further develop and grow SEECEL. After this

comprehensive evaluation, the partner states will expect some changes to be developed and

implemented. It is imperative that this process is done transparently and information should be

actively communicated and feedback should be solicited by the partners. The main findings from

the evaluation can be classified in the following categories

Strategic Objectives & Policy

There is ample evidence that strong capacity has been developed since 2009 within

SEECEL in the area of entrepreneurship learning. SEECEL should continue its valuable

work and expand beyond the regional dimension to include the rest of European Union

members. Furthermore, SEECEL could also play a more active role in the Mediterranean

region and Eastern Partnership Countries (EaP) since there was already strong

expression of interest from other non-EU countries to participate in SEECEL activities.

SEECEL has the strong potential to become part of the infrastructure solution for EU to

implement SME and entrepreneurship policies and practices

The SEECEL strategy should remain the same because it works well for the SEECEL

member states.

Communication

Develop a communication strategy that will improve awareness and branding for

SEECEL’s vision. It is our recommendation that a comprehensive media development

plan includes a public media campaign possible with short videos that address issues such

as Entrepreneurial Skills, Entrepreneurial Culture, Entrepreneurial Mentality, and all that

can benefit the community as a whole, the economy in areas such as unemployment, and

the individual practitioners.

Communication between the active SEECEL project participants, for example, SEECEL

Management/Team and Teachers, SC, and others close to the organization needs

improvement in bidirectional communication. It is important to have clear and concise

processes and tools to funnel recommendations, opportunities and issues back to SEECEL

for action. SEECEL should be clearer and distinct to communicate its strategies and

policies as well as provide feedback to teachers and other implementation stakeholders. It

also needs to communicate better with Ministry and other agency stakeholders.

The SEECEL website and the CoP are already in place and provide access to information

and resources. Future developments should build on the existing infrastructure to allow for

more users to engage with the online tools and benefit from SEECEL activities.

Excellent examples of building entrepreneurial learning in SEECEL member states need

to be better documented and shared. Such success stories can increase awareness among

Page 58: 2nd External Evaluation Report of SEECEL activities

Confidential 58

the general public and that will have an impact on a cultural change that will embrace

Entrepreneurial Skills Learning.

Governance

It is important to engage more SEECEL member states in developing the agenda for SC

meetings, so that it becomes more relevant to the needs of countries and stakeholders.

A more active role should be given to business representatives, chambers of commerce,

and SMEs, which can lead to more holistic approach to SEECEL strategic objectives. For

this, there is a need to develop some new activities which strengthen cooperation between

the business sector and education sector.

As the program grows and scales, it might be useful to consider to establish local

entrepreneurship coordination mechanisms in SEECEL member states to facilitate the

implementation and monitoring of SEECEL goals and achievements.

Program Implementation

Continue the dedicated work related to the SBA for Europe and use findings from the TNA

to further develop programs that target each country’s needs.

There is a need for using English as the language of communication for SEECEL activities.

However, as project activities scale and more schools engage in each member state, it will

be important to consider ways of addressing the language barrier to allow for more schools

and students to engage.

It is important to continue and expand the involvement of the local and local SMEs in

SEECEL activities. The program should also facilitate and encourage and expand

participation by parents, as we have seen in many cases and schools visited. If the kids

who are involved in the program receive encouragement by their parents, they will be more

willing to get involved and invest more time. This will also snowball to other parents who

will eventually support their children to participate and eventually the whole community

will be involved.

Program continuity is a challenge that needs to be addressed. Students participate in the

program only for one year after which time the program and infusion of knowledge ends

for them. It is a concern that the interaction of the students and the program ends when

there is an opportunity to learn more about Entrepreneurial Skills Learning. Our

recommendation is to extend the program progressively to more years until it becomes part

of the national curriculum.

A clear plan on how scaling up to more schools and probably full country implementation,

should be developed be each SEECEL member state. A toolkit and a service to partner

countries could be provided that will entail strategic planning but also training for decision

makers, policy makers, ministry officials, school principals, and teachers.

Page 59: 2nd External Evaluation Report of SEECEL activities

Confidential 59

Many teachers expressed the need to increase the interaction between schools with

student exchanges and exchange visits. In future activities the process for more

exchanges and the financing of such activities should be considered carefully to ensure

the best value for money.

School Related

Given the important role of education, as part of the entrepreneurship learning ecosystem,

SEECEL should continue to work closely with SEECEL member states, to include more

schools into the program and perhaps provide some continuity of the program to allow

students to participate in the program for more than one academic year.

Also, maybe address all levels of education to include more teachers, elementary and

high schools, and universities, and perhaps involve more countries into the program to

pass on the knowledge and experiences to other regions in the European Union.

The process of joining SEECEL and managing project implementation at school level,

requires specific skills. Offering project management training to teachers and principals

will be useful and will empower them to better implement and monitor project activities.

Teachers who were involved in one year, should be able to share their knowledge and skills

with others, in a way that will transfer their knowhow to colleagues and more schools and

communities. This knowledge and experiences should be captured and shared with all

current and future school participants. Tacit learning is one of the most effective ways to

learn from other’s experiences.

Community of Practice

A useful feature for the CoP could be a rating system for teachers to rate the lesson plans,

and also a way to show which lesson plans and activities were downloaded more often.

This can provide tips to teachers on how to choose lesson plans and also adapt them into

their teaching and context.

The TASK BOX on the CoP, should be further used and expanded to allow for teachers to

receive prompt feedback on their work. Teachers need to receive feedback on the lesson

plans they develop and also on their reports and project activities. SEECEL member states

and local teacher training institutes should collaborate to better support teachers during

project implementation.

The complete package for training of teachers, mentors and stakeholders can be developed

using eLearning technologies to allow for flexibility of delivery and possibility to reach

out to all countries.

Regarding the online learning, it would be valuable to establish a process for using learning

analytics, integrating it with the online community, and future eLearning developments, to

provide detailed data on how the resources are used, and how teachers communicate and

receive feedback.

Page 60: 2nd External Evaluation Report of SEECEL activities

Confidential 60

Teacher and Stakeholder Training

There is a need for a structured and formalized Teacher training that takes place prior to

the program being implemented at school level. Teacher training should be required for

all schools that decide to join the program.

Instead of 1-2 day workshops, it will be good to offer a whole week or even longer summer

courses, to ensure teachers get the needed knowledge and skills to implement EL in their

classroom practices.

For the future designs of teacher training, it will be important to have two types of Mentors.

The School Mentors who will support new schools coming into the program and Mentors

from the local communities and SMEs who can support the learners and the teachers during

the implementation stage of their idea development.

Special effort and support should be provided at the policy level to ensure that EL is

integrated in country school curricula. Therefore, training should be provided not only to

teachers but also to school principals, inspectors, teacher educators, academics, training

agencies, SBA experts, and ministry of education officials.

A Certification system could be developed with local ministries and teacher training

agencies, so that teachers who participate in the program can be certified and earn extra

points on their CV, something that can act as a motivation on teacher engagement.

Monitoring and Evaluation

There is a need to establish a benchmarking process to help determine how successful the

SEECEL program is. It would be useful to have a process to collect data on the long term

impact of Entrepreneurship Learning on SEECEL member states. For example, what

happens to the program graduates, how successful they are in their communities, if they

transfer the knowledge to others, and if they facilitate similar learning activities in their

communities?

Establish a process to identify success stories and give recognition at the Teacher, Student,

School or the Systemic Level. It will be useful to develop an alumni database that; a) It

keeps track of what happens to the individuals graduating from these programs, and b) It

registers these graduates contribution towards achieving the overall goals and objectives

set by SEECEL. The initial benchmarking should be done from existing data as well as

through an original data gathering research that should be undertaken by a third party to

ensure integrity of data gathering. This information could be also used to perform a

comparative analysis with similar programs run by other organizations and government

agencies.

Page 61: 2nd External Evaluation Report of SEECEL activities

Confidential 61

References Flash Eurobarometer 354 (2012). Entrepreneurship in the EU and beyond. Online.

http://ec.europa.eu/public_opinion/flash/fl_354_sum_en.pdf (accessed on 13th June 2016)

Europe (2020). A strategy for smart, sustainable and inclusive growth. Online.

Available from

http://ec.europa.eu/eu2020/pdf/COMPLET%20EN%20BARROSO%20%20%20007%20-

%20Europe%202020%20-%20EN%20version.pdf (accessed 23rd June 2016)

European Commission (2013). Communication from the commission to the European Parliament,

the Council, the European Economic and Social Committee and the Committee of the Regions.

Online.

Available from https://ec.europa.eu/transparency/regdoc/rep/1/2012/EN/1-2012-795-EN-F1-

1.Pdf (accessed 23rd June 2016)

European Commission (2015). European Innovation Scoreboards. Online.

http://ec.europa.eu/growth/industry/innovation/facts-figures/scoreboards/index_en.htm (accessed

on 13th June 2016)

European Union (2006). Recommendation of the European Parliament and of the Council of 18

December 2006 on key competences for lifelong learning. Online.

http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32006H0962 (accessed on 13th

June 2016)

Eurydice (2016). Entrepreneurship Education at School in Europe: Eurydice Report. Online.

https://webgate.ec.europa.eu/fpfis/mwikis/eurydice/images/4/45/195EN.pdf (accessed on 13th

June 2016)

Johnson Cornell University (2015). The Global Innovation Index: Country ranking. Online.

https://www.globalinnovationindex.org/content/page/data-analysis/ (accessed on 13th June 2016)

Martinez, C., Levie, J., Kelley, D. J., Aemundsson, J. and Schott, T. (2008). Global

entrepreneurship monitor special report: A global perspective on entrepreneurship education and

training. Online.

http://www.babson.edu/Academics/centers/blank-center/global-research/gem/Documents/gem-

2010-special-report-education-training.pdf (accessed on 13th June 2016)

McCoshan, A., Lloyd, P., Blakemore, M., Gluck, D., Betts, J., Lepropre, M. and McDonald, N.

(2010). Towards Greater Cooperation and Coherence in Entrepreneurship Education. Online.

Page 62: 2nd External Evaluation Report of SEECEL activities

Confidential 62

http://docplayer.net/13314512-Towards-greater-cooperation-and-coherence-in-entrepreneurship-

education.html (accessed on 13th June 2016)

PISA (2012). Creative Problem Solving: Students’ skills in tackling real-life problems (Volume

V). Online. http://www.oecd.org/pisa/keyfindings/pisa-2012-results-volume-v.htm (accessed on

13th June 2016)

SEECEL (2013). Strategic Plan 2013 – 2016: Think to Act Entrepreneurially. Online.

Available from

http://www.seecel.hr/UserDocsImages/Documents/SEECEL%20Strategic%20Plan%202013%20

2016.pdf (accessed 23rd June 2016)

Stake, R. E. (2011). Qualitative research: Studying how things work. New York: The Guilford

Press.

Thematic Working Group on Entrepreneurship Education (2014). Final Report. Online.

http://ec.europa.eu/education/policy/strategic-framework/archive/documents/entrepreneurship-

report-2014_en.pdf (accessed on 13th June 2016)

Page 63: 2nd External Evaluation Report of SEECEL activities

Confidential 63

Annexes

Page 64: 2nd External Evaluation Report of SEECEL activities

Confidential 64

List of Interviews

Country Name Role Name of organization

Former

Yugoslav

Republic of

Macedonia

Valentin

Miričevski

vice-dean, professor;

EL coordinator

Faculty of Natural Sciences

and Mathematics

Former

Yugoslav

Republic of

Macedonia

Elizabeta

Jovanovska-

Radanovik

adviser, SBA team

member/coordinator

Vocational and Educational

Training Centre

Former

Yugoslav

Republic of

Macedonia

Bojan Jovanovski assistant professor,

SBA team member

Faculty of Mechanical

Engineering Ss. Cyril and

Methodius

Former

Yugoslav

Republic of

Macedonia

Nada Stoimenova adviser for primary and

secondary school, SBA

team member

Ministry of Education and

Science

Former

Yugoslav

Republic of

Macedonia

Mile Boškov adviser Business Confederation of

Macedonia

Former

Yugoslav

Republic of

Macedonia

Igor Jurukov adviser Bureau for Development of

Education

Former

Yugoslav

Republic of

Macedonia

Emilija

Hadzivasileva

teacher, EL coordinator Gymnasium Panče

Arsovski

Former

Yugoslav

Republic of

Macedonia

Zorica Zavirovska teacher Gymnasium Panče

Arsovski

Former

Yugoslav

Republic of

Macedonia

Milena Ignjatova teacher, EL coordinator Primary School Lazo

Angelovski

Former

Yugoslav

Republic of

Macedonia

Valentina Petrova teacher Primary School Lazo

Angelovski

Page 65: 2nd External Evaluation Report of SEECEL activities

Confidential 65

Bosnia and

Herzegovina

Radmila

Jakovljević

adviser Agency for Pre-primary,

Primary and Secondary

Education

Bosnia and

Herzegovina

Vesna Puratić expert officer, SBA

expert

Ministry of Civil Affairs

Bosnia and

Herzegovina

Dragan Milović Assistant Minister/SBA

coordinator

Ministry of Foreign Trade

and Economic Relations

Bosnia and

Herzegovina

Danica Vasilj adviser, SBA expert Agency for Pre-primary,

Primary and Secondary

Education

Bosnia and

Herzegovina

Lucija Mikulić head master, teacher,

EL coordinator

Primary School Ruđer

Bošković

Bosnia and

Herzegovina

Jasna Spajić teacher Primary School Ruđer

Bošković

Bosnia and

Herzegovina

Jasminka Petric teacher, EL coordinator Gymnasium fra Grga

Martić

Bosnia and

Herzegovina

Anita Lukenda professor, EL

coordinator

Faculty of Natural Science,

Mathematics and

Educational Science

Bosnia and

Herzegovina

Ivana Vasilj assistant Faculty of Natural Science,

Mathematics and

Educational Science

Montenegro Ivana Mrvaljević adviser Ministry of Education and

Sports

Montenegro Andjela Pušonjić adviser, SBA team,

TNA

Directorate for

development of SMEs

Montenegro Nevena Čabrilo senior adviser, SBA

team

Bureau for Education

Services

Montenegro Dragutin Ščekić SBA expert, EL

coordinator

Primary School Veljko

Drobnjaković

Montenegro Nenad Meštrović teacher, EL coordinator Gymnasium Kotor

Montenegro Lidija Vasić teacher, EL coordinator Primary School Njegoš

Montenegro Marija Samardžić teacher Primary School Njegoš

Croatia Dijana Bezjak former assistant

Minister of

Entrepreneurship and

crafts, board member

Ministry of Regional

Development and EU

Funds

Croatia Želimir Kramarić CCE is one of the co-

founder of SEECEL

The Croatian Chamber of

Economy

Croatia Tajana Kesić Šapić director Centre for

entrepreneurship,

innovation and

technological development

Croatia Antonija Mršić SC member Ministry of

Entrepreneurship and

Crafts

Page 66: 2nd External Evaluation Report of SEECEL activities

Confidential 66

Croatia Dragica Karaić SC member, Danube

strategy coordinator

Ministry of

Entrepreneurship and

Crafts

Croatia Vesna Hrvoj Šic senior adviser, SBA

expert

Ministry of Science,

Education and Sports

Croatia Jadranka Bjelica Director/Head teacher Primary School Fran Krsto

Frankopan

Croatia Majda Granić project coordinator Primary school Bartol

Kašić

Croatia Marijan Grbac head master of school Primary school Bartol

Kašić

Albania Evisi Kopliku SC member SBA coordinator

Mirela Andoni SC member Ministry of Education and Science, Institute for Educational Development - IED

Serbia Katarina Obradović

Jovanović

SC member SBA coordinator

Radovan Živković SC member Education

EU and other institutions

INSTITUTION POSITION NAME

DG GROW Internal Market, Industry, Entrepreneurship and SMEs, European Commission

Policy Officer Simone Baldassarri

DG EAC Education and Culture

Policy Officer Helene Skikos

DG EMPL Head of Unit Ana Carla PEREIRA

ETF Senior Specialist - Thematic Policy Unit

Anthony Gribben

OECD Head of the OECD Investment Compact

Alan Paić & Annita Richter

Regional Cooperation Council Senior Expert for Smart Growth Vanja Ivošević

Page 67: 2nd External Evaluation Report of SEECEL activities

Confidential 67

Tables from Educational Institutions Survey Results

The number of respondents who answered survey varied among certain survey questions. Initial

questions were answered by 78 respondents. The respondents from educational institutions were

asked to indicate level of satisfaction/dissatisfaction with the 18 distinct project features, on the

five point Likert scale (1- very dissatisfied – 5 very satisfied). The Cronbach's alpha for the 18

item scale was 0.940, indicating the excellent scale reliability. According to the findings presented

in Table 7, the respondents were the most satisfied with the: Student participation in activities

(M=4.59, SD=0.88); Experiences from peer learning activities (M=4.55, SD=0.93); SEECEL’s

response to problems that came up during implementation (M=4.54, SD=0.97); and The

program’s contribution to the school’s entrepreneurial culture (M=4.52, SD=0.88). It is apparent

that the respondents were satisfied with program features which were directly linked to the school

and student involvement and SEECEL’s role in the programme. Nevertheless, the respondents

were the least satisfied with the: Feedback we received on lesson plans (M=3.62, SD=1.17); The

orientation program for school management (M=3.99, SD=1.03); and Overall benefits for the

local community (M=4.04, SD=1.02). The project features that were not recognized as successful

deal with support and training received from training institutions and actual impact of the project

on the local community.

Table 7. Level of the respondents’ satisfaction/dissatisfaction with the distinct project features

M SD

1. The training I received from SEECEL and its partners 4.12 1.09

2. Quality of the lesson plans developed as part of involvement in SEECEL

activities 4.14 .92

3. Feedback we received on lesson plans 3.62 1.17

4. Opportunity to implement lesson plans in teaching 4.28 .88

5. Opportunity to implement certain activities from lesson plans in teaching 4.38 .92

6. The orientation program for school management 3.99 1.03

7. Experience from developing your organization’s action plans 4.29 .94

8. Experiences from piloting the action plan 4.36 1.00

9. Experiences from peer learning activities 4.55 .93

10. SEECEL’s response to problems that came up during implementation 4.54 .97

11. Communication and cooperation with other people in your school involved in

the project 4.23 1.03

12. Collaboration with Mentor Schools 4.17 1.08

13. Student participation in activities 4.59 .88

14. Overall benefits for teachers 4.20 1.01

15. Overall benefits for students 4.48 .91

16. Overall benefits for the local community 4.04 1.02

17. The program’s contribution to the school’s entrepreneurial culture 4.52 .88

Page 68: 2nd External Evaluation Report of SEECEL activities

Confidential 68

18. Overall participation in SEECEL activities 4.52 .86

The respondents were asked to indicate level of agreement/disagreement with the 8 statements

describing impact of their participation in SEECEL activities, on the five point Likert scale (1-

strongly disagree – 5 strongly agree). The Cronbach's alpha for the 8 item scale was 0.887,

indicating the good scale reliability. According to the findings presented in Table 8, the

respondents recognized the positive impact of the activities on all listed aspect of school work and

organizational culture. The project had the strongest impact on the: Improvement of creativity

among students (M=4.55, SD=.631) and Improvement of the entrepreneurship culture in their

school/organization (M=4.39, SD=.691). The respondents believed that the project activities had

least impact on Teaching improvement in their schools (M=4.03, SD=.664). Furthermore, the

respondents listed some additional benefits of their participation in SEECEL activities such as:

new contacts, improved cooperation between teachers and with parents, new tools and new school

curriculum, benefits for students (preparation for lifelong learning, work opportunities with local

businesses, improved motivation and initiative).

Table 8 Level of the respondents’ agreement with the statements on positive impact of the

participation in SEECEL activities.

M SD

1. The entrepreneurship culture in my school/organization has improved 4.39 .691

2. Teaching has improved 4.03 .664

3. Learning has improved 4.12 .676

4. Creativity among students has improved 4.55 .631

5. School management team has improved 4.38 .769

6. Creativity among teachers has improved 4.22 .783

7. Students are more engaged in the classroom 4.30 .773

8. The overall entrepreneurship eco-system culture has improved (Classroom,

school, and local community culture) 4.28 .802

The respondents which were not using CoP platform were asked to specify reasons for not using

it. Their answers varied between use of other means of communication (emails, social networks,

and phone conversations), lack of understanding on how to use platform and what to talk about

under the CoP discussion forums, lack of computers and internet at school, to setup of the CoP

platform which is not aligned to teachers’ needs. As one respondent wrote: ‘The CoP might have

been more and better utilized if there had been spaces for subject teachers (not just the PI's

managers and coordinators) to collaborate, devise LPs and other activities, exchange best

practices’.

The respondents were asked to evaluate CoP’s ease of use through 6 statements, on the five point

Likert scale (1- strongly disagree – 5 strongly agree). The Cronbach's alpha for the 6 item scale

Page 69: 2nd External Evaluation Report of SEECEL activities

Confidential 69

was 0.892, indicating the good scale reliability. According to the findings presented in Table 9,

the respondents appreciate the User friendly features (M=4.34, SD=.871) and Ease of use

(M=4.29, SD=.805). Still, certain percentage of respondents cannot use the platform without

instructions (M=3.95, SD=1.082).

Table 9 Level of the respondents’ agreement with the statements on CoP’s ease of use

M SD

1. It is easy to use 4.29 .805

2. It is easy to understand 4.23 .844

3. It is fast 4.22 .875

4. It is user friendly 4.34 .871

5. It is flexible 4.25 .811

6. I can use it without instructions 3.95 .082

Respondents were asked to evaluate SEECEL website ease of use through 6 statements, on the

five point Likert scale (1- strongly disagree – 5 strongly agree). The Cronbach's alpha for the 6

item scale was 0.903, indicating the excellent scale reliability. According to the findings presented

in Table 10, the respondents appreciate the User friendly features (M=4.33, SD=.909) and

Flexibility (M=4.28, SD=.806) of the SEECEL website.

Table 10 Level of the respondents’ agreement with the statements on SEECEL website ease of

use

M SD

1. It is easy to use 4.23 .921

2. It is easy to understand 4.27 .859

3. It is fast 4.22 .934

4. It is user friendly 4.33 .909

5. It is flexible 4.28 .806

6. I can use it without instructions 4.05 1.015

On the five point Likert scale (1- very dissatisfied – 5 very satisfied), the respondents were asked

to indicate level of satisfaction/dissatisfaction with the program related communication and

collaboration through 4 statements. The Cronbach's alpha for the 6 item scale was 0.727,

indicating the acceptable scale reliability. According to the findings presented in Table 11, the

respondents were very satisfied with collaboration with SEECEL staff (M=4.49, SD=1.045).

Page 70: 2nd External Evaluation Report of SEECEL activities

Confidential 70

Table 11 Level of the respondents’ satisfaction/dissatisfaction with the program related

collaboration.

M SD

1. Collaboration with SEECEL staff 4.49 1.045

2. Collaboration with the teacher training agency 3.94 1.162

3. Collaboration with other schools involved in the project 4.14 1.030

The respondents were asked to indicate level of agreement/disagreement with the 6 statements

describing different types of additional support for project participation. The Cronbach's alpha for

the 6 item scale was 0.700, indicating the acceptable scale reliability. According to the findings

presented in Table 12, the respondents recognized need for support in a variety of areas. The most

commonly recognized areas that require support are: Peer learning (M=4.30, SD=.908) and

Training (M=4.26, SD=.974). Educational institutions received additional support for the project

participation through teacher training, exchange of good practices f-2-f and through social

networks, teachers' manuals for lesson plans delivered by school subject counsellors. Furthermore,

the respondents commented that they would benefit from the online EL teacher training course

under the CoP platform.

Table 12 Level of respondents’ agreement/disagreement on need for different types of support in

this project.

M SD

1. Resources 4.15 .792

2. Training 4.26 .974

3. Support 3.92 1.149

4. Funding 4.23 1.093

5. Time 4.15 1.030

6. Peer learning 4.30 .908

Finally, the respondents were asked to indicate level of agreement/disagreement with the 5

statements dealing with project challenges. The Cronbach's alpha for the 5 item scale was 0.817,

indicating the good scale reliability. According to the findings presented in Table 13, most of the

respondents did not perceive listed challenges as problematic. Still, the respondents believed that

Lack of funding for student and teacher participation in peer learning activities (M=2.94,

SD=1.213) and Lack of time for teachers to integrate entrepreneurship

learning in the classroom (M=2.93, SD=1.328) may be somewhat more problematic than other

listed challenges. Also, the respondents mentioned that participation was difficult due to everyday

obligations which they have in the classroom and some other administrative issues which they are

facing in their schools. One of the complaints focused on the amount of administrative work

involved with the project participation.

Page 71: 2nd External Evaluation Report of SEECEL activities

Confidential 71

Table 13 Level of the respondents’ agreement with the listed project challenges.

M SD

1. The project is not aligned with the school curriculum 2.71 1.260

2. We did not receive adequate support 2.11 1.042

3. Lack of funding for project activities 2.73 1.148

4. Lack of funding for student and teacher participation in peer

learning activities 2.94 1.213

5. Lack of time for teachers to integrate entrepreneurship

learning in the classroom 2.93 1.328

Page 72: 2nd External Evaluation Report of SEECEL activities

Confidential 72

Tables from Steering Committee and Stakeholder Survey Results

The survey respondents were asked to indicate how strongly they agree or disagree with the 7

statements related to SEECEL, on the five point Likert scale (1- strongly disagree – 5 strongly

agree). The Cronbach's alpha for the 7 item scale was 0.890, indicating the good scale reliability.

According to the findings presented in Table 14, the respondents in average agreed or strongly

agreed with all of the statements. They most strongly agreed with the statements that SEECEL’s

activities are contributing to the development of entrepreneurship culture in the region (M=4.80,

SD=.422; M=4.60, SD=.548) and that SEECEL supports the alignment of national policies with

EU recommendations and policies related to entrepreneurial learning (M=4.80, SD=.422;

M=4.60, SD=.548).

Table 14 Level of the respondents’ agreement with the statements related to SEECEL.

Steering Committee Stakeholders

M SD M SD

1. The results which have been achieved by SEECEL in the period

2013-2016 are satisfactory 4.70 .483 4.60 .548

2. SEECEL’s activities are contributing to the development of

entrepreneurship culture in the region 4.80 .422 4.60 .548

3. SEECEL supports the alignment of national policies with EU

recommendations and policies related to entrepreneurial

learning 4.80 .422 4.60 .548

4. The strategic goals and approaches in SEECEL’s 2013-2016

strategic plan and key policies are relevant to the mission of

SEECEL 4.60 .516 4.40 .894

5. The SEECEL activities are relevant to its goals 4.60 .516 4.40 .894

6. The approaches identified in SEECEL’s 2013-2016 strategic

plan, addressed the conditions, needs and interests of key

stakeholders 4.40 .516 4.60 .548

7. The approaches identified in SEECEL’s rolling/ongoing work

plan addressed the conditions, needs and interests of key

stakeholders 4.20 .919 4.60 .548

Furthermore, on the five point Likert scale (1- strongly disagree – 5 strongly agree) the respondents

were asked to indicate how strongly they agree or disagree with the 20 statements about

appropriateness, effectiveness, and relevance of SEECEL. The Cronbach's alpha for the 20-item

scale was 0.934, indicating the excellent scale reliability. According to the findings presented in

Table 15, the respondents in average agreed or strongly agreed with all of the statements. They

most strongly agreed with the statements that SEECEL’s work supports structured regional

cooperation (M=4.80, SD=.414) and that SEECEL’s work is in line with EU policies (M=4.80,

SD=.414). Although the participation in SEECEL activities was high, the respondents indicate

Page 73: 2nd External Evaluation Report of SEECEL activities

Confidential 73

that in the future they would benefits from more participation in SEECEL activities (M=4.00,

SD=.816; M=3.80, SD=1.304)

Table 15 Level of the respondents’ agreement with the statements about appropriateness,

effectiveness, and relevance of SEECEL.

Steering Committee Stakeholders

M SD M SD

1. The SEECEL activities in the 2013-2016 strategic plan were

effective in pursuing its mission. 4.60 .516 4.80 .447

2. SEECEL’s rolling work plan was effective in pursuing its

mission. 4.40 .516 4.80 .447

3. The peer-learning activities were implemented at a

satisfactory level. 4.60 .516 4.60 .548

4. The peer-learning activities contributed to the development

of a culture of entrepreneurship 4.40 .966 4.60 .548

5. SEECEL activities facilitated the exchange of good practice

among stakeholders 4.50 .972 4.40 .548

6. SEECEL had the necessary resources and capacity to

undertake the activities required to achieve its strategic

goals. 4.10 1.287 4.00 1.225

7. SEECEL’s key stakeholders were satisfied with the

approaches and activities used to achieve its mission. 4.40 .516 4.40 .548

8. There was a fair participation among stakeholders in

SEECEL activities 4.00 .816 3.80 1.304

9. The targets and measures of success set out in the strategic

plan were realistic. 4.30 .483 4.20 .447

10. SEECEL’s work is in line with EU policies. 4.90 .316 4.60 .548

11. SEECEL’s work is of benefit to my country’s national

development. 4.60 .516 4.60 .548

12. SEECEL’s work is of benefit to my organization’s

development. 4.10 .876 4.00 1.225

13. SEECEL’s work is of benefit to regional development. 4.50 .527 4.60 .548

14. SEECEL’s work supports structured regional cooperation. 4.80 .422 4.80 .447

15. The structured regional cooperation run by SEECEL

promotes exchange of knowledge and good practices across

the region 4.80 .422 4.60 .548

16. SEECEL’s work contributes substantially to the

development of Human Capital in the region (SBA) 4.70 .483 4.60 .548

17. SEECEL supports the institutionalization of regional

dialogue and cooperation in the area of lifelong

entrepreneurial learning within the Small Business Act

(SBA) under principles 1 and 8

4.80 .422 4.60 .548

18. The Training Needs Analysis (TNA) is an important tool for

understanding the needs of SMEs in the region 4.30 .823 4.20 .447

19. The Training Needs Analysis (TNA) supports national

developments in the area of entrepreneurship 3.90 1.101 4.40 .548

20. The Training Needs Analysis (TNA) contributes to the

development of entrepreneurship in the region 4.00 1.155 4.00 1.225

Page 74: 2nd External Evaluation Report of SEECEL activities

Confidential 74

The respondents from educational institutions were asked to indicate level of

satisfaction/dissatisfaction with the 6 distinct features related to SEECEL work, on the five point

Likert scale (1- very dissatisfied – 5 very satisfied). The Cronbach's alpha for the 6-item scale was

0.754, indicating the acceptable scale reliability. According to the findings presented in Table 16,

the respondents in average were satisfied or very satisfied with all of the statements. The

respondents were the most satisfied with the Collaboration with SEECEL management team

(M=4.70, SD=.483; M=4.80, SD=.447) and Collaboration with SEECEL staff (M=4.70, SD=.483;

M=4.60, SD=.548).

Table 16 Level of the respondents’ satisfaction/dissatisfaction with the listed features.

Steering Committee Stakeholders

M SD M SD

1. Collaboration with SEECEL management team 4.70 .483 4.80 .447

2. Collaboration with SEECEL staff 4.70 .483 4.60 .548

3. Collaboration with SEECEL Steering Committee Members 4.60 .516 4.00 1.000

4. Collaboration with other stakeholders in the country and

region within the framework of SEECEL activities 4.00 1.155 4.20 .447

5. SEECEL adopted feedback from Steering Committee

members and integrated it in its activities 4.30 .949 3.80 .837

6. The Steering Committee meetings were engaging and

productive 4.40 .699 3.60 .894

The respondents were asked to indicate how strongly they agree or disagree with the 5 statements

related to challenges that SEECEL faces, on the five point Likert scale (1- strongly disagree – 5

strongly agree). The Cronbach's alpha for the 5-item scale was 0.704, indicating the acceptable

scale reliability. According to the findings presented in Table 17, the respondents on average were

neutral or agreed that SEECEL lacks sufficient funding (M=3.40, SD=.966; M=3.20, SD=.837) and

disagreed that the project is not aligned with the national life-long learning strategy of their

country (M=2.10, SD=1.197; M=1.80, SD=.447). In addition, the respondents listed some

additional challenges that SEECEL is facing. The listed challenges are: need for increase of the

pilot institutions network, need for more concrete results (e.g. SEECEL should develop model of

entrepreneurial education), more promotion activities in partner countries.

Page 75: 2nd External Evaluation Report of SEECEL activities

Confidential 75

Table 17 Level of the respondents’ agreement with the statements about challenges that SEECEL

faces.

Steering Committee Stakeholders

M SD M SD

1. Lack of sufficient funding 3.40 .966 3.20 .837

2. The project is not aligned with the national

entrepreneurial learning strategy in my country 2.10 1.197 1.80 .447

3. The project is not aligned with the national life-long

learning strategy of my country 1.90 .738 2.00 1.225

4. Lack of sufficient communication among stakeholders 2.70 1.160 3.00 1.414

5. Lack of adequate support 2.30 .949 3.20 1.643

The survey respondents were asked to indicate how strongly they agree or disagree with the 7

statements related to changes that are needed in order to achieve better results, on the five point

Likert scale (1- strongly disagree – 5 strongly agree). According to the findings presented in Table

18, the respondents recognized that following changes are needed: Training for all stakeholders

(M=4.50, SD=.527; M=4.60, SD=.548) and need for Additional resources for SEECEL activities

(M=4.00, SD=.943; M=4.60, SD=.548). The respondents disagreed that there is a need for

Changes in organization and functionality of SEECEL’S management and governance (M=2.50,

SD=1.345; M=2.80, SD=1.095). In addition, the respondents commented that it would be

necessary to establish coordination body in each partner country and assure that steering committee

activities are more transparent to all stakeholders.

Table 18 Level of the respondents’ agreement with the statements about changes that are needed

in order to achieve better results.

Steering Committee Stakeholders

M SD M SD

1. Changes in organization and functionality of SEECEL’S

management and governance 2.50 1.354 2.80 1.095

2. Change in the role of the Steering committee 2.90 1.287 3.40 1.140

3. Additional resources for SEECEL activities 4.40 .699 4.20 .837

4. Training for all stakeholders 4.50 .527 4.60 .548

5. Additional funding 4.00 .943 4.60 .548

6. Better communication 3.00 1.155 3.40 1.517

7. More time for implementing activities 3.40 1.174 4.00 1.225

Page 76: 2nd External Evaluation Report of SEECEL activities

Confidential 76

Respondents were asked to indicate how strongly they agree or disagree with the 3 statements

related to SEECEL’s future, on the five point Likert scale (1- strongly disagree – 5 strongly agree).

According to the findings presented in Table 19, the respondents on average strongly agree that

SEECEL should become the European centre and/or agency for entrepreneurial learning

(M=4.40, SD=.699; M=3.60, SD=1.673).

Table 19 Level of the respondents’ agreement with the statements about SEECEL’s future.

Steering Committee Stakeholders

M SD M SD

1. SEECEL should become the European centre and/or

agency for entrepreneurial learning 4.40 .699 3.60 1.673

2. SEECEL should focus more on the strategic level of

entrepreneurial learning 4.00 .816 3.80 1.304

3. SEECEL should work closely with training institutes

for implementing the program in schools and the

education system in general 4.10 .994 3.60 1.140

Page 77: 2nd External Evaluation Report of SEECEL activities

Confidential 77

Executive Summary for Grant Contract 2013/316-501

This executive summary presents the findings and recommendations regarding the external

evaluation of the SEECEL activities under Grant Contracts 2013/316-501 conducted at the SEE

level (in SEECEL member states). The aims of the evaluation were:

To document SEECEL’s progress against SEECEL’s mission and vision, its key policies

and the strategic targets set out in the 2013-2016 strategic plan.

To review and assess the relevance and appropriateness of the 2013-2016 strategic plan to

SEECEL’s mission, and the specific objectives of the current project.

To recommend changes of strategy and emphasis that should be incorporated into the

SEECEL’s 2017-2020 strategic plan.

Data were collected and analyzed using both quantitative and qualitative methods. In summary,

the data collected were from the following sources:

Review of the literature, documents, policies, reports, past evaluations, and EU documents.

Study visits in 4 countries (e.g. Croatia, Bosnia and Herzegovina, Montenegro, and the

Former Yugoslav Republic of Macedonia) and interviews with local stakeholders and

implementing institutions.

Skype interviews, phone interviews and/or questionnaires from, Albania, Serbia, and other

relevant stakeholders.

Two online surveys distributed to education institutions, Steering Committee members and

other stakeholders.

Interviews and focus group discussions with SEECEL team

Overall Recognition

SEECEL’s work is aligned with a number of European policy documents, strategies, and

initiatives. The EU2020 strategy places big emphasis on developing the Human Capital of Europe.

Recognizing that the large majority of enterprises in Europe are SMEs, building their capacity and

promoting entrepreneurship is of paramount importance. In 2006, the European Commission had

identified the ‘sense of initiative and entrepreneurship’ as one of the eight key competences for

lifelong learning. The 2008 Small Business Act (SBA) for Europe, the 2012 Communication on

Rethinking Education, the Entrepreneurship Action Plan 2020, the New Skills Agenda for Europe,

and the very recent EntreComp: The Entrepreneurship Competence Framework 2016, all place

entrepreneurship education and entrepreneurial learning at the center of the debate for improving

competitiveness, combating unemployment and supporting growth and sustainable development.

Furthermore, entrepreneurship is a key driver in the European Commission’s Investment Plan for

Europe (Juncker Plan) and is also integrated in the United Nation’s 2030 Agenda for Sustainable

Development.

Page 78: 2nd External Evaluation Report of SEECEL activities

Confidential 78

After detailed data collection and analysis, it became apparent that SEECEL has achieved its

objectives and that its strategic goals. In some occasions success exceeded the expectations. Data

analysis identified some issues that when addressed will constitute SEECEL as one of the most

respected, effective, and professional regional organizations in the field of Entrepreneurial

Learning. Below a summary of the most important findings are presented. For the full findings

and recommendations, please refer to the full report.

Successes

Detailed data collection and analysis demonstrated that SEECEL is successful in creating

the foundation for a developed entrepreneurship culture across the region. Already there is

expression of interest from countries beyond the region and the EU to participate and

collaborate with SEECEL. Furthermore, SEECEL fosters alignment with the EU policies,

strategies and actions in the field of entrepreneurial learning.

SEECEL has been extremely successful. As a result, the European Commission has

recognized SEECEL as a good practice in strategic regional cooperation, good practice for

conceptual solutions in the field of entrepreneurial learning as a key competence and good

practice in line with the first principle of the Small Business Act for Europe. SEECEL has

become a reference point both for EU Member States and the pre-accession countries and

was awarded with the following awards:

o Creators for Centuries reward and recognition for the year 2014 for contribution

to the development of entrepreneurship in South East Europe, awarded to SEECEL

by an independent international committee during the Regional summit of

entrepreneurs of the South East Europe the best 300;

o European Enterprise Promotion Award (EEPA) as National Winners in 2014

for Promoting the Entrepreneurial Spirit, awarded to SEECEL by the European

Commission;

o The Champion of Regional Cooperation of 2013, awarded to SEECEL by the

Regional Cooperation Council for its active and dynamic approach to regional

cooperation and contribution to the SEE 2020 preparation process;

o The European Projects Awards IPA 2013 - for the project entitled ''Developing

the Entrepreneurial Society in Western Balkans and Turkey'', in the category

“ongoing” projects and best actors in Regional and Local Development (2013);

o The Knowledge Economy Network (KEN), an international Best Practice Award

for "original and successful practice in any domain of knowledge society and

economy development" and especially for "good practice in successful regional

cooperation in training and education”.

The work of SEECEL is based on a successful model. Implementing the policy and

providing the field training and school implementations, is what makes SEECEL different

from other similar projects.

Page 79: 2nd External Evaluation Report of SEECEL activities

Confidential 79

SEECEL strongly supports and encourages networking and regional cooperation. SEECEL

had a big impact in member states at the policy level. Without SEECEL some of these

countries could not be at the same level when it comes to entrepreneurship learning.

Since 2009, SEECEL has developed in-depth knowledge and core expertise in the area of

entrepreneurship learning. As a result, SEECEL is a member of many expert groups at the

European Commission level working in the area of entrepreneurial learning.

All stakeholders are very satisfied with the staff and management team of SEECEL. The

staff and management were very supportive to countries in implementing programs and

reaching their targets.

Teachers, principals, and school staff interviewed, seemed to be very satisfied with the

program and the benefits of entrepreneurial learning. Teachers clearly indicated that their

greatest motivation and reward in participating in the program was the impact the program

had on learners.

SEECEL contributed substantially to the progress towards the Small Business Act for

Europe which SEECEL member states demonstrate. Particularly, with the human capital

dimensions (1&8) which are the focus of SEECEL.

As it was clearly stated in the OECD report “SME Policy Index: Western Balkans and

Turkey 2016”, SEECEL member states have made substantial progress towards the

implementation of the SBA, which is a good indication of the hard and dedicated work of

SEECEL and its team.

The Training Needs Assessment (TNA) is an important tool that contributes substantially

towards the evidence-based policy making. TNA supports SEECEL member states in

developing the training and support mechanisms their SMEs need to succeed and grow.

SEECEL participates in several EC expert and advisory groups such as the groups for

Transversal skills (DG EaC), Indicators (DG EaC), EnreComp (DG JRC), and WEGATE

– EU WE platform (DG GROW).

Challenges

Given the fact that SEECEL has already made substantial contribution in the region and

beyond, the European Commission should provide long-term support to SEECEL to

continue this very important work.

One of the challenges is due to the absence of a mandate to the Steering Committee

Members from some SEECEL member states. National governments need to provide such

a mandate because that will be the vehicle that will facilitate getting SEECEL related

activities, budgets, and other resources into the national budgets and that can support

SEECEL and member states strategic objectives.

The evidence-based model, tools and resources on Entrepreneurial Learning developed by

SEECEL, can only be implemented successfully with the support of teacher training

institutes in SEECEL member states.

All activities and communication is conducted in English. As the project expands it needs

to have a plan for allowing local stakeholders to operate in their local language.

Page 80: 2nd External Evaluation Report of SEECEL activities

Confidential 80

During program implementation, it is important to include local experts from the countries

in which the program is implemented.

None of the schools had prior experience with entrepreneurial learning and how to

implement it in their teaching practices. Schools would have been more efficient and

effective if more training was provided in advance of implementation activities.

Participating in teacher training is not required for schools that join the program which

leads to discrepancies in program implementation. The training offered is insufficient and

the actual implementation of the training (delivering to the teachers) needs to be improved.

In order to increase its reach and represent multiple stakeholders, and have more impact on

the local, national, and regional level the Steering Committee needs to include active

representatives and perhaps mentors from the business community.

The history of the region and the political complexities related to this history, present

certain challenges, particularly when there is a need to get stakeholders from various

countries to collaborate.

Recommendations

Since 2009, SEECEL has developed in-depth knowledge and capacity in the area of

entrepreneurship learning. SEECEL makes an important contribution towards

entrepreneurial learning, and it is important to continue and expand its work. Furthermore,

SEECEL could also play a more active role in the Mediterranean region and Eastern

Partnership Countries (EaP) since there was already strong expression of interest from

other non-EU countries to participate in SEECEL activities.

SEECEL has strong potential and the capacity to become part of the infrastructure solution

for EU, and it has developed an evidence-based model, tools and processes to implement

SME and entrepreneurship policies and practices.

The communication strategy of SEECEL needs to be adjusted so that it improves the

awareness and branding for SEECEL’s vision and role.

Many resources are already available on the SEECEL website and the Community of

Practice. As the project grows, it is important to build mechanisms to allow for more users

to engage with the online tools and benefit from SEECEL activities and interaction with

the SEECEL community. Also, the TASK BOX on the Community of Practice can be

further developed to allow for teachers to receive prompt feedback on their work.

Examples of integrating entrepreneurial learning in SEECEL member states and good case

studies and success stories need to be better documented and shared. Good success stories

have the potential to increase awareness among teachers, parents, policy-makers and the

general public about the many benefits of Entrepreneurial Learning.

Business representatives, chambers of commerce, and SMEs, should be given a more active

role. Some actions that will strengthen the cooperation between the business sector, the

education sector and the broader community need to be designed and implemented.

Education plays an important role in the entrepreneurship learning ecosystem. As SEECEL

member states progress towards integrating entrepreneurial learning in their schools, they

Page 81: 2nd External Evaluation Report of SEECEL activities

Confidential 81

need to begin drafting solid plans on how to scale up from a few schools to the whole

education system.

As the program grows, SEECEL should use elearning technologies to develop the training

of teachers, mentors and stakeholders to allow for flexibility of delivery and possibility to

reach out to as many countries as possible.

It is extremely important that teacher training is required for all schools that decide to join

the program. Teacher training needs to be well structured, and offered at school level in

parallel with ongoing teacher support.

SEECEL member states and their respective ministries of education and teacher training

authorities can develop a certification system which will certify teachers who complete the

training and implement the program and also act as a reward system for teachers and

schools.

In order to properly monitor the long term impact of entrepreneurial learning, it is important

to establish a benchmarking process including metrics and other tools to collect long term

data.

The full implementation of the Small Business Act for Europe should be continued. The

very important work of the Training Needs Assessment should be expanded and the

findings from the assessment need to be used by education authorities to develop custom

programs that will target each country’s specific needs.

Page 82: 2nd External Evaluation Report of SEECEL activities

Confidential 82

Executive Summary for Grant Contract 2013/334-013

This executive summary presents the findings and recommendations regarding the external

evaluation of the SEECEL activities under Grant Contract 2013/334-013 conducted at the SEE

level (in SEECEL member states). The aims of the evaluation were:

To document SEECEL’s progress against SEECEL’s mission and vision, its key policies

and the strategic targets set out in the 2013-2016 strategic plan.

To review and assess the relevance and appropriateness of the 2013-2016 strategic plan to

SEECEL’s mission, and the specific objectives of the current project.

To recommend changes of strategy and emphasis that should be incorporated into the

SEECEL’s 2017-2020 strategic plan.

Data were collected and analyzed using both quantitative and qualitative methods. In summary,

the data collected were from the following sources:

Review of the literature, documents, policies, reports, past evaluations, and EU documents.

Study visits in 4 countries (e.g. Croatia, Bosnia and Herzegovina, Montenegro, and the

Former Yugoslav Republic of Macedonia) and interviews with local stakeholders and

implementing institutions.

Skype interviews, phone interviews and/or questionnaires from, Albania, Serbia, and other

relevant stakeholders.

Two online surveys distributed to education institutions, Steering Committee members and

other stakeholders.

Interviews and focus group discussions with SEECEL team

Overview

Entrepreneurship is a key priority for the European Commission. The valuable work of SEECEL

is in line with a number of European policy documents, strategies, and initiatives. The European

Commission had identified the ‘sense of initiative and entrepreneurship’ as one of the eight key

competences for lifelong learning. The 2008 Small Business Act (SBA) for Europe, the 2012

Communication on Rethinking Education, the Entrepreneurship Action Plan 2020, the New Skills

Agenda for Europe, and the very recent EntreComp: The Entrepreneurship Competence

Framework 2016, all place entrepreneurship education and entrepreneurial learning at the center

of the debate for improving competitiveness, combating unemployment and supporting growth

and sustainable development.

The specific objective of the contract under evaluation was to further develop the lifelong

entrepreneurial learning system with special focus on ISCED 3 level general part of secondary

education through identified set of strategic goals in line with Human Capital Dimension of the

SBA for Europe. Although the report on this evaluation is for a separate project contract and

activities it is visible that SEECEL manage it as a part of a whole process in building

Page 83: 2nd External Evaluation Report of SEECEL activities

Confidential 83

entrepreneurially literate societies. Data collection and analysis clearly demonstrated that SEECEL

has achieved its objectives. The key successes, challenges, and recommendations, are presented

below.

Successes

This was the first pilot cycle for the general part of upper secondary schools. ISCED 3

schools have participated for the first time in SEECEL activities and implemented

entrepreneurial learning in their classrooms. Teachers participated in specially designed

training, used the entrepreneurial learning outcomes, developed and implemented lesson

plans, reflected on their learning, and adopted innovative assessment methods.

From all data collected, it is clear that the work of SEECEL contributes substantially to

promoting an entrepreneurship culture in the SEECEL member states.

Several stakeholders highlighted the fact that entrepreneurial learning pilot activities gave

students and teachers opportunities to engage with innovative teaching and learning and

real entrepreneurial experiences. These kinds of experiences are anticipated for everyone

during the educational cycle in the Entrepreneurship 2020 Action Plan.

ISCED 3 teachers and school staff are satisfied with the staff and management team of

SEECEL. Furthermore, they demonstrated strong interest on the project and the area of

entrepreneurial learning.

From the teachers’ point of view, the benefits for students are many and that as a result of

participation in project activities, learners were more engaged and that their creativity and

innovative thinking has improved.

As a result of SEECEL activities, local companies opened their doors to students from

ISCED 3 schools, having a positive impact on the entrepreneurship ecosystem: the school,

the community and SMEs.

Challenges

The evidence-based model, tools and resources on Entrepreneurial Learning developed and

implemented need to be further supported.

SEECEL needs to work closely with partners and teacher training institutes in SEECEL

member states to ensure that teachers receive the support they need to successfully

implement the project.

ISCED 3 schools did not have any prior experience with entrepreneurial learning and how

to design lesson plans and activities that will bring new experiences to their students. There

is a need for more systemic and systematic training and support to be provided to teachers,

before and during the implementation.

Page 84: 2nd External Evaluation Report of SEECEL activities

Confidential 84

Recommendations

SEECEL has contributed substantially to the development and implementation of

entrepreneurial learning in ISCED 3 schools and it should continue to expand its work.

The SEECEL website and the Community of Practice are very valuable tools for teachers

and schools. Access to this tools needs to be expanded to allow more teachers to use them

in their teaching practice.

Good case studies coming out from ISCED 3 schools need to be well documented and

shared, to help raise awareness on the importance and value of entrepreneurial learning.

All ISCED 3 schools that decide to join the program, need to participate in formal

systematic training for the teachers and the school leadership team.

For successfully scaling up the implementation of entrepreneurial learning in schools,

systemic education reform efforts need to be aligned with the values and principles of

entrepreneurial learning.