Antonio MocciExternal evaluation

Ede - the Netherlands, 11 December 2009

Impacts

UtilitySustainability

SocietyEconomyEnviron-ment

Project

EvaluationRelevance Efficiency

Effectiveness

OutputsInputsObjectives

Outcomes/ Results

Needsproblems

issues

Evaluation criteria

Objectives for evaluating INTENT

Consider the process of result production, of output realisation

Analyse products in comparison to planned results

Consider the impact of project outcomes on partner organisations and on VET systems.

Working process was characterised by:

Balanced responsibilities

Respect for individual expertise and interests

Use of ICT (Web and virtual workplace).

Process

A two level organisation to achieve two different objectives

Process

Results and outputs production

Confrontation with stakeholders

Products

Three main products:

The unit profile on International Entrepreneurship

Training programme in five sub modules

Assessment criteria and portfolio of International Entrepreneurship

A robust dissemination programme ;

Stakeholder involvement;

Network interrelation

Products

Some characteristics of products

The Unit profile is articulated according to the structure and the requirements of ECVET: this means the declination of a set of knowledge, skills and competence required to perform in an international context effectively and appropriately when interacting with others.

Applicable to different geographical contexts and to different sectors.

Products

Some characteristics of products

The Training programme is articulated in five submodules with specifications for the KSC structure of each sub module and for its articulation in didactic units. Pre requisite are indicated as well, in terms of personal motivation, abilities and previous intercultural experiences. Materials for participants and teachers are listed according to three levels: beginners, intermediate and advanced.

ProductsSome characteristics of products

The Assessment criteria set standards for the assessment of trainees in the programme; they also set guidelines for certification of the training programme.

The Portfolio International Entrepreneurship Competence provides employers and employees, trainers and trainees a record of progress in key attributes of International Entrepreneurship Competence. It has three parts:A Passport of International Entrepreneurship Competence which records all formally assessed competences,A Biography of International Entrepreneurship Competence in which assessees may record personal experiences and encounters that may have contributed to their international and intercultural development,A Dossier of Evidence of International Entrepreneurship Competence in which assessees may keep any documentary or recorded evidence of their progress and actual competence to date.

Project Impact

Internal impact External impact

On partner organisations

On VET systems

Internal impact seems to be stronger and more visible in relation to four areas: first, the integration of the results (or part of them) into qualification systems. Second, the integration of part of results into teachers’ and trainers’ “toolbox”. Third, the extension from the trade sector to other sectors and to different professional roles (workers, managers, entrepreneurs). Fourth, the extension to both IVET and CVET, although continuous training seems to be a more promising

field of development.

Project Internal Impact

Points of attention: Existing results (UP, TP, AC) need to be translated in national languages. Adaptation needs to consider national legal frameworks and standards. The assessment criteria have to be adapted to the target group (students in the

formal education system or non-formal system).Strong points: Flexibility of the solution adopted to design and deliver the training programme. Easy transferability of the unit profile and of training programme to other

sectors and to different professional levels. The process of producing results involved several target groups (institutions,

companies, VET providers), and this added value to results. Relevance of intercultural issues and competences in the process of building the

European Union (Active citizenship). The outputs’ structure make possible the implementation of EQF principles

(levels 3, 4, 5). Teachers, trainers and professionals have the possibility to use and customize

training materials and assessment criteria. Training materials can be addressed to a variety of target groups, including

entrepreneurs.

Project Internal Impact

Intent products - the unit profile, the training programme and the assessment criteria - provide for concrete and tangible tools to

be utilised for using and applying ECVET.

The concrete application of the ECVET tools and principles, the establishment of a credible network to be considered as a reference for system development, the capacity to transfer the INTENT approach into EQF application through NQF upgrading, seem to be the most outstanding impact effects this partnership produced on VET systems and represent the concrete added value INTENT gave to European lifelong learning.

Project External Impact

Relevance: To what extent are the project objectives justified in relation to needs? Can their raison d’être still be proved? Do they correspond to local, national and European priorities ?

In relation objectives formulated in the application form and articulated by the partnership in the inception phase, it is reasonable to say that INTENT was in line with the European policies finalised at improving mobility of students and workers. Consistency is particularly relevant with respect to the ECVET and EQF implementation policies. National priorities and frameworks have been considered.

Conclusions

Effectiveness: To what extent have the objectives been achieved? Have the interventions and instruments used produced the expected effects? Could more effects be obtained by using different instruments?

Objectives have been achieved and planned results produced. Though time schedules had to be modified sometimes (for example, the research activity took longer than expected), outputs have been produced and planned effects were obtained. Results exploitation process was intense in the final part of the project, but it did not finish, according to what project partners declared. Beside internal effectiveness (the capacity to achieve objectives) it seems that the project had also external effectiveness (the capacity to produce results able to satisfy beneficiaries’ expectations).

Conclusions

Utility: Are the expected or unexpected effects globally satisfactory from the point of view of direct or indirect beneficiaries ?

Though external evaluation does not deal explicitly with beneficiaries’ satisfaction, the analyses carried out by partners on training participation testify a visible interest and approval from participants. As for indirect beneficiaries (partner organisations and key actors) collected opinions indicate remarkable satisfaction for work methodology and results usability. Some recurrent indications (such as the need for considering national frameworks and the need for translation in national language) may improve results’ utility.

Conclusions

Sustainability: Are the results and impacts including institutional changes durable over time? Will the impacts continue when public funding is over ?

The impact effects realised by the partnership seem to have activated a process of change and further development of outcomes which is going on in many partner countries. Durability over time seem guaranteed by partners’ commitment and by their roles as VET policies planners and implementers.

Conclusions

Thank you for your attention !!!