“Thematic Priority 3”

Draft 19.12.02

Evaluation of IP + NoE

Evaluation planning1st Call

•1st stage: •2nd stage:•Other instruments:

Call IP/SMEs

•1st stage: •2nd stage:

Call P2/P3

•1st stage: •2nd stage:•Other instruments:

06/3/03 beginning April26/6/03 2nd week July10/4/03 end May / beg June

10/4/03 beginning May03/9/03 mid September

24/4/03 3rd week May16/9/03 2nd week October24/4 + 16/9 3rd week May +

Oct.

Deadline Evaluation period

• Stage 1 : Outline proposal - not “incomplete” - all criteria will be applied

• Stage 2 - S&T and financial details, including work/financial plan for first 18 months;further/more details through hearing - all criteria will be applied

Two Stage Evaluation...

Evaluation procedures for the new instruments

Common approach

•Eligibility Criteria •Criteria & thresholds

•External experts:min 5

Priority 3 approach

• 2 stage submission & evaluation

• Hearings at 2nd stageReduced time to contract, i.e: finalise Description of Work or JPA, administrative and financial plan)

• Stage 1 : Outline proposal - not “incomplete” - all criteria will be applied

• Stage 2 - S&T and financial details, including work/financial plan for first 18 months;further/more details through hearing - all criteria will be applied

Two Stage Evaluation...

Evaluation issues...Integrated Projects

• Relevance to call 3/5

• Potential impact 3/5

• S & T excellence 4/5

• Quality of the consortium 3/5

• Quality of the management 3/5

• Mobilisation of the resources 3/5 24/30

Common for all priorities

IP evaluation criteria

A) Relevance to the objectives of the call, in terms of:

– scope of the IP vis à vis the WP/call (including political objectives - Lisbon, Gothenburg, ERA…)

– support to transformation of industry, addressing real bottlenecks and key problems

– deliverables: breakthrough in new applicable knowledge (new products, processes and services)

– ambition (critical mass; relationship and coherence between modules; multi-sectoral approach….)

– appropriateness of the use of an IP: flagship project? particular case for “IP for SMEs” ?

IP Evaluation criteria (cont)

B) Potential impact, in terms of:

– Competitiveness (i.e: new markets and new products; impacts on industry; clear relevance for SME intensive sectors in the case of “IP for SMEs”)

– Support to EU policies (e.g:health, employment, sustainability, enlargement, etc..) as well as

Standardisation/regulations– Potential impact at international level– Commitment of partners related to exploitation of

potential impacts (corresponding to their strategy?)– Science & Society issues including gender

IP Evaluation criteria (cont)

C) S & T excellence, in terms of:

– S &T objectives, content and achievements (work plan, milestones & deliverables); progress beyond state-of-the-art;

– multidisciplinary, efficient and coherent approach (key to achieving the objectives)

– clear implementation plan; interrelation between the various components of the IP to ensure radical innovation (TT, education, IPR aspects, etc)

– recognition of ERA excellence

IP Evaluation criteria (cont)

D) Quality of the consortium, in terms of:

– expertise of partners (suited to role in the project)– efficient groupings of partners– active industrial involvement, including SMEs

(substantial in the case of “IP for SMEs”)– allocation of activities and scheduling

(complementarity and effectiveness) – high level commitment of partners (credible,

realistic, in line with organisations’ long term strategy) …

– … based on a clear consortium agreement

IP Evaluation criteria (cont)

E) Quality of the management, in terms of:

– effectiveness of the organisational, management and governance structure vis à a vis the “complexity” of IP’s activities: co-ordination and follow-up; performance indicators; procedures for decision-making and conflict resolution; allocation and distribution of responsibilities; articulation between the different modules including use of knowledge and dissemination /exploitation, communication strategies…..

– evolution of the partnership during life of project– knowledge management, including IPR issues,

education and training, communication, etc.

IP Evaluation criteria (cont)

F) Mobilisation of resources, in terms of:

– availability of necessary resources (critical mass): human/financial/equipment

– balance and cost-effectiveness – industrial funding– consistency of the financial plan – potential to mobilise complementary funds, during

and after the project to achieve expected results– key personnel involvement

• Relevance to call 3/5

• Potential impact 3/5

• Degree of integration & the J P A 4/5

• Excellence of the participants 3/5

• Organisation and management 3/5 20/25

Common for all priorities

Evaluation issues...Networks of Excellence

NoE evaluation criteria

A) Relevance to the objectives, in terms of:

– scope of the NoE vis à vis the WP/call– support to competitiveness and sustainable

development through strengthening of scientific and technical excellence

– appropriateness of the use of an NoE: real needs for EU industry and society to integrate activities and research teams in the field?

– ambition towards ERA (overcome fragmentation) – deliverables: progressive/durable integration; world-

wide recognised ‘virtual’ centre of excellence

NoE Evaluation criteria (cont)

B) Potential impact, in terms of:

– effective integration of research capacities in the field and strategic relevance to ERA

– addressing problems of knowledge of tomorrow– socio-economic/industrial impact– spreading of excellence within/beyond the NoE – skills development/generation– contribution to Community societal objectives– potential for TT and dissemination towards industry

and in particular SMEs, in particular contribution to standards

NoE Evaluation criteria (cont)

C) Quality of the integration, in terms of:

– appropriateness of the JPA:

° (1) integrating activities; ° (2) jointly executed research (S&T excellence,

coherence of research activities); ° (3) spreading of excellence, including through

education and training° (4) management activities

– adequacy of the number/profile of researchers to be integrated with NoE’s objectives

– degree of commitments from partners

NoE Evaluation criteria (cont)

D) Excellence of the participants, in terms of:

– expertise of partners (suited to role in the network and achievement of potential impacts)

– high level commitment of partners (credible, realistic) during and after the project

– past experience of networking with reference to management capacitities

– quality of the grouping, expertise, experience and complementarity, in particular potential for TT and dissemination towards industry, in particular SMEs

– possible impact on international cooperation

NoE Evaluation criteria (cont)

E) Organisation and Management, in terms of:

– effectiveness of the organisational, management and governance structure vis à a vis the “complexity” of NoE’s activities: co-ordination and follow-up; performance indicators; response to emerging research needs; procedures for decision-making and conflict resolution(consortium agreement); allocation and distribution of responsibilities, communication strategies…

– Management of resources, and evolution of the partnership during the life of the network

– knowledge management– adequacy of the requested grant

Evaluation procedures for the other instruments

Common approachsingle stage

•Eligibility Criteria •Criteria & thresholds

•External experts:min 3

Specific Targeted Research Projects

• Relevance to call 3/5

• S & T excellence 4/5

• Potential impact 3/5

• Quality of the consortium 3/5

• Quality of the management 3/5

• Mobilisation of the resources 3/5 21/30

Common for all priorities

Evaluation issues...Co-ordination Actions

• Relevance to call 3/5

• Quality of the co-ordination 4/5

• Potential impact 3/5

• Quality of the consortium 3/5

• Quality of the management 3/5

• Mobilisation of resources 3/5 21/30

Common for all priorities

Evaluators’ profile: a key point for IPs…..

– S/T expertise not sufficient !

Ability to evaluate also ‘integrated’ set of activities:

– potential impact (on competitiveness, sustainable development, etc)

– critical mass; financial engineering, IPR issues, training, etc.); – risks; management, communication ...

Evaluators’ profile: a key point for NoEs...

Ability to evaluate:– integration in the field; – potential impact (on ERA, competitiveness,

sustainable development, etc); – management (financial, human resources)– …

Which other expertise (apart of previous experience in networking) is

needed?

Evaluators’ profile: as usual for the other

instruments

Ability to evaluate:

– scientific & technical excellence; – potential impact (on competitiveness,

sustainable development, etc); – management (financial, human resources)– balance of research consortia and proposed

resources