Monitoring of the SEE-ERA.NET Pilot Joint CALL (PJC) and the implementation of the Lessons learnt in the SEE-ERA.NET PLUS JOINT CALL

Marion Haberfellner

Centre for Social Innovation (ZSI), Vienna

2

Overview of Monitoring Activities:

• Monitoring Plan established and external monitoring expert contracted (Jan 07)

• Thematic conferences monitored: Belgrade conference attended, questionnaires analysed and presentation of results at Bucharest Rep-SEE meeting (Feb-Mai 07)

• PJC documents and data studied and preliminary analysis (Jun-Oct 07)

• PJC survey: questionnaire elaborated and finalised in cooperation with SEE-ERA.NET consortium and INTAS (Sep-Oct 07)

• Sofia quality assurance meeting, Thessaloniki Steering Board (Oct 07): discussing PJC and PJC survey

• Interim monitoring report (Nov 07) – including PR monitoring, PJC monitoring and PJC survey analysis

• End monitoring report (August 2009)

3

Remarks on PJC results: country participation

Country No teams in

eligible proposals% of SEE eligible

teamsSEE inhabitants

in Mio% of SEE

inhabitantsRatio: teams/Mio

inhabitants

Albania 61 4,30% 3,60 1,55% 16,94

Austria 59 4,15% 8,20 3,54% 7,20

Bosnia Herzegovina 83 5,85% 4,55 1,96% 18,24

Bulgaria 125 8,80% 7,32 3,16% 17,08

Croatia 142 10,00% 4,49 1,94% 31,63

France 44 3,10% 63,71 27,49% 0,69

FYROM 118 8,31% 2,05 0,88% 57,56

Germany 73 5,14% 82,04 35,40% 0,89

Greece 192 13,52% 10,71 4,62% 17,93

Hungary 62 4,37% 9,96 4,30% 6,22

Montenegro 50 3,52% 0,68 0,29% 73,53

Romania 83 5,85% 22,28 9,61% 3,73

Serbia 227 15,99% 10,15 4,38% 22,36

Slovenia 101 7,11% 2,01 0,87% 50,25

SUM 1420 100,00% 231,75 100,00%  

4

Remarks on PJC results: country participation• In absolute numbers, most teams have participated from Serbia (227), Greece

(192) and Croatia (142)

• Number of teams compared to number of inhabitants per SEE country: Montenegro, Slovenia and FYROM show highest participation to population ratio

• All WBC countries have a high participation rate. Due to:- PJC targeted at these countries- consortium condition: at least one WBC per application- but shows also that ALL WBC took advantage of the PJC and that there is huge interest for developing scientific cooperation in the region and with EU members

• EU member states:- Slovenia, Greece and Bulgaria show highest participation – closer to region, established scientific networks- other EU member states relatively lower participation, especially the bigger countries FR, DE – low budget, consider awareness raising measures for RJC in these countries

5

PJC Survey - Results: Comments on positive aspects

• Established cooperation with colleagues in the region

• Perspective of future projects with higher budgets

• Knowledge and results sharing

6

RECOMMENDATIONS/Funding and Call Budget

Better steering of financial resources of the call: countries should foresee reserves to prop up their share of call funding, if necessary. Such adding of financial resources would allow funding more and the most excellent projects, and prevent exhaustion of funds.

Another alternative to avoid exhaustion of funds would be a “real common pot” scheme, where each country transfers its contribution to the call in a common pot. In this scheme the joint funds would be used irrespective of national origin and be distributed to the most excellent top ranking project proposals. The principle of “juste retour”, where each country receives the funds back that it invests in such a call, could not be guaranteed any more. But administrative procedures and project selection procedures would be much simplified.

Increase funding per project and the call budget overall (feedback of participants)

7

RECOMMENDATIONS/Procedures

Consider simplifications of the electronic submission for project

applications (too much information requested for size of projects,

duplication of requested information on the submission system)

Consider establishing one single multilateral contract per project, to

reduce administrative procedures for scientists.

Adapt the number of different instruments to the call budget; keep

the call simple and avoid offering a too broad range of instruments. Too

many instruments add complexity and increase administrative cost and

procedures.

8

RECOMMENDATIONS/Procedures

Make the rules of the game sufficiently clear and avoid

changing them during call implementation, project selection and

contracting (virtual common pot concept, eligibility and evaluation

criteria) All of this needs to be made sufficiently clear and published with

the call document. Evaluation criteria should be adapted to the funding instrument.

Transparency of evaluation should be increased overall.

9

Remarks on PJC results: „Rules of the game“

• Not all important “rules of the game” for the PJC call were fully

made clear to the applicants.

• Examples:

- which entities are eligible for receiving funding

- project selection: VCP procedure has not been made clear in

call guidelines

- Summer Schools: rule of 50% minimum participation from WBC

had not been outlined in call guidelines

10

Remarks on PJC results: procedural weaknesses in the evaluation

• Some assessment/evaluation procedures of the PJC were not

fully appropriate.

• Examples:

- the same assessment (evaluation) criteria were applied for all

three different instruments (RP, NP, SS)

- different weights were applied for evaluation criteria

- discrepancies between two evaluations: when shall a third

evaluation be requested?

- technical: an evaluator mixed up the evaluation sheets of two

projects for SS

11

RECOMMENDATIONS/PROCEDURES:EVALUATION

Consider establishing a scientific council (SC), composed of well

renowned scientists.

This council would support the project selection process by screening

those remote evaluations for appropriateness, which show high

discrepancies in scores and by checking and adjusting the ranking list of

proposals selected for funding.

Call: in relation to the size of the call in financial terms and in terms of

projects to be funded. (small scale call: few colleagues, might consist of

scientists not originating of countries involved in the consortium so as to

keep this institution neutral)

12

RECOMMENDATIONS/PR and Awareness raising activities:

Promotional conferences for a call: useful tool for consortium

building and networking. Adapt the number and duration of

such conferences to the size of call; (a call with a modest

budget & a limited number of projects to be funded: promotional

measures accordingly be limited and well targeted at countries

with low participation in the pilot call)

Take additional promotional measures for the call in those

countries, where turnout of scientists was rather low, budget was

not exhausted and which participation is essential for network-

building for scientists from WBC (e.g. DE, FR).

13

RECOMMENDATIONS/Success rates:

Better steering of the success rate: consider implementing a

future call in a two step procedure. In a first step only short

outline proposals of few pages, in which the scientific idea and

approach is briefly described, shall be requested. These outline

proposals shall be evaluated and only best ranked project

consortia be invited to submit a full proposal. This procedure

helps steering the success rate, reduces administrative effort on

the side of scientists and avoids frustration.

Another option for steering the success rate would be to have a

more narrow topical focus of the call.

14

RECOMMENDATIONS/Success rates:

In general it seems sensible to make a more profound analysis of

possible topics of the call, by considering for example the

scientific strengths of the WBC. Results of ongoing networking

projects for the WBC region (e.g. INCO-NET WBC) should be

taken into account here.

Support of a call implementation agency (INTAS) proved

essential for the pilot call and will be sensible also for the success

of a larger scale joint call. Synergies with other regional

ERA.NETs funded under FP7 should be considered here.

15

Remarks on PJC results: Success rate

• Success rate is rather low.

Certainly it can be justified with the argument of the Pilot phase (testing for

interest in such an activity); to a lesser extent with the limited effort for preparing a

proposal.

• For the Real Joint Call the expectations will be a bit different. Target a better

success rate, e.g. by calling for still more focussed topics, by limiting the number

of different instruments, etc.

• Team success rate:

- in absolute figures Greece, Serbia, FYROM most successful

- in relation to eligible teams France, Germany and Hungary;

- countries with high number of eligible teams pay to a certain extent price of this

success: teams cannot be funded, exhaustion of funds - VCP

16

Remarks on PJC results: Success rate per instrument

19 7 5 31

215 70 36 321

0%

20%

40%

60%

80%

100%

Research Projects Networks Summer Schools Total

Funded projects Eligible projects

17

Remarks on PJC results: Success rate per country

Country No teams in

funded proposalsTeam success

rate fundedNo teams in

eligible proposalsTeam success

rate eligible

Albania 8 5% 61 13%

Austria 8 5% 59 14%

Bosnia & Herzegovina 8 5% 83 10%

Bulgaria 13 8% 125 10%

Croatia 14 9% 142 10%

France 8 5% 44 18%

Former Yugoslav Republic of Macedonia 15 10% 118 13%

Germany 11 7% 73 15%

Greece 18 12% 192 9%

Hungary 9 6% 62 15%

Montenegro 6 4% 50 12%

Romania 6 4% 83 7%

Serbia 18 12% 227 8%

Slovenia 13 8% 101 13%

Sum 155   1420 11%

18

SEE-ERA.NET contacts

Monitoring teamManfred Spiesberger (external expert)Email1: m.spiesberger@openmedicalinstitute.orgEmail2: manfred_spiesberger@yahoo.de

Centre for Social Innovation:Marion Haberfellner (previously Peter B. Mayr)

CNRS:Jean-Luc TeffoCeline-Delacourt Gollain

National Authority for Scientific Research in RomaniaIulia Mihail, Anca Ghinescu, Alexandra Gurau

19

Lessons learnt for the SEE-

ERA.NET PLUS Joint Call

2020

LESSONS LEARNT FROM THE PJC 1/1

FUNDING Funding increased – for the complete call budget and for each individual

project

Not feasible: Virtual common pot: No national reserves for propping up of funds due to financial crisis

PROCEDURES Only one instrument (research projects) one single multilateral contract per project Rules of the game clear

However: Different national eligibility criteria

2121

LESSONS LEARNT FROM THE PJC 1/2

EVALUATION Scientific Council established

PR/AWARENESS RAISING ACTIVITIES PR increased for countries with low turnout (France, Germany)

SUCCESS RATES Two-step procedure Reduction to two topics Selection of topics: results of consultations in the wbc-inco.net

project

22

SEE-ERA.NET PLUS contact details

Coordinator:

Centre for Social Innovation (ZSI)Marion Haberfellnersee-era.net@zsi.atmhaberfellner@zsi.atLinke Wienzeile 246A-1150 Vienna, Austria

Phone: + 43 1 49 50 442 - 67Mobile: + 43 699 114 802 80Fax: + 43 1 49 50 442 – 40

http://plus.see-era.nethttp://www.see-era.net