evaluation of research and development in estonia …...procedure of evaluation tallinn mai 2017...

ChristianEnssHeidelbergUniversity

Evaluation of Research andDevelopment in Estonia 2010-1017

Regular Evaluation May 2017

Panel MembersOleJannikBjerrum DenmarkAnneBoddington UKAlineBonami FranceVerityJoyBrown UKAlgimantas Čepas ItalyJenniferClareGreen UK

AndrewE.Clark FranceBarbaraEkbom SwedenChristianEnss GermanyGeoffreyMichaelGadd UKHansGellersen UKJan-GunnarPersson DenmarkMartinHalliwell UKFrands Herschend Sweden

Boye Jensen DenmarkPeterJonker NetherlandsSimoKnuuttila FinnlandRonaldJMaughan UKIanMcConnell UKMarialena Nikolopoulou UKElianne KristinRiska,FinnlandIsabelTorres,UK

TallinnMay2017

Fields of the Evaluation

1. Naturalsciences (10units)

2. Engineeringand technology (4units)

3. Medicaland health sciences (6units)

4. Agricultural and veterinary sciences (4units)

5. Social sciences (5units)

6. Humanities and the arts (9units)

Procedure of Evaluation

Tallinn

Mai2017 visitsatunitswithhearing andlaboratoryvisits39units at6universities,4 R&Denterprisesand11 otherinstitutionsofhighereducation

Reportbasedon:

o self-assessmentsprovidedbyunits

o hearingswithmembersofunits

o visitsoflaboratoriesandinfrastructures

Jõgeva

TartuTõravere

Why bother doing an evaluation of research and development?

Arethingsgoingintherightdirection?

Arethereproblemssomewhere?

Howcanresearchanddevelopmentbemademoreefficient?

Wheredooneneedtomakeadjustments?

Assessment of Output

Assessment of Output

What is the Output of Research and Development

Notpapersandnot impactinJournals!!!!

itsknowledgeandnewproducts

and well-educatedpeople

Assessment Criteria and Indicators

o Scientificimpactofresearch

o Sustainabilityandpotentialofresearch

o Societalimportanceofresearch

Organisationof Researchand DevelopmentAct

Universities Act,

Basedon

Scientificimpactreferstothesizeandimpactofthedirect R&Doutput– publications andindustrialpropertyitems– inaninternationalfield-specificcomparison.

Indicatorsofscientificimpact:Numberofpublicationsandtheirclassification

Impact ofscientificarticles

Number ofindustrialpropertyitems

R&Doutcomes(publications)withhighestimpact(assessmentbytheinstitution);

Significantadditionalinformationintermsofscientificimpact(assessmentbytheinstitution)

Scientific Impact of Research

SustainabilityisassessedthroughtheabilityoftheinstitutiontomaintainR&D atasufficientlevelinthecorrespondingfieldinthemediumterm(sevenyears).

Theprincipalfocusofsustainabilityisonhuman andfinancialresources.Foruniversities,R&Dlinkagestodoctoralstudiesisalsoconsideredunderthiscriterion.

Potentialconsiderstheinternal andexternalresourcesandopportunitiestotheinstitutionthathaveyettobedevelopedorutilized.

Indicatorsofsustainabilityandpotential:Thecomposition oftheR&Dstaff;

Thenumber ofdoctoralstudents andgraduates;

TheamountandstructureofR&Drevenue;

Theadequacy andstateofinfrastructure;

R&D-relatedcollections;

Sustainability and Potential of Research

SocietalimportanceofresearchreferstotherelationshipbetweentheR&Dandcurrentandpotential(research,environmental,economic,societal/cultural,etc.)concerns.

Coherencewithsociety maybebestachievedbycarryingoutappliedresearchthatiscurrentlyimportantatthenationallevelor,inthelong-termperspective,potentialR&Dwhichcouldhaveanimpactontheenvironment,theeconomy,and/orcommunityandculture,even atthegloballevel.

Indicatorsofsocietalimportanceofresearch:Societalimportanceofresearchmaybedescribedbytheindicatorsofcriteria5.1and5.2,incasetheyareassociatedwithaspecificaspectofsocietalimpact(e.g.howdoctoralgraduateshavehadanimpactonthesociety(basedonfacts,notassumptions)).

R&Drevenuesfromservicecontractswithenterprisesorthegovernment;counsellingactivitiesforthestateorenterprisesoncurrentsocietalissues;transferofR&Doutcomesandpopularizationtosociety;publiclyavailabledatabases,publications,products/servicesasoutcomesofR&Dactivitiesorotherfactsthatindicatesocietalimportanceofresearch.

Societal Importance of Research

Some Observations Part 1o thequalityofresearchanddevelopmentinEstoniaisgenerallyhighandmany

oftheunitsevaluatedforthisreportarestrongcompetitorsonaninternationallevel

o thenumber ofverysuccessful researchgroups iscertainlyimpressive withafewbeingoutstanding

o atsomeinstitutions,therearegroups andunits ofsubcriticalsize

o overallamountoffunding formostunitsseemssatisfactorybutappearstoofractured

o theabsenceofstablecorefundingimpactsmostunitsnegatively becauseofdifficultiestomaintaininfrastructuresandtoallowthemtokeepthenecessarylevelofqualifiedstaff

o theproportionofcompetitivefundingisgenerallytoolarge

o someinstitutionsdidundergomajorrestructuring

o Interdisciplinarycentersareformedattheexpenseofdisciplines

o researchspectrumisoverallsomewhatnarrow

o thereisatrendtohavemoretenure-trackpositions,buttheirseemsnogeneralplan tomakethistransition

o infrastructure oftheevaluatedunitsisgenerallyquitegoodandinmanycasesofhighinternationalstandard

o infrastructure hashugelybenefitedfromEuropeanstructuralfunds,whichwillrunoutsoon

o theequipment offewgroups seemstobefaroutdated andclearlyneedsupgrading

Some Observations Part 2

o PhDstudentsarewelltrainedand,inmostcases,appeartodoexcellentresearch

o financialsupportofstudents isquitedifferent in differentunits

o thedemand forthreepublicationstobegraduatedappearsarbitrary

o publicoutreachactivitiesexistasomeuniversitiesbutshouldbeincreased

o theconceptofR&Denterprises (institutes) as pipelinebetweenbasicresearchandthecommercialmarketisworkingwell

o thestrongbias forapplied researchisdangerous inlongterms

o onlyasmallfraction ofacademicpositionsisfilledbytop-levelforeign scientists

Some Observations Part 3

Natural Science The team

GeoffreyGaddBiology

HansGellersenComputerScience

AlineBonamiMathematics

ChristianEnssPhysics

SoftwareTechnologyandApplicationsComp.Center

TartuObservatory

Universityof Tartu

TallinnTechnicalUniversity

Estonian Biocentre

TallinnUniversity

Estonian Universityof LifeSciences

Cybernetica AS

CenterofFoodTechnology

NationalInstituteofChemicalPhysicsandBiophysics

Units ScientificImpact

SustainabilityPotential

SocietalImportance

verygood good satisfactory

Natural Science General Comments

o focusoncomputerscience,lifescience,environmentalscienceandchemistry

o strongbiasforappliedresearch

o mathematicandphysicsisarathersmallportion

o Interdisciplinaryiscurrentlyahottopic

o Infrastructure ismostlyverygood,duetoEuropeanstructuralfunds

o PhDstudentsmostly seemtoenjoy whattheyaredoingandarewellsupported

o onlyasmallfraction ofacademicpositionsisfilledbytop-levelforeign scientists

o provide longtermfunding forlargescaleinfrastructures

o reduce theimpactofmetrics infundingdecisions

o establishaprogramforinfrastructurefunding

o keepfundingbasicresearchwithout requirementsofapplicationandindustrialpartners

o extentvisitorprogram tostrengthenlongtermrelations

o givehigherpriorityforstartingfaculties inprojectfundingandinfrastructurecalls

o include panel membersinthe preparation ofanevaluation

o Establishasteeringboardforevaluations:researchers,industryrepresentatives,

externalexperts,councilmembers,ministry

o computerplatformneedstobefixed

Recommendations to the Estonian Research Council

Recommendations to The Ministry of Education and Research

o demandlong-termstrategicplansfromallinstitutionsincludingresearchprofiles,appointmentplans,infrastructureneeds,organizationalandeducationalaspects

o providebroadsupportforbasic researchandeducation

o increasecorefundingandkeeplevelofcompetitivefunding

o free researchersfromunnecessarilydetailedregulationsandqualityassessments

o supporthighlevelinternationalappointmentsinaspecialprogram

o providesustainedlong-termsupportforlarge-scaleinfrastructures

o establish aprogramforinfrastructuretosubstitute theEuropeanstructuralfunds

o considersteppingupEstonianshareinimportantinternationalinfrastructures

Thank you

EstonianResearchCouncil:DorisPavlov,Eva-LiisaOtsus,Kadri Raudvere, Maarja Sillaste,Karmen Kert,Priit Tamm,Andres Koppel

Fellowpanelmembers:OleJannikBjerrum,AlineBonami,VerityJoyBrown,Algimantas Čepas,JenniferClareGreen,AndrewE.Clark,BarbaraEkbom,GeoffreyMichaelGadd,Frands Herschend,HansGellersen,Frands Herschend MartinHalliwell,Boye Lagerbon Jensen,PeterJonkers,SimoKnuuttila,RonaldJMaughan,IanMcConnell,Marialena Nikolopoulou,Jan-GunnarA.Persson,Elianne KristinRiska,IsabelTorres