evaluation of research and development in estonia …...procedure of evaluation tallinn mai 2017...
TRANSCRIPT
ChristianEnssHeidelbergUniversity
Evaluation of Research andDevelopment in Estonia 2010-1017
Regular Evaluation May 2017
Panel MembersOleJannikBjerrum DenmarkAnneBoddington UKAlineBonami FranceVerityJoyBrown UKAlgimantas Čepas ItalyJenniferClareGreen UK
AndrewE.Clark FranceBarbaraEkbom SwedenChristianEnss GermanyGeoffreyMichaelGadd UKHansGellersen UKJan-GunnarPersson DenmarkMartinHalliwell UKFrands Herschend Sweden
Boye Jensen DenmarkPeterJonker NetherlandsSimoKnuuttila FinnlandRonaldJMaughan UKIanMcConnell UKMarialena Nikolopoulou UKElianne KristinRiska,FinnlandIsabelTorres,UK
TallinnMay2017
Fields of the Evaluation
1. Naturalsciences (10units)
2. Engineeringand technology (4units)
3. Medicaland health sciences (6units)
4. Agricultural and veterinary sciences (4units)
5. Social sciences (5units)
6. Humanities and the arts (9units)
Procedure of Evaluation
Tallinn
Mai2017 visitsatunitswithhearing andlaboratoryvisits39units at6universities,4 R&Denterprisesand11 otherinstitutionsofhighereducation
Reportbasedon:
o self-assessmentsprovidedbyunits
o hearingswithmembersofunits
o visitsoflaboratoriesandinfrastructures
Jõgeva
TartuTõravere
Why bother doing an evaluation of research and development?
Arethingsgoingintherightdirection?
Arethereproblemssomewhere?
Howcanresearchanddevelopmentbemademoreefficient?
Wheredooneneedtomakeadjustments?
Assessment of Output
Assessment of Output
What is the Output of Research and Development
Notpapersandnot impactinJournals!!!!
itsknowledgeandnewproducts
and well-educatedpeople
Assessment Criteria and Indicators
o Scientificimpactofresearch
o Sustainabilityandpotentialofresearch
o Societalimportanceofresearch
Organisationof Researchand DevelopmentAct
Universities Act,
Basedon
Scientificimpactreferstothesizeandimpactofthedirect R&Doutput– publications andindustrialpropertyitems– inaninternationalfield-specificcomparison.
Indicatorsofscientificimpact:Numberofpublicationsandtheirclassification
Impact ofscientificarticles
Number ofindustrialpropertyitems
R&Doutcomes(publications)withhighestimpact(assessmentbytheinstitution);
Significantadditionalinformationintermsofscientificimpact(assessmentbytheinstitution)
Scientific Impact of Research
SustainabilityisassessedthroughtheabilityoftheinstitutiontomaintainR&D atasufficientlevelinthecorrespondingfieldinthemediumterm(sevenyears).
Theprincipalfocusofsustainabilityisonhuman andfinancialresources.Foruniversities,R&Dlinkagestodoctoralstudiesisalsoconsideredunderthiscriterion.
Potentialconsiderstheinternal andexternalresourcesandopportunitiestotheinstitutionthathaveyettobedevelopedorutilized.
Indicatorsofsustainabilityandpotential:Thecomposition oftheR&Dstaff;
Thenumber ofdoctoralstudents andgraduates;
TheamountandstructureofR&Drevenue;
Theadequacy andstateofinfrastructure;
R&D-relatedcollections;
Sustainability and Potential of Research
SocietalimportanceofresearchreferstotherelationshipbetweentheR&Dandcurrentandpotential(research,environmental,economic,societal/cultural,etc.)concerns.
Coherencewithsociety maybebestachievedbycarryingoutappliedresearchthatiscurrentlyimportantatthenationallevelor,inthelong-termperspective,potentialR&Dwhichcouldhaveanimpactontheenvironment,theeconomy,and/orcommunityandculture,even atthegloballevel.
Indicatorsofsocietalimportanceofresearch:Societalimportanceofresearchmaybedescribedbytheindicatorsofcriteria5.1and5.2,incasetheyareassociatedwithaspecificaspectofsocietalimpact(e.g.howdoctoralgraduateshavehadanimpactonthesociety(basedonfacts,notassumptions)).
R&Drevenuesfromservicecontractswithenterprisesorthegovernment;counsellingactivitiesforthestateorenterprisesoncurrentsocietalissues;transferofR&Doutcomesandpopularizationtosociety;publiclyavailabledatabases,publications,products/servicesasoutcomesofR&Dactivitiesorotherfactsthatindicatesocietalimportanceofresearch.
Societal Importance of Research
Some Observations Part 1o thequalityofresearchanddevelopmentinEstoniaisgenerallyhighandmany
oftheunitsevaluatedforthisreportarestrongcompetitorsonaninternationallevel
o thenumber ofverysuccessful researchgroups iscertainlyimpressive withafewbeingoutstanding
o atsomeinstitutions,therearegroups andunits ofsubcriticalsize
o overallamountoffunding formostunitsseemssatisfactorybutappearstoofractured
o theabsenceofstablecorefundingimpactsmostunitsnegatively becauseofdifficultiestomaintaininfrastructuresandtoallowthemtokeepthenecessarylevelofqualifiedstaff
o theproportionofcompetitivefundingisgenerallytoolarge
o someinstitutionsdidundergomajorrestructuring
o Interdisciplinarycentersareformedattheexpenseofdisciplines
o researchspectrumisoverallsomewhatnarrow
o thereisatrendtohavemoretenure-trackpositions,buttheirseemsnogeneralplan tomakethistransition
o infrastructure oftheevaluatedunitsisgenerallyquitegoodandinmanycasesofhighinternationalstandard
o infrastructure hashugelybenefitedfromEuropeanstructuralfunds,whichwillrunoutsoon
o theequipment offewgroups seemstobefaroutdated andclearlyneedsupgrading
Some Observations Part 2
o PhDstudentsarewelltrainedand,inmostcases,appeartodoexcellentresearch
o financialsupportofstudents isquitedifferent in differentunits
o thedemand forthreepublicationstobegraduatedappearsarbitrary
o publicoutreachactivitiesexistasomeuniversitiesbutshouldbeincreased
o theconceptofR&Denterprises (institutes) as pipelinebetweenbasicresearchandthecommercialmarketisworkingwell
o thestrongbias forapplied researchisdangerous inlongterms
o onlyasmallfraction ofacademicpositionsisfilledbytop-levelforeign scientists
Some Observations Part 3
Natural Science The team
GeoffreyGaddBiology
HansGellersenComputerScience
AlineBonamiMathematics
ChristianEnssPhysics
SoftwareTechnologyandApplicationsComp.Center
TartuObservatory
Universityof Tartu
TallinnTechnicalUniversity
Estonian Biocentre
TallinnUniversity
Estonian Universityof LifeSciences
Cybernetica AS
CenterofFoodTechnology
NationalInstituteofChemicalPhysicsandBiophysics
Units ScientificImpact
SustainabilityPotential
SocietalImportance
verygood good satisfactory
Natural Science General Comments
o focusoncomputerscience,lifescience,environmentalscienceandchemistry
o strongbiasforappliedresearch
o mathematicandphysicsisarathersmallportion
o Interdisciplinaryiscurrentlyahottopic
o Infrastructure ismostlyverygood,duetoEuropeanstructuralfunds
o PhDstudentsmostly seemtoenjoy whattheyaredoingandarewellsupported
o onlyasmallfraction ofacademicpositionsisfilledbytop-levelforeign scientists
o provide longtermfunding forlargescaleinfrastructures
o reduce theimpactofmetrics infundingdecisions
o establishaprogramforinfrastructurefunding
o keepfundingbasicresearchwithout requirementsofapplicationandindustrialpartners
o extentvisitorprogram tostrengthenlongtermrelations
o givehigherpriorityforstartingfaculties inprojectfundingandinfrastructurecalls
o include panel membersinthe preparation ofanevaluation
o Establishasteeringboardforevaluations:researchers,industryrepresentatives,
externalexperts,councilmembers,ministry
o computerplatformneedstobefixed
Recommendations to the Estonian Research Council
Recommendations to The Ministry of Education and Research
o demandlong-termstrategicplansfromallinstitutionsincludingresearchprofiles,appointmentplans,infrastructureneeds,organizationalandeducationalaspects
o providebroadsupportforbasic researchandeducation
o increasecorefundingandkeeplevelofcompetitivefunding
o free researchersfromunnecessarilydetailedregulationsandqualityassessments
o supporthighlevelinternationalappointmentsinaspecialprogram
o providesustainedlong-termsupportforlarge-scaleinfrastructures
o establish aprogramforinfrastructuretosubstitute theEuropeanstructuralfunds
o considersteppingupEstonianshareinimportantinternationalinfrastructures
Thank you
EstonianResearchCouncil:DorisPavlov,Eva-LiisaOtsus,Kadri Raudvere, Maarja Sillaste,Karmen Kert,Priit Tamm,Andres Koppel
Fellowpanelmembers:OleJannikBjerrum,AlineBonami,VerityJoyBrown,Algimantas Čepas,JenniferClareGreen,AndrewE.Clark,BarbaraEkbom,GeoffreyMichaelGadd,Frands Herschend,HansGellersen,Frands Herschend MartinHalliwell,Boye Lagerbon Jensen,PeterJonkers,SimoKnuuttila,RonaldJMaughan,IanMcConnell,Marialena Nikolopoulou,Jan-GunnarA.Persson,Elianne KristinRiska,IsabelTorres
