monday, july 23rddevelopment, the impact of research, collaboration, and thematic specialization....

14
Monday, July 23rd Bibliometrics Reviewed: History, Institutionalization, and Concepts Stefan Hornbostel, German Centre for Higher Education Research and Science Studies (DZHW), Germany The emergence of bibliometrics is closely linked to the growth of scientific information in the 20th century and to what de Solla Price called the evolution from “little science to big science”. Initially, bibliometrics and its early concepts were oriented towards library access, bibliographic databases, and information services. However, since the 1960s other disciplines, especially the sociology of science, inspired the development of a new and interdisciplinary understanding of bibliometrics. In the 1970s and 1980s the increasing information needs on behalf of science policymakers boosted the institutionalization of bibliometrics as an own field of research, while at the same time this new application context necessitated new concepts. Little by little, a specific bibliometric methodology aiming to be suitable for today’s applications such as formula- based funding systems, assessments, evaluations etc. came into being. The lecture will present this development process and, thereby, demonstrate common concepts of bibliometrics. Introduction to Bibliometric Data Sources (Part 1 and Part 2) Wolfgang Glänzel, Centre for R&D Monitoring (ECOOM) & Dept. MSI, KU Leuven, Belgium Juan Gorraiz, Bibliometrics and Publication Strategies, University of Vienna, Austria This talk is about the specific requirements for bibliographical data sources to be met in regard to their suitability for bibliometric application. Furthermore relevant issues like coverage, representativeness and selection criteria are considered. Any appropriate bibliography can potentially serve as data source for bibliometric purposes, however, comparative studies and large-scale analyses require large standardized data sources like bibliographic databases. After providing some background information, the main features of bibliographic databases are discussed with special focus on the question, which of them are useful, essential or even indispensable for bibliometric use (most databases are rather designed for information retrieval). In particular, the pros and cons of the three major multidisciplinary data sources – Web of Science, SCOPUS and Google Scholar – are discussed. Focus points of the second part are: a) to show that the selection of the data sources will determine the outcome of your bibliometric analysis; b) how to check and enrich your data in order to extend the data pool for bibliometric analysis; and c) to consider that data sources are continuously changing their structure (number of indexed sources).

Upload: others

Post on 10-Oct-2020

3 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Monday, July 23rddevelopment, the impact of research, collaboration, and thematic specialization. Options and limitations of the indicators introduced will be discussed. Subject Classification

Monday,July23rdBibliometricsReviewed:History,Institutionalization,andConceptsStefanHornbostel,GermanCentreforHigherEducationResearchandScienceStudies(DZHW),GermanyTheemergenceofbibliometricsiscloselylinkedtothegrowthofscientificinformationinthe20thcenturyandtowhatdeSollaPricecalledtheevolutionfrom“littlesciencetobigscience”.Initially,bibliometricsanditsearlyconceptswereorientedtowardslibraryaccess,bibliographicdatabases,andinformationservices.However,sincethe1960sotherdisciplines,especiallythesociologyofscience,inspiredthedevelopmentofanewandinterdisciplinaryunderstandingofbibliometrics.Inthe1970sand1980stheincreasinginformationneedsonbehalfofsciencepolicymakersboostedtheinstitutionalizationofbibliometricsasanownfieldofresearch,whileatthesametimethisnewapplicationcontextnecessitatednewconcepts.Littlebylittle,aspecificbibliometricmethodologyaimingtobesuitablefortoday’sapplicationssuchasformula-basedfundingsystems,assessments,evaluationsetc.cameintobeing.Thelecturewillpresentthisdevelopmentprocessand,thereby,demonstratecommonconceptsofbibliometrics.IntroductiontoBibliometricDataSources(Part1andPart2)WolfgangGlänzel,CentreforR&DMonitoring(ECOOM)&Dept.MSI,KULeuven,BelgiumJuanGorraiz,BibliometricsandPublicationStrategies,UniversityofVienna,AustriaThistalkisaboutthespecificrequirementsforbibliographicaldatasourcestobemetinregardtotheirsuitabilityforbibliometricapplication.Furthermorerelevantissueslikecoverage,representativenessandselectioncriteriaareconsidered.Anyappropriatebibliographycanpotentiallyserveasdatasourceforbibliometricpurposes,however,comparativestudiesandlarge-scaleanalysesrequirelargestandardizeddatasourceslikebibliographicdatabases.Afterprovidingsomebackgroundinformation,themainfeaturesofbibliographicdatabasesarediscussedwithspecialfocusonthequestion,whichofthemareuseful,essentialorevenindispensableforbibliometricuse(mostdatabasesareratherdesignedforinformationretrieval).Inparticular,theprosandconsofthethreemajormultidisciplinarydatasources–WebofScience,SCOPUSandGoogleScholar–arediscussed.Focuspointsofthesecondpartare:a)toshowthattheselectionofthedatasourceswilldeterminetheoutcomeofyourbibliometricanalysis;b)howtocheckandenrichyourdatainordertoextendthedatapoolforbibliometricanalysis;andc)toconsiderthatdatasourcesarecontinuouslychangingtheirstructure(numberofindexedsources).

Page 2: Monday, July 23rddevelopment, the impact of research, collaboration, and thematic specialization. Options and limitations of the indicators introduced will be discussed. Subject Classification

Somebasicdatabasefeaturesarealsointroducedexemplarilyfromdifferentproducts.Adistinctionismadebetweensubject-specificandmultidisciplinarydatabases.Inaddition,subject-specificdatabases(e.g.“MathSciNet”,“SciFinder”,ADS),patentdatabases(e.g.“DerwentInnovationsIndex”,Espacenet(PATSTAT))orpilotprojectsforcitationindexingontheweb(e.g.“BASE”,“CiteseerX”–allbasedonopenaccessarchives)arepresentedandexaminedcriticallyregardingtheirdataenrichmentpotentialinbibliometricanalyses.Somefinalrecommendationsondatavalidation,ontheimportanceofuniqueidentifiersandontheuseof„subjective“datasourceslikeCRISwillclosethistalk.ScientometricIndicatorsinUse:anOverviewSybilleHinze,GermanCentreforHigherEducationResearchandScienceStudies(DZHW),GermanyWolfgangGlänzel,CentreforR&DMonitoring(ECOOM)&Dept.MSI,KULeuven,BelgiumTheuseofscientometricindicatorsdatesbacktothe1960sand1970sintheUnitedStateswherethefirstScienceIndicatorsreportwaspublishedin1973.Sincethenavarietyofindicatorsemergedaimingatreflectingvariousaspectsofscienceandtechnologyandtheirdevelopment.Thepresentationwillgiveanoverviewofrelevantindicatorsandtheiruseinsciencepolicyandrelatedrealms.Thespecificfocuswillbeonindicatorsusedinthecontextofresearchevaluation.Inparticularindicatorsappliedtomeasuringresearchperformanceatthevariouslevelsofaggregationi.e.themacro,mesoandmicrolevelwillbeintroduced.Arangeofaspectsreflectingresearchperformancewillbeaddressedsuchasresearchproductivityanditsdynamicdevelopment,theimpactofresearch,collaboration,andthematicspecialization.Optionsandlimitationsoftheindicatorsintroducedwillbediscussed.SubjectClassificationinBibliometrics–Theory,Approaches&LimitationsStephanGauch,GermanCentreforHigherEducationResearchandScienceStudies(DZHW)&HumboldtUniversitätzuBerlin,GermanyClassificationsshapethewaysweperceiveboththeobjectsclassifiedaswellasdifferencesbetweenobjects.Naturally,theyshapebibliometricanalyses,especiallysowhenreplicationandconsistencyarekeynecessities.Theyalsoare,tonosmallextent,aproductofdiscourseandthereforeaspecialformofconventionthatcanbothbeenablingaswellaslimiting.Inthecourseofthissessionwewillapproachclassificationsfromatheoreticalaswellaspracticalperspective.Amongthequestionsaddressedare“Whatdoesitmeantoclassify?”,“Howcanwedeterminewhatmakesa*good*classification?”,“Whattodoifanobjectcanbeclassifiedtomultipleclasses?”etc.Thesessionaimsatprovidingadeeperandmoreinformedinsighttoparticipantsaboutwhatitreallymeanstouseclassifications,encouragingbothacriticalmindsetaswellasprovidingpracticaladvice.BibliometricMethodsinSubjectDelineationWolfgangGlänzel,CentreforR&DMonitoring(ECOOM)&Dept.MSI,KULeuven,Belgium

Page 3: Monday, July 23rddevelopment, the impact of research, collaboration, and thematic specialization. Options and limitations of the indicators introduced will be discussed. Subject Classification

Subjectdelimitationhasbecomeacentralissueinso-called"domainstudies".Sciencepolicyaddressesnewemergingorcomplexinterdisciplinarytopicsthedelineationofwhichisparticularlydifficult.Thedelineationofthesetopicsordomainsis,ononehand,stronglyrelatedwithinformationretrievalsinceoftenrathertraditional"searchstrategies"usingcorejournals,keywordsandphrasescanbeappliedbut,ontheotherhand,goalsandmethodsofadvancedsubjectdelineationessentiallydifferfromthoseofusualretrieval.Propersubjectdelineationisalsonecessarytofindcorrectreferencestandardsforbenchmarkingtheresearchperformanceoftheactorsinthetopicunderstudy.Thefirstpartofthelecturewillfocustraditionaltechniquesthatcaneasilybedevelopedforandusedintheonlineversionsofbibliographicdatabases.Thesecondpartwillintroduce"bibliometrics-aided"retrieval.Oneofthemainmethodologicalcharacteristicsofbibliometrics-aidedretrievalisthatbibliometricsallowsincluding'metric'componentsinsearchstrategies.Inthecourseofthelectureitwillbeshownhowlexicalandcitation-basedcomponentscanbeusedtograduallyextendtheoriginalcore(orseed)ofsurelyrelevantdocumentspreviouslyobtainedfromtraditionalliteraturesearches.TheWebofScienceofferstheoptionofrelatedrecords(basedonbibliographiccoupling)whileScopususeskeywords.Resultscanbefilteredbytheirrelevanceandadditionalrelateddocumentscanbeaddedtothecoresetusingthresholds.Theapplicationofdirectcitationlinksormoreadvancedtextualsimilaritiesisagainreservedforarathersmallgroupofuserswithaccesstocustomdata.Inthiscase,too,thresholdscanbesettofilternoiseandtocontrolprecisionandgranularity.TheEvolutionofCitationDatabaseswithFocusonNewDevelopmentsJeffClovis,ClarivateAnalyticsIn1955,EugeneGarfieldpublishedapaperinScienceentitled"CitationIndexesforScience:ANewDimensioninDocumentationthroughAssociationofIdeas."Today,Dr.Garfieldstandsasapioneerinthefieldofbibliometrics,andhislegacyofamultidisciplinary,citationdatabasehasgrownovertheyearstoencompassnewtechnologiesnotenvisioned58yearsago.Thesenewtechnologieshavemovedcitationdatafromprinttoelectronicformat,andultimatelyintoaWeb-basedenvironmentofhypernavigation,optimisticandcontext-sensitivelinking,andbeyond.ThispaperwillreviewthewayinwhichthecitationindexeshaveevolvedtobecomepartoftheWebofScienceplatform-anintegratedplatformofnetworkedresourcesutilizedby9,400researchlibraries,governmentagenciesandcorporationsworldwide.StartingwiththeimportanceofDr.Garfield'seffortsinbibliometricsthatultimatelyledtothedevelopmentoftheISIplatform,thispaperwillthenfocusonthreecriticalareas:howtheplatformmeetsthegrowingneedforaccesstomultidisciplinarycontentintoday'sresearchenvironment;thedevelopmentofourfoundation,subjectspecialtyandregionalindexesthatallowcross-contentsearching;andnewtechnologies-theauthentication,access,androutingmanagementsystemsintegratedintoWebofScience.

Page 4: Monday, July 23rddevelopment, the impact of research, collaboration, and thematic specialization. Options and limitations of the indicators introduced will be discussed. Subject Classification

Tuesday,July24thDesigningEffectiveQueriesforDocumentRetrievalStephanGauch,GermanCentreforHigherEducationResearchandScienceStudies(DZHW)&HumboldtUniversitätzuBerlin,GermanyThequalityofbibliometricapproaches,bothexplorativeaswellasevaluative,isstronglyinfluencedbythewaysearchqueriestobibliometricdatabasesareconstructed.Thisbecomesapparentwhenbeginningscholarsandpractitionersofbibliometricsareshockedwhentheylearnthatthescientificfieldortopictheythoughtcouldbecoveredbyasimplesearchtermisfarbettercoveredbypagesandpagesofcarefullyselectedandintricatecombinationsofsearchterms,journalsetsandclassifications.Inthissessionwewillexploregoodpracticeexamplestodesign“effectivequeries”.Participantswillbeshownhowtogetthemostfromexpertknowledge,howtoiterativelyoptimizequeries,howtocarefullyusetruncatingtechniquesoftermstocovermoregroundandhowtoavoidpitfallssuchasover-optimizationorqueriesthatare“toofuzzyaroundtheedges”.DataCleaningandProcessingChristineRimmert,InstituteforInterdisciplinaryStudiesofScience-AGBibliometrie,BielefeldUniversity,GermanyThequalityofbibliometricanalysesisheavilydependingonappropriatedhandlingoftherelevantrawdatafields.Dependingonthelevelofaggregationandthetargetobjectsunderstudy,variousissuesofaccuracycancomeupwithcitationlinksandseveraldataelements(documenttype,author,institution,country,journal,fieldanddiscipline).WewillhaveacloselookattherelevantdatafieldsinmoderncitationdatabaseslikeWebofScienceorScopustoseeiftheyare“readytouse”fordoingallkindsofbibliometricstudies.Mainproblemsofdataqualitywillbeshownandmajortypesoferrorsandtheirconsequenceswillbediscussed.Standardisation,verificationandtheintroductionofidentifierscanhelptoovercomeproblemsofdataquality.DataprocessingapproachesoftheGermancompetencecentreforbibliometricswillbedemonstrated.JournalImpactMeasures(Part1andPart2)WolfgangGlänzel,CentreforR&DMonitoring(ECOOM)&Dept.MSI,KULeuven,BelgiumJuanGorraiz,BibliometricsandPublicationStrategies,UniversityofVienna,AustriaTheseminaronimpactmeasureswillfirstshedlightonthebestknownandmostcontroversialindicator,namelyGarfield’sJournalImpactFactor.Itsstrengthsandweaknessesaswellasitscorrectusewillbediscussedthoroughly.Moreoverthecorrespondinganalyticaltool,ThomsonReuter’sJournalCitationReportswillbedemonstrated.AlternativeimpactmeasureslikeEigenfactormetrics,SJRandSNIPhavebeenintroducedwithinthelastyearsandwillbepresentedinthesecondpartofthistalktocompletethepicture.

Page 5: Monday, July 23rddevelopment, the impact of research, collaboration, and thematic specialization. Options and limitations of the indicators introduced will be discussed. Subject Classification

SubjectNormalizationforCitationAnalysisWolfgangGlänzel,CentreforR&DMonitoring(ECOOM)&Dept.MSI,KULeuven,BelgiumSubjectnormalisationforcitationanalysisisafundamentalrequirementforcitationanalysisinamultidisciplinaryenvironment.Recentlytwofundamentalapproachesexist,theso-calledsource-andcitingsidenormalisation,or,usinganotherterminology,theaprioriandaposteriorinormalisation.Bothmethodswillbeintroducedanddescribed.Althoughtheapriorinormalisationrepresentsamoreadvancedmethodology,itsapplicationisreservedforarathersmallgroupofusers.Thereasonistheaccesstoandtheprocessingofthecompletedatabase(WebofScienceorSCOPUS)sinceinthisapproachcitationshavetobenormalisedbeforetheyarecounted.Knowledgeaboutthisnormalisationtechniqueis,however,importantbecausethisfuture-orientedmethodologyisalreadyappliedbylargerbibliometriccentres.Thesecondmethodisratherconservative,butcanbeappliedbyanyuserwhohasaccesstotheonlineversionoftheWebofScienceorSCOPUS.Themaincharacteristicofaposteriorinormalisationisthatcitationcountsarenormalisedaftercountingonthebasisofproperreferencevalues.Advantagesanddisadvantagesofbothmethodsarediscussedandexamplesforthesecondapproacharecalculated.ProductPresentation:InCitesJeffClovis,ClarivateAnalyticsInCitesisacitation-basedevaluationtoolforacademicandgovernmentadministratorstoanalyzeinstitutionalproductivityandbenchmarkoutputagainstpeersandaspirationalpeersinanationalorinternationalcontext.Withcustomizedcitationdata,globalmetrics,andmultidimensionalprofilesontheleadingresearchinstitutions,over9,000sitesglobally,you’llgetcomprehensiveinsightintoyourperformance.BuiltonthefoundationofWebofScience,InCitesusesthemostthorough,accurate,andobjectivedataavailable.InthisshorttutorialwewillfocusonaquicktourhighlightingEuropeaninstitutions.AllattendeeswillreceivecomplimentaryaccessfortheESSSsessions.

Page 6: Monday, July 23rddevelopment, the impact of research, collaboration, and thematic specialization. Options and limitations of the indicators introduced will be discussed. Subject Classification

Wednesday,July25thBestPracticesinBibliometricsMeasuringScience:EvaluationandMappingofScientificResearchTonvanRaan,CentreforScienceandTechnologyStudies(CWTS),LeidenUniversity,TheNetherlandsWepresentanoverviewofthelatestdevelopmentsin‘measuringscience’basedonbibliometricmethods.Ourcentraltopicistheroleofcitation-andconcept-networksandtheircombinationasanaturalbasisforboththeconstructionofperformanceindicatorsaswellastheconstructionofsciencemaps.Wepresentreal-lifeexamplesofpracticalapplicationsofadvancedbibliometricmethodsintheevaluationandmappingofuniversities,departmentsandinstitutes.Theseapplicationsalsoofferindividualscientistsinstrumentstoexploretheirownresearchfield.Weexplainhowcluster-basednormalizationisusedtotackletheproblemofthelargedifferencesincitationdensitywithinfields.ThestrategicpotentialofsciencemappingbasedonnewbibliometricinstrumentssuchastheVoS-viewerandCitNetExplorerisshownbyrecentworkonSleepingBeauties.FinallywediscussthenewversionoftheLeidenRankingincomparisonwithotherprominentuniversityrankings.BibliometricServicesattheUniversityofViennaJuanGorraiz,BibliometricsandPublicationStrategies,UniversityofVienna,AustriaBibliometricsisidealforlibrarianstodevelopandprovideinnovativeservicesforbothacademicandadministrativeuniversitystaff.TheBibliometricsandPublicationStrategiesDepartmentinViennahasbeenimplementedwithintheLibraryandArchiveServicesoftheUniversityofVienna.Itcanserveasarolemodelforotheracademiclibrarianswhowishtobecomemoreengagedinthisfieldorevenplantoimplementaccordingservices.Thispresentationgivesanoverviewofallbibliometricservicesofferedbythedepartmentandwillthenfocusonthoserelatedtoindividualevaluationandparticularlytoprofessorialappointments.TheViennaUniversitybibliometricapproachreliesonavarietyofbasic,simpleindicatorsandfurthercontrolparametersinordertoaddressthemulti-dimensionalityoftheproblemandtofostercomprehensibility.Our“topcountsapproach”allowsanappointmentcommitteetopickandchoosefromaportfolioofindicatorsaccordingtotheactualstrategicalignment.Furthermore,controlandadditionaldatahelptounderstanddisciplinarypublicationhabits,tounveilconcealedaspectsandtoidentifyindividualpublicationstrategiesofthecandidatesorindividualresearcherstobeevaluated.Bibliometricsonlyshinesalightonquantitativeaspectsandshouldneverbeappliedirrespectiveofthegivenqualitativecontext.PolicyUseofBibliometricEvaluationanditsRepercussionsontheScientificCommunitywithFocusonResearch,Technology,Patents,DevelopmentandKnowledgeTransferKoenraadDebackere,KULeuven,Belgium

Page 7: Monday, July 23rddevelopment, the impact of research, collaboration, and thematic specialization. Options and limitations of the indicators introduced will be discussed. Subject Classification

Modernsciencepolicyfirmlyreliesonbibliometricdata&indicatorstoassessthescientificperformanceofresearchinstitutions,researchgroupsandevenindividualresearchers.Inaddition,benchmarkingthescientificperformanceofcountriesandregionsisanotheritemontheagendaofevaluativesciencepolicy.Duringthepresentation,therepercussionsofthispolicyuseofbibliometricevaluationwillbedealtwithalongthreelinesofthoughtandreflection.First,recenttrendsandinsightsintodataandindicatoruseforevaluativesciencepolicywillbehighlighted.Second,anoverviewofcurrentpolicyframeworkswillbepresented,takingintoaccounttherecenttrendtolinkscientificperformancetoso-calledsmartspecializationpolicies.Third,wewillreflectuponthemultifacetedimpactthosetrendshave(ormayhave)onthescientificcommunityand(inthelimit)thebehaviorofindividualscientists.ResearchImpactAssessmentAcrossDomains-CombiningStandardsandBibliometricsDataStephanGauch,GermanCentreforHigherEducationResearchandScienceStudies(DZHW)&HumboldtUniversitätzuBerlin,GermanyThereexistsalong-standingtraditionoflinkingbibliometricdatatoothertypesofsources,e.g.patentdocumentsto“linksciencetoinnovation”.Withtheadventofalternativemetricstherehasbeenasurgeofsuchexpansiveactivities.Insomecases,uniqueidentifiersandAPIsareinplacetomakethelifeofaresearcher(relatively)easy.Everythingisneatandclean.Atleastonthesurface–butthat’sadifferentissue.Inothercases,allthereisisugly,dirtyandmessydata–andsometimesnoteventhat.Itisthesecases,aresearcherneedstouseawiderarrayoftools.Butwait…there’smore.Thetasktounlocksuchnewdatasourcesisnotlimitedtotherealmofthetechnical.Ifindicatorsarebuiltontopofthislinkeddata,thereremainsthechallengetoconstruct/uncovermeaningandrelevanceofsuchnumbers.Ataskaseasilyoverlookedandneglected,asitisfundamental.Inthissession,youwillgetawalkthroughofsuchachallengeusingtheexampleoftechnicalstandardsasnewdatasource.Thereareanumberofquestionstothelinkbetweenstandardsandpublications:Howmuchscienceisthereintechnicalstandards?Aretherefeedbackeffectstobeingcitedinstandards?Maybeyougetmorecitationsfromthis!Whatisthemeaningofthistypeofcitationcounts?Justtonameafew.Youwillbepresentedwithanarrayoftoolstomakeyourlifeeasier,howtoplanresearchdesignsbeyondbibliometricstouncoverthemeaningofsuchcounts,whyitisimportanttounderstandthedomain-specificcitationculturesandpractices,typicalpitfallsinopeningupnewdatasources,and,enpasssant,learnsomethingabouttheinnerworkingsoftechnicalstandardsandthestandardisationsystem.BibliometricsinPractice:HowtoGenerateReportsforInstitutionsDanielTorres-Salinas,UniversidaddeGranada(EC3metrics,ResearchEvaluationUnit),SpainInaninstitutionalcontextandataprofessionallevel,oneofourmaintasksistocarryoutbibliometricreports.Thesestudiesareessentialbecausetheyareusedbymanagerstomakedecisions(distributionoffunds,recruitmentofpersonnel,planningofresearchlines,etc.).Inthistalkwewillexplainhowtomakeaglobalbibliometricreportofaninstitution,weuseasacasestudytheUniversityofGranada.Wefocusonthesetopics:

Page 8: Monday, July 23rddevelopment, the impact of research, collaboration, and thematic specialization. Options and limitations of the indicators introduced will be discussed. Subject Classification

1)Generalconsiderations:target,selectionofindicators,objectives,etc.);2)whatsourcesofinformationcanbeused;3)Howtocontextualizeandinterprettheindicators;4)Howtocomparetheresultswithotherinstitutions(Benchmarking);5)Howtomakegraphsandtables;and6)Disseminationofresultsanddata.TheApplicationContextofResearchAssessmentMethodologiesHenkF.Moed,IndependentresearcherandscientificadvisorThelecturepresentsaseriesofkeynotionsfromthespeaker’snewmonograph“AppliedEvaluative Informetrics (Springer, 2017)”. First, an analytical distinction is madebetween four domains of intellectual activity in a research assessment process: thedomains of policy, evaluation, analytics, and data collection. Next, the lecture defendsthe idea of a multi-paradigmatic, value-free informetrics, and argues that, althoughevaluative assumptions (“values”) on what constitutes research performance play acrucialroleinresearchassessment,thisroleoftenremainsimplicit,andsuchvaluesareextra-informetric, in the sense that their validity cannot be grounded in informetricresearch.Aseriesofexamplesillustrateshowthechoiceofindicatorsinanassessmentprocessisinfluencedbypolicyobjectives,andhowtechnicalindicatorconceptsfitintothedevelopers’wider–social,culturalandhistorical–context.Itisfurtherarguedthatevaluative frameworks are indispensable but often missing in research assessment.Finally,thelecturecriticallyreflectsontheassumptionsunderlyingcurrentpracticesinthe use of informetric indicators in research assessment, and proposes a series ofalternativeapproaches,indicatingtheirprosandcons.TopicProminenceinScience–aNewBibliometricApproachtoIdentifyHigh-DynamicResearchTopicsTomaszAsmussen,CustomerConsultant,ElsevierAsresearchisbecomingincreasinglyinterdisciplinaryandinternationalitbecomesmoredifficulttoidentifyexistingresearchstrengthanduncovernewemergingresearchfieldswithahighrelevancyglobally.ElsevierandSciTechStrategieshavepartneredtodevelopascientificallyfoundedandyetpracticalbibliometricapproachtoidentifyresearchTopicsusingdirectcitationanalysisandfurtherassigningaProminenceindicatortodetermineitsmomentum.ByapplyingthisbibliometricapproachtothefullScopusdatasetsince1996roughly97,000stableresearchtopicshavebeenidentifiedandassignedaProminenceindicatorbyweighing3metrics(citationcount,viewscount,journalimpact)forpapersclusteredinaTopictodetermineitsmomentum.ThefactthataTopic’sProminencehighlycorrelateswithfundingmakesithighlyrelevantforresearchmanagersandresearchersalikeandthusProminenceprovidesadata-drivenindicationintowhichresearchtopicsarethebesttoinvesttimeandeffortsin.TopicProminenceinScienceisavailableinSciValandcanbeappliedtocountries,universities,researchersorgroupsofresearchersformorethan8,500researchinstitutionsgloballytorunafullresearchportfolioanalysis,inseconds.Dimensions-LinkingPublicationsandCitationswithGrants,PatentsandClinicalTrialstoWidentheViewTomLickiss–SeniorResearchConsultant,ÜberResearchPublicationsandtheirassociatedcitationsareagoodproxytomeasuretheimpactfromandwithinthescientificcommunity,butthisisonlyonefacetofamuchlargerpicture.

Page 9: Monday, July 23rddevelopment, the impact of research, collaboration, and thematic specialization. Options and limitations of the indicators introduced will be discussed. Subject Classification

Dimensionshasbeenrealisedtoaddressthisissue,andtoprovidegreaterandmoreopenlyavailableinsightsacrosstheresearchlifecycle.Byintegratingmorecontenttypes(likegrantstocaptureprojectbasedfunding,clinicaltrialsfortranslationintoclinicalcures,patentstoprovidethecommercialtranslationofresearchactivitiesandevenpolicydocumentstoreflectwhereresearchresultsmadeitintopolicies)wecanchangethewaysinwhichweaccessandunderstandthescholarlylandscape.Dimensionsdoesnotjustputmoredocumentsinadatabase,italsolinksthemconsistentlytogethertoallowa‘trajectoryview’attheinstitutional,researcherortopiclevel,enablinguserstogatherabroaderandmorequalitativeunderstandingoftheassociatedresearchactivity.Takingthisbroaderapproachtoresearchinformationalsoprovidestheopportunitytodevelopnewscientometricapproaches,usingamoremultifacetedviewtoanalyseinput,resultsandimpactfromvariousangles.InthepresentationwewillexplainthebasicconceptsandusecasesofDimensions,anddiscusshowresearcherswhoareinterestedtoworkwithorbuildonourdatacandoso.

Page 10: Monday, July 23rddevelopment, the impact of research, collaboration, and thematic specialization. Options and limitations of the indicators introduced will be discussed. Subject Classification

Thursday,July26thTheApplicationofNetworkAnalysisinScienceStudies:CommonTheoreticalBackgroundforBroadApplicationsBartThijs,CentreforR&DMonitoring(ECOOM),KULeuven,BelgiumNetworkanalysisinscientometricsprovidesapowerfulsetoftoolsandtechniquestouncovertherelations,structureanddevelopmentamongdifferentactorsinscience.ItisoftenreferredtoasMappingofScienceandcanbeappliedtoallentitiesassociatedwithsciencelikedisciplines,journals,institutionsandresearchers.Thislecturewillfocusmainlyondifferentmeasuresofrelationsbetweenentitiestacklingbothontheclassicalapproachesasonthenewtechniquesofnetworkanalysisinanapplication-orientedapproachwithinasolidtheoreticalframework.Relationsbasedoncitationsandreferencesincludebibliographiccoupling,co-andcross-citation.Otherdirectlinksbetweenentitiesincludeco-authorship,institutionalcollaborationorinternationalcollaboration.Alsolexicalapproacheslikeco-wordanalysisandtextminingwillbetackled.Eachofthesemeasureshastheirownpropertieswhichcanhavestrongimplicationsontheapplicabilityoftheanalyticaltechniques.Inordertoimprovethedistinctivecapabilitiesofthesemeasuresnewhybridapproacheshavebeenproposed.Thelecturewillalsodealwithseveralanalyticaltoolsandvisualizationtechniquesthataresuitableforcapturingtheunderlyingstructure.Clusteringtechniqueslikek-meansorWard’shierarchicalclusteringareproventechniquestoclassifytheentitiesmodularityclusteringhasbecomeapopularalternative.ResearchCollaborationMeasuredbyCo-AuthorshipWolfgangGlänzel,CentreforR&DMonitoring(ECOOM)&Dept.MSI,KULeuven,BelgiumCo-authorshipcanbeusedasaproxyforresearchcollaborationathigherlevelsofaggregation,e.g.,inthecaseofinstitutionalorinternationalcollaboration.Butevenatthelevelofresearchteamsandindividualscientists,co-authorshippatternsrevealimportantinformationaboutmainactorsandtheirroleinthenetworkofscholarlycommunication.Inthefirstpartofthelecturetheanalysisofco-authorshipnetworksatthemicro,mesoandmacrolevelisdescribed.Thestrengthofco-authorshiplinksamongindividualscientists,institutionsorcountriescanpreferablybedeterminedusingappropriatesimilaritymeasures.Co-authorshipnetworkscanreadilybevisualisedapplyingsuitablesoftwarethatisavailableandfreefornon-commercialuse.Inthesecondpart,bibliometricindicatorsfortheanalysisofresearchcollaborationatthemesoandmacrolevelwillbeintroduced.Itwillbeshownhowindicatorsandsimilaritymeasurescanbecalculatedusingthe“analyseresults”and“citationreport”toolintheonlineversionoftheWebofScience.TechnologyMaturity–UsingBibliometricMethodstoAssessTechnologyReadinessMariadeKleijn-Lloyd*,SVPAnalyticalServices,ElsevierGovernmentsandresearchfundersareincreasinglytryingtoensureresearchtheysponsorleadstoeconomicimpactandtechnologybreak-throughs.Insteadofthinkingin

Page 11: Monday, July 23rddevelopment, the impact of research, collaboration, and thematic specialization. Options and limitations of the indicators introduced will be discussed. Subject Classification

termsofresearchdisciplines,theyreframethequestionto‘grandchallenges’ortotechnologieslikenanotechnologies,artificialintelligenceorrobotics.Andontopoftheusual‘whereistheexcellenceandwhocollaborateswithwho’,questionsareaskedonthematurityortechnologyreadiness.Wewillpresentamethodandcasestudyusingbibliometricstoinvestigateresearchintechnologiesandassesstechnologyreadiness.

Page 12: Monday, July 23rddevelopment, the impact of research, collaboration, and thematic specialization. Options and limitations of the indicators introduced will be discussed. Subject Classification

Friday,July27thFocusTopicDay:Bibliometrics&OpenAccessTheDifferentFlavoursofOpenAccessBernhardSchubert,OpenAccessOffice,UniversityofVienna,AustriaThispresentationintroducestheconceptofOpenAccesstoscholarlypublicationsandthevarious"roads"leadingtowardsit.WewillalsobelookingattransformationalbusinessmodelsfromClosedtoOpenAccessandatservicesinstitutionscanoffertosupportthisdevelopment.OpenAccessandPublicationPracticesinAstronomyandMathematicsNielsTaubert,BielefeldUniversity,GermanyInastronomyandmathematicsalargeshareofpublicationsisfreelyaccessibleonlineviadisciplinaryorsubject-specificrepositories.Referringtoanempiricalstudyofincludingbibliometricanalysisandin-depthinterviews,thecontributionexaminestheroleofself-archivedmanuscriptsinthecommunicationsystemofthetwodisciplines.Theanalysisshowsthatrepositoriesactasasecondchannelofdisseminatingresearchinadditiontojournals.Moreover,itreconstructshowrepositoriesarebeingusedbyauthorsandreaders.Inbothfieldsauthorseven-handedlyself-archivetheirmanuscriptsinpartnotonlybeforethepublicationappearsinajournalbutevenbeforepeerreviewiscompleted.Thishappensfordifferentreasons,includingtheimprovementofaccessibility,theprotectionofpriority,andtoincreasethechancesofgettingtheresearchpublishedinajournal.Earlyself-archivingbeforecompletionofpeerreviewdefactobypassestheevaluationprocedurewhichisapreconditionfortrustinpublishedresearch.Therefore,itisaskedwhetherreadersdealwithsuchpre-printsinaspecificway,takingtheirpotentialnon-peer-reviewednatureintoaccount.Thereconstructionshowsthattheusabilityofself-archivedmanuscriptsresultsfromspecificroutinesamongthereaders:Theyinterpretcontextualinformationofpre-prints,undertaketestsofplausibility,usetheauthornameasaproxyfortrust,limitthecitationofpre-prints,anddistinguishbetweentrustworthyandnon-trustworthycomponentsofapre-print.Thus,theroutinesofthereadersarecomplementarytotheroutinesofauthorsandare–tosomeextent–shapedbyepistemiccharacteristicsoftheparticularfield.BibliometricsandOpenAccessEricArchambault,1science&Science-Metrix,CanadaAcademicshavebeenexaminingtheOpenAccess(OA)modelforscientificpublicationsforyears.ThoughithasbeennotedrepeatedlythatOAavailabilityisrisingsteadily,currentmeasurementisoftenlimitedbyvaguedefinitionsandexistingbibliographicdatabases.Oneofthechallengesistodistinguishmoreclearlybetweendifferenttypesofavailability(e.g.goldorgreen),whileaccountingfor,butnotconflating,otherimportantdimensionssuchastimetoavailability,transiencyandlicensetype.Anothermajorchallengeliesinmeasurementperse.Measuringopenaccessavailabilitycaninvolve

Page 13: Monday, July 23rddevelopment, the impact of research, collaboration, and thematic specialization. Options and limitations of the indicators introduced will be discussed. Subject Classification

computingthetotalnumberofavailablearticlesandthepercentageofavailablepapers.Thispresentationwillexaminecurrentmeansofretrievingopenaccessarticles(thenumeratorinthepercentage)andthedatabasescurrentlybeingusedtocountthetotalnumberofavailablepapers(thepercentage’sdenominator).AdvocatesofOAhavebeenarguingthatbyincreasingaccessibility,OAwouldenhancetheimpactofresearch.ThispresentationwillreviewcurrentevidenceontheOAcitationadvantageincludingtheargumentsraisedbyitsdetractors.DisentanglingGoldOpenAccess:DisciplinaryandCountryEffectNicolasRobinson-Garcia,INGENIO(CSIC-UPV),UniversitatPolitècnicadeValència,Spain;UniversidaddeGranada(EC3metrics),Spain/DanielTorres-Salinas,UniversidaddeGranada(EC3metrics,ResearchEvaluationUnit),SpainInthisseminarwewilldiscusscurrentpublicationtrendsingoldOpenAccess(OA).Thepurposeistodevelopafullunderstandingoncountrypatterns,OAjournalscharacteristicsandcitationdifferencesbetweengoldOAandnon-goldOApublications.Forthis,wewillfirstreviewcurrentliteratureregardingOpenAccessanditsrelationwithitsso-calledcitationadvantage.Startingwithachronologicalperspectivewewilldescribeitsdevelopment,howdifferentcountriesarepromotingOApublishing,anditseffectsonthejournalpublishingindustry.Wewilldeepentheanalysisbyinvestigatingtheresearchoutputproducedbydifferentunits.First,wewillfocusontheproductionofcountrieswithaspecialemphasisoncitationanddisciplinarydifferences.GoldOApublishingisbeingencouragedinmanycountriesasopposedtoGreenOA.Wewilldiscusshowthisaffectsresearchers’publicationpatternsandwhetheritensuresanallegedcitationadvantageasopposedtonon-goldOApublications.

Page 14: Monday, July 23rddevelopment, the impact of research, collaboration, and thematic specialization. Options and limitations of the indicators introduced will be discussed. Subject Classification

Saturday,July28thAltmetrics:StateoftheArtandFutureWorkIsabellaPeters,Leibniz-InformationszentrumWirtschaft(ZBW),GermanyThelecturewillpresentthecurrentstateoftheartofaltmetricsresearchanditsmajorfindings.Itwillparticularlyfocusonstudiesonthecoverageandintensityofaltmetricsaswellasonthetheoriesandmodelstryingtoestablishatheoreticalbackgroundofaltmetrics.Researchgapswillbeidentifiedandcurrentdevelopmentsandinitiativesthataimatbringingaltmetricsintopractice(e.g.,NISO,EU,LIBER)willbepresented.PracticalApplicationsofAltmetricsNicolasRobinson-Garcia,INGENIO(CSIC-UPV),UniversitatPolitècnicadeValència,Spain;UniversidaddeGranada(EC3metrics),SpainThistalkaddressespracticalissues,limitationsandproblemswemightencounterwhenretrievingandprocessingaltmetricdata.Wewillexplorewhatitiswhattheindicatorsprovidedbythemainaltmetricprovidersofferandhowthesecanbeusedinameaningfulwaytomakeinformativeandsensibleanalyseswhichoffercouldofferanaddedvaluetoourinstitutions.Thefirstpartofthetalkwillfocusonthediscussionofthepeculiaritiesandtechnicalitiesrelatedtohowaltmetricdataisprovidedtotheuser.Thesecondpartwillprovideexamplesandshowempiricalapproachesthatcanbeappliedtoinformonthesocialmediaattentionofresearchers’activity.