metrics and the university professor...
TRANSCRIPT
Metrics and the University Professor
Submissions
Free webinars from Elsevier related to metrics, Scopus and SciVal and the new metric called CiteScore, which is a journal‐level metric.
Included in the Elsevier webinars are others under the Library Connect channel with a librarian focus.
https://www.brighttalk.com/channel/9995/library‐connect
SciVal Next
SciValWhat is required? Institutionally
o metrics that support their position Individually
o metrics that support promotion (with peer review)o metrics that support grant applications o but never to be used for individual performance evaluations
SciVal Benchmarking
o must comparing like with likeo must be normalized for discipline or fieldo must be contextualized
Transparency
o We need to explain our methodology for accountability
Terminology
o Explain how the resources define the parameters they use
o Important to understand the data that leads to the data.
SciValProcess and Lessons Learned
The Benchmarking section of SciVal is where you need to go to generate comparisons between individuals or groups, and research areas.
SciVal and Scopus work hand‐in‐hand so you can look at an author profile in Scopus; locate the author and then select the “Analyse Author Output” link and then subject area tab you can see a breakdown of the subject headings assigned to that author. These subject headings are very broad.
SciVal identifies these subjects as “Research Areas” which are broken down into more detail. You cannot add the broad subject heading as a “Research Area” and so must select one of the more detailed subject headings. I chose to use “Research Area” (whatever that was) with the miscellaneous subheading which worked well.
SciValProcess and Lessons Learned
• In SciVal you can compare that author against all the authors writing in that research area as a whole.
• If you are searching for an author with a number of names you can make those names a group and compare that group against the subject area or any number of individual authors.
SciValProcess and Lessons Learned
SciVal will automatically produce nice reports for each search but in an individual PDF file. If you need to condense the reports, another option is to export the information to excel and then cut and paste into a condensed report.
Date range for benchmarking is limited to 1996 onwards. There are numerous options for selecting criteria for benchmarking which
can be easily selected from the dropdown menus provided. Results can be retrieved in chart or table form. Both can be exported to excel.
Benchmarking on SciVal
WofS Next
Web of ScienceA few lessons learned
InCites Next
Moscovitch InCites Report
InCites Step by Step• Step 1 (Optional):
• Look up author using advanced search in Web of Science. Identify research areas and multiple names.
• Step 2: Analytics > People • > By Attributes > Person Name or ID• Search for author with all possible names• For each result, click on the drop down arrow, choose Research Area and preview the results to determine the most appropriate categories.
• Compare with WofS results from step 1• Step 3: Analytics > People
• > By Attributes > Person Name or ID• > By Research Output > Research Area• Limit the results for each Research Area identified as relevant in Step 2 – Do one at a time
• Create a baseline to amalgamate all the name variations. • Update results with all the chosen Research Areas together for an overall baseline for the author that removes incorrect but similar names.
InCites Step by Step
• Step 4: Analytics > Research Areas• > By Attributes > Research Area• Run a search for each of the relevant Research Areas to generate a comparator for the results in Step 3
• Step 5: Analytics > Research Areas• > By Attributes > Research Area• > By Research Output > Person Name or ID• To double check results, you can also run individual searches by Research Area and then limit each one by ALL the author name variants
• Step 6: • Choose indicators of interest (gear symbol)• Export CSV file
Google Next
Google Scholar Analysis
• Tasked with finding h‐index for candidates using GoogleScholar
• No profile? Publish or Perish used• Software that retrieves and analyzes academic citations usingGoogle Scholar
• Provides a number of metrics, including the h‐index
Benefits: • Captures more citations (including non‐English language
publications) • Profile owner has more control over content associated
with profile
Issues: • “dirty data” name disambiguation issues, questionable
quality of citation • Not designed to capture interaction beyond citations
Publish or Perish
One of the major issues: name disambiguation
• Depending on the uniqueness of the name, disambiguation can be difficult• Search strategy seems to be most effective with Anglophone North American names• Time consuming to run searchers • Google Scholar does not have dedicated fields for searching research area or affiliation• A lot of fine‐tuning tips available, but a deep understanding of the faculty’s research is needed
Issues Next
Issues • Name disambiguation inaccuracies• Institutional affiliation data missing • Cross‐field comparisons are unfair • Numbers create a false sense of reliability
Messy data…
Best Practices• Offer training instead of results• Get someone else to re‐run the analysis & compare results• Check attributed publications against the researcher’s CV • Compare InCites to SciVal• Include caveats in your report• Discourage exclusive reliance on bibliometics• Wherever possible, Involve the researcher being profiled
Reccomended reading:Diana Hicks, Paul Wouters, Ludo Waltman, Sarah de Rijcke, et al.. "The Leiden Manifesto for Research Metrics" Nature Vol. 520 (2015). https://www.nature.com/polopoly_fs/1.17351!/menu/main/topColumns/topLeftColumn/pdf/520429a.pdf
