positioning universities

8/6/2019 Positioning Universities

1/34

PRIME-ENID Winter School - 6novembre 2007 - Roma

The positioning of universities

Jean Thves, Michel Zitt, Ghislaine Filliatreau

Observatoire des Sciences et des Techniques (OST)


2/34


1. Positioning indicators vs Ranking and/or

evaluation

2. OEU project

3. Bibliometrics as a tool for positioninguniversities

4. To conclude


3/34


Positioning the universities vs ranking/evaluating them.

Ranking and/or evaluation approach: notionof effectiveness of the institution (theinstitution as itself : one shot )

Positioning approach: notion of measuring

its position within a given environment(the institution as a strategic actor) andalong a trajectory


4/34


Positioning indicators : overall frame

In this rationale, the indicators aim at characterisingthe elements of the national innovation system,considering it is made of differentiated, autonomous andstrategic agents (Barr, 2005)

firstly, the position of the actors () are as

important as formal inputs and outputs for theirperfomance ; secondly, the benchmark with auniversal production function () is replaced by the ideaof helping individual actors to position themselves in a

complex and fragmented institutional space (Lepori,Barr, Filliatreau, 2007)


5/34



characterising the elements of the national

innovation system

Applied to universities : universities are a

component of the research system in achanging context (triple helix context(Leydesdorff & Etzkowitz) mode 2 of

research context (Gibbons et al.).


6/34


Differentiated, autonomous, strategic agents

Applied to universities, as autonomous and strategic agents , universities need toolsto know itself, its environment in order to adapt to new research modes, researchfunding opportunities etc

The positioning indicator frame supposes also a trajectory within the environment

considered.

Three leverages for autonomy for research entities (Felt et al.) :

the higher education system environment vis--vis its head ministry institution(here ministries)

The university vis--vis other universities

The components of the universities themselves

The positioning approach aims at describing these 3 levels.



7/34


position themselves in a complex and fragmented institutional space

Universities need to position themselves (to know at which interactions it is) inorder to adapt its responses vis--vis :

Research actors (other institutions) / policy makers (for new regulations,

new rules, new funding opportunities) Funding sources : region / nation / EU / private

Human resources (staff) and students (which are a source of money (director indirect), a source of knowledge producers and which can become a

label (if international visibility)



8/34


Positioning versus Ranking or Evaluation

Universities, in order to know if they are efficient are often ranked

(or evaluated for funding rate (ex: RAE or the Australianexperience (see Butler))

Differences between ranking focus and positioning focus could be

analysed through the answers of three main questions regarding the measurement of universities:

Who makes the indicators ?

Who uses/needs indicators ? What are the expected effects of the indicators?


9/34



Until now, rankings (see Shangha ranking) are

made to evaluate the results of theuniversities.

Answers of questions are :

Who makes : clients / competitor / ministries forfunding

Who uses : clients , i.e. students, companies,funders/ uses depend on results

What are the effect : universities try to do better (ex. Shangha) i.e. adapt to the criteria (ex. evaluationprocess RAE or Autralian univ (Butler 2003) ;universities use their good rank for attractivity


10/34



Positioning indicators have the ambition to better

understand the organization of the overallsystem and could be used by the measured actors for strategic management

Answers of questions are : Who makes : researchers, universities themselves,

policy makers (as principals)

Who uses : same / uses independant of results What are the effect : strategic analysis of results /

policy making / research steering / enhancedunderstanding of institution, system


11/34


The OEU : a PRIME project that aimed at investigating theoverall positioning of the universities

The Observatory of the European University (OEU) has elaborated amatrix through which 5 main dimensions (F, HR, AO, TM and Gov.)

were analysed through the filter of 5 transversal issues : autonomy,strategic capabilities, attractiveness, differentiation profile andterritorial embedding

The approach is a strategic one :

Work has been done with the university research managers andPRIME researchers so as to build both a better knowledge of highereducation environment (for the researchers) AND new tools for

analysing their institution (for the University research managers)

This double approach is in line with the positioning focus


12/34


13/34


Funding * autonomy

Indicators to be built have to answer to these questions

Interest for researchers and research managers, whateverthe answer isthere are no good or bad result but a

better understanding of the university budget

FUNDING

OMY

KQ1.1

What is the degree of freedom the government enjoys in the use of government funding?

How significant is the portion of non-governmental funding that goes to research?


14/34


The examples of bibliometric indicators.

- Bibliometrics is the quantitative study of the community interactions, basedon the contents of publication outputs that are key elements in the real life of

scientific communities.

- Bibliometrics is all the more practicable than outputs are codified:

- - patents, strong codification by law

- - publications in peer-reviewed journals: quasi-norms in presentation, rhethorics,references

- - other types of documents, however with less strong codification, may beexploited: articles in lower-standard journals, reports,

- - non-standard documents: websites especially academic

- The level of information is not warranted by the level of codification(patents!)


15/34


Bibliometrics as a tool for positioninguniversities

The Tool

Thomson (ISI) databases or more recently Scopus combine:

- multidisciplinary coverage of scientific literature (including selection criteria for journalsfor Thomson)

- information on all authors and especially all affiliations

- information on cited references

Weaknesses:

- coverage of social and humanities, adding to different modes of communication in thosedisciplines (books, conferences)

- in comparison with specialized databases, lack of controlled vocabulary and of

elaborated classification scheme, replaced by proxies (Thomson: classification ofjournals)

- and, whatever the database, the unification problem ../..


16/34


The tool is an image of the activity of researcherswithin universities

- 1. Activity of scientists in a university (a lab) is multidimensional, andinvolves several types of customers for several outputs (see compass

card of research , CSI-Mines) many of them in written form with variouslevels of codification.

- - education: courses and material for students (Mission I)

- - publication/commmunication for academic colleagues and others (Mission II)

- - patents and expertise for industry

- - publications, reports and expertise for government- - id for non government bodies e.g. associations of patients

- - public information, citizens awareness

- 2. Activity of scientists is conducted within a community of colleagues

- - intervening at all stages of the publication cycle (getting funds, collaborate,discussing article, submitting, be read, be cited)

- - intervening in other aspects of self-organisation of communities (career, etc.)

- (see Latour, Wouters)


17/34


Examples of bibliometric indicators and

their interpretation in terms of positioning the university :

Three main categories of indicators Power (nb of pub) Performance (nb of cit)

Positioning (co-pub) : position of the actor in aparticular network (collaborative network (ratio andspectrum of partnerships), disciplinary network)


18/34


More is better Power (size-dep) Output Volumes or market

shares of pub.

Id. for cit (visibility)

Performance (size-

indep)

Throughput Productivity, pub

and cit, individ. HI,

global DEA perf

bibliometric perf. Impact (cit.),relative impact &

akin measures

What is better? Descriptive Specialization/

variety

Network position(e.g. collab.)

Specialization

Collab. Ratios

PatternsCentrality,

Betweenness


19/34

The coverage issue of the database


Biologie

fondamentale19,3

Mathmatiques

3,7

Sciences de

l'univers

7,5

Sciences pour

l'ingnieur

14,9

Physique

13,8

Chimie

18,1Biologie applique-

cologie

8,3

Recherchemdicale

33,2


20/34


Specialisation index : an indicator that describes the university scientificproduction activity, whatever the size, the overall performance etc

It is not said if one discipline is better than another (a research policyquestion), hence, a higher specialisation index in one disciplinebetter or not for the university

Butregarding national policies, or funding structure of the university,the index allows to position the university within the more globalenvironment (the region, the nation, the EU etc)

The environment (the market ) is important to position the universityactivities


21/34


22/34

The bidimensional positioning


Indice de spcialisation et impact relatif

(compte fractionnaire)

Biologie

fondamentale

Recherche

mdicale

Biolo gie applique -

cologie

Chimie

Physique

Sciences de

l'univers

Sciences pour

l'ingnieur

M athmatiques

M ultidisciplinaire

0

0,5

1

1,5

0 0,5 1 1,5

Indice de spcialisation

Impactrelatif5

ans


23/34


Co-publications activity (networks of partners for scientific publications).

Is for a university better to collaborate more or less, with whom, inwhich disciplineetc: this answer from the university manager willbe different than the answer from the research policy analyst or fromthe funding agency. The position for each actors will depend on

the location of the actors within the given environment of theuniversity considered (see Bourdieu : the actor speaks from itslocation within the scientific field, and considering its scientificcapital - which definition comes from intrisic rules or norms for this

specific field)


24/34


% of articles of Univ A 1996 1997 1998 1999 2000 2001

en mono-signature 19,8 28,9 19,8 24,0 25,2 22,6

en copublication 80,2 71,1 80,2 76,0 74,8 77,4

en copublication Univ A 13,0 14,2 15,6 15,6 14,8 14,9

en copublication internationale 44,1 46,1 43,3 48,7 50,9 51,8

en copublication europenne 25,5 25,2 23,2 26,2 26,4 28,9

Total 100,0 100,0 100,0 100,0 100,0 100,0

Total en nombre d'articles 2106 2027 2101 2024 2051 1764

% d'articles de la France 1996 1997 1998 1999 2000 2001en mono-signature 30,8 29,4 27,8 26,2 24,6 22,2

en copublication 69,2 70,6 72,2 73,8 75,4 77,8

en copublication France 47,5 47,7 49,3 49,7 49,7 51,2

en copublication internationale 32,8 34,7 35,7 37,5 39,6 41,6

en copublication europenne 16,3 17,4 17,9 19,0 20,0 21,3Total 100,0 100,0 100,0 100,0 100,0 100,0

Total en nombre d'articles 45865 47340 48639 48544 49330 43082


25/34


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

Biologie fondamentale

FR

A

US

A

DE

U

GB

R

CH

E JPN BEL CAN ITA CEI ESP NLD

SW

E AUS CSK DNK

190 99 51 27 23 20 15 15 14 11 10 8 7 7 5 5

Recherche mdicaleFR

AUS

AGB

RDE

UCH

E BEL ITASW

E CAN NLD JPN ESP AUT AUS DNK BRA

145 33 19 18 13 10 9 8 6 6 5 5 4 3 3 3Biologie applique -

cologieFR

AUS

ADE

UCA

NGB

R JPN ITA BEL CHE ESPMD

GCH

N AUS CSKNO

R PAKPR

T

28 8 8 6 6 4 2 2 2 2 2 1 1 1 1 1 1

ChimieFR

ADE

U USAGB

R ITACH

E JPN ESP CEI BEL CAN DZAMA

R AUS NLD POL

133 54 48 31 27 14 14 10 8 8 8 6 6 6 6 5

PhysiqueFR

AUS

ADE

UGB

R ITACH

E POL BELNO

R CEI HUN SLQSW

E BRAGR

C PRT

134 79 68 53 49 42 29 25 22 21 20 17 16 16 14 13

Sciences de l'universFR

AUS

ADE

UGB

R ITACH

ECA

N BEL ESP NLD CEI IND AUS JPNSW

E POL

79 25 20 15 10 7 6 6 6 5 4 4 4 3 3 3

Sciences pour l'ingnieur

FRA

USA

DEU ITA

CHE NLD

GBR CEI AUT POL BEL

MAR

GRC

CHN ESP IND

40 14 11 10 6 5 5 4 3 3 3 2 2 2 2 2

MathmatiquesFR

AUS

A ITADE

UCH

NCH

EGB

R POL BELBR

A CAN CEI ESP IND JPN MEXNLD

TUR

11 6 3 3 2 1 1 1 1 1 1 1 1 1 1 1 1 1

Multidisciplinaire FRA USA DEU GBR ITA BEL CHN JPN AUS CHE ESP CEI DNK FIN ISR MDG

10 9 4 3 1 1 1 1 1 1 1 0 0 0 0 0

Toutes disciplinesFR

AUS

ADE

UGB

R ITACH

E BEL JPN CEI POL ESP CANSW

E NLDNO

R BRA

646 286 205 141 111 97 62 52 45 43 42 39 37 32 28 26


26/34


Activity index : in which set of articles the university produces the most.

The life of scientific articles is then dependent on its visibility : it can never be cited, orcited some times or very visibile and cited a lot.

Glnzel recalls that there are at least 15 different reasons for citing an article.

Paying homage to pioneers, giving credit for related work, identifying methodology,

providing background reading, correcting ones own work, correcting the work ofothers, criticising previous work, substantiating claims, alerting the forthcoming work,providing leads to poorly disseminated etcwork, authenticating data, identifyingoriginal publication in which an idea/concept was discussed, disclaiming work ofothers, disputing priority claims of others

This cited life of the article place it in a set of articles with the same life. We define 6sets of articles from the 5% most cited to the 40% not cited.


27/34


28/34


The issues of normalization (discipline or specialisation, before geographicalnormalization) is a positioning approach. The same set of data, with adifferent normalization, tells a different story on the university analysed :

normalizing is focusing on a set of components; it structures theenvironment.

You can have very good results in a specific set of data (articles) but overallbeing less good or average, alltogether.

Barr says co-constructing indicators : the context of the number (the resultof the measure) has been discussed by the actors before measuring. Theindicator is elaborated without number (it the context/the environmentwhich is being structured through the construction of the indicator)

Once the environment is structured, the university within this structuredenvironment can be analysed (positioned both in the given environment andwithin its own trajectory).


29/34


To conclude

Univ

State

Industry

Public

NGO

Society

Source : adapted from Etzkovitz, Leydesdorff

The university is embeddedwithin an evolving environment.


30/34


Univ/PRO

Univ

State

Industry

Public

NGO

Society

Source : adapted from Callon, Laredo

The university interactswith the components of this environment.


31/34


32/34


Univ/PRO

1.Collaborations

2. Human resources :

fluxes, mobility3. Hybrid forms: joint

labs mixtes, spin-offs,

entrepreneurial univ.

Univ

State

Industry

Public

NGO

Public

The university is embedded in several networkswith its researchpartners in this environment.


33/34


We have seen technical examples to illustrate the general idea ofthe concept of positioning indicators.

The interest of this frame is that we enlarge the focus and considerindicators are also a good tool for knowing the environment and thelinks between the environment and the university.

Universities (and researchers) need more and more indicators thatare able to tell a new story on an old system ; the frame of positioning indicators is adapting data and indicators with this

aim. It serves both parties.


34/34


http://www.obs-ost.fr

positioning universities

Documents