composite indicators. methodology and applications to r&d...
TRANSCRIPT
![Page 1: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/1.jpg)
1
Composite indicators.Methodology and Applications to
R&D indicators
by Andrea SaltelliEC, JRC
Workshop on Entrepreneurship Indicators:
R&D and Technology
Copenhagen, 30 and 31 October 2008
![Page 2: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/2.jpg)
2
Joint Research
Centre
![Page 3: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/3.jpg)
3
Whereabouts
![Page 4: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/4.jpg)
4
![Page 5: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/5.jpg)
5
Methodology from: Joint OECD-JRC
handbook.
•5 years of preparation,
•2 rounds of consultation with OECD high level statistical committee,
•finally endorsed March 2008 with one abstention
![Page 6: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/6.jpg)
6
• Composite indicators’ fortune
![Page 7: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/7.jpg)
7
How many of them are around?
Searching “composite indicators”
Scholar Google October 2005 992
June 2006 1,440
May 2007 1,900
October 2008 3,030
![Page 8: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/8.jpg)
8
Use: an example, from the Economist web edition, October 15
Business travel ranking (The most and least attractive destinations for business travellers)Distance-learning MBAs(A ranking of the top distance-learning MBA programmes)Democracy index (The world's most and least democracticcountries)Operational risk ratings (Rating the risk of doing business in 150 markets)E-readiness (The best countries for conducting electronic business)Global house-price indices (Tracking house-price inflation around the world)
![Page 9: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/9.jpg)
9
Another example, about Mauritius, Economist October 16
THE 1.3m people of Mauritius love to prove famous people wrong. On independence from Britain in 1968, pundits such as a Nobel prize-winning economist, James Meade, and a novelist, V.S. Naipaul, did not give much of a chance to this tiny, isolated Indian Ocean island 1,800km (1,100 miles) off the coast of east Africa. Its people depended on a sugar economy and enjoyed a GDP per person of
only $200. Yet the island now boasts a GDP per person of $7,000, and very few of its people live in absolute poverty. It
once again ranks first in the latest annual Mo Ibrahim index, which measures governance in
Africa. And it bagged 24th spot in the World Bank’s global ranking for ease of doing business—the only African country in the top 30, ahead of countries such as Germany and France. How does it pull it off?
![Page 10: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/10.jpg)
10
Ideally, a composite indicator should be based on:
‐Solid theoretical framework,
‐Underlying data of good quality,
‐Tested methodology.
When these conditions are met, it can be used for advocacy and analysis.
Saltelli, A., 2007, Composite indicators between analysis and advocacy, Social Indicators Research, 81 , 65‐77.
![Page 11: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/11.jpg)
11
• The steps to build CIs
![Page 12: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/12.jpg)
12
A good technical preparation for a CI can make it more robust (to uncertainties in data, weights,…) more resilient (remain relevant over time), more defensible (in dialogue with stakeholders…)
![Page 13: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/13.jpg)
13
Step 1. Developing a solid theoretical framework
What is badly defined is likely to be badly measured …
Examples …
The challenges are:• To integrate a broad set of (probably conflicting) points of view while keeping within a manageable construct. A model. a storyteller a representation/narrative meaningful to a community of peers
![Page 14: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/14.jpg)
14
E‐business readiness (EC)
Definition: Capability of a company to engage in electronic transactions with the objective of increasing business competitiveness.
Aim: evaluation of a composite indicator to monitor country progress in the implementation of the i2010 initiative...
![Page 15: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/15.jpg)
15
33 countries: EU‐27, US, JP,CH, IS, NO, TR
Summary innovation Index
Innovation drivers, to measure the structural conditions required for innovation potential
Innovation & entrepreneurship, to measure the efforts towards innovation at the microeconomic level
Knowledge creation, to measure the investments on human factors and on R&D activities, considered as the key elements for a successful knowledge-based economy
Input Indicator
Innovation Drivers Knowledge CreationInnovation & Enterpreneurship
A theoretical framework consisting of 5 groups was discussed and agreed.
![Page 16: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/16.jpg)
16
Application, to measure the performance, expressed in terms of labour and business activities, and their value added in innovative sectors
Intellectual property, to measure the achieved results in terms of successful know how, especially referred to high-tech sectors
33 countries: EU‐27, US, JP,CH, IS, NO, TR
Output Indicator
Application Intellectual Property
Summary innovation Index
![Page 17: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/17.jpg)
17
Step 1. Developing a solid theoretical framework
After Step 1. the developer should have…• A clear understanding and definition of the multidimensional phenomenon to be measured. • A nested structure of the various domains. • A list of selection criteria for the underlying indicators, e.g., input, process, output.
![Page 18: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/18.jpg)
18
Step 2. Selecting indicators
A composite indicator is above all the sum of its parts…
Excerpt: The strength of a composite indicator can largely depend on the quality of the underlying data. […]. The theoretical framework should guide the choice of the underlying indicators. The selection process can be quite subjectiveand therefore should involve stakeholders.
![Page 19: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/19.jpg)
19
a1 - % of firms that use Interneta2 - % of firms that have web/home pagea3 - % of firms using ≥ 2 security facilitiesa4 - % of employees using computera5 - % of firms with broadband connectiona6 - % of firms with LAN
b1 - % of firms purchasing products/serv. via Internetb2 - % of firms receiving orders via Internetb3 - % of firms with IT linked with other internal ITb4 - % of firms with IT linked with external ITb5 - % of firms using Internet for bankingb6 - % of firms selling products via Internet
E-business readiness composite indicator
Adoption of ICT Use of ICT
Enterprise survey 2007 (Eurostat), yearly since 2003, Adoption and Use composite indicators
![Page 20: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/20.jpg)
20
a1 - % of firms that use Interneta2 - % of firms that have web/home pagea3 - % of firms using ≥ 2 security facilitiesa4 - % of employees using computera5 - % of firms with broadband connectiona6 - % of firms with LAN
Adoption of ICT
![Page 21: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/21.jpg)
21
b1 - % of firms purchasing products/serv. via Internetb2 - % of firms receiving orders via Internetb3 - % of firms with IT linked with other internal ITb4 - % of firms with IT linked with external ITb5 - % of firms using Internet for bankingb6 - % of firms selling products via Internet
Use of ICT
![Page 22: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/22.jpg)
22
Summary Innovation Index• Data from Eurostat Science and Technology Indicators and
Community Innovation Survey (CIS).
• A set of 53 indicators analyzed (FA)
• A reduced list of 26 indicators was discussed by developers and members of GSO (an expert group) with DG ENTR.
![Page 23: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/23.jpg)
23
Step 2. Selecting indicators
After Step 2. the developer should have…
• Checked the quality of the available data.• Discussed the strengths and weaknesses of each selected indicator.• Metadata
![Page 24: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/24.jpg)
24
Step 3. Multivariate analysis
Analysing the underlying structure of the data is an art …
a1 1 a2 0.85 1 a3 0.83 0.87 1 a4 0.77 0.81 0.89 1 a5 0.56 0.55 0.72 0.72 1 a6 0.64 0.61 0.66 0.73 0.56 1 b1 0.50 0.72 0.69 0.63 0.40 0.54 1 b2 0.61 0.80 0.75 0.74 0.62 0.54 0.86 1 b3 0.43 0.39 0.54 0.54 0.39 0.48 0.10 0.29 1 b4 0.35 0.26 0.41 0.30 0.31 0.37 0.16 0.24 0.75 1 b5 0.70 0.49 0.48 0.45 0.60 0.31 0.09 0.38 0.21 0.13 1 b6 0.55 0.75 0.78 0.77 0.59 0.57 0.77 0.83 0.44 0.24 0.29 a1 a2 a3 a4 a5 a6 b1 b2 b3 b4 b5
Component Initial Eigenvalues
Total % of variance Cumulative % 1 7.242 60.35 60.35 2 1.523 12.69 73.04 3 1.178 9.82 82.86 4 0.554 4.61 87.47 5 0.512 4.26 91.73 6 0.385 3.20 94.94 7 0.242 2.01 96.95 8 0.131 1.09 98.04 9 0.098 0.82 98.04 10 0.064 0.54 99.40 11 0.043 0.36 99.76 12 0.029 0.24 100.00
0
1
2
3
4
5
6
7
8
0 2 4 6 8 10 12 14
Component number
Eige
nval
ues
![Page 25: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/25.jpg)
25
Step 4. Imputation of missing data.
The idea of imputation could be both seductive and dangerous …Almost all datasets contain missing data.
E-businessCountry a1 a2 a3 a4 a5 a6 b1 b2 b3 b4 b5 b6
AT 0.97 0.78 0.84 0.53 0.72 0.34 0.42 0.18 0.49 0.19 0.85 0.02
BE 0.97 0.72 0.82 0.59 0.86 0.45 0.43 0.18 0.49 0.18 0.87 0.02
BG 0.75 0.31 0.54 0.20 0.61 0.16 0.03 0.01 0.17 0.05 0.44 0.01
CY 0.88 0.47 0.67 0.42 0.69 0.20 0.12 0.07 0.44 0.06 0.54 0.01
CZ 0.95 0.71 0.77 0.40 0.77 0.31 0.22 0.09 0.31 0.08 0.87 0.01
DE 0.95 0.78 0.90 0.61 0.80 0.47 0.52 0.24 0.52 0.19 0.76 0.05
DK 0.97 0.84 0.93 0.80 0.39 0.36 0.33 0.62 0.21 0.93 0.04
EE 0.94 0.62 0.74 0.39 0.78 0.26 0.13 0.07 0.24 0.11 0.93 0.03
EL 0.93 0.60 0.52 0.37 0.72 0.35 0.08 0.06 0.71 0.01
ES 0.94 0.49 0.76 0.49 0.90 0.27 0.16 0.08 0.30 0.11 0.81 0.02
EU27 0.93 0.63 0.80 0.49 0.77 0.34 0.29 0.15 0.41 0.14 0.77 0.03
FI 0.99 0.81 0.97 0.70 0.91 0.47 0.19 0.15 0.53 0.15 0.91 0.03
FR 0.96 0.57 0.82 0.52 0.89 0.35 0.43 0.11 0.76 0.02
![Page 26: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/26.jpg)
26
Three common approaches to deal with missing data:
•case deletion (removes either country or indicator from the analysis)
•single imputation (e.g. Mean/Median substitution, Regression, etc.)
•multiple imputation (e.g. Markov Chain Monte Carlo algorithms).
![Page 27: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/27.jpg)
27
Step 5. Normalisation of data
Avoid adding up apples and oranges …
RankingStandardizationRe-scaling Distance to reference countryCategorical scales
…
![Page 28: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/28.jpg)
28
Step 6. Weighting and aggregation
The relative importance of the indicators can become the substance of a negotiation …
Weights based on statistical modelsPrincipal component/Factor analysis Data envelopment analysis Regression approach Unobserved components models
![Page 29: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/29.jpg)
29
Step 6. Weighting and aggregation
Weights based on opinions: participatory methods
Budget allocation Public opinionAnalytic hierarchy process Conjoint analysis
![Page 30: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/30.jpg)
30
Step 6. Weighting and aggregation
Weights based on Multi Criteria Analysis
To know more buy:
![Page 31: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/31.jpg)
31
Space of alternatives
Including/excluding variables
Normalisation
...
ImputationWeights
Aggregation
Performance index
Italy GreeceSpain
10
20
30
40
50
60
Step 7. Robustness and sensitivity
Uncertainty analysis can be used to assess the robustness of composite indicators …
![Page 32: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/32.jpg)
32
Monitoring e-Business Readiness of European enterprises
Sensitivity analysis of countries position
![Page 33: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/33.jpg)
33
To know more buy:
Or read: Saisana M., Saltelli A., TarantolaS. (2005) Uncertainty and Sensitivity analysis techniques as tools for the quality assessment of composite indicators, Journal of the Royal Statistical Society - A, 168(2), 307-323.
![Page 34: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/34.jpg)
34
y = 0.7691x + 20.249R2 = 0.6979
40
50
60
70
80
90
100
40 50 60 70 80 90 100
Composite Le arning Inde x
Econ
omic
and
Soc
ial W
ell-b
eing
Step 8. Links to other variablesComposite indicators can be linked to other variables and measures
Comparing effectively complex dimensions: Canadian welfare and Canadian Composite Learning Index
![Page 35: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/35.jpg)
35
From LudgerWößmann, Contribution of Education and Training to Innovation and Growth, see:
www.education-economics.org
![Page 36: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/36.jpg)
36
Step 9. Back to the details
De‐constructing composite indicators can help extend the analysis …
Construction (sec F)
Wholesale and retail trade (sec G)
Provision of short-stay accommodation (sec H)
Motion picture, video, radio and television activities (sec O)
Transport, storage and communication
(sec I)
Real estate, Renting and Business activities (sec K)
Manufacturing (sec D)
All sectors
0.15
0.20
0.25
0.30
0.35
0.40
0.50 0.55 0.60 0.65 0.70 0.75 0.80 0.85 0.90 0.95
ICT Adoption
ICT
Use
All countries by sector
![Page 37: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/37.jpg)
37
Monitoring e-Business Readiness of European enterprises
Enterprise survey 2007 (Eurostat), yearly since 2003, Adoption and Use composite indicators
2006Use Adoption
![Page 38: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/38.jpg)
38
AU
IL
SKEL LT
PT
LU
ATIEFR
NLBE
UKUS DE
JP
CH
DK
HU
CYES MT
EEIT
CZSI
IS
CA
HRPL
LVRO
BG
SE
FI
NO
TR
0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
-4.0 -3.0 -2.0 -1.0 0.0 1.0 2.0 3.0 4.0 5.0 6.0
Average grow th rate of SII (2003-2007)
2007
Sum
mar
y In
nova
tion
Inde
x
Sw eden Innovation leaders Innovation follow ersModerate innovators Catching-up countries Turkey
Dotted lines show EU performance.
THE 2007 SUMMARY INNOVATION INDEX (SII)
![Page 39: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/39.jpg)
39
Step 10. Presentation and dissemination
![Page 40: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/40.jpg)
40
COMPARATIVE ANALYSIS OF INNOVATION PERFORMANCE of THE
EUROPEAN COUNTRIES
0.08
0.23 0.23 0.24 0.25
0.25 0.26
0.26 0.27 0.29 0.
31 0.33 0.33 0.
350.
360.
360.
36 0.37
0.44 0.
47 0.47 0.48
0.48 0.49 0.50 0.
53 0.55 0.57 0.59 0.60
0.61 0.62 0.64 0.
670.
73
0.18 0.19
0.45
0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
TR RO LV B GHR P L SK P T EL HU LT M TES CY IT SI CZ NO A U EE CA EU B E FR NL A T IE IS LU US UK DE JP DK IL FI CH SE
THE 2007 SUMMARY INNOVATION INDEX (SII)
![Page 41: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/41.jpg)
41
Why the ten steps?
To combat lack of transparency, which makes them and the associated the policy inference, arbitrary and objectionable.
The three pillars of a well-designed CI are:Solid theoretical framework,Underlying data of good quality,Tested methodology .
Methodology should include assessment of the CI robustness.
CI can be refuted by practitioners(examples of WEF GCI and Ecological Footprint)
![Page 42: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/42.jpg)
42
•No more methodology. The KEI study
![Page 43: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/43.jpg)
43
Giuseppe Munda and Michaela Saisana
![Page 44: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/44.jpg)
44
Main research questions:
• Is it possible to measure the knowledge economy?• What are the drivers of the knowledge economy?• How does knowledge economy relate to other
complex dimensions?• Is it possible to reduce the total number of individual
indicators of KEI conceptual framework without loosing any relevant information?
• Are rankings useful at all for deriving policy suggestions?
![Page 45: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/45.jpg)
45
Figure 1. KEI Conceptual framework of a Knowledge-based Economy
Knowledge-based Economy Index
A1. Production and
diffusion of information and communication
technology (ICT)
A2. Human resources, skills and creativity
A3. Knowledge
production and diffusion
B2. Social performance
A4. Innovation,
entrepreneurship and creative destruction
B1. Economic Outputs
C1. Internationalization
Economic impact of ICT (A1a) Internet use by firms (A1b)
Internet use by individuals (A1c) Government ICT (A1d)
General education (A2a) Human resource in S&T education (A2b)
Skills (A2c) Creativity (A2d)
Mobility (A2e)
Research and experimental development (A3a) Patents (A3b)
Bibliometrics (A3c) Knowledge flows (A3d)
Total investment in intangibles (A3e)
Entrepreneurship (A4a) Demand for innovative products (A4b)
Financing of innovation (A4c) Market innovation outputs (A4d)
Organisational indicators (A4e)
Income (B1a) Productivity (B1b) Employment (B1c)
Environmental (B2a) Employment and economic welfare (B2b)
Quality of life indicators (B2c)
Trade (C1a) Knowledge production and diffusion (C1b)
Economic structure (C1c) Human resources (C1d)
![Page 46: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/46.jpg)
46
1. Production and diffusion of ICT
2. Human resources, skills and creativity
3. Knowledge production and diffusion
4. Innovation, entrepreneurship and creative destruction
5. Economic outputs
6. Social performance
7. Internationalisation
![Page 47: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/47.jpg)
47
We do not trust one model to build the Knowledge Economy Index, thus we opt for a “multi-modellingprinciple”
![Page 48: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/48.jpg)
48
Methodological scenarios for the development of the KEI composite
Aggregation function Additive
(linear) Multiplicative (geometric averaging)
Non-compensatory multi-criteria analysis
All (total 29) • • • Sub-dimensions included One-at-a-time excluded • • •
All (total 7) • • • Dimensions included One-at-a-time excluded • • •
Preserved • • • Pillar Structure Not preserved • • • z-scores • • Min-max • • Normalisation Raw data • Factor analysis • • • Equal weighting • • • Weighting Data envelopment analysis •
The frequency matrix of a country’s rank in each of the sevendimensions and the overall KEI was calculated across the ~2,000 scenarios~2,000 scenarios.
![Page 49: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/49.jpg)
49
Ran
k 1
Ran
k 2
Ran
k 3
Ran
k 4
Ran
k 5
Ran
k 6
Ran
k 7
Ran
k 8
Ran
k 9
Ran
k 10
Ran
k 11
Ran
k 12
Ran
k 13
Ran
k 14
Ran
k 15
Ran
k 16
Ran
k 17
Ran
k 18
Ran
k 19
Ran
k 20
Ran
k 21
Ran
k 22
Ran
k 23
Ran
k 24
Ran
k 25
Ran
k 26
Ran
k 27
Ran
k 28
Ran
k 29
Sweden 54 46Denmark 55 30 14Luxembourg 36 4 14 25 4 7 7 4Finland 18 23 29 9 11 11USA 11 32 2 4 39 9 4Japan 4 7 18 32 36 4UK 2 5 16 38 39Netherlands 86 4 4 7Ireland 4 61 14 4 9 9Austria 18 50 18 7 7Belgium 11 4 11 57 16 2France 4 14 18 11 54EU15 4 57 39EU25 4 4 14 32 39 7Germany 7 79 4 7 4Slovenia 7 41 38 14Estonia 4 36 25 21 11 4Malta 7 13 9 21 23 27Cyprus 36 7 4 23 23 7Spain 4 4 32 25 29 7Czech. Rep. 4 7 30 39 5 7 7Latvia 20 36 11 21 7 5Italy 29 18 9 29 9 7Greece 4 4 4 29 18 21 7 14Lithuania 4 41 13 32 11Hungary 2 13 13 57 2 14Portugal 4 4 7 11 61 14Slovakia 4 7 18 71Poland 100Legend:Frequency lower 15%Frequency between 15 and 30%Frequency between 30 and 50%Frequency greater than 50%
Knowledge Economy Index
![Page 50: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/50.jpg)
50
Median and associated 5th and 95th percentiles for the rank distribution
3 34
56 6
89
1011
1213 13
15
1718 18
19 19
22 2223
24 2426
2728
29
10
4
8
12
16
20
24
28
Swed
enDe
nmar
kLu
xem
bour
gFi
nlan
dUS
AJa
pan
Unite
d Ki
ngdo
mNe
ther
land
sIre
land
Aust
riaBe
lgiu
mFr
ance
EU15
EU27
Ger
man
ySl
oven
iaEs
toni
aM
alta
Cypr
usSp
ain
Czec
h Re
p.La
tvia
Italy
Gre
ece
Lith
ouan
iaHu
ngar
yPo
rtuga
lSl
ovak
iaPo
land
2004
Kno
wle
dge
Econ
omy
Inde
x (ra
nk)
Same results as before, different way of presenting them
![Page 51: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/51.jpg)
51
Spearman rank correlation coefficients between KEI and its seven dimensions (median ranks across ~2,000 simulations)
Prod
uctio
n an
d di
ffus
ion
of IC
T
Hum
an re
sour
ces,
skill
s and
cr
eativ
ity
Kno
wle
dge
prod
uctio
n an
d di
ffus
ion
Inno
vatio
n, e
ntre
pren
eurs
hip
and
crea
tive
dest
ruct
ion
Econ
omic
out
puts
Soci
al p
erfo
rman
ce
Inte
rnat
iona
lisat
ion
KEI 0.90 0.95 0.91 0.40 0.68 0.78 0.63 Production and diffusion of ICT 0.87 0.88 0.42 0.54 0.58 0.41 Human resources, skills and creativity 0.90 0.44 0.60 0.68 0.53 Knowledge production and diffusion 0.37 0.42 0.69 0.55 Innovation, entrepreneurship and creative destruction 0.17 0.06 0.04 Economic outputs 0.59 0.48 Social performance 0.61 Internationalisation 1.00 All coefficients are significant ( 29,05.0 =< np ).
All dimensions play a role but surely the less influential seems to be the ‘innovation, entrepreneurship and creative destruction dimension’.
![Page 52: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/52.jpg)
52
•
atbe
cycz
dedk
ee
esEU15EU25
fifr
gr
hu
ie
it jp
lt
lu
lv
mt
nl
pl
pt
se
si
sk
uk
0
50
100
150
200
250
1357911131517192123252729
KEI (median rank)
Gro
ss d
omes
tic p
rodu
ct p
er c
apita
in
purc
hasi
ng p
ower
sta
ndar
ds (P
PS)
Is KE relevant at all for a good overall economic performance?
The answer seems to be “YES”. Except Germany, Italy and Spain which have a high level of GDP per capita without any particular good performance on a KE.All the other high level GDP countries seem to perform well in the KEI composite (Luxembourg probably an outlier)
![Page 53: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/53.jpg)
53
at
be
cy
cz
de
dkeees
EU15EU25
fi
frgr
hu
ie
it
jp
lt
lu
lv mt
nlpl
pt
se
sisk
uk
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
1357911131517192123252729
KEI (median rank)
New
PhD
s (p
er th
ousa
nd p
opul
atio
n ag
ed 2
5-34
)
at
be
cy
cz
de dk
ee
es
EU15EU25
fi
fr
gr
hu
ie
it
jp
lt
lu
lv
mtnl
pl
pt
se
si
sk
uk
0.0
0.1
0.1
0.2
0.2
0.3
0.3
1357911131517192123252729
KEI (median rank)
Per
cent
age
of w
orki
ng a
ge p
opul
atio
n (1
5-64
) w
ith a
tert
iary
edu
catio
n (IS
CED
5 a
nd a
bove
)
at
becy
czde
dk
eees
EU15EU25
fi
fr
gr
hu
ieit
jp
lt
lulv
mt
nl
pl pt
si
sk
uk
0.0
0.1
0.1
0.2
0.2
0.3
0.3
1357911131517192123252729
KEI (median rank)
Par
ticip
atio
n in
life
long
lear
ning
per
wor
king
ag
e po
pula
tion
(25-
64)
While the number of Ph.Ds seems to play a role in explaining the success of a KE (all countries with an high number of Ph.Ds, except Portugal, are top countries in the KEI median ranking), the same result does not apply to percentage of working population with a tertiary education (very clear the case of Italy where the number of working population with a tertiary education is extremely high, but the number of Ph.Ds is small). Participation to life long learning seems also to be a success factor, although not for all top countries.
In sum, the human capital theory seems to be corroborated by the KEI results roughly.
PhD
LLL
Tertiary Ed.
![Page 54: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/54.jpg)
54
at
becy
cz
de dkee
es
EU15EU25
fi
fr
gr
hu
ieit
jp
ltlu
lv
mtnl
plpt
se
si
sk
uk
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
1357911131517192123252729
KEI (median rank)
Gro
ss d
omes
tic e
xpen
ditu
re o
n re
sear
ch a
nd
expe
rimen
tal d
evel
opm
ent (
GER
D) t
o gr
oss
dom
estic
pro
duct
(GD
P)
Countries which invest in research are top countries in a KBE. Research is a key driver for a KBE surely, thus the endogenous growth idea (the Schumpeterian theory) seems also corroborated.
Checking the Schumpeterian theory:
‘Gross domestic expenditure on research and experimental development’relates with KEI median.
![Page 55: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/55.jpg)
55
at
be
cycz de
dk
eees
EU15
EU25fi
frgr
hu
ie
it jp
ltlu
lv
mt
nl
pl
pt
se
si
skuk
40
45
50
55
60
1357911131517192123252729
KEI (median rank)
Env
ironm
enta
l Per
fora
mnc
e (B
2A, s
core
)
These arguments seem corroborated by the KEI measure. In fact, as one can see, no clear relationship between environmental performance and a KBE exists.
Is a KE an eco-efficient one?The Jevons’ paradoxteaches us that an ▲ in efficiency in using a resource leads, in the medium to long term, to an ▲ in consumption of that resource (rather than a ▼). No Kuznets curve …
![Page 56: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/56.jpg)
56
at
be
cy
cz
de
dk
ee
es EU15EU25
fi
fr
gr
hu
ie
it
jp
lt
lu
lv
mt
nl
pl
pt
se
si
sk
uk
0.00
0.02
0.04
0.06
0.08
0.10
0.12
1357911131517192123252729
KEI (median rank)
Lon
g te
rm u
nem
ploy
men
t rat
e
Does a KE influence unemployment?
In the short term technological progress and job creation are conflicting objectives (Kok report on Europe) but they might be compatible in the long period if a right balance (i.e. compromise) between flexibility and employment security is found.
Compatibility between technological progress and job creation. All top countries in the KEI measure are presenting an extremely low long term unemployment rate.
![Page 57: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/57.jpg)
57
atbe
cycz
de
dk
ee
es
fi
fr
gr
hu
ie
it
jp
lt
lu
lv
mt
nl
pl
pt
se
si
sk
uk
1
5
9
13
17
21
25
29
33
37
41
45
49
1357911131517192123252729
KEI (median rank)
Hum
an D
evel
opm
ent I
ndex
External benchmark:Human Development Index.
The relationship found seems again a corroboration of the human capital theories, in fact the correlation between KEI and the HDI is extremely high (though possibly spurious!).
![Page 58: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/58.jpg)
58
•JRC‐IPTS (Seville) work on companies R&D expenditures scoreboard.
![Page 59: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/59.jpg)
![Page 60: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/60.jpg)
60
- The EU Industrial R&D Investment Scoreboard: analysis of 1000 EU and 1000 non-EU top investing companies in R&D
- The EU Survey of Business R&D- Economic and policy analysis of corporate R&D.
Industrial Research and Innovation at JRC - Seville http://iri.jrc.ec.europa.eu/
![Page 61: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/61.jpg)
61
Some Results (1): Nature of the R&D investment gap
88 % 12 %EU - US
Sectoral composition effect Underinvestment effect
Total: -1.8%
Breakdown of EU-US gap in R&D intensity into sectoral composition and underinvestment effects.
Note: In this figure, only companies of similar R&D size are considered, i.e. the top EU (391) and the US (563) with R&D investment above a common threshold (€23 million in the latest year).Source: JRC-IPTS calculations (2008) - Analysis of 2007 EU Industrial R&D Investment Scoreboard
- EU's R&D intensity deficit is largely explained by the different industrial structure (sectoral composition effect).
Industrial Research and Innovationhttp://iri.jrc.ec.europa.eu/
![Page 62: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/62.jpg)
62
- R&D rises productivity much more in high-tech sectors than in low-tech ones
Some Results (2): Econometrics of R&D & firm productivity
Industrial Research and Innovation at JRC Seville
-6-4
-20
-6-4
-20
-8 -6 -4 -2 0 -8 -6 -4 -2 0
High Low
Medium Total
Pro
duct
ivity
/Em
ploy
ee
R&D stock/employeeGraphs by R&D Intensity sector groups
R&D Stock/Employee vs. Productivity/Employee
Source: European Commission, JRC –IPTS (2008) Analysis of 2007 EU Industrial R&D Investment Scoreboard
![Page 63: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/63.jpg)
63
Industrial Research and Innovationhttp://iri.jrc.ec.europa.eu/
More info on the R&D investment scoreboard from
![Page 64: Composite indicators. Methodology and Applications to R&D ...rosted.nu/attachments/File/2008/Indicators... · Ideally, a composite indicator should be based on: ‐Solid theoretical](https://reader034.vdocuments.site/reader034/viewer/2022050407/5f84622690fc484f3d341fe0/html5/thumbnails/64.jpg)
64
More reading at (just google composite indicators):
http://composite-indicators.jrc.ec.europa.eu