oecd world forum on key indicators statistics, knowledge and policy palermo, 10-13 november 2004...

OECD World Forum on Key IndicatorsOECD World Forum on Key Indicators

Statistics, Knowledge and PolicyStatistics, Knowledge and Policy

Palermo, 10-13 November 2004Palermo, 10-13 November 2004

““Indicators and benchmarking as a support Indicators and benchmarking as a support to the decision-making process: the Italian to the decision-making process: the Italian experience in active employment policiesexperience in active employment policies ””

Aviana BulgarelliAviana Bulgarelli

Director General for Vocational Training and Guidance Director General for Vocational Training and Guidance Policies, Ministry of Labour and Social Policies, Italy.Policies, Ministry of Labour and Social Policies, Italy.

Benchmarking in the public decision-making Benchmarking in the public decision-making processprocess

Benchmarking can be used for two types of tasks:Benchmarking can be used for two types of tasks:

for a coordinated, consistent and systematic for a coordinated, consistent and systematic analysis of performanceanalysis of performance, possibly tied to a , possibly tied to a system of targets/objectives, with the definition system of targets/objectives, with the definition of a set of significant and feasible indicators;of a set of significant and feasible indicators;

for an for an identification identification of the most effective of the most effective intervention intervention policiespolicies, models and instruments , models and instruments for achieving the targets and pinpointing for achieving the targets and pinpointing good/best practicesgood/best practices to be transferred in to be transferred in various contexts.various contexts.


Benchmarking processBenchmarking process

1.1. Identification of macro-objectives and corresponding Identification of macro-objectives and corresponding performances to measure the success of implemented performances to measure the success of implemented policies.policies.

2.2. Identification of a system of elementary variables/indicators Identification of a system of elementary variables/indicators related to the macro-objectives.related to the macro-objectives.

3.3. Comparison between the national/regional situations Comparison between the national/regional situations examined and the reference benchmark. examined and the reference benchmark.

4.4. Understanding and explaining reasons for the differences Understanding and explaining reasons for the differences observed in performance and identifying actions to be observed in performance and identifying actions to be undertaken.undertaken.

5.5. Monitoring policy implementation, analysing benchmarking Monitoring policy implementation, analysing benchmarking results and, possibly, correcting the entire process.results and, possibly, correcting the entire process.


Key problems in public policiesKey problems in public policies

multiple objectives and, at the same time, specific multiple objectives and, at the same time, specific intervention instruments and policies;intervention instruments and policies;

alternative objectives and problems to identify the alternative objectives and problems to identify the corresponding trade-offs;corresponding trade-offs;

difficulty finding connections between specific difficulty finding connections between specific policies/instruments available and the different objectives policies/instruments available and the different objectives identified;identified;

plurality of stakeholders contributing to the identification plurality of stakeholders contributing to the identification of objectives and to the implementation and monitoring of of objectives and to the implementation and monitoring of policies.policies.

Fixing objectives and guidelines for

the EU with clear time-tables Establishing

indicators and benchmarks to compare best-

practicesTranslating the guidelines into

national and regional policies

Periodic monitoring,

evaluation, peer review as a mutual learning process

European Employment Strategy (EES) and European Employment Strategy (EES) and benchmarkingbenchmarking

The open method The open method of coordinationof coordination

Benchmarking has often been used as a “surrogate” for Benchmarking has often been used as a “surrogate” for evaluation, affecting therefore the decision-making process.evaluation, affecting therefore the decision-making process.The EES is an example of multi-level strategic programming The EES is an example of multi-level strategic programming and represents the reference framework of a complex and represents the reference framework of a complex benchmarking exercise based on the benchmarking exercise based on the open method of open method of coordinationcoordination..

European Employment Strategy (EES) and European Employment Strategy (EES) and benchmarkingbenchmarking

Example from the guidelines for the employment policies Example from the guidelines for the employment policies of the Member States in the EES (Council Decision of 22 of the Member States in the EES (Council Decision of 22

July 2003)July 2003)

Overall employment rate: 67 % in 2005 and 70 % in 2010Overall employment rate: 67 % in 2005 and 70 % in 2010

Employment rate for women: 57 % in 2005 and 60 % in 2010Employment rate for women: 57 % in 2005 and 60 % in 2010

Employment rate for older workers (55 to 64): 50 % in 2010Employment rate for older workers (55 to 64): 50 % in 2010

Overarching objective: full employmentOverarching objective: full employment

Specific guideline: 4. Promote development of human Specific guideline: 4. Promote development of human capital and lifelong learningcapital and lifelong learning

By 2010:By 2010:at least 85 % of 22-year olds in the EU should have at least 85 % of 22-year olds in the EU should have completed upper secondary educationcompleted upper secondary education

the EU average level of participation in lifelong learning the EU average level of participation in lifelong learning should be at least 12,5 % of the adult working-age should be at least 12,5 % of the adult working-age population (25 to 64 age group).population (25 to 64 age group).

European Employment Strategy (EES) and European Employment Strategy (EES) and benchmarking: the Italian experiencebenchmarking: the Italian experience

EES process EES process Maastricht Stability Pact process Maastricht Stability Pact process

but with a greater degree of freedom:but with a greater degree of freedom:

Social and labour policies are more complex in terms Social and labour policies are more complex in terms of objectives, target population and effectiveness of of objectives, target population and effectiveness of intervention toolsintervention tools

Complex decision-making mechanism involving a large Complex decision-making mechanism involving a large number of stakeholders at different decision-making number of stakeholders at different decision-making levels with high political, and managerial responsibilitylevels with high political, and managerial responsibility

However, the EES has had an influence on the action of policy-However, the EES has had an influence on the action of policy-makers:makers:

at a national levelat a national level

at a regional levelat a regional level

at a local/provincial level in some casesat a local/provincial level in some cases

Reform process of Reform process of labour market in labour market in

ItalyItaly

stronger emphasis on specific targets (female and older stronger emphasis on specific targets (female and older population) even when the priority does not fully population) even when the priority does not fully correspond to local labour market conditionscorrespond to local labour market conditions

multiplicity of stakeholders at many decision-making multiplicity of stakeholders at many decision-making levels with strong vertical and horizontal subsidiarity (the levels with strong vertical and horizontal subsidiarity (the statistical information system reflects this complexity)statistical information system reflects this complexity)

the objective of complying with the Lisbon benchmark has the objective of complying with the Lisbon benchmark has required the creation of many technical units involving a required the creation of many technical units involving a large number of stakeholderslarge number of stakeholders

Implementation of Implementation of labour monitoring labour monitoring

systemsystem


Influence of EES and Influence of EES and EU benchmarking modelEU benchmarking model


Redirecting policies: the case of lifelong learning (I)Redirecting policies: the case of lifelong learning (I)

Lisbon benchmark: 12.5% by 2010Lisbon benchmark: 12.5% by 2010Italy: 4.7% participation rate in 2003Italy: 4.7% participation rate in 2003EU average: 9.7% participation rate in 2003EU average: 9.7% participation rate in 2003

Indicator for lifelong learning = participation rate of population Indicator for lifelong learning = participation rate of population aged 24-65 in permanent and continuing trainingaged 24-65 in permanent and continuing training

Italian context: Italian context: strong disparitiesstrong disparities among generations and among generations and among regions (especially in southern Italy) among regions (especially in southern Italy) ++ time and time and budget constraints.budget constraints.

Problem:Problem:

Significant impact on policy-makers, but little room for Significant impact on policy-makers, but little room for manoeuvre manoeuvre

shifting resources towards lifelong learning policies to the shifting resources towards lifelong learning policies to the detriment of alternative intervention policies would have detriment of alternative intervention policies would have political and organisational consequences.political and organisational consequences.


Redirecting policies: the case of lifelong learning (II)Redirecting policies: the case of lifelong learning (II)

According to Eurostat’s continuing vocational training survey According to Eurostat’s continuing vocational training survey (CVTS2), (CVTS2), 24%24% of Italian enterprises carry out training of Italian enterprises carry out training activities compared to an EU average of activities compared to an EU average of 62%.62%.

Key factor:Key factor: majority of small and medium-sized enterprises majority of small and medium-sized enterprises and traditional sectors, with little interest to offer training to and traditional sectors, with little interest to offer training to employees:employees:

How to make the most of available How to make the most of available resources?resources?

Investing in human capital has no tangible effects in the short Investing in human capital has no tangible effects in the short termterm

Greater incidence of training costs (both direct and indirect)Greater incidence of training costs (both direct and indirect)Smaller organisational and logistical capacitySmaller organisational and logistical capacityUncertainty regarding returns of trainingUncertainty regarding returns of training

Introduce new forms of interventions rather Introduce new forms of interventions rather than financial incentivesthan financial incentives


Redirecting policies: the case of lifelong learning (III)Redirecting policies: the case of lifelong learning (III)

Creation of the Bilateral funds for continuing vocational Creation of the Bilateral funds for continuing vocational training – training – Fondi paritetici interprofessionali per la Fondi paritetici interprofessionali per la

formazione continua formazione continua

Direct involvement of social partnersDirect involvement of social partners

(managing of financial resources, planning and (managing of financial resources, planning and directing interventions, organising and directing interventions, organising and integrating local and sector needs, monitoring integrating local and sector needs, monitoring the outcomes of activities)the outcomes of activities)

encouraging a greater dissemination of the training encouraging a greater dissemination of the training culture and activities (especially with regards to culture and activities (especially with regards to smaller-sized enterprises) smaller-sized enterprises)

steering financing towards initiatives more in line with steering financing towards initiatives more in line with the actual requirements of companiesthe actual requirements of companies

Aims:Aims:


Influence of benchmarking at a local levelInfluence of benchmarking at a local level

The experience of The experience of Local Action Plans for employment Local Action Plans for employment is is a typical example, where many local authorities have set up a typical example, where many local authorities have set up integrated strategic programming tools for training, integrated strategic programming tools for training, education and labour, within the framework of EES education and labour, within the framework of EES guidelines.guidelines.

The The Italian ProvincesItalian Provinces are required to play a leading role in are required to play a leading role in active labour policies and have also been involved in the active labour policies and have also been involved in the benchmarking system proposed with the EES.benchmarking system proposed with the EES.

Indicators, benchmarking and evaluationIndicators, benchmarking and evaluation

Main methodological and technical problemsMain methodological and technical problemsCoherence and comparabilityCoherence and comparability: better no comparison than : better no comparison than a “wrong” one a “wrong” one (e.g.: LLL indicator of Continuous Labour Force (e.g.: LLL indicator of Continuous Labour Force Survey Survey poor statistical visibility of short training courses) poor statistical visibility of short training courses)

RelevanceRelevance: in a multi-level programming system, important : in a multi-level programming system, important to understand “who has to decide what” and to provide to understand “who has to decide what” and to provide suitable cognitive support suitable cognitive support (e.g.: Local Action Plans in Italy)(e.g.: Local Action Plans in Italy)

Timeliness 1Timeliness 1: indicators reflect an “out-of-date” vision of the : indicators reflect an “out-of-date” vision of the economy economy always always ex-postex-post and often available after a long and often available after a long delay delay (e.g.: monitoring process of EES implementation)(e.g.: monitoring process of EES implementation)

Timeliness 2Timeliness 2: acceleration of economic processes requires a : acceleration of economic processes requires a switch from reactive to proactive approach switch from reactive to proactive approach Can we Can we construct a common system (methodologies and/or construct a common system (methodologies and/or parameters) for estimating indicators?parameters) for estimating indicators?

Transferability:Transferability: differences between contexts in which differences between contexts in which policies are implemented can have a great impact on their policies are implemented can have a great impact on their effectivenesseffectiveness

Indicators, benchmarking and evaluationIndicators, benchmarking and evaluation

Relationship between benchmarking and policy evaluationRelationship between benchmarking and policy evaluationHow can we obtain information actually enabling to steer How can we obtain information actually enabling to steer policy choices? policy choices?

How can we overcome theHow can we overcome the ex-post ex-post nature of benchmarking?nature of benchmarking?

How can we haveHow can we have a system consistent with the specific level a system consistent with the specific level of the decision-making process?of the decision-making process?

DebateDebate::

Can we raise benchmarking from a tool for comparing and Can we raise benchmarking from a tool for comparing and analysing policy performance to an input for the evaluation analysing policy performance to an input for the evaluation process? process?

oecd world forum on key indicators statistics, knowledge and policy palermo, 10-13 november 2004...

Documents