The performance

of the public sector

Pierre PestieauCREPP, University of Liège,

CORE, PSE and CEPR

2

Outline

1. Introduction

2.The performance approach and the concept of best practice

3. Measuring productive efficiency

4. The performance of social protection

5. Conclusion

3

1. Introduction

Measuring and ranking: a must

People do it anyway but badly Transparency and governance Yardstick competition – Open Method of Coordination (OMC)

Important distinction between the public sector as a whole and its components

Problem of aggregation Technical link between outcomes (outputs) and resources (inputs)

The performance is to be measured by the extent to which the preassigned objectives are achieved.

4

2. The performance approach and the concept of best practice

The public sector is a set of more or less aggregated production units (social security administration, railways, health care, education, national defence, social protection,…)

Each unit is supposed to use a number of resources, within a particular setting, to produce a number of outputs

Those outputs are related to the objectives that have been assigned to the production unit by the principal, the authority in charge

Approach used here: productive efficiency and to measure it, the efficiency frontier technique is going to be used

5

Productive efficiency is just a part of an overall performance analysis. It has two advantages:

It can be measured It is a necessary condition for any other type of

objectives

Main drawback: it is relative

Based on a comparison among a number of rather similar production units

Its quality depends on the quality of the observation units.

6

Set of observations Best practice frontier

Non parametric method: DEA (data envelopment analysis)

Parametric method

Comparative advantage

Illustration with one input/one output

7

output

input

Set of comparable observations

Figure 1

8

output

input

Parametric

Figure 2

9

output

input

Non Parametric

Figure 3

10

output

input

bc

A B

a

t+1

t

Figure 4

11

Technical progress:

Efficiency in t:

in t + 1:

aA

A

bB

B

Change in efficiency: ca -

12

Motivation of efficiency study: performance improvement

Factors of inefficiency:

Exogenous (location) Endogenous (low effort) Policy related (ownership, competition)

13

3. Measuring productive efficiency. Conceptual and data problems

Two problems.

Weak link between the inputs used and the expected outcomes

Confusion between lack of data and conceptual difficulties

Research strategy. Two areas quite typical of public spending: education and railways transports; how performance should be measured if data availability were not a constraint?More precisely, when listing the outputs and the inputs, assume that the best evidence one can dream of is available.

14

3.1. The best evidence

Inter-country comparison.

Importance of institutional, political and

geographical factors.

15

Ideal data Outputs Passenger kilometres Comfort and punctuality Freight tons and kilometers - bulk - containers - others Delivery quality and punctuality Equity of access Passengers per seat Inputs Labor (disaggregated) Equipment (disaggregated by type and by

quality) Tracks (length and quality) Energy (sources) Environment Geography, stage length Autonomy Competition or contestability Price discrimination Community service obligation Observations Very large number of years and countries

Railways

16

Ideal data

Output Acquired skills (of sample of 18y. old individuals)

- math, science, reading - foreign languages Direct employability Indirect employability (through college) Happiness Contribution to R and D

Input Teachers (level and quality) Staff Building, equipment Spatial distribution of schools Skills at the end of the primary education level

Environment Competition between networks Competition with private schools Role of the family Unemployment rate, economic growth Pedagogical technique

Observations Large number of countries and years

High schools

17

3.2. Actual studies

Most qualitative variables are missing.

Difference between developed and less developed countries.

Focus on financial variables.

18

Ideal data Data used

in recent studies Outputs Passenger kilometres v Comfort and punctuality ~ Freight tons and kilometers v - bulk ~ - containers ~ - others ~ Delivery quality and punctuality ~ Equity of access – Passengers per seat ~ Inputs Labor (disaggregated) v Equipment (disaggregated by type and by

quality) v

Tracks (length and quality) ~ Energy (sources) ~ Environment Geography, stage length ~ Autonomy ~ Competition or contestability ~ Price discrimination ~ Community service obligation ~ Observations Very large number of years and countries Too small

Note: v = OK; ~ = more or less; – = unavailable

Railways

19

Ideal data Recent studies

Output Acquired skills (of sample of 18y. old individuals)

- math, science, reading v - foreign languages – Direct employability – Indirect employability (through college) ~ Happiness – Contribution to R and D ~

Input Teachers (level and quality) ~ Staff ~ Building, equipment v Spatial distribution of schools – Skills at the end of the primary education level –

Environment Competition between networks Competition with private schools ~ Role of the family – Unemployment rate, economic growth ~ Pedagogical technique ~

Observations Large number of countries and years

~

Note: v = OK; ~ = more or less; – = unavailable

Education

20

Sector and authors Number of units

Type and period of data

Number of outputs and

inputs Method

Mean efficiency degrees

Remarks and other findings

EducationRhodes and South-wick (1988)

64 public and 57 private

universities in US

Panel annual 1971, 1974,

1981

5 outputs5 inputs

Non parametric

About 88% a year

- Private universities have slightly hither efficiency scores, for everyyear considered

RailwaysOum & Yu (1991)

21 railways companies

Annual data 1 output1978-1988

Parametric 1 each year - Limited evidence has been found for a relationship between the share of state in capital and cost efficiency- Positive correlation appears between cost efficiency and the importance of the cantons’s participation in the deficit of firms

Filippini & Maggi (1991)

57 railways under mixed ownership

Annual data1985-1988

1output3 inputs

+2 network characteristics

Non parametric

81% - Tendered services have higher efficiency scores that non-tendered ones.

Productive efficiency comparative studies of public and private firms

21

Is it worth the amount of time?Yes, but with caution

Technical efficiency is just one aspect of efficiency.

Lack of quantitative variables may distort the results.

For education importance of employability.

22

4. Measuring the performance of the public sector as a whole

Ideally:

Data on happiness (average and distribution) with and without social protection or at least on how the welfare state fulfils its objectives: health, education, employment, poverty alleviation, inequality reduction;

Data on inputs.

23

Actually:

Data on indicators of social inclusion (or exclusion);

Data on social spending.

24

Three issues:

Aggregation: DEA or SPI,

Scaling: (0,1) or average or goalposts,

Use of inputs: performance versus

inefficiency.

25

Table 1: Indicators of exclusion. Definition and correlations

Definition

POV At-risk-of-poverty rate

INE Inequality

UNE Long term unemployed

EDU Early school leavers

EXP Life expectancy

Correlation

POV INE UNE EDU EXP

POV 1.000

INE 0.912 1.000

UNE 0.420 0.409 1.000

EDU 0.668 0.782 0.252 1.000

EXP -0.069 -0.098 0.084 -0.203 1.000

Source: The five indicators are taken from the Eurostat database on Laeken indicators (2007).

26

Table 2: HDI normalization and SPI1 - 2004

POV INE UNE EDU EXP SPI1 Rank AT 0.80 0.87 0.93 0.99 0.57 0.83 2 BE 0.60 0.82 0.33 0.89 0.53 0.63 8 DE 0.50 0.72 0.04 0.88 0.58 0.54 10 DK 1.00 0.97 0.96 1.00 0.07 0.80 4 ES 0.10 0.54 0.48 0.25 0.91 0.46 13 FI 1.00 0.95 0.76 0.99 0.00 0.74 6 FR 0.70 0.77 0.37 0.82 0.87 0.70 7 GR 0.10 0.31 0.00 0.79 0.51 0.34 14 IE 0.00 0.56 0.87 0.86 0.35 0.53 11 IT 0.20 0.41 0.35 0.55 1.00 0.50 12 LU 1.00 0.90 0.98 0.86 0.35 0.82 3 NL 0.90 0.82 0.87 0.82 0.54 0.79 5 PT 0.00 0.00 0.57 0.00 0.00 0.11 15 SE 1.00 1.00 0.96 1.00 0.90 0.97 1 UK 0.30 0.49 1.00 0.79 0.47 0.61 9 Mean 0.55 0.68 0.63 0.77 0.51 0.63

27

Difference in shadow prices

SPI1 SPI2

POV -0.02 -0.03

INE -0.05 -0.04

UNE -0.04 -0.05

EDU -0.006 -0.010

EXP 0.06 -0.003

Correlation: 0.9Dependent on irrelevant alternatives.

SPI1 and SPI2

28

DEA with same input:- DEA1: 0.921- DEA2: 0.990

DEA is not invariant to non linear transformation.- DEA3: 0.992

29

Figure 1: DEA1 frontier

q1

q20

C

A

B

D

E

D*

E*

F*

F

30

DEA1 DEA2 DEA3 Scores rank Scores rank Scores rank

AT 0.995 7 0.988 9 0.999 7 BE 0.892 12 0.983 12 0.972 14 DE 0.886 13 0.984 10 0.975 13 DK 1.000 1 1.000 1 1.000 1 ES 0.939 8 0.997 7 0.996 8 FI 1.000 1 1.000 1 1.000 1 FR 0.937 9 0.997 7 0.995 9 GR 0.795 14 0.981 13 0.969 15 IE 0.900 10 0.976 14 0.995 10 IT 1.000 1 1.000 1 1.000 1 LU 1.000 1 1.000 1 1.000 1 NL 0.900 10 0.984 10 0.995 10 PT 0.565 15 0.959 15 0.980 12 SE 1.000 1 1.000 1 1.000 1 UK 1.000 1 1.000 1 1.000 1

Mean 0.921 0.990 0.992

Note: DEA1, DEA2 and DEA3 results correspond to HDI, Afonso et al. and “goalspot” normalization data respectively.

Table 3: DEA efficiency scores. 2004

31

SPI1 SPI2 SPI3 DEA1 DEA2 DEA3 SPI1 1.000 SPI2 0.894 1.000 SPI3 0.959 0.883 1.000 DEA1 0.801 0.643 0.750 1.000 DEA2 0.669 0.517 0.598 0.903 1.000 DEA3 0.583 0.576 0.405 0.679 0.656 1.000

Table 4: Correlations between indexes

32

Measuring performance or efficiency

Problem: weak link between social spending and education, health, unemployment.

Ranking modified

33

DEA1 DEA1 Scores rank Scores rank

AT 0.995 7 AT 0.917 8 BE 0.892 12 BE 0.809 12 DE 0.886 13 DE 0.769 13 DK 1.000 1 DK 0.824 11 ES 0.939 8 ES 1.000 1 FI 1.000 1 FI 0.943 6 FR 0.937 9 FR 0.924 7 GR 0.795 14 GR 0.752 14 IE 0.900 10 IE 1.000 1 IT 1.000 1 IT 0.988 5 LU 1.000 1 LU 1.000 1 NL 0.900 10 NL 0.864 9 PT 0.565 15 PT 0.444 15 SE 1.000 1 SE 1.000 1 UK 1.000 1 UK 0.825 10

Mean 0.921 Mean 0.871

Table 5 DEA efficiency scores without and with social expenditures as input. 2004

34

Race to the bottom?

Test of convergence SPI1 and Malmquist decomposition

35

SEFI

DK

DENL

ATFR

LUBEGR

UK

ITIE

PT

ES

y = -1.2741x + 1.0326

R2 = 0.8024

-1%

0%

1%

2%

3%

4%

5%

6%

7%

8%

9%

0,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0,9 1,0

SPI1 - 1995

Gro

wth

rat

e o

f S

PI1

(1

995-

2004

)Figure 6: Convergence of SPI1

36

ITES

GR UK

BE

FR DENL

IE

AT

DK

FILUPTSE

y = -0.0862x + 0.0853

R2 = 0.9468

-1%

0%

1%

2%

3%

4%

5%

0,4 0,5 0,6 0,7 0,8 0,9 1,0 1,1

DEA1 1995

Ave

rag

e E

ffci

cie

ncy

ch

ang

e 19

95-

2004

Figure 7: Convergence of DEA1 according to “technical efficiency” change

37

5. Conclusion Yes for efficiency measures when the

production technology is well understood.

Caution when the technology is unclear and environmental variables are missing.

For the welfare states, ranking performance is preferable.

DEA is to be preferred over SPI.

No clear guidelines on the choice of scaling.