National Accounts Working Party3-5 October 2007

Paris

OECD handbook on the measurement of volume output of health and education

Paul Schreyer, OECD/STDSandra Hopkins, OECD/ELS

Contents

• Background

• General concepts

• Education

• Health

• Way forward

Background: OECD Project

• Strong and continued demand for output measures of education and health by policy-makers

• European Regulation• Project started in 2005, endorsement by CSTAT• Builds on previous work: Eurostat Handbook on Volume

and Prices, Atkinson Report, country experiences• Cooperation with the UKCeMGA and Eurostat• Financial support by INSEE (France), Government of

Norway, United Kingdom• Workshops in London (2006) and Paris (2007)• Objectives:

– OECD Handbook by end 2008– Data development

Background: An old question – what is new?

1. Joint work with sector specialists

• Elaborated jointly with OECD’s specialised networks– Network of education experts– Network of health experts

• Both networks have strong interest in measuring appropriate volume output

Background: An old question – what is new?

2: Joint treatment of temporal and spatial dimensions

• Education and health PPPs are of great importance to analysts

• PPPs and national accounts have to be consistent

• Handbook deals with both dimensions in parallel

Background: An old question – what is new?

3: Joint treatment of non-market and market production

• Even for market producers of education and health services, price-volume splits are not obvious

• In particular, quality adjustment is difficult in both cases

• Handbook emphasises non-market production and volume indicators but not exclusively – the principles should be the same for market and non-market production

Concepts and terminology

• Distinction must be made between inputs, outputs, outcomes -

• Best explained by way of a graph

Inputs

Labour, capital, intermediate

inputs

Environmental factors

Information about outcome is a

possible tool for quality adjustment

Process without explicit quality

adjustment

Process with explicit quality

adjustment

Example education: number of

pupils/pupil hours by level of education

Example education: quality-adjusted number

of pupils/pupil hours by level of

education

Example health: number of complete

treatments by type of disease

Example health: quality-adjusted

number of complete

treatments by type of disease

Outputs

Information about outcome is a

possible tool for quality adjustment

Direct outcome

Indirect outcome

Knowledge and skills as measured by

scores

Health status of population

Future real earnings,

growth rate of GDP,

well-rounded citizens

etc.

Outcomes

Inhereted skills, socio-economic background, etc.

Hygene, lifestyle, infrastructure etc.

Inputs

Labour, capital, intermediate

inputs

Environmental factors

Information about outcome is a

possible tool for quality adjustment

Process without explicit quality

adjustment

Process with explicit quality

adjustment

Example education: number of

pupils/pupil hours by level of education

Example education: quality-adjusted number

of pupils/pupil hours by level of

education

Example health: number of complete

treatments by type of disease

Example health: quality-adjusted

number of complete

treatments by type of disease

Outputs

Information about outcome is a

possible tool for quality adjustment

Direct outcome

Indirect outcome

Knowledge and skills as measured by

scores

Health status of population

Future real earnings,

growth rate of GDP,

well-rounded citizens

etc.

Outcomes

Inhereted skills, socio-economic background, etc.

Hygene, lifestyle, infrastructure etc.

If outcome indicators are used for quality adjustment, they:

•Should control for any other factors that affect outcome for

consumers (e.g. socio-economic background of pupils,

environmental impact on health)•Should only capture marginal effect of process on outcome

Quality adjustment

• First step towards capturing quality change is the correct stratification, i.e., the comparison of products with the same or at least similar characteristics.

• However, matching of services has its limits.• Also, stratification should be able to capture effects of

substitution• However, avoid treating goods or services as substitutes

that are in fact different products• Explicit quality adjustment may make it necessary to

invoke outcomes

Cost and value weights: principles

• In a market context, changes in the price or quantity of products are weighted by their expenditure share reflecting relative valuation by consumers/producers

• In a non-market context, only cost observations are available and there is no guarantee that cost weights reflect relative valuation by consumers

Cost and value weights: principles

• 2 possibilities to deal with this problem:– Assume that on average, cost shares reflect also

relative valuation by consumers– Impute relative valuation by consumer but

• total value of non-market output ≠ costs;• difficult measurement issues;• asymmetry with regard to treatment of other products• not within the scope of national accounts although value

weights are useful for welfare analysis

• Handbook recommends use of cost weights

Cost and value weights: practice

• Note:– Compiling cost or value information in the

required classification is not a trivial task– Example: no data may be available on the

cost or value of medical care by disease because pricing mechanisms, or cost accounting are not defined over episodes of treatment

Education

ISIC rev 4 classes ISCED-97 levels of education 8510 Pre-primary and primary education Levels 0 and 1 8521 General secondary education Levels 2 and 3 oriented general 8522 Technical and vocational secondary education Levels 2 and 3 oriented vocational and technical 8530 Higher education Levels 4, 5 and 6 8541 Sports and recreation education

Not classified in ISCED-97 levels of education 8542 Cultural education 8549 Other education n.e.c.

8550 Educational support activities Not explicitly mentioned in ISCED-97 levels of education

For

mal

In

form

al

Sup

port

Scope of education services

Handbook covers only formal education servicesFocus is on secondary education

Inputs

Process without explicit quality

adjustment

Process with explicit quality adjustment

Direct outcome

Indirect outcome

Knowledge and skills as measured by

scores

Knowledge status of

population

Scope National accounts Welfare, policy analysis

Inhereted skills, socio-economic background, etc.

No of teachers, capital,

intermediate inputs

Future real earnings,

growth rate of GDP,

well-rounded citizens

etc.

Outcomes

Environmental factors

Outputs

Quality-adjusted activities/processes = transfer of skills and

knowledge by level of education

Number of pupils/pupil hours

by level of education, and

by other relevant characteristics

Information about scores as a tool for quality adjustment

Summary of proposed measures:1) Stratification

Minimum stratification Preferred stratification

Level 0 Pre-primary education All classes

Level 1 Primary education or first stage of basic education

Normal classes or pupils

Special classes or handicapped pupils

Level 2Lower secondary or second stage of basic education

Normal classes or pupils

Special classes or handicapped pupils

Level 3 Upper secondary educationGeneral + pre-vocational

Vocational

Level 4Post-secondary non-tertiary education

General / vocational if available

Summary of proposed measures:1) Stratification (contd)

Minimum stratification Preferred stratification

Level 5BMore practical and occupation-specific programmes tertiary education

All classes or by professional purpose

Level 5A + 6More theoretically-based programmes tertiary education

By fields of education and/or prestige of education unit, or by equivalences of degrees

Adult and other informal education

Adult and other education, anticipating extension of “education” content in ISIC rev 4, class 8540.

Adult general education

Adult vocational education

Computer training

Driving lessons

Music lessonsOther cultural and artistic lessonsSport lessons

Recreational lessons

Education support activitiesAccording to what will be retained in class 8550 of ISIC rev.4

Other education activities

Summary of proposed measures for education services: 2) variables

Stratum Quantity Quality (educational) CommentPre-primary education Pupil-hours NonePrimary education: normal

Pupils Contribution to scoresThe sub-stratification normal / special could be replaced by coefficients reflecting the extra costs for social services provided to handicapped pupils

Primary education: special or handicapped pupilsLower secondary: normalLower secondary: special or handicapped pupilsUpper secondary education: general + pre-technical or pre-vocational

Entry education status has to be controlled for, this can be with the help of a model

Upper secondary education: vocational

Relative future real earnings and employment rate if no scores available

Only incremental revenues must be considered - real earnings and employment rate “without teaching” have to be subtracted from total earnings or employment

Post-secondary non-tertiary educationMore practical and occupationally specific programmes tertiary educationMore theoretically based programmes tertiary education

Credits (ECTS) as 1st bestCombination of time-lagged degrees as 2nd bestEnrolled tudents as 3rd best

Differentiation by field of education Relative “value” of level of degrees could be estimated from labour market

Different concepts but close figures in practice.

Education services: conclusions and questions

• Stratification can go a long way towards constructing volume indices – but are process measures an acceptable proxy for a full quality adjustment?• A mix of quality-adjustment approaches is suggested in the Handbook – e.g., scores for secondary education, degrees or a human capital approach for tertiary education. Would a single approach be preferable?

Health services

1. Aggregation by disease or illness

Aggregation of quantities of services:• Health volume output can be measured at 2 levels:

disease or institution1. Aggregation by disease or illness • Ideally, health volume output should be measured by

complete treatments by disease as this is the product which an individual purchases from a health provider.

• Complete treatment refers to the pathway that an individual takes through heterogeneous institutions – offices of doctors, hospitals, medical laboratories etc. – in order to receive full and final treatment for a disease or condition.

1. Aggregation by disease or illness

Benefits: – “Our concern should be not where the money comes

from and where it goes but what it buys.” (Triplett 2001)

– The summing of points of contact with the health system to estimate a complete treatment means that if clinical practice changes over time, and is associated with a change in the cost of providing the service, this will be reflected in the output measure e.g movement to day-only surgery and non-invasive types of surgery.

1. Aggregation by disease or illness

Problems:– In SNA, total output of an activity is based on

summing up outputs of various service providers. Principle is directly applicable only if the service provider is the same during the whole treatment.

– Demanding data requirements e.g. linking patient treatment across providers, ability to determine the beginning & end point of treatment

– Cost of illness studies require disease specific price indexes for conversion into volumes. Difficult in a nonmarket system

2. Aggregation by institutions

Diagnosis Related Groups (DRGs) aggregate across a hospital treatment, usually acute episodes only

– There is no international DRG system

– Aggregation across other providers is problematic e.g. doctors, psychiatric hospitals etc.

2. Aggregation by institutions

• Development and harmonisation of classification systems is required to ensure improvements in compatibility and comparability of health volume output both temporally and spatially. Developments proposed include a classification of health care products and international harmonisation of DRG systems for both inpatients and outpatients.

2. Aggregation by institutions

• In the shorter run, it is possible to aggregate health volume output using currently existing DRG systems for hospital outputs, Resource Utilisation Groups for nursing home outputs and summing up activities in outpatient services.

Quality adjustment

• Ideally, health volume output should be adjusted for the improvement in health outcomes which are due to the introduction into the health industry of new treatments as well as improvements in the existing practices.

• Wealth of outcome measures and an ‘industry of quality measurement’ which compiles and records health outcomes, but at this stage the quality indicators which could be applied for adjustment to health volume output are rudimentary and under development

• Developments should include consensus on what indicators should be used for quality adjustment and the role of some quality issues, e.g. waiting times, on health outcomes. Choice of quality indicators should emphasise internationally comparable and consistent measurement.

Way forward

•Presentation of draft report to health experts next week•Possible input to Eurostat seminar November 2007•Revision of report, and inclusion of education PPPs•During 2007/08: work of Taskforce on Health PPPs (supported by European Commission)•End 2008: complete draft report

•2009 and beyond: OECD will seek mandate to begin empirical implementation