quasi-market reforms in employment and training services - cedefop

Quasi-market reforms in employment and trainingservices: first experiences and evaluation results

Ludo Struyven, Geert Steurs

In:

Descy, P.; Tessaring, M. (eds)

Evaluation of systems and programmesThird report on vocational training research in Europe: background report.

Luxembourg: Office for Official Publications of the European Communities, 2004(Cedefop Reference series, 57)

Reproduction is authorised provided the source is acknowledged

Additional information on Cedefop’s research reports can be found on:http://www.trainingvillage.gr/etv/Projects_Networks/ResearchLab/

For your information:

• the background report to the third report on vocational training research in Europe contains originalcontributions from researchers. They are regrouped in three volumes published separately in English only.A list of contents is on the next page.

• A synthesis report based on these contributions and with additional research findings is being published inEnglish, French and German.

Bibliographical reference of the English version:Descy, P.; Tessaring, M. Evaluation and impact of education and training: the value of learning. Thirdreport on vocational training research in Europe: synthesis report. Luxembourg: Office for OfficialPublications of the European Communities (Cedefop Reference series)

• In addition, an executive summary in all EU languages will be available.

The background and synthesis reports will be available from national EU sales offices or from Cedefop.

For further information contact:

Cedefop, PO Box 22427, GR-55102 ThessalonikiTel.: (30)2310 490 111Fax: (30)2310 490 102E-mail: [email protected]: www.cedefop.eu.intInteractive website: www.trainingvillage.gr

http://www.trainingvillage.gr/etv/Projects_Networks/ResearchLab/

www.cedefop.eu.int

www.trainingvillage.gr

Contributions to the background report of the third research report

Impact of education and training

Preface

The impact of human capital on economic growth: areviewRob A. Wilson, Geoff Briscoe

Empirical analysis of human capital development andeconomic growth in European regionsHiro Izushi, Robert Huggins

Non-material benefits of education, training and skillsat a macro levelAndy Green, John Preston, Lars-Erik Malmberg

Macroeconometric evaluation of active labour-marketpolicy – a case study for GermanyReinhard Hujer, Marco Caliendo, Christopher Zeiss

Active policies and measures: impact on integrationand reintegration in the labour market and social lifeKenneth Walsh and David J. Parsons

The impact of human capital and human capitalinvestments on company performance Evidence fromliterature and European survey resultsBo Hansson, Ulf Johanson, Karl-Heinz Leitner

The benefits of education, training and skills from anindividual life-course perspective with a particularfocus on life-course and biographical researchMaren Heise, Wolfgang Meyer

The foundations of evaluation andimpact research

Preface

Philosophies and types of evaluation researchElliot Stern

Developing standards to evaluate vocational educationand training programmesWolfgang Beywl; Sandra Speer

Methods and limitations of evaluation and impactresearchReinhard Hujer, Marco Caliendo, Dubravko Radic

From project to policy evaluation in vocationaleducation and training – possible concepts and tools.Evidence from countries in transition.Evelyn Viertel, Søren P. Nielsen, David L. Parkes,Søren Poulsen

Look, listen and learn: an international evaluation ofadult learningBeatriz Pont and Patrick Werquin

Measurement and evaluation of competenceGerald A. Straka

An overarching conceptual framework for assessingkey competences. Lessons from an interdisciplinaryand policy-oriented approachDominique Simone Rychen

Evaluation of systems andprogrammes

Preface

Evaluating the impact of reforms of vocationaleducation and training: examples of practiceMike Coles

Evaluating systems’ reform in vocational educationand training. Learning from Danish and Dutch casesLoek Nieuwenhuis, Hanne Shapiro

Evaluation of EU and international programmes andinitiatives promoting mobility – selected case studiesWolfgang Hellwig, Uwe Lauterbach,Hermann-Günter Hesse, Sabine Fabriz

Consultancy for free? Evaluation practice in theEuropean Union and central and eastern EuropeFindings from selected EU programmesBernd Baumgartl, Olga Strietska-Ilina,Gerhard Schaumberger

Quasi-market reforms in employment and trainingservices: first experiences and evaluation resultsLudo Struyven, Geert Steurs

Evaluation activities in the European CommissionJosep Molsosa

Quasi-market reforms in employment and training services: first experiences

and evaluation resultsLudo Struyven, Geert Steurs (1)

AbstractIn this paper we focus on quasi-market reforms for employment and training services for job seekers.Open tendering is the standard form for the introduction of market forces in several countries, with acompetitive bidding process used to determine with which provider an order is placed for a particularservice. It is an alternative to the more traditional method of subsidising providers, where a country’s owngovernment department/service, in a protected legal position, receives a guaranteed share of govern-ment funding. The focus on quasi-market reforms means that we will not cover the broader evaluationissues of active labour-market and training policies, in so far as they are disconnected from the mode ofservice delivery. Quasi-market changes are only just being put into place but this does not mean that anykind of evaluation is impossible. We start from a theory of quasi-markets as developed by Le Grand andBartlett (1993) which specifies the conditions quasi-markets have to meet if they are to succeed. In ouranalysis, we will make a preliminary assessment of the extent to which those conditions appear to be metin practice.

(1) Ludo Struyven is working as a research manager in the Education and Labour Market Section at the Higher Institute of LabourStudies, Catholic University of Leuven, Van Evenstraat 2E, B-3000 Leuven; e-mail: [email protected]. Geert Steurs is working as a project manager at IDEA Consult, Congresstraat 37-41, bus 3, B-1000 Brussels; e-mail:[email protected].

Table of contents

1. Executive summary 223

2. Introduction 225

3. The role of quasi-markets 226

3.1. The economic rationale for competitive service delivery 226

3.2. Definition of quasi-markets 226

3.3. Quasi-markets as allocation model of service delivery 226

3.4. Quasi-markets: do they work? 228

3.5. Linking contestability and market forces to the activation of benefit claimants 229

4. Overview of reforms 230

4.1. The four countries at a glance 230

4.2. Objectives of the reforms 231

4.2.1. Outcomes and costs 231

4.2.2. Choice 231

4.2.3. Other motives 232

4.3. Market forces in the broader institutional and policy context 232

4.3.1. The Swedish context of the 1980s 232

4.3.2. Australia, the Netherlands and the UK: institutional reforms in

the second half of the 1990s 232

4.3.2.1. Relationship between functions 232

4.3.2.2. Scope for market forces in implementing functions 233

4.3.2.3. Market forces and the concept of the welfare state 233

4.3.3. Conclusion 234

5. Evaluation framework for quasi-market mechanisms 236

5.1. Criteria for summative evaluation 236

5.1.1. Efficiency 237

5.1.2. Responsiveness 237

5.1.3. Quality 237

5.1.4. Choice 237

5.1.5. Equity 238

5.2. Criteria for formative evaluation 238

5.3. Available evaluation studies 240

5.3.1. Australia 240

5.3.2. The Netherlands 241

5.4. Summary 242

6. Formative evaluation 243

6.1. Market structure 243

6.1.1. Separation between purchaser and provider 243

6.1.2. Provider competition 243

6.1.2.1. The relationship between tendering procedures and provider competition 243

6.1.2.2. Price versus quality competition 244

221

6.1.3. Player’s market 245

6.1.3.1. Concentration and increases in scale 246

6.1.3.2. Difficult entry to the market 247

6.1.3.3. The position of the former public employment service 248

6.1.4. The position of the purchaser 248

6.1.4.1. Degree of monopoly at the purchaser side 248

6.1.4.2. Degree of centralisation at the purchaser side 249

6.2. Information and transparency 250

6.2.1. Lack of transparency 250

6.2.2. Contractual compliance monitoring in Job Network 250

6.2.3. Performance measurement in Job Network 251

6.3. Transaction costs 252

6.4. Motivation: the combination of input and output financing 253

6.4.1. Input and output financing 253

6.4.2. The Dutch payment system 253

6.4.3. Output financing and training 255

6.4.4. Impact of the outcome-based focus on provider behaviour 255

6.5. Creaming and parking 256

7. Summative evaluation 257

7.1. Efficiency 257

7.2. Responsiveness and choice 258

7.2.1. Application of the job seeker classification instrument 258

7.2.2. Choice of provider 259

7.2.3. Client satisfaction 259

7.3. Quality 259

7.3.1. Services of the Australian Job Network 260

7.3.2. Training within Job Network 260

7.3.2.1. The amount of training 260

7.3.2.2. Outcome based payment and incentives to provide training 261

7.4. Net impact and equity of the Australian Job Network 261

7.4.1. Gross outcomes 261

7.4.2. Sustainability of outcomes 262

7.4.3. Before-and-after comparisons 263

7.4.4. Provider comparisons 263

7.4.5. Net impact and equity 263

8. Conclusions 265

8.1. Assessment of evaluation strategy 265

8.2. Main findings 265

8.3. Lessons for other countries 268

List of abbreviations 269

References 270

Quasi-market reforms in employment and training services: first experiences and evaluation results

List of tables and figures

TablesTable 1: The key functions in employment placement and spread among government organisations

(situation in 2002) 233

Table 2: Objectives, instruments and players in market forces in four countries 234

Table 3: Criteria for summative evaluation of quasi-market reforms 237

Table 4: Conditions and criteria for the functioning of quasi-markets for social policy 239

Table 5: Tendering procedure 244

Table 6: Current situation 245

Table 7: C4 concentration ratio 247

Table 8: Success rates of established versus new providers (Job Network 1) 247

Table 9: Market types based on the number of providers and purchasers 249

Table 10: Payment structure and competition 254

Table 11: Cost-effectiveness of Job Network 258

Table 12: Gross outcomes of Job Network 262

Table 13: Empirical evidence concerning the impact of quasi-markets in Australia (1998-02)

and the Netherlands (2000-02) 265

Table 14: Empirical evidence concerning the conditions for quasi-markets in Australia (1998-02)

and the Netherlands (2000-02) 266

Table 15: Advantages and disadvantages of two market types based on tendering practice in

Australia and the Netherlands 267

FiguresFigure 1: Traditional model of service allocation 227

Figure 2: Quasi-market model 227

Figure 3: Voucher model 228

In this paper we focus on quasi-market reformsfor employment and training services for jobseekers. Open tendering is the standard form forthe introduction of market forces in several coun-tries, with a competitive bidding process used todetermine with which provider an order is placedfor a particular service. It is an alternative to themore traditional method of subsidising providers,where a country’s own government department/service, in a protected legal position, receives aguaranteed share of government funding. Thefocus on quasi-market reforms means that wewill not cover the broader evaluation issues ofactive labour-market and training policies, in sofar as they are disconnected from the mode ofservice delivery. Quasi-market changes are onlyjust being put into place but this does not meanthat any kind of evaluation is impossible. We startfrom a theory of quasi-markets as developed byLe Grand and Bartlett (1993) which specifies theconditions quasi-markets have to meet if they areto succeed. In our analysis, we will make apreliminary assessment of the extent to whichthose conditions appear to be met in practice.

This paper reports primarily on evaluation ofthe Australian quasi-market model, which is inmany ways the most radical in the world. Closestto Australia are the reforms in the Netherlands. Inthe UK (New Deal programmes) and Sweden(training), the model has been adopted only partlyand with divergent features along the employ-ment and training programmes. A system anal-ysis is provided of quasi-market arrangements inthese countries. Subsequently, the evidencecomes mainly from the Australian Job Networkand, to a lesser extent, the emerging private rein-tegration market in the Netherlands.

The experiences from the Australian and otherquasi-market reforms support the view that themarket model for providers of reintegrationservices is superior to that operating previously.However, there is no hard empirical evidence forthis. It is extremely difficult to distinguish rigor-ously between the effect of market forces andother changes that may have taken place simul-taneously, such as changes in the benefit system

or in the labour market. But this can not be ajustification for the lack of external (scientific)evaluation research on these reforms.

In Australia, the most significant gain is thatthe market system leads to more cost-effectiveperformance: services cost less, with a more orless similar net impact as previous programmesand sufficient responsiveness and quality.Weaker points are the very limited choice for jobseekers and the erosion of the level of services,especially with regard to job-specific training.However, findings on net impact and quality aretentative and should be added to other findingson the amount of training services, the financialincentives and their role for disadvantaged jobseekers. It was found that financial incentives,focusing on employment outcomes, have anadverse effect on the equal treatment of jobseekers. The incentive structure favours thepursuit of short-term employment outcomes overtraining and education, although training andeducation may, in the longer-term, be of greaterbenefit to job seekers.

In the Netherlands, it is not yet clear how farthe arrangements to foster responsiveness andchoice between providers may work well. TheDutch reintegration market tends to becomeself-regulating on quality: at the moment, there isno evidence of increased efficiency nor equity.

Despite these mixed findings, there is morepositive evidence on the conditions required forthe functioning of quasi-markets. Typical prob-lems concerning information and transparencyand transaction costs can be overcome by refine-ments in structure, tendering mechanisms, imple-mentation and monitoring arrangements. In thisrespect, the experiences are not similar, asshown by the market patterns in Australia andthe Netherlands. Australia has one centrally oper-ating purchaser, in contrast to the Netherlandswhere there are several purchasers operating in adecentralised manner. They represent contrastingmarket types, with both advantages and disad-vantages.

The evaluation review of emerging markets foremployment and training services highlights the

1. Executive summary

central role played by incentives. Therefore, itmakes sense to give much more attention in eval-uation research to the relationship between

tendering and market conditions on the one handand the behaviour of both providers and jobseekers on the other.

Evaluation of systems and programmes224

Countries are increasingly turning to private organi-sations to implement activation and reintegrationmeasures aimed at job seekers. In several OECDcountries, this is currently taking the form of amarket forces policy, based on the creation of amarket for employment placement and reintegra-tion. The mechanism used is that of opentendering, with reliance on private bodies (partially)replacing government organisation for employmentplacement, training and reintegration. The centralissue of this study deals with the experiencesacquired with market forces. Four pioneering coun-tries are examined: Australia, the Netherlands,Sweden and the UK. The key questions are asfollows. Within which structural reforms is the intro-duction of market forces placed? What are theintended objectives of the market forces? Is there agenuine market for reintegration services? Whatabout the former state provider? What are theimpact of the tendering rules and the incentivestructure on this provider? What are the effects ofmarket forces on the workings of the labour market,in particular the position of the job seekers? What isthe impact on the services provided? Whathappens to the amount and the nature of training?

Particular attention is given in this study toevaluating the implementation and (potential)impact, based on both formative and summa-tive evaluation. Hence, we examine to whatextent during implementation formative orsummative evaluation has taken place and inwhat way this has already influenced furtherimplementation.

This paper is structured as follows. InChapter 3, we deal primarily with the economicrationale of quasi-markets, and summarise theevidence so far with similar reforms in relatedpolicy sectors of other human services. We thengive a short overview of the main reforms andpolicy objectives for the quasi-market reforms,followed by a discussion about the institutionaland policy context of the benefit and activationsystems of the countries under consideration. InChapter 5 we introduce framework for evaluation,based upon the theory of quasi-markets. Subse-quently, we look in more detail at the empiricalfindings concerning formative (Chapter 6) andsummative evaluation (Chapter 7). In Chapter 8,we discuss concluding points.

2. Introduction

3.1. The economic rationale forcompetitive service delivery

Market forces are used to open up governmentorganisation to the market. In this context, weunderstand market forces as being a means ofachieving public goals not as shifting financingfrom the public sector to the private. Traditionally,services aimed at labour reintegration have beenheavily monopolised by government provision. Ina system of market forces, the services areoffered by a wide field of providers, with bothNGOs and profit-making organisations (e.g.employment agencies) operating alongside thegovernment provider. Market forces do not meanthat activities are ‘left’ to the market, but that thegovernment has the services provided by privatemarket parties. The government can also continueto play its own role as provider. The introductionof market forces stems from the pressure toimprove efficiency and responsiveness in thepublic sector. In economic literature, the role ofcompetition and contestability is threefold:(a) using competitive pressure to constrain activ-

ities and drive producers to keep pricesdown;

(b) organising according to the best known tech-niques of production;

(c) innovating to avoid economic annihilation(Webster and Harding, 2000).

Markets with a plausible threat of entry may beenough to be described as contestable markets.Government funding is justified on the groundsthat purely private provision will result insub-optimal consumption levels where positivesocial externalities are present and economicjustice principles are relevant. Therefore, a broaddistribution of education, health and socialservices may be required, broader than wouldarise under purely private arrangements.However, while public funding is justified, it doesnot immediately follow that the government mustalso ‘provide’ all or any of these services. In theliterature this model has been called quasi-markets.

3.2. Definition of quasi-markets

In a quasi-market, the relationships betweenpurchasers and providers are structured in acommercial way. This produces a shift frombureaucratic monopoly providers towards inde-pendent, competitive providers: ‘The governmentbureaucracies responsible for providing theseservices were broken up, being replaced by statepurchasers who contracted with independentproviders to provide the service concerned’(Le Grand, 2001). Decisions are made on the basisof the market criteria of price, quality and deliverytime. As in an ordinary market, competition existsbetween various providers, but these are not,however, necessarily driven by profit maximisa-tion. On the demand side, the available budget isdistributed by a purchasing agency. Le Grand andBartlett define this new mode of public servicedelivery as an internal or ‘quasi’-market: a marketin the sense of independent agents competingwith one another for custom from purchasers, buta quasi-market in that, unlike in a normal market,purchasing power comes not directly fromconsumers but from the state (Le Grand, 2001).Other differences from normal markets are theappointment of an agent to act as a purchaser onbehalf of the final consumer and – in some policysectors – a preponderance of non-profit or evenpublic providers. The new delivery system took offin many countries in the 1980s and 1990s in socialpolicy fields like health, education and housing.According to the theory of quasi-markets, marketforces have two main goals: on the one hand, toincrease effectiveness and, on the other hand, toincrease the responsiveness of the providers andthe freedom of choice for consumers (Le Grandand Bartlett, 1993).

3.3. Quasi-markets as allocationmodel of service delivery

The introduction of a quasi-market for publicfunded services can be considered as a specific

3. The role of quasi-markets

mode of service delivery, to be distinguishedfrom privatisation or contracting out. With privati-sation a former public sector function is movedpermanently into the private sector with achange in the owner’s structure and in the way offinancing. Contracting out is often understood asoutsourcing from the public provider; in abroader sense contracting out means that theformer public sector function is subject of privatesector competition, where the public providercan continue to act as one of the competingactors. In this sense contracting out stands for aquasi-market arrangement. Finally, there is thevoucher system, which enables the client orcustomer to shop around between providers(Derksen et al., 2000; Thuy et al., 2001; Greenet al., 2001; Struyven et al., 2002). These modelscan be seen as different allocation models:resourcer oriented (the traditional government’sprovider); provider and purchaser oriented(quasi-market model) and customer oriented(voucher model).In each model the following roles are distin-guished: the resourcer role, the purchaser role,the provider role and the user role. In the tradi-tional model, publicly funded services areprovided by the public provider, with some partsusually being contracted out to other providers.

In this case, the government is combining the roleof resourcer, purchaser and provider. In thequasi-market model, a clear distinction is madebetween the purchaser (principal) and theprovider (agent). In the voucher model, funds aregoing directly to the user, who can make a choicebetween different providers in the market. In thismodel, the user (or customer) is acting as apurchaser of services.

Quasi-market reforms in employment and training services: first experiences and evaluation results 227

Figure 1: Traditional model of service allocation

(Central government)

Public provider

Contacting out

User/client

Public, non-profit +Private providers

Figure 2: Quasi-market model

Central government

Central purchaser

Non-profit + Private actorsPublic actor

User/Client

3.4. Quasi-markets: do theywork?

Market forces cannot simply be equated with lesssocial protection. What is striking is that thecountries in which quasi-market arrangementsare implemented, are involved in far-reachingreforms to systems of benefits, activation andemployment placement (Struyven et al., 2002).The application of market forces can be seen asa means of making social security manageable:not by breaking down rights to benefits, but byreorganising implementation. Rather than ‘freemarket forces’, the issue is much more one of‘managed market forces’ or even ‘controlledmarket forces’ (see the concept of managedliberalisation in van der Veen, 2000). The questionis not therefore ‘government or market’ but of‘which policy mix of government and market?’.

Here we are mainly concerned with quasi-markets in typical publicly funded sectors, suchas education, health and other human services.British experiments in these areas are among themost radical of those tried. In his review of 2001,Le Grand reports on the evidence so far withthese quasi-markets. The most striking conclu-sion from this review is how different they are(Le Grand, 2001). In the British health care marketreforms, there seems to have been little overallmeasurable change. The quasi-market delivered

relatively little of the benefits that its advocateshoped, but also generated few of the disastersthat its critics predicted. In contrast, the market ineducation does seem to have brought significantchanges in behaviour in British schools. Therehave been gains in efficiency and choice, but alsosignificant equity losses (see below with regard tothe evaluation criteria). According to Le Grand,there are a number of possible explanations forthe relative failure in health care and the relativesuccess in education. First, the incentive struc-ture was relatively weak in health and relativelystrong in education (e.g. reward for success,penalty for failure). Second, the constraints facedby the agents were different (e.g. degree ofautonomy; flexibility to respond to demand pres-sures). Finally, the motivations of the activeagents played a role. In hospitals, the dominantmotivation is quality of the care provided byspecialist consultants and their professionaladvancement. In education, school ‘managers’were more preoccupied by the financial successor failure of their institution. Le Grand concludesthat, for the introduction of quasi-markets intopublic service delivery significantly to changebehaviour in the relevant actors, a number ofconditions have to be fulfilled. In this paper wewill examine how far these conditions are presentin the emerging employment and training servicesmarkets.


Figure 3: Voucher model

Central government

Non-profit + Private actorsPublic actor

User/Client

3.5. Linking contestability andmarket forces to theactivation of benefitclaimants

The practice of market forces is rooted in thebroader activation concept (European Commis-sion, 1998). The shared goal of the reforms is toallow social integration to take place more viaparticipation in the labour process than purelyby redistributing income through benefits. Theefforts of the job seeker, the reintegrationcompany and the benefits agency have to begeared towards the same goal. The premise of‘work first’ (UK) or ‘work before income’ (the -Netherlands) figures in each country. Incentivesor financial stimuli deeply influence the nature of

the mutual relationships between the playersinvolved. Reintegration companies are steeredtowards achieving the placement standard andjob seekers are judged on their willingness toaccept an activation offer or a job. The outcometakes precedence. The underlying reasoning isthat steering based on placement results is moreeasily based on competition between providers;after all, their role is ultimately that of mediatorwith the demand for labour. The benefits agencyis given more autonomy and responsibility inexchange for result commitments, while theduties of the individual job seeker as ‘benefi-ciary’ are now also considered. Broad unanimityexists concerning the goal, i.e. linking the rightto a benefit to participation in an integrationroute.


4.1. The four countries at aglance

The Australian Job Network is an internationalmodel for market forces in employment place-ment. The model has been in place since1 May 1998 for all the employment placementand reintegration services. Two tendering proce-dures have already taken place, in 1997 (JobNetwork 1) and in 1999 (Job Network 2). The thirdround is planned for March 2003. This case studyis targeted at three labour-market services whichwere deployed in the market. ‘Job matching’consists of placement services for all registeredjob seekers, including vacancy recruitment andhelping job seekers to find a job. ‘Job searchtraining’ is intended for job seekers with amoderate degree of disadvantage and is aimed atimparting ‘job search skills’. ‘Intensive assis-tance’ involves individual counselling for the mostdisadvantaged job seekers. Dramatic institutionalreforms were carried out at the same time as theintroduction of Job Network. The folding of theCommonwealth Employment Service and theformation of Centrelink, an autonomous govern-ment organisation which provides a wide range ofservices to citizens through the ‘one-stop shop’format, formed part of this. The Department forEmployment, Workplace Relations and SmallBusiness (DEWRSB) acts as sole purchaser.Centrelink refers job seekers on to the providers:Employment National (successor to theCommonwealth Employment Service), privateproviders and NGOs.

The Dutch government has set itself the aim ofcreating a ‘purely private reintegration market’. Atnational level, the principle of open tendering hasbeen generalised for job seekers and the disabled(first round in 2000; second round in 2001). Forthose on benefits, at municipal level the evolutiontowards ‘transparent purchase’ of the reintegra-tion services present on the market is still verygradual. The ‘privatisation’ of labour reintegrationforms one of the pillars of the Structure of workand income implementation law (SUWI law),

which entered into force on 1 January 2002. It ishoped, through intervention in the administrationorganisation, to reduce the sharp increase innumbers of disabled people receiving disabilitybenefits. The population of job seekers and thedisabled is part of the responsibility of the newbenefits agency (UWV or Employees’ InsuranceBenefits Agency); benefit recipients remain undermunicipal responsibility. Paying out benefits – andassociated tasks such as establishing the right tobenefits – remains within the public domain. Rein-tegration as a whole is transferred into the privatedomain. The reintegration component of thepublic employment placement organisation(Arbeidsvoorziening) is transferred to the autono-mous public company KLIQ. Several basic tasks,such as registration and direct placement areoffered jointly for both groups via the one-stopshop of the Centres for Work and Income (CWI),which are taking over some of the tasks of theformer job centres. The Netherlands has manybuyers as a result of shared commissioning by theUWV and the municipalities.

The Welfare to Work programme of the BritishLabour government to a large extent continuesthe trend of involving private parties in imple-menting the various New Deal programmes. TheNew Deals fall under the auspices of the Employ-ment Services, which itself retains a number oftasks regarding intake and counselling. TheEmployment Service is an integrated benefits andemployment placement organisation and isorganised along the one-stop shop conceptthrough a network of Jobcentres Plus. Theservice operates as an autonomous componentof the new Department for Work and Pensions.Since 1998, the Employment Service has beenorganising individual tendering procedures perNew Deal programme (for young people, thelong-term unemployed, single parents, etc.) andper region. What exactly is carried out by themarket can vary from one region to another. Inaddition, the Department for Work and Pensionssets up pilots for employment zones and actionteams. These are initiatives for the placement andreintegration of job seekers in a well-defined area

4. Overview of reforms

of high structural unemployment. The benefitsfunction is also privately organised.

Sweden has for a long time emphasised activelabour-market measures, which involves devotinga great deal of attention to training. The Swedishcase study concentrates on the introduction ofmarket forces in organising the supply of trainingfor job seekers. In contrast to the other countries,this reform did not take place within the Employ-ment Service. It was in two phases, starting in1986 when organisation of training was reformedfor the first time, with separation of demand andsupply. Publicly financed training became thecompetence of the National Labour Market Boardand 24 County Labour Market Boards. A newagency, AmuGruppen, was formed, with the taskof providing training courses. At the same time,the market for publicly financed training courses,via decentralised tendering, was opened up topublic and private training providers. In 1993, thesecond phase of the reforms was implemented.The autonomous government agency was trans-formed into an independent state enterprise (nowLernia), which has to compete under the sameconditions with other players in the trainingmarket.

4.2. Objectives of the reforms

In the theoretical literature about market forces,two main aims are usually identified: to increaseeffectiveness and to increase the responsivenessof providers and freedom of choice forconsumers. These aims partly correspond withthe aims of active labour-market policies ingeneral: to improve the employability of theunemployed, increase the efficiency of jobsearching (in contrast with passive policy, whichprovides benefits for those without work), securejob outcomes and improve equity (ProductivityCommission, 2002).

4.2.1. Outcomes and costsWhen studying the objectives for market forces interms of reintegration, the motive of technical effi-ciency and cost control seems to rise to thesurface in all countries. In addition, countries alsoaim for greater allocation efficiency using a newservice concept (Australia) or a more flexiblesupply of training (Sweden). Australia and the UK

both emphasise the importance of outputs andoutcomes (results). The British system, moreover,is highly target-driven. In the Netherlands, marketforces – via a reorganisation of the entire imple-mentation structure – are focused on controllingthe problem of disability benefits (under the WAO)which has dominated social security policy forover a decade. It is hoped that market forces willreduce those claiming benefits under the WAO(by making the employer the principal) andincrease those leaving the benefits system underthe WAO (by making the benefits agency theprincipal). Result steering and market forces gohand-in-hand in this case.

The creation of open competition is a keyelement in all countries. This is explicit in thepolicy theory of the Netherlands: the creation of apurely private reintegration market and trans-parency in allocating public resources function asthe central objectives. Neither the Netherlandsnor the UK use market forces in the context ofmajor government budget savings. In Australia,savings as a motive for market forces were alsonot officially on the agenda but, in practice,significant savings do seem to have been made.The Job Network system was actually introducedby abolishing a number of existing labour-marketprogrammes. Moreover, the governmentmanaged to bring the cost price to a little lowerthan it had previously been, using assumed pricemargins (see below).

4.2.2. ChoiceExcept what theory would lead us to suppose,relatively little reference is made to the secondaim in the four case studies: the aim of increasingthe options for the consumer. The logic is thatwhen the government offers services as amonopolist, job seekers cannot make their ownchoices because there are no alternatives. Themarket should be better able to guarantee indi-vidual freedom of choice but little of this can beseen in present practice. It should however besaid that experiments are under way withperson-related or person-following budgets (e.g.personal job account in the employment zones;person-related reintegration budget in theNetherlands). Australia does refer to the goal ofgreater freedom of choice but, in practice, thisdoes not seem to work well (see below). Onepossible explanation for the missing dimension offreedom of choice is that market forces systems


are heavily geared towards controlling reintegra-tion supply via regulation of providers. One otherexplanation is that the activation context (byemphasising rights and obligations) and thejob-seeking population (as a result of the pres-ence of less independent target groups) lendthemselves less to this.

4.2.3. Other motivesA further consideration concerns the operation ofthe government provider. In a market discussion,we could expect dissatisfaction with the govern-ment provider to be the main reason for reformsand the goal of shifting the production oflabour-market services to the private sector to bea response to the problems of government provi-sion. This does not seem to feature in officialdiscussions in the countries in question, thoughsome reference is made in the appeal for strongprivate involvement. It is likely that in practice ithas been an important breeding ground forgreater openness with respect to more intensemarket forces.

Finally, there is the political-ideological motive.Market forces are increasing being introduced, withAustralia, New Zealand, the UK and the US playinga leading role. Nonetheless, the ideological argu-ment has to be put into perspective when lookingat the prior development process in Australia or theUK. More decisive in nature is the question of howfar there is a potential market for it.

4.3. Market forces in the broaderinstitutional and policycontext

The most striking aspect of the countries underconsideration here is that the introduction ofmarket forces is associated with a dramatic insti-tutional reform of the broader system of benefits,activation and employment placement. Inessence, it affects the structure of organisationand incentives. The choice for market forces isnot self-evident. In this respect, we draw adistinction between Sweden and the three othercountries (Struyven, et al., 2002).

4.3.1. The Swedish context of the 1980sThe context for development of market forcesdiffers. For instance, the initial driving force

behind Swedish training policy appears to havebeen increasing unemployment, which created anincreasing need for training that could not be metby AmuGruppen, the state provider. In addition,the Swedish authorities wanted to make thegovernment provider more cost efficient byexposing it to competition. However, in theSwedish example, market forces seem to becontained within a process of institutionalchange. This sought a clearer distinction betweeneducation and training geared towards the labourmarket, which led to the shift in responsibility forthe former Amu centres from the Department ofEducation to the Department of Employment.

4.3.2. Australia, the Netherlands and the UK:institutional reforms in the second halfof the 1990s

These wider institutional reforms are found in thethree other countries and the case studies showthe extent to which Australia, the Netherlandsand the UK have recently become involved infar-reaching reforms which are not yet complete.The motives for transferring to market forces arerooted in diagnosis of the existing institutions: thelegitimacy of keeping certain tasks in the publicdomain (or not), the nature and efficiency ofincentives concealed within them, the relation-ships between inter-related players. The struc-tures vary in the countries studied with differ-ences in the organisational division betweenfunctions, the scope for market forces in theimplementation of functions and the new conceptof the welfare state.

4.3.2.1. Relationship between functionsThe relationship between the two functions ofbenefits and placement represents a fundamentaland constantly recurring field of tension inlabour-market policy. The OECD usually identifiesthree main functions for policy, irrespective ofwho is assuming them: the benefit payment func-tion, the placement function and the referral func-tion (to labour-market programmes) (Fay, 1997).Developments in market forces allow us to iden-tify three additional functions:(a) the transparency function with respect to the

vacancies market, specifically via the pres-ence of a broad public vacancy databaseand/or private databases;

(b) the tendering function: in present-day marketforces, this means putting out to tender activ-


Australia The Netherlands United Kingdom

Benefit Centrelink – UWV

Employment Service (ES)– Municipalities

Direct placement By tender CWI Jobcentre Plus (= ES)

Intake and referral Centrelink CWI Jobcentre Plus (= ES)

Vacancy market Central department

– Private market

transparency function – CWI ES

Tendering Central department – UWV – ES

– Municipalities – Central department

Reintegration By tender By tender By tender

ities in the field of employment placementand reintegration;

(c) the reintegration function: this means fulfillingand/or implementing activities in the field ofreintegration (introduction, training, etc.,through to placement and after-care).

Two sub-functions can be distinguished aspart of the placement function: direct placementand reintegration. This means that six key func-tions can be identified. Table 1 examines how thefunctions are distributed in the new setting oforganisations.

In Australia and the UK, powers are distributedbetween the central department and one inte-grated government organisation: Centrelink(Australia) and the Employment Service (UK) withthe network of Jobcentres Plus. This develop-ment contrasts with the new structure inthe Netherlands, where the benefits and place-ment functions are (remain) divided between twodifferent organisations: the UWV and the CWI.The special feature of the Dutch situation is thattwo other organisational mergers are takingplace: first at the level of the benefits agenciesand, second, at the level of the intake and referralfunction. In the new configuration, the intake andreferral function is the job of the CWI, whereasthese tasks used to be exercised in a fragmentedmanner by the employment agencies, the admin-istration agencies and the municipal socialservices.

4.3.2.2.Scope for market forces in implementingfunctions

Differences exist in the extent to which functionsare left to the market. In this respect, Australiagoes the furthest, with typical tasks related toemployment placement by a public institutionreduced to a minimum. In the Netherlands, intakeencompasses a broader procedure and the bene-fits agency can also continue to carry out casemanagement (client follow-up right through toplacement). In the UK too, the role of the Employ-ment Service remains more significant; thoughthis varies depending on the region. However, theUK is also experimenting on a territorial basiswith tendering out both reintegration and benefittasks (Employment Zones).

4.3.2.3.Market forces and the concept of thewelfare state

Other differences emerge from the broaderperspective of social protection. It is clear thatthe reforms in Australia push the mutual connec-tion between result-orientation, free competitionand conditional benefits (mutual obligation) to theforefront. In the Netherlands and the UK, officialpolicy discussions are moving in the same direc-tion, although it cannot be said that a policy ofless social protection is being pursued. Rather,we are seeing strong emphasis on the mutualrights and obligations of activation bodies andjob seekers.


Table 1: The key functions in employment placement and spread among governmentorganisations (situation in 2002)

Australia The Netherlands United Kingdom Sweden

4.3.3. ConclusionThe introduction of the market mechanism doesnot stand alone, but is part of the search for a morecoherent benefits and activation system. Thissystem, in turn, is related to changes in theconcept of the welfare state. Institutional reformrepresents one of the major pre-conditions formarket forces. This, however, complicates evalua-tion of the impact of the market forces mechanismbecause the introduction of the market mechanismis difficult to isolate from the other system changes.

Table 2 presents a brief summary of the fourcase studies in terms of the following characteris-tics: objectives, type of market mechanism,commencement date, principal, frequency,purchaser/provider separation, type of marketforces, definition of the assignment, target groupsin the specification, quality monitoring body, roleof the social partners, referral body, choice for theprovider, choice for the customer and vacancies.Most of these points will be discussed in moredetail later in the paper.


Table 2: Objectives, instruments and players in market forces in four countries

Objectives – Geared towards outcomes

– Efficiency/savings– Individual services

of high quality

– Purely private reintegration market

– Disability benefits cost control

– Transparency inallocation of resources

– Target-driven– Geared towards result– Cost control

– Cost control/ savings

– Flexibilisation oftraining supply

Market mechanism

Open tendering Open tendering (in UWV; in municipalitiesstill exceptionally)

Open tendering (in theory)

4 types: Open (usually),restricted, negoti-ated, selective

Introduction of opentendering

Generalised since 1997

Generalised since 1999(UWV) Municipalities:still being developed

Generalised since 1998 Generalised since1993

Principal One procedure throughcentral department(Commonwealth)

– Disabled (OccupationalDisability InsuranceAct) + unemployed(Unemployment Insur-ance Act): throughbenefits agency (UWV)(Central ZBO –Autonomous ManagingBody)

– Benefit claimants:municipalities

– New Deal: Through regions ofEmployment Servicewithin uniform procedure

– Employment Zones +Action Teams: centraldepartment

– Regions: byCounty LabourBoards withinuniform framework

– Additional: LocalEmploymentServices

Frequency One national tenderevery three years

– one tender annually(UWV)

– municipal tenders atany time

Tenders per programmeevery three years

Tenders perregion annually

Purchaser/providerdistinction

Yes – UWV: yes– Municipalities: some

still in role of provider(municipal companies)

Yes Yes


Type ofmarket

Monopsony, buyers’market

Towards quasi-market Intermediate form Monopsony perregion

Definition ofassignment

Five separate services:job matching; job search training;intensive assistance;new enterprise incentivescheme; project contracting(harvest labour)

– UWV: one reintegrationservice in five stages:reintegration plan,diagnosis; supply reinforcement,mediation; placement andafter-care

– Municipalities: variouslyspecified

Specified Specified bytraining

Targetgroups inspecification

Not specified 22 target groups (Occupational DisabilityInsurance Act/UnemploymentInsurance Act)

Separate plans perNew Deal target group:young people; 25+;single parents; partnersof the unemployed;disabled people; 50+

No target groupsspecified

Quality monitoringbody

Principal – Principal– Branch organisation– Advisory council (Work

and Income Council)

– Principal– Adult Learning

Inspectorate

– Principal– National Labour

Market Board

Social partners

No structured advice,but bilateral consultation

Until 2001: relativeinvolvement of sectoralcouncils

From 2002: advisorycouncil (RWI - Work andIncome Council)

New Deal Task Force Structured adviceat national andregional levels

Customerreferrals

Centrelink UWV (in practice still byfive benefits agencies) +municipalities

Jobcentre Plus Employmentagency

Compulsoryparticipationby provider

Yes No Yes Yes

Choice forcustomerbetweenproviders

In theory yes, limited in reality

If possible, choicebetween severalproviders

Choice of options No choice

Vacancies Compulsory use ofregistered vacancies innational vacancydatabase; providerbrings in most vacancies

Free vacancy recruitmentSeveral databases(private)

Free vacancy recruitment

–

Our focus on quasi-market reforms for employ-ment and training services for job seekers meanswe will not cover broader evaluation issues ofactive labour-market and training policies, in sofar as they are disconnected from the mode ofservice delivery. Quasi-market changes are onlyjust being put into place but this does not meanthat any kind of evaluation is impossible. Westart from a theory of quasi-markets as devel-oped by Le Grand and Bartlett (1993) whichspecifies the conditions quasi-markets have tomeet if they are to succeed. In our analysis, wewill make a preliminary assessment of the extentto which those conditions appear to be met inpractice.

This paper reports primarily on evaluation ofthe Australian quasi-market model and, to alesser extent, the Netherlands. There are severalreasons for this. In the first place, the AustralianJob Network is in many ways the most radical inthe world. Closest to Australia are the reforms inthe Netherlands. In the UK, the model has beenadopted only partly and with divergent featuresalong the employment and training programmes.Also, the models in both the Netherlands andthe UK have only recently undergone a similarreform. As a consequence, and in contrast to inAustralia, there is no systematic review of theevidence relating to the quasi-market inthe Netherlands or the UK; existing evaluationstudies did not suit a meta-analysis, especially onimpact evaluation. For these reasons we willfocus mainly on the evaluation of Job Network;for the other experiences we rely on our ownoverview of the four countries as far as this isuseful for the implementation evaluation(Struyven et al., 2002). For Australia andthe Netherlands, an attempt has been made togather as much reliable evaluation evidence aspossible from different sources. The last para-graph of this chapter summarises the main evalu-ations available for both Australia and the Nether-lands.

5.1. Criteria for summativeevaluation

The first question on evaluation is what might bemeant by ‘success’, or, what are the appropriatecriteria for the evaluation of quasi-marketreforms? A further distinction can be madebetween different types of evaluation, such asformative and summative evaluation. Accordingto evaluation literature (Patton, 1982; Clarke,1999), formative evaluations aim to providinginformation for programme improvement, modifi-cation and management. The question is ‘Howcan the programme be improved?’, whereas thequestion for summative evaluation is ‘Should theprogramme be continued, and if so, at whatlevel?’. Summative evaluations (or impact evalu-ations) aim to determine programme results andeffects, especially for the purpose of makingmajor decisions about programme continuation,expansion, reduction and funding. These twotypes of evaluation are not mutually exclusive.Over time, a particular programme or reformmight be involved in activities from differenttypes of evaluations, starting from procurementand probity audit, monitoring and formativeevaluation to summative evaluation andmeta-evaluation.

The aim of quasi-markets can be both effi-ciency and responsiveness, and freedom ofchoice and equality: ‘The aim was to improve effi-ciency, choice and responsiveness, and to raisequality standards – and without any adverseconsequences, especially for equity’ (Le Grandand Bartlett, 1993). This provides four criteria forevaluation and a more recent paper by Le Grand(2001), added quality as a fifth. The corre-sponding five criteria can be understood ascriteria for summative evaluation, stated inTable 3.

Each of the criteria is further explained andillustrated by one or more key indicators, withoutmaking any claim to be exhaustive.

5. Evaluation framework for quasi-market mechanisms

5.1.1. Efficiency Efficiency can be interpreted in many ways, butLe Grand and Bartlett distinguish two basicconcepts that underlie most of these differentinterpretations: the total costs of service deliveryand the costs of providing any given quantity orquality of a service (productive or technical effi-ciency). One can further distinguish betweendifferent forms of technical efficiency. Technicalefficiency is achieved when a given output isproduced using minimal deployment of input(input efficiency) or when maximum output isachieved using a given quantity of input (outputefficiency).

5.1.2. Responsiveness Increasing responsiveness can be regarded asthe pursuit of a greater allocative efficiency.Allocative efficiency refers to the output whichbest fits in with the needs and preferences of theend users as well as the locality in which theyoperate.

5.1.3. Quality Contrary to Le Grand and Bartlett’s theory, qualitycannot be left out when discussing the evaluationof public service delivery. This seems to be recog-nised in Le Grand’s recent overview of evaluationfindings of quasi-markets (Le Grand, 2001). Qualityis a broad concept with many interpretations. Oneof the many applications, as used by the EuropeanFoundation for Quality Management, coversprocess and organisation criteria (enablers) as wellas results criteria. For the evaluation of quasi-markets, quality indicators can be based on theservice (kind and intensity of service), the client’sperception (client satisfaction, complaints proce-dures) and (micro)-economic impact (gross andnet placement ratios, drop-out ratio).

5.1.4. Choice The logic is that when the government offersservices as a monopolist, clients cannot maketheir own choices because there are no alterna-tives. The market should be better able to guar-antee individual freedom of choice. After all, the


Criterion Indicator Aim

Efficiency Cost of service provision To minimise total costs

Value for money To minimise costs of providinga given quality or quantity

Responsiveness Individual needs and preferences To make the service more flexible Local needs and sensitive to needs

Quality (process, Kind of services To enhance both processoutput, outcome) Intensity of services and output of the service

Client satisfactionComplaints proceduresGross placement ratioNet placement ratioDrop-out ratio

Choice Choice of service As an end in itselfChoice of provider for clients As an instrument for achieving Choice of clients for providers the other aims

Equity Concentration on those with Increased efficiency, responsiveness the greatest needs and choice without adverse consequences

in terms of increased inequityTo avoid creaming

Source: Le Grand and Bartlett (1993); Le Grand (2001)

Table 3: Criteria for summative evaluation of quasi-market reforms

market mechanism is sustained by differences inindividual preferences. As stated by Le Grandand Bartlett, a concern for choice may be justi-fied as an end in itself, or as an instrument forachieving other policy aims. Theoretically, onecan distinguish between the choice of service,the choice of providers (for clients) as well as thechoice (of clients) for providers.

5.1.5. Equity Equity rarely appears explicitly as one of thepolicy objectives of the quasi-market reforms, asLe Grand and Bartlett noticed: ‘no doubt becausethey are being driven by a government for whichequity considerations are not necessarily a highpriority’ (Le Grand and Bartlett, 1993). The keyconcern when providing social services is toconcentrate on those with the greatest needs.Le Grand and Bartlett define an equitable serviceas one whose use is determined primarily byneed and not by irrelevant factors such asincome, socioeconomic status, gender or ethnicorigin. The more care resources an individualrequires to bring his or her level of welfare up tosome predetermined level, the greater is his orher need. The aim is to avoid creaming andparking (see below) of the easy-to-place clients.

5.2. Criteria for formativeevaluation

Another set of criteria, from a formative evaluationviewpoint, can be derived from the conditions forthe functioning of markets. According to Le Grandand Bartlett, the following five conditions must besatisfied in order to be able to function as amarket: there must be a market structure based oncompetition and prices, with accurate andcomplete information, with moderate transactioncosts, a balanced motivational structure andwithout adverse consequences in terms ofcreaming. Theoretically, a number of problems canarise in these markets. These problems relate tothe market structure, information and trans-parency, transaction costs and risk selection.

The main problems relate to competition andprice as the basic components of a market struc-ture:(a) dominant providers exploit their monopoly

power due to a lack of competition among

providers, or else large purchasers exploittheir monopoly power due to a lack ofcompetition among purchasers.

(b) the relationship between the two dominantplayers on both sides can be too intimate, sothat the system could become one simply ofdecentralised budgets, with a managementcontract between purchaser and provider, butwith no real competition on either side;

(c) providers are driven out of business, thesystem fails to attract new entrants andprevent the ‘exit’ of efficient providersbecause a dominant purchaser may sour rela-tionships with providers and lower theirmorale and motivation;

(d) prices are not formed directly betweenproducers and consumers, so do not reflectuser preferences, and are ‘administered’ ornegotiated between purchasers andproviders;

(e) providers do not necessarily face hard budgetconstraints because of their specific owner-ship status.

Besides the problems of the market structure,providers may look for strategies to reduceuncertainty or competition, for example throughmergers and collaboration. These strategies canoccur when it is difficult for purchasers to monitorquality and to carry out checks because ofincomplete information. Providers may engage inwhat Williamson (1975, 1985) calls opportunisticbehaviour, exploiting their informational advan-tage to reduce costs at the expense of quality.Following Le Grand and Bartlett (1993), there aretwo kinds of opportunistic behaviour to which thetheoretical literature draws attention: moralhazard and adverse selection. Moral hazardoccurs where providers put fewer resources intothe provision of the service than is consistentwith the terms of their contract. Becausecontracts are incomplete, i.e. they do notprescribe in full detail what providers must dounder all possible circumstances, providers canreduce their costs by cutting their services withinthe limits of the contract. Employment serviceproviders for instance may be obliged by contractto have regular contact with job seekers, but theymay be inclined to limit these contacts to aminimum to reduce their costs (contact over thephone instead of face-to-face contact). Adverseselection occurs where providers possess certain


characteristics that may adversely affect theprovision of the service and that are known tothem but that they do not reveal to the purchaser.Because of this asymmetric information, thecontracting conditions offered by the purchaserwill target the ‘average quality provider’.Providers which can offer better quality, though ata higher price, will not be interested while forproviders offering poorer quality, the contract isattractive. As a consequence, the purchasercontracts the providers he does not want.

In theory both problems could be prevented inthe formulation of the contract and by continuousmonitoring, but this will increase administrationand transaction costs. Fully specifiedcost-per-case contracts would then replace theincomplete block contracts. The transactionswhich take place in quasi-markets are often quitecomplex and multi-dimensional, given thecomplexity of the social services. Transactioncosts may be particularly acute in the presence ofuncertainty and risk. Both purchasers andproviders, which operate in an unpredictablecontext of labour demand and supply, are unableto specify all future contingencies in a writtencontract. This problem is known as the incom-plete contract: contracts are deliberately left

incomplete and reflect a willingness to cooperaterather than the precise allocation of risks, respon-sibilities and rewards (similar to the implicitcontract under the pre-market system) (Walchet al., 1997).

Quasi-markets may also be subject to theproblem of risk selection: the provider’s selectionof the most profitable clients. This problem, alsoknown as cream-skimming, discriminates againstthe more expensive users: the long-term unem-ployed, the low-educated, minority groups, etc.The predominance of users’ interests inproviders’ financial or organisational interests canalso occur in relationship with the purchaser asan agent acting on behalf of users.

The question when evaluating the reforms is towhat extent the conditions are present in order tofunction as a quasi-market for employment andtraining services. Table 4 summarises the criteriawhich can be derived from the identified prob-lems for each of the five conditions.

The conditions and related criteria or indicatorsfor the functioning of quasi-markets for socialpolicy can be explained as follows:(a) market structure; various providers and

purchasers must be present; new providersmust gain easy access; as well as existing


Conditions Criteria

Many providers and many purchasersCompetition Easy entry for new providers

MarketExit Possibility of exit

structureMarket prices Prices are formed directly

Equal treatment to providers with different ownership status

AccurateAccess to cheap and accurate information

InformationComplete

Formulation of contractsMonitoring of quality

Transaction CertaintyModerate transaction costs for purchaser and provider

costs Transparency Tendering and contract conditionsNew entrants

Motivated by financial

Motivation considerations (providers) Balance profit/not for profit motivationMotivated by needs of Purchaser acting on behalf of usersusers (purchasers)

Cream Needs of users determineMeasures to avoid creaming

skimming services/ treatment

Source: Based on Le Grand and Bartlett (1993); van der Veen (2000); Trommel et al., 2001

Table 4: Conditions and criteria for the functioning of quasi-market for social policy

monopoly structures, space must at least becreated for a counterbalance, for example byalso allowing local authorities to function aspurchasers;

(b) information; easy access to information aboutcosts and quality must be available. This canbe organised by setting up one umbrellaservice to act as an information manager forall the required aspects;

(c) transaction costs; transactions are complexand multi-dimensional and require a certainspecialisation; together with a high level ofuncertainty (e.g. concerning what will beneeded in the future) this can lead to highcosts. Transaction costs should not be higherthan the costs of the previous system;

(d) motivation; providers must also be guided byfinancial interests and motivated to anticipatemarket signals. Purchasers must be moti-vated to work towards the welfare of the user(above their own organisational interests);

(e) cream skimming (creaming); the user or clientreceives a free service, but discrimination canoccur with respect to ‘more expensive’ users.This can be remedied by linking the need tothe remuneration offered, for example bygiving poorly educated people greaterweighting in standardisation or financing.

5.3. Available evaluation studies

For empirical analysis of quasi-market reforms inthis paper, we can rely upon own research carriedout for the Ministry of Employment in the FlemishCommunity of Belgium and the OECD (Struyvenand Steurs, 2002; Struyven et al., 2002). Theresearch consists of literature review and qualita-tive case studies, based on an analysis of writtensource material and 47 interviews with importantplayers in the four countries (2). These sourcesprovided the main input for the formative evalua-

tion in this paper. In addition to this research, theanalysis has been extended with a review of morerecent documents and evaluation studies (up toDecember 2002) and specific information gath-ered by telephone interviews or e-mail correspon-dence. Studies have been screened on availablefindings for both formative and summative evalu-ation, though much more information about thedetailed functioning and impact evaluation of thequasi-market seems to be available fromAustralia. As the only parallel to Australia is theemergence of a private reintegration market inthe Netherlands, the overview in this paper ismainly based on empirical evidence from bothcountries. However, one must consider that thereis, so far, no comparable monitoring and evalua-tion available from the Netherlands (3). For thesereasons the summative evaluation review isbased on evidence from Australia only. Of partic-ular interest in this paper is the importance oftraining and the impact of quasi-market reformsin job brokerage and reintegration services on theprovision of training. The evidence on this pointfrom the Australian Job Network draws heavily ona memo prepared for us by the Australian Evalu-ation and Programme Performance Branch of theDepartment (4).

5.3.1. AustraliaAt the introduction of Job Network there wasconsiderable public interest in the process forreviewing such a radical shift in delivery arrange-ments. A comprehensive evaluation strategy wasdeveloped which involved three reporting stages,recognising that the reforms would take severalyears to develop and mature, as well as an inde-pendent review (5).

The first two stages of the official evaluationreported in May 2000 and May 2001. Thesereports were largely based on performance underJob Network’s first contract. Stage 1 focused onthe implementation of Job Network and its earlyoperation, while stage 2 examined equity of


(2) Face-to-face interviews conducted in the period between June and October 2001 with the authorities, the market players, thesocial partners and independent experts.

(3) After publication of this paper, perhaps progress will have been made with regard to monitoring and evaluation data.(4) With thanks to Mr Philip Gatenby of DEWR.(5) There has also been a Probity Adviser’s report on the first Job Network tender in 1999, as well as a performance audit under-

taken by the Australian National Audit Office, which we do not consider further here. Finally, the OECD review on Australia’slabour market policies was for the most part devoted to the Job Network (OECD, 2001).

access to assistance and outcomes, early indica-tors of the impact of assistance (on the likelihoodof leaving income support) and regional perfor-mance (DEWRSB, 2000a; 2000b). The thirdreport, published in May 2002, provides informa-tion on the lessons learned from evaluating JobNetwork over the period of its operation sinceMay 1998 (DEWR, 2002b). In assessing effective-ness, the Stage 3 evaluation examined thesustainability of employment outcomes, theimpact of the major services in improvingemployment prospects, how well Job Network isendorsed by its clients, its responsiveness tospecial needs and its macro-economic impacts.The evaluation also considered the relative contri-bution of factors that can explain the level ofeffectiveness achieved, and how Job Networkhas progressed in the light of its design princi-ples. This paper is mainly based on the evalua-tion findings of the third evaluation report.

The evaluation strategy also included an inde-pendent review of the policy framework underpin-ning Job Network. This Independent Review ofthe Job Network was conducted by the Produc-tivity Commission and was published inJune 2002 (Productivity Commission, 2002). Thiscommission had to examine critically andcomment on the framework for deliveringlabour-market assistance, including the applica-tion of the purchaser-provider model to employ-ment assistance, the roles of the relevant players,areas where the model could be improved andthe possible scope for applying the model toother types of services. In conducting its review,the Commission had regard for research alreadyundertaken, but also drew on community inputand called for submissions from interest groupsand the broader community.

Apart from the report published by the Produc-tivity Commission, most evaluation studies havebeen carried out within government by theDepartment of Employment and Workplace Rela-tions (DEWR) as the department responsible foradministering the Job Network. This gives rise toconcerns about the independence of the findings.To date no external scientific evaluations havebeen carried out. A general problem here is thelack of data. The Department is not willing torelease data for scientific research, mainly citingreasons of confidentiality. This explains the lackof hard empirical evidence so far on the effective-

ness of Job Network. The Productivity Commis-sion also notes in its report that ‘Increased trans-parency would also help to increase the exposureof the Job Network to the discipline of externalscrutiny by independent researchers’.

5.3.2. The NetherlandsIn the Dutch reintegration market, the Ministerhas undertaken to monitor developments closelythrough an annual evaluation of SUWI (EersteKamer, 2001). Evaluation plays a prominent partin the Dutch process. As is true for policy ingeneral, the Dutch government devotes consider-able resources to various types of internal andexternal evaluation research.

In contrast with the evaluation concernsexpressed by the Dutch authorities in general, tothe end of 2002 only monitoring studies areavailable on the emerging reintegration market,with evaluation studies of tendering procedures.External studies on the emergent market struc-ture contain a process evaluation of the marketarrangements in practice. These studies werecommissioned by the Department of socialAffairs and Employment, first published in 2000(Dijk et al., 2001) and updated in 2002 (Mevissenet al., 2002). In addition, central indicators arebeing developed by the benefit institution(UWV), the municipalities and the advisory body(RWI), but there is no common integrated statis-tical system (see below). UWV tendering hasbeen evaluated, in particular to consider thegeneral efficiency of tendering and contracting(Cordia et al., 2001; LISV, 2001; UWV, 2002).Furthermore, an economic overview of theevolution of the Dutch reintegration market sincethe beginning of the 1990s has been published(Groot et al., 2002). This study, carried out bythe University of Amsterdam, was commis-sioned by the Department of Economic Affairsand mainly based on available documents andinterviews. Finally, a country specific study ofthe Netherlands was published in the specialJob Network issue of the Australian Journal ofLabour Economics (Struyven and Steurs, 2003).This study was based on original field research(document analysis and interviews) concerningthe implementation in 2000-02 (Struyven andSteurs, 2002).


5.4. Summary

In this chapter we have derived two sets ofcriteria from the theory of quasi-marketsexpounded in Le Grand and Bartlett. The first setof criteria relates to the functioning of thequasi-market; the second is derived from theobjectives of market forces policy. The first setconstitutes formative evaluation, the lattersummative evaluation. In the next chapters, the

evidence concerning the quasi-market reforms inthe employment and training sector are struc-tured following this evaluation framework. Themain questions for formative evaluation concernthe rationale in design and implementation. Towhat extent are the quasi-market arrangementsconsistent with theoretical principles? To whatextent are they relevant to the needs of thelabour-market actors? For summative evaluation,the main question concerns the impact on qualityand effects: do the reforms meet the objectives?


6.1. Market structure

Probably the most important structural conditionsfor quasi-markets to function properly relate tomarket structure. In the cases we consider in thispaper, market competition has been introduced bymeans of open tendering. This requires a separa-tion between purchaser and provider, not only asan institutional setting, but also in the minds of theactors involved. Subsequently, in order to gain theadvantages of competition, various providers(supply side) and purchasers (demand side) mustbe present. Tendering procedures and contractconditions may have an important impact on thedegree of provider competition. Because of theimportance of a level playing field betweenproviders, we also pay attention to the position ofthe former Public Employment Service. A finalissue in relation to the market structure which weaddress is the importance of price competition asopposed to quality competition.

6.1.1. Separation between purchaser andprovider

Tendering requires a strict separation betweenthe purchaser and the provider, known as thepurchaser/provider split, in order to avoidjudge/judged situations.

In all cases studies, the split between purchaserand provider is inherent in the model. Nevertheless,the new institutional setting does not mean that theculture of tendering is automatic. This is mostclearly noticeable in a situation where marketforces are a recent phenomenon. Experiences inthe Netherlands show that, in addition to structuralchanges, a change in mentality is also needed withrespect to dealing with preferential relationshipsfrom the past. This is perhaps one of the moststubborn transitional problems which a govern-ment has to face when shaping market forces.

6.1.2. Provider competitionProvider competition is necessary if the advan-tages of tendering are to be grasped (Fay, 1997;

Derksen et al., 2000). A competitive supply sidewill, however, often be missing when marketforces are introduced into a field where serviceshave traditionally been organised chiefly by thegovernment, as in the case of employmentplacement and reintegration. The design of thetendering procedure and the content of thecontracts can help in the development of asufficiently broad providers’ field (Domberger,1998). After a discussion of these factors, wetake a closer look at price competition in rela-tion to quality concerns. Table 5 presents thetendering procedure in the four countries indiagram form.

6.1.2.1. The relationship between tenderingprocedures and provider competition

The first factor which influences the developmentof provider competition is the scope and durationof the contracts. On the one hand, contracts canbe too small to stimulate the supply side and, onthe other hand, the scope of the contract can beso large that only large providers can participate.For this reason, activities can be groupedtogether or split up into smaller units to interestenough providers, or collaboration can bepermitted. This is the case in the Netherlands(‘tranches’ in the UWV tendering) as well asthe UK (different New Deal tenders). The length ofthe contract is also important. A longer durationenables the providers to recoup their invest-ments. For instance, in Australia some potentialnew participants found the term of 19 monthsunder Job Network 1 too short, which led them todecide not to participate. However, too long aterm can prevent potential new participants fromentering the market. For this reason, the CountyLabour Market Board in Sweden, for example,rarely makes use of the possibility open to it ofextending contracts.

Another factor which influences the number ofproviders is the specificity of investments whichare required to provide the service. For instance,the provision of activation and reintegration

6. Formative evaluation

services requires the deployment of highlyspecific human capital. Organising the provisionof certain training courses is also associatedwith considerable start-up costs. Experience inSweden shows that competition exists particu-larly in training courses which can be offered bythe providers in return for relatively low fixedcosts (administration, IT, etc.), while incapital-intensive sub-markets (welder, mechan-ical engineer, etc.), far fewer entered the market.

A third factor is related to quality require-ments. A great deal of attention is – rightly –devoted to the quality aspects of reintegrationservices and training. The quality requirements

which are imposed can, however, become defacto obstacles to entry to the market. Forexample, the experience requirement in theDutch UWV tendering process means that newparticipants do not stand a chance (see below).

6.1.2.2. Price versus quality competitionIf the quality of a product or service to beprovided can be unambiguously determined, theprice can be used as the only selection criterionin the tendering process. This condition is underno circumstances fulfilled when it comes toemployment placement and reintegrationservices. As well as the price, the proposed



New Deals: tenderswith local purchasing

Tendering with one UWV tendering: organisationsTendering

Type of central purchasing tendering with central

with localtendering

organisation purchasing organisation Employment Zones:purchasing

tendering with centralorganisations

purchasing organisation

UWV tendering: qualified open tendering in two rounds

New Deal Programme:(Usually) open

ProcessOpen tendering qualified open

tendering inin one round

Municipal tendering:tendering

one roundstill being developed,

in two rounds

so far few open tenders

Job Network 1: if qualitycriteria are met, primarily price Price and was important, UWV tendering: price quality

Price versusfixed prices for and quality competition,

Fixed prices, competition,

qualityIntensive Assistance but price never most

therefore 100 % with price

competitionimportant criterion

quality competitionweighted

Job Network 2: 75 % at 30-40 %quality, 25 % price, with minimum prices for Intensive Assistance

UWV procedure:New Deals:

Duration ofConsiderable; leads to

limited by two-phaselimited by three months

Transaction review of procedure procedure but

introduction of regarded ascosts with proposal for

registration andtwo-phase too long

‘roll-over’ of contractsoffer per individual

procedurecontract

Table 5: Tendering procedure

quality must also be a major selection criterion.Relevant and proven experience and expertiseare one possible criterion. But this may implyanother trade-off with the intensity of competi-tion, because too much emphasis on provenexpertise rules out the entry of new players to themarket. Moreover, past performance may beunreliable, because outcomes achieved maydepend on many, and sometimes random,factors. It may be very hard to differentiate objec-tively between providers that have differentcapacities to generate outcomes. If bidderscannot adequately signal and pre-commit todeliver higher quality, then the major basis fordiscriminating between tenders bids will be price.Accordingly, while price competition at thebidding stage can be an important feature ofpurchaser-provider arrangements, it cannot begiven full reign when the purchaser does notknow how much they are buying or the overallquality of the service.

In each of the countries examined, it is clearthat the introduction of market forces cannot berelated to the introduction of price competition,as quality plays an important role. In the UK, fixedprices are used in tendering, for each of the threeprogrammes under consideration. Since theselection criteria are related to the quality onoffer, we can talk of 100 % quality competition. InAustralia, the price criterion was given aweighting of 25 % in the second and most recent

tendering process, while the quality offered wasgiven a weighting of 75 %. In principle, thisenabled higher prices for Job Network providersthat might achieve greater outcomes than theaverage. Free prices apply for job matching andjob search training while for intensive assistanceprices were subject to competitive bidding, but afloor price was set. The floor price protectedquality, but to a large degree it also set the pricefor many Job Network providers as they biddown to the floor. Therefore, the ProductivityCommission, in its independent review of the JobNetwork, recommends that the department setdefault prices for Job Network services (Produc-tivity Commission, 2002). This has actually alsobeen proposed by the government in its discus-sion paper on the new arrangements for the thirdtender to come (DEWR, 2002a). In the Nether-lands, free price competition exists, but the priceis not the most important criterion in any singlecontract. When tendering for training in Sweden,the price is given a weighting of 30-40 % and theprice is often used as the first selection criterion.

6.1.3. Player’s marketIn this section, we deal with the actual develop-ments which have taken place on the supply sidein the countries under consideration, including adiscussion of the position of the former publicprovider. Table 6 gives an overview of the(changes in the) situation in the four countries.


Table 6: Current situation


Number – 205 organisations – UWV contracts: – New Deals: Very extensive providers at 2 114 sites 33 main contractors 2 500 contracts market

– NGOs, private sector, – Total market >650 – Total market:public sector companies 1 800 approved

– Private sector, municipal organisationscompanies

Public – Public company, – In transition as public Public Employment – Lernia (formerlyprovider: fully conforms company with Service as AmuGruppen)(former) to market guaranteed budget benefits agency = public company,Public – Market share until end 2001, and provider privatised inEmployment sharply reduced then privatised of first route two phasesService – Long tradition of – Gradual reduction of phase (variable) (1986, 1993),

contracting out compulsory purchase fully conformsto private players from Public Employment to market

Service – Declining marketshare

6.1.3.1. Concentration and increases in scaleWhat developments actually took place in thecountries considered? In Australia, 306 providerswere selected in the first round of tendering (JobNetwork 1), who offered their services at over1 000 sites. The second tender reduced thenumber of candidates to 205, while the numberof sites rose to 2 114. Providers had thus becomelarger and the supply side more concentrated. InJob Network 2, the four largest providerstogether accounted for roughly a third of themarket for each of the services provided. Jobsearch training is the least concentrated marketsegment, but concentration increased mostsignificantly for these services (see Table 7). Theconcentration in intensive assistance fell sharply

because Employment National withdrew almostcompletely as a provider in this market, while itheld 42 % of the market under Job Network 1.

In the Netherlands, the number of contractedproviders increased from 33 after the firstUWV-tendering to 41 after the second UWVround. This meant there was a reduction in theconcentration, though this remained high. Thefour largest providers had a combined marketshare of almost 59 % after the first round, andthis was still more than 47 % after the secondround (calculations based on data provided byLISV). Note that the UWV-market represents onlya sub-market within a broader market structurethat is constituted by the UWV, the municipalitiesand the employers (Struyven and Steurs, 2003).



Other public None Municipal companies, Educational (municipal) providers educational establishments establishments educational

establishments

NGOs – Sharply increasing NGOs are classed Less known about – Wide variety ofmarket share as public organisations them, do not really players

– Previously already stand out – Trade union active as provider organisations

– Professionalisation, – Former employeesscale increases and of AmuGruppenconcentration – Entry and exit

– Joint ventures depending on scope of government budget

Private – Sharply rising – From pure organiser Employment Zones:providers market share to provider conscious choice

– Previously already – Increase in scale and for close involvementactive as provider alliances, e.g. of private players

– Few new members with insurers as well (Working Links = as niche players public-private

joint venture)

General – Market concentration – Purely private market – Developing market Entry in urbanisedtrends – Relatively few entrants with sub-markets – Concentration regions and for

(medically-oriented, movement non-capital-work-oriented) through tendering intensive training

– Driven by insurers of complete – Very mobile market packages

Market share Employment National: KLIQ: 17 %, UWV market, Employment Service Lernia (formerly by public 8 % 50 % municipal market as principal AmuGruppen): provider (expected) 25 %

In order to guarantee a varied supply, rules aresometimes applied regarding a minimum numberof providers. For instance, the departmentresponsible for allocating contracts in Australiaassumes that at least 5 providers are present ineach region and that none of the providers will beallocated more than 50 % of the total contractedvolume in each region. In the Netherlands, a rulestates that if a contract is split up into several‘tranches’, a reintegration company cannot beawarded more than one tranche. In the UK too,when contracts are allocated, a sufficiently broadmix of market players is ensured.

6.1.3.2. Difficult entry to the marketOverall, it also seems that entry to the market hasremained relatively limited, especially by the largeproviders. Under Job Network 1 in Australia, onein five providers was new, while under JobNetwork 2 fewer than one in six of the providerswas new. Most providers, even the genuinelyprivate parties, had already been involved asproviders previously – after tendering – in thelabour-market programmes that can be regardedas the forerunners of Job Network. In the Nether-lands, 18 new companies entered the market inthe second tendering round, but together theyonly obtained a market share of about 10 %.

This limited entry to the market is related,among others, to the selection criteria, which

include the expertise and experience of thepotential providers. The Australian National AuditOffice observed in its analysis of the results of thefirst tender that organisations which at that timewere in receipt of funding for providing casemanagement or other, related employment place-ment services, achieved a much better result thanother candidate providers. This is clearly illus-trated by the figures in Table 8.

While the average success rate after the firstround of tendering was 30 %, it was significantlyhigher for already established providers (43 %)and much lower for new providers (14 %). In spiteof this, almost 45 % of the candidates were newproviders.

In the Netherlands, requirement for experienceexcludes new entrants to the market. It is envis-aged that 20% of the contracts can be allocatedwithout tendering, for example to new playerswho put forward an innovative approach. InSweden, new players are entering the market,particularly in the urbanised regions and fortraining courses which are not capital-intensive.This can be explained by the widely fluctuatingscope of demand, which means that new playersdo not run the risk of investing in the setting up ofexpensive training programmes.


Job Network 1 Job Network 2

Job Matching 32 35

Job Search Training 21 31

Intensive Assistance 65 34

* The C4 concentration ratio corresponds with the sum of the market shares of the four largest providers. The market share iscalculated on the basis of the contracted volume.

Source: Davidson, 1998; DEWRSB, 2000a.

Table 7: C4 concentration ratio*

Established providers New providers Total

Number of applicants 569 447 1 016

Successful applicants 243 63 306

Success rate 43 % 14 % 30 %

Source: ANAO (1998)

Table 8: Success rates of established versus new providers (Job Network 1)

(%)

6.1.3.3. The position of the former publicemployment service

The introduction of market forces has had asignificant influence on the status and role of the(former) public employment service. In Australia,the Netherlands and Sweden, the introduction ofmarket forces was (partly) responsible for reformsto the public employment service because it isimportant to create a level playing field in whichproviders are treated equally.

In Australia, the Commonwealth EmploymentService was closed down and a new companyset up: Employment National. This new publiccompany still only has the function of provider ofemployment placement and reintegrationservices, where it has to compete with otherproviders. The remaining tasks of the Common-wealth Employment Service were assumed by thedepartment and by Centrelink. Whereas Employ-ment National still played an important role in JobNetwork 1, its market share has fallen sharply inJob Network 2; for intensive assistance it hasactually fallen from 42 % to 1 %. The transition toa public company which has to compete on anequal footing with other providers has been verydifficult, involving substantial staff cuts. As aresult of the sharply declining market share in thepublicly financed placement and reintegrationmarket, the company recently developed activi-ties for the private market. But the Minister forFinance and Administration has announced thatthe marketable parts of Employment National willbe sold and the remainder of the companywound up on 30 June 2003.

In the Netherlands and Sweden the publicprovider was not abolished, but reformed. Inthe Netherlands, KLIQ is being privatised and allthe reintegration activities of the employmentservice are being made its responsibility. KLIQ isalso a purely commercial enterprise, which nolonger has any privileged relationship with thegovernment. Privatisation has been accompaniedby staff cutbacks and financial problems. Asimilar story can be told about AmuGruppen, thegovernment provider of training courses inSweden. This organisation was privatised in twostages, into a company operating fully in line withthe market and whose shares are now held 100 %by the Swedish government. AmuGruppen – nowknown as Lernia – also saw its market share in thepublicly financed training market decline sharply,

leading to reductions in staff numbers. An increas-ingly large percentage of its turnover now comesfrom activities for the private sector.

6.1.4. The position of the purchaser

6.1.4.1. Degree of monopoly at the purchaserside

A crucial determinant of the operation of marketmechanisms in social service delivery is thedegree of monopoly that characterises the clientside. In a quasi-market it is usually not the directuser who exercises the choices concerningpurchasing decisions; they are delegated to athird party such as a government department orauthority in employment and training (Le Grandand Bartlett, 1993). In some cases, there is amultitude of purchasers and a multitude ofproviders. The opposite situation is a tendencytowards bilateral monopoly, with a singlepurchaser and a single provider. In between liesituations in which there are few purchasers andmany providers, or few providers and manypurchasers (Walch, 1995).

Practice in the various countries varies, theposition of the purchaser being important.Australia and the Netherlands are the twoextremes: Australia has only one purchaser; inthe Netherlands, the tendering practice tendstowards a quasi-market with a large number ofbuyers and providers. In between lies the UKsystem, which is characterised by manyproviders and few purchasers (EmploymentService and Department of Work and Pensions)which operate several tender processes. Table 9represents the different market types.

The Dutch situation is related to the impor-tance attached to the municipalities forlabour-market policy, as well as to employers,who have been made financially responsible forsickness and disability benefits (Struyven andSteurs, 2002; 2003). With regard to the popula-tion of job seekers, the purchase structure is leftup to the autonomous government institution(UWV) and to the municipalities. The differenti-ated purchaser structure has a greater chance ofleading to a quasi-market with a multiplicity ofpurchasers and suppliers. However, Struyvenand Steurs point out that in 2001–02 this marketstructure had not yet introduced much differenti-ation on the purchaser side. The most importantpurchaser, UWV, still very much has the char-


acter of a monolithic regional body invitingtenders; purchasing by municipalities is still at anearly stage (Struyven and Steurs, 2002). Ingeneral the municipalities have not yet built upenough expertise for their own purchasingpolicy. Municipalities are used to operating as‘benefit factories’ and now they have to trans-form themselves into ‘reintegration directors’.Often, there is a lack of knowledge of both theclient and the labour market. One key difficulty isthat it is not yet fully in the interests of themunicipalities to have an open market. They areplaying a double role because they have publiccompanies themselves providing jobs under theJob Seekers Reintegration Act and the SocialEmployment Act. In principle, these municipalcompanies have to compete in a way that is fullyin accordance with the market, which means thatthey are also registered for VAT. It seems that apurchasing policy is only a possibility for thelarger municipalities. Only a minority of munici-palities organise an open bidding process, whileothers operate more selective tendering proce-dures (Mevissen et al., 2002).

6.1.4.2. Degree of centralisation at the purchaserside

Another determinant in the functioning of marketmechanisms is the degree of centralisation. Toavoid problems of coordination while developingmarket processes, governments tend towards theestablishment of tendering frameworks, uniformprocedures and contracts, performance measure-ment systems, etc. This tendency reflects pres-sure for centralisation, whereas the specificationand fine-tuning of tendering conditions andcontracts tend towards decentralisation.According to Walch, there is a threefold distinc-

tion in the organisational structure of a marketmodel: the corporate core responsible forstrategy and policymaking; the client side forservices responsible for setting and monitoringstandards; and the service provider, who actuallydelivers services (Walch, 1995).

The early experiences in the four case studiesindicate that there is no single way of resolvingthe tension between centralisation and decen-tralisation. The tendering mechanism can bestructured from the central policy level or asmaller territorial policy level. In that respect, thefour case studies examined exhibit major differ-ences. In Australia, competences are stronglyconcentrated, with the central department inCanberra acting as sole purchaser as well aspolicy-maker. This contrasts with the decen-tralised model in the Netherlands and Sweden,where the regional level (County Labour Boardsin Sweden) or the local level (municipalities inthe Netherlands) have an inalienable role aspurchaser and principal. The situation inthe Netherlands (UWV tendering) and the UK(New Deals) is closer to the centralist model,although regional departments have more impacton the tendering process.

Decentralised purchasing organisations offer adual advantage: they are closer to the field sothat they are better able to assess needs and,since the market has more buyers, the relation-ship between buyers and providers is morebalanced. The use of a uniform tendering proce-dure and model agreement is important: thiscontributes not only to greater transparency ofthe tendering process, but also to lower transac-tion costs. Each of the three countries mentionedattempts to standardise the procedures.


ProvidersFew Many

Few Bilateral monopoly/oligopolyMonopsony/oligopsonye.g. the Australian Job Network

Purchasers

Many Monopoly/oligopolyFree (quasi-)markete.g. the Dutch reintegration market

Source: Based on Walch, 1995

Table 9: Market types based on the number of providers and purchasers

6.2. Information and transparency

Following the quasi-market theory, markets onlyoperate efficiently when both sides of the markethave access to cheap and accurate information,particularly concerning the costs and the quality ofthe service concerned. Le Grand and Bartlett(1993) point out that ‘Providers must be able tocost their activities so as to be able to price themappropriately. Purchasers must be able to monitorthe quality of the service they are purchasing, soas to limit the opportunity for providers to reducecosts by lowering quality’. The information defi-ciencies stem from the outcome-based focus ofthe market model. An outcome-based approachdoes not need to specify what processes are usedby providers to obtain jobs for their clients, ascompetition over time should shift providers to thebest methods (Productivity Commission, 2002).However, it may still be in the interests of thepurchaser to discover what processes are actuallymost effective and to diffuse them among theirsuppliers. Furthermore, information deficienciesmay underlie problems in price competition andjob seekers’ choice. Finally, transparency isconsidered to be an important policy principle,where the provision of public funds is involved(Productivity Commission, 2002).

6.2.1. Lack of transparencyIn the case studies of Australia and the Nether-lands, there is poor information about what worksin practice. This is seen as a disadvantage ofpayment based on outcomes (ProductivityCommission, 2002). Publicly available informationabout what providers actually do with their clientsis relatively poor, This is especially so with thecontent and intensity of services, such as theimportance of training where the available moni-toring and evaluation documents offer fewdetails. Information sharing and coordinationbetween providers are limited. Job seekersstruggle to make choices because of a generallack of transparency in the system.

In the Netherlands, little attention is yet beingpaid to addressing the problem through datacollection (Struyven and Steurs, 2002). In general,computerisation is lagging behind. The provision ofinformation and facilities to monitor reintegrationefforts have not been able to keep up with therapidity of the changes. On the purchaser side, the

UWV tendering procedure has created more trans-parency in their corresponding market segment(Cordia et al., 2001). A uniform tendering proce-dure, model agreement and predetermined selec-tion and award criteria also increased the trans-parency of the procedure compared with the firsttender and contributed to lower transaction costs(UWV, 2002). One crucial question is whether theUWV tendering procedures are focused in aone-sided way on the new inflow of unemploymentand disability insurance benefit claimants. Place-ment and drop-out figures have an important partto play in the payment of fees; it might, therefore,be expected that there would be much more trans-parency in the validity and reliability of the place-ment figures used. In the municipalities marketsegment, several monitoring initiatives have beenset up in the context of different programmes andfunding mechanisms, but without any coordina-tion. According to an external monitoring reportordered by the government, the municipalities aresuffering from an overdose of monitoring and eval-uation (Mevissen et al., 2002). This can beexplained by the absence of an overall integratedemployment and client information system.

In contrast to the Netherlands, Australiaalready has an employment information andservice delivery system. This system providesextensive information about Job Networkmembers (including information on their perfor-mance) and services to job seekers. Despite this,a general lack of transparency is mentioned bythe Productivity Commission’s report and recog-nised by the Government’s response (ProductivityCommission, 2002; DEWR, 2002b). The Commis-sion recommends that provision of information tojob seekers about Job Network and the associ-ated referral system be enhanced to allow jobseekers greater scope for informed choice.

6.2.2. Contractual compliance monitoring inJob Network

The Australian Department acting as a purchaserhas in place an extensive regime for monitoringand quality assurance. It makes use of codes ofconduct, declarations of intent and intensive assis-tance support plans as well as the Job Networkcontracts themselves. According to DEWR, theJob Network Code of Conduct is the centralfeature of consumer protection under JobNetwork. The aim of the Code is to produce thebest outcomes for job seekers and employers by


developing a high-quality, continuously improvingservice that engenders ethical behaviour betweenall parties. All Job Network members are requiredto meet the minimum standards set out in theCode, which forms part of their contract with theCommonwealth. Organisations tendering for inten-sive assistance in the second round were requiredto submit declarations of intent that described therange of services they would provide to jobseekers. The declaration of intent forms a part of aprovider’s contract with DEWR (6).

Many providers perceive increasing complianceburdens that direct them away from their main goalof placing disadvantaged job seekers in employ-ment (Struyven et al., 2002). The use by providersof the Integrated Employment Service is verycomplicated. Clearly, there will always be a needfor a level of compliance monitoring that canensure that outcomes paid for are in fact delivered,and detect and deter unintended behaviours byproviders. But too strict a monitoring regime willimpose a high administrative burden on both thedepartment and providers, and has the potential toreduce flexibility and deter innovation. The Produc-tivity Commission Report raises the idea of an inde-pendent agency to oversee the Job Network anddepartment’s management of it. Such an agencycould address issues of transparency and account-ability, and ameliorate any problems of powerimbalance and distrust between the Job Networkplayers and the department. For the Commissionthis is a conditional recommendation ‘if significantproblems do continue’ under the new contractperiod (Productivity Commission, 2002). TheCommission also recommends that all data on JobNetwork programmes that is not confidential bemade available for independent scrutiny by otherresearchers.

6.2.3. Performance measurement in JobNetwork

To assess the performance of providers, theAustralian Department has designed a star rating

model, to be used to renew contracts and to payoutcomes. The model uses a set of performanceindicators and associated weightings based on theperformance indicators outlined in the Job Networkcontracts (7). A provider’s actual performance isassessed against its expected performance,adjusted to take account of variations in client mixand local labour-market conditions. Scores aredistributed between one and five stars such that70 % of providers in a region are rated at threestars or better. The star ratings system has occu-pied a central place in the operation of the market.

The star rating appears to have increased JobNetwork’s focus on securing outcomes for jobseekers. Stakeholders have both welcomed andcriticised this approach. Supporting a focus onoutcomes is based on two arguments:(a) a good way to secure a lasting job is to

accept any job in the short-term. In otherwords, there are more likely to be lastingbenefits for job seekers if the providerfocuses on securing a job placement ratherthan on spending time building up skills, orprogressively addressing any barriers toemployment. But critics claim that short-termemployment may not necessarily in the longterm interest of some disadvantaged jobseekers (DEWR, 2002c, pp. 152-153);

(b) a greater weight to outcomes for more disad-vantaged job seekers combats incentives forproviders to work most intensively with theeasiest to place candidates.

Nevertheless, many participants are critical ofthe model, mainly because of concerns that itlacks transparency and measures performanceon too narrow a basis or with too much error(Struyven et al., 2002; Productivity Commission,2002). Job Network providers do not know thespecification of the model. They are also mixed intheir understanding of where the ‘goalposts’ are.There are cases where the rules have beenchanged after providers have geared their busi-nesses to meet the original rules. This in turn has


(6) Each provider must draw on its declaration of intent to draft an intensive assistance support plan that specifies the activitiesand services it will provide to each individual job seeker remaining on their caseload 13 weeks after they commence. While theintensive assistance support plan is not legally binding between the provider and the job seeker, DEWR checks the provider’sdelivery of activities as specified in the support plan as a part of its contract monitoring activities.

(7) For example, in the case of intensive assistance, the current weightings are as follows: 40 % for interim outcomes (13-weekoutcomes) as a proportion of commencements, 30 % for final outcomes (26-week outcomes) as a proportion of commence-ments, 10 % for the proportion of participants placed in a job and 20 % for the proportion of interim outcomes going to disad-vantaged job seekers.

impacts on business confidence, certainty andinvestment. The Commission’s report recom-mends that full details of the star rating model bemade publicly available, including periodicassessments of its technical validity. ‘Trans-parency will help alleviate some of the distrust ofthe model and allow better models to beconstructed in the future’. The evaluation alsorecommends that all declarations of intent (andsimilar documents that specify services to beprovided to job seekers by individual providers)as well as information on contracts (nature,extent and cost; compliance monitoring) shouldbe made public (Productivity Commission, 2002).

Another critical point is what is actuallymeasured by the star rating. Placement outcomescannot be assumed to be a measure of the overallquality of service provision. Performance is onlyone aspect of quality. The key problem is how todefine and measure an outcome.

6.3. Transaction costs

The transaction costs associated with setting upa tendering process, both for purchaser andprovider, are often underestimated (Domberger,1998). Transaction costs for purchasers include,for example, the costs of writing and publishingthe specifications, evaluating the offers, drawingup model contracts, completing negotiations andmonitoring the agreements. Typically, all thisrequires a great deal of senior management timeand a level of expertise that may not always bepresent within the body which will be responsiblefor the tendering process. Providers also incurtransaction costs, for instance when preparingtheir bids. In each of the case studies, the trans-action costs emerged as an important area ofattention.

In Australia, the preparation of the twoprevious tenders, evaluating the offers andcontract processing required the deployment ofhuge numbers of people and vast resources bythe purchaser. But most providers in the JobNetwork also regard competitive tendering asexpensive and time-consuming. The tenderdocuments are very complex, as are thecontracts. The tender process — preparation oftender bids, assessment of bids and finalisationof contracts — disrupts service flows noticeably

during the second contract. Renewing the marketthrough a global tender also contributes to busi-ness uncertainty among providers and confusionamong job seekers and employers. The govern-ment has therefore proposed that around 60 % ofthe existing capacity be rolled over under a thirdcontract for the best-performing providers, withthe remaining share being subject to tenderbidding (DEWR, 2002a). The contract rollover willpartly avoid the high disruption costs to jobseekers of Job Network 2, but since manyproviders may still be bidding for capacity at theEmployment Services Area level, the contractingtransaction costs may not be appreciably lowerthan previously. This suggests that the costsinvolved in a hybrid rollover/tender model meanthat it too is not sustainable.

An alternative, proposed by the ProductivityCommission (though not followed by the govern-ment), is to use licensing as the basis for entry intothe industry (Productivity Commission, 2002,p. XLI). Licensing would be dependent on aprovider meeting certain minimum standards andagreeing to various contract provisions (such as acode of conduct, undertaking assessment of jobseekers and whatever activity testing was agreed).The standards set would be similar to those estab-lished by the department for pre-tender qualifica-tion. Renewal of the licence would be tied toperformance, so that the vital role performed byexisting arrangements in driving poorer suppliersfrom the market would be maintained.

In the Netherlands and the UK, attempts aremade to limit transaction costs by working with aprocedure in two rounds. In a first stage compa-nies come forward as candidates for the statedtarget groups. In order to be invited to take part inthe second round, predetermined, objectiveselection criteria have to be met. In a secondstage, the purchaser invites the selected reinte-gration companies to submit a tender. Contractsare awarded on the basis of predetermined awardcriteria. This implies savings for both the principaland the potential providers. Another advantage ofthis selection procedure is that a larger number ofproviders will be interested in applying, given therelatively limited efforts which have to be made tosubmit an offer. However, if only a limited numberof providers can actually submit an offer afterselection, this could result in higher pricesbecause the competition is limited.


6.4. Motivation: the combinationof input and output financing

The balance of motivation is affected by the intro-duction of market incentives (Le Grand, 2000). Inorder to influence the provider’s behaviour and totackle information asymmetries, much attentionhas been paid to the payment structure. The rela-tionship between the principal and the providers ischaracterised by a number of information imbal-ances which influence the payment structure used(Canoy and Vollaard, 1999). These informationasymmetries relate in the first instance to thefactors which determine the costs of reintegration:the technology and the efforts that have to bemade by the employment placement consultant(provider). The technological capacity of anemployment placement consultant relates, forexample, to its access to information networks, itsknowledge of the demand for labour or its knowl-edge of reintegration opportunities. Clearly, theseproviders will know more about their technologicalcapacity than the government. The efforts of theemployment placement consultants also deter-mine the extent to which they use their capacitiesefficiently, for example the amount time devoted toeach client. The government could monitor theseefforts. But the result of the efforts in the form ofthe number of reintegrated benefit claimants willbe more easily seen by the government, makingan abstraction of the measurement problems andof whether the output is as desired. Informationasymmetries may also arise because the employ-ment placement provider has a clearer insight intothe profile of the unemployment pool for which areintegration programme is being developed. Thatprovider can then focus on the most easily placedcandidates (creaming).

6.4.1. Input and output financingThese information asymmetries create problemswhen it comes to paying providers. The govern-ment will want to structure funding arrangementsin such a way that providers are encouraged toachieve efficient and effective reintegration. Thetheory puts forward a combination of input andoutput financing as a solution: the employmentplacement consultant receives an ‘initial fee’ in

advance per person to be placed and a ‘place-ment fee’ afterwards if the efforts made result inplacement of the job seeker in a job. Through theoutput financing part, the providers are encour-aged to provide services which produce results.The input financing results in a fair spread of riskbetween the principal and the contractor.

It is clear from the case studies that thepayment structures used in practice often fit inwith the theoretical solution. The providersreceive an initial fee as soon as a number ofminimum requirements are satisfied (for example,the formulation of an ‘activity agreement’ forintensive assistance in Australia). In the event thatthe counselling results in an ‘outcome’, typicallydefined as placement of the job seeker in a job,the providers receive a placement bonus. Sincethe placement of a job seeker in a permanent jobis better than placement in a temporary job, theplacement bonus is sometimes divided up, with ashare only being paid when the job seeker hasbeen working for a certain number of weeks. Forinstance, an ‘interim placement fee’ is awardedfor intensive assistance in Australia if the jobseeker remains in work for 13 weeks and a ‘finalplacement fee’ after 26 weeks. A comparablesystem applies to the employment zones inthe UK. More details of the payment structure aresummarised in Table 10.

6.4.2. The Dutch payment systemThe Dutch payment system deserves specialattention for two reasons. The first is that providersmust ‘guarantee’ a placement percentage (= resultcommitment) in their offers, which also applies asa selection criterion. Second, providers may earn abonus if they do better than they promised. Thepayment consists of three parts: a fixed payment,a placement fee and a bonus, and depends on thefollowing parameters:N = the number of clients commencedC = the contracted price per client (specified by

providers in their bids)f = the fixed payment percentage (specified by

UWV in its call for tenders) (8).Pg = the ‘guaranteed’ placement percentage

(specified by providers in their bids) Pr = the realised placement percentage


(8) The more difficult the target group, the higher the fixed payment, in order to combat creaming.

The ‘fixed payment’ (like the commencementfee in Australia) is paid for every client whoactually starts a programme, independent ofthe placement percentage. It is equal to N*C*f.The ‘placement fee’ is outcome based. It isequal to N*C*(1-f)*Pr. The final part is ‘thebonus’, which is also outcome-based but whichalso depends on the comparison between therealised placement percentage and the guaran-teed placement percentage. It is equal toN*C*(1-f)*Pr*(1-Pg)/Pg.

Providers are rewarded if they do better thanthey are contractually obliged to. This is not thecase within Job Network, where there is no bonuspayment. The bonus formula also implies that a

higher guaranteed placement percentage has anegative impact on the size of the bonus. But theinvitation to tender specifies minimum placementpercentages which vary for each target group.Higher minimum placement percentages apply tothe target groups which are easier to place in work,in order to counteract creaming.

The total payment is equal toN*C*(f+(1-f)*Pr/Pg). This formula clearly showsthat if the realised placement percentage ishigher than the guaranteed placementpercentage (i.e. Pr > Pg), a company receivesmore than the price submitted (N*C) (9); but thereverse is true if a provider fails to place as manyclients as contracted (i.e. when Pr < Pg) (10).


Table 10: Payment structure and competition


Input versus Job matching: UWV tendering: Combination of Only input output only output financing combination of input input and output financingfinancing and output financing, financing

Job search training: plus bonus if combination of input performances are better and output financing than contractually

stipulatedIntensive assistance: combination of input,interim outcome and final outcomefinancing

Measures Providers cannot Large number of New Deals: providersto combat refuse job-seekers target groups, cannot refuse creaming off individually tendered job-seekers; large number

Intensive assistance: of packages with different job-seekers divided Minimum placement pricesinto various categories percentages per

target group Employment zones:output financing

The more difficult dependent on length the target group, of unemploymentthe higher the fixedcompensation

Choice for Theoretically yes, For larger target Some job-seekers Job-seekers job-seekers in practice barely groups, choice between do have options are referred on

several providers by Public EmploymentService Office

(9) Suppose a company offers to help 100 clients to find a job (N = 100) for a price of EUR 1 000 per client (C = EUR 1 000). Theguaranteed placement percentage is 55 % (Pg = 0.55). The fixed payment for the target group is 40 % (f = 0.4). If the companyis able to place 60 clients (Pr = 0.6), it will receive a total payment of EUR 105 455.

(10) If the company can only place 50 clients in a job (Pr = 0.5), the total payment will be EUR 94 545.

6.4.3. Output financing and trainingOne important condition for the use of outputfinancing is that the intended result can be clearlyand objectively defined. This is not easy fortraining. The purchaser might only pay the trainingprovider if the job seeker successfully completesthe course or, to direct incentives towards trainingwhich really helps job seekers to find a job, if thecompleted training results in employment.

There are, however, a number of disadvan-tages to such an approach. Training providerswould be encouraged to keep job seekers in theircourse longer than may be necessary such as incases where the job seeker was offered a jobbefore the end of the training. Providers wouldalso be financially penalised by early leavers.

If a job seeker finds a job after completing atraining course, it is not clear how it can be provedthat finding the job was the consequence of thetraining. This helps to explain why Sweden onlyenvisages input financing for training providers.Nevertheless, the expert group on efficient trainingprogrammes for the unemployed is of the opinionthat the National Labour Market Board should try todevelop mechanisms where the payment forproviding vocational training is linked with perfor-mance standards (Näringsdepartementet, 2000).The two possibilities they suggest are to link thepayment to individual knowledge goals or to link abonus to the percentage of participants who haveemployment after the training programme.

Another disadvantage of output financing oftraining services is that providers may not bewilling to take many risks and may invest too littlein expensive training for the job seekers they arecounselling, given that they are not certain ofbeing able to recover these costs through aplacement fee.

The absence of specific training funding in theoutcome-based payment structure seems to be aproblem in Australia, where there is widespread –if anecdotal – evidence that the amount oftraining provided has been massively reduced bythe Job Network providers (see below). However,training can be individually compensated, as inthe Netherlands.

6.4.4. Impact of the outcome-based focus onprovider behaviour

The star rating model, as applied in the AustralianJob Network, has a significant impact on providerbehaviour. A profit maximising contracted agencywill aim for the best outcomes as measured by thecontract indicators, but not necessarily the bestoverall outcome. In any system designed aroundoutcome payments, providers will work to theincentive structure built into the contract. Ideallythe contract should incorporate all importantoutcomes. This is not always easy to achieve: ‘[...]competition for “stars” [is] continuing to take thefocus away from “what is best for the client” andinstead emphasising “how can I get a payableoutcome?”’ (Mission Australia in ProductivityCommission, 2002, p. 3.13). This goes against theintegrity and the spirit of the Job Network (Ibid.,11.20). There are many possible manifestations ofthis in the Job Network. Where there is any discre-tion in taking or subsequently excluding clients,incentive payments based on simple performanceoutcomes may lead to the taking on of those jobseekers most likely to achieve payable outcomes(creaming). In addition, providers may avoid jobplacements that last less than 13 weeks for inten-sive assistance clients or less than 15 hours for jobmatching clients, these being the relevant thresh-olds for outcome payments specified in thecontract. Job Network providers may also haveincentives to manipulate the system by ‘parking’(providing little assistance to) clients with lowemployment probabilities, creating temporary arti-ficial jobs that maximise placement payments, androtating people through them. Job outcomes perse may underemphasise or ignore the wage level,duration or quality of a job.

In the Netherlands, a debate is going onwhether to increase the weight of the placementfee and to move on to a ‘no cure no pay’ system,at least for the job seekers who are relatively easyto reintegrate (11). This may cause several disad-vantages: providers will raise their prices, they willmeet problems of liquidity, and they will invest lessin the hard-to-place job seekers or in excessivetraining for the job seekers. Until now, training hasbeen individually compensated for in the Nether-lands. In the future this may change since the idea


(11) An estimated 70 % of the total number of job seekers.

is to finance training only if the provider managesto find a job for the job seeker (SZW, 2002).

6.5. Creaming and parking

Competition within quasi-markets in combinationwith output financing can give rise to ‘creaming’:the providers select the job seekers who are theeasiest to place, in order to increase their profits.Job seekers at the greatest distance from thelabour market are left out in the cold. Whilecreaming refers to the practice of selecting jobseekers who are the easiest to place, parkingrefers to the practice of providing little assistanceto those with either insurmountable or highbarriers to obtaining work, or who have low likeli-hoods of achieving payable outcomes. But lowlevels of assistance are not necessarilyattributable to the provider alone: parking mightalso be a behavioural response by demotivatedjob seekers (self-parking).

The contracts and payment structures as usedin the countries under consideration contain anumber of elements which are explicitly intendedto counteract these phenomena. First, providerstypically cannot reject the job seekers who arereferred on to them: this is the case in Australiabut not in the Netherlands.

Furthermore, it is also usual to classify the jobseekers into various categories, depending ontheir distance from the labour market, and toadjust both the amount and the relative weight ofthe commencement fee and the outcome fee. Forinstance, job seekers for intensive assistance inAustralia are divided into two categories, with the(minimum) price being higher for the category ofthose most difficult to place in work. However, inJob Network 2, the number of categories wasreduced from three to two, which increases thechance of creaming within the categories. Inthe Netherlands, 22 target groups were coveredby the UWV tendering process in 2001, a numberof these being grouped by sector. Tendering tookplace at the level of the (partly) sector-specifictarget groups. This is intended to providecustomers with more tailor-made services, aswell as counteracting creaming within excessivelylarge groups of people. In the (target group-specific) New Deal programmes in the UK, theservices put out to tender are divided into a large

number of packages depending on their durationand content, which restricts the possibilities forcreaming within those packages.

In addition, contracts with providers in theNetherlands include a type of result commitmentin the form of a placement percentage (seeabove). The minimum placement percentagesvary for each target group: higher minimumplacement percentages apply to the targetgroups which are easier to place in work.

Finally, the initial fee for the target groupswhich are more difficult to place is higher than theplacement fee (see above). A form of positivecreaming applies within the Employment Zones:the placement fees are higher for job seekerswho have been out of work for over three years,so that providers can concentrate their efforts onthose most in need.

The problem of parking could be overcome bymoving to a fee-for-service system in which any ofa wide range of service types would simply bepaid for by government when delivered to jobseekers. In that case, parked job seekers wouldreceive services because it would then be in theinterests of providers to maximise service delivery,rather than outcomes. However, over-servicingand poorly targeted service delivery are well-known risks of a fee-for-service system.

By defining the target groups as specifically aspossible and also by region, the aim is to delivermore tailored services to job seekers. This canalso be seen as a way of preventing creamingwithin wider populations. Finally, it fosters smallerspecialised companies to enter the market.Nevertheless, a majority of providers inthe Netherlands consider the large number oftarget groups as negative. According to the mainDutch purchaser, less differentiation in targetgroups may be expected during the third roundbecause most of the job seekers have recentwork experience and can be easily reintegrated(UWV, 2002).

From a formative evaluation point of view, itcan be stated that the case countries are fullyaware of the danger of creaming, and have there-fore incorporated a large number of stipulationsand refinements in the design of the marketsystem. In the summative evaluation the questionis whether these adjustments do actually avoidcreaming, parking and related problems ofinequity.


7.1. Efficiency

There is widespread evidence from a number ofauthoritative studies that market systems are themost efficient (Savas, 2000). Savings seem mostlikely to be made in the case of simple and repet-itive services such as refuse collection andcleaning. In more complex services, such ashealth and social care, there seem to be fewsavings to be made (Deakin and Walch, 1996).However, some scepticism is justified, as ‘manystudies rely on assertion or on surveys ofmanagement perception’ (Ibid.). For employmentand training services there is mixed evidenceabout possible efficiency gains.

For the Netherlands, there is as yet noevidence of increased efficiency in the reintegra-tion market compared with the preceding period.Cost savings do not play a role, as the overallbudget for reintegration activities was increasedduring the reform period (Mevissen et al., 2002).The total volume of business is estimated to beEUR 770 million, accounting for 160 000 jobseekers (Groot et al., 2002). Prices are rising, withan average of EUR 4 700 a trajectory in 2002,compared with EUR 3 500 in 1998 provided bythe former public employment service Arbei-

dsvoorziening (Ibid.). So one cannot yet concludethat the market model in the Netherlands hasbecome more cost-efficient, since prices arehigher compared with the pre-market period.

Evidence from the Australian Job Networkseems to confirm the expected gain in cost-effi-ciency. According to the final evaluation report, theintroduction of Job Network has resulted in asubstantial reduction in the cost of achievingemployment outcomes (DEWR, 2002c). JobNetwork costs per employment outcome (12) havebeen the lowest achieved in the past decade: atabout AUD 5 000-6 000 since mid-1998, comparedto between AUD 10 000 and AUD 16000 in themid-1990s and AUD 8000-9000 in the early 1990s(Table 11).

In 2000–01, costs per employment outcomewere AUD 560 for job matching, AUD 1 390 forjob search training and AUD 5 440 for intensiveassistance. The costs for the second and thirdwere dramatically below those of the comparableprogrammes that they replaced: respectively,AUD 2 600 for job clubs and AUD 12 100 for thegroup of programmes replaced by intensiveassistance. The substantial efficiency gains madeby Job Network have also translated intoimproved cost-effectiveness, measured in terms

7. Summative evaluation

(12) These are gross outcomes.

Period Cost per employment outcome (AUD)

Early 1990s 8 000 – 9 000

Mid 1990s 10 000 – 16 000

Late 1990s 5 000 – 6 000

Type of service Working Nation Job Network

Job matching – AUD 560

Job search training 2 600 (a) 1 390

Intensive assistance 12 100 (b) 5 440

(a) Job Club(b) Average for seven programms replaced by Intensive Assistance

Table 11: Cost-effectiveness of Job network

of costs per net impact (DEWR, 2002c,p. 126-127) (13). The improved cost-effectivenessof Job Network is largely a function of greaterefficiency rather than greater effectiveness, sincethe net impact of Job Network services wasfound to be similar to the net impact of theprevious labour-market programmes (see below).

The independent review by the ProductivityCommission gives some additional figures whichallow an evaluation of the efficiency of JobNetwork (Productivity Commission, 2002,p. 5.24). For the four years from 1996/97 to1999/00, the Government allocated more thanAUD 5.4 billion to fund all labour-market assis-tance and entry-level training programmes;AUD 1.6 billion for 1996/97, and an average ofAUD 1.3 billion per year thereafter (of which JobNetwork programmes have averaged aboutAUD 0.75 billion per year for the past threeyears). This compared to annual average fundingin 1994/95 and 1995/96 of AUD 3.7 billion.

7.2. Responsiveness and choice

A key indicator of Job Network’s responsivenessis its capacity to target services to job seekers.Therefore, we first examine application of the jobseeker classification instrument which streamsjob seekers to the different services providedwithin Job Network. A subsequent issue ofresponsiveness we pay attention to is the choiceof provider made by job seekers. Finally weaddress two other indicators, i.e. job seeker andemployer perceptions of quality, by a shortpresentation of the results of the customer satis-faction surveys.

7.2.1. Application of the job seekerclassification instrument

If job seekers can be reliably identified andstreamed to the service that is most likely tomake a difference to their employment prospects,resources can be targeted to where impact ishighest. Under current arrangements, effectivetargeting relies to a large extent on the applica-tion and accuracy of the job seeker classification

instrument. This aims to identify job seekers whoare most likely to remain unemployed or becomelong-term unemployed.

The application of the instrument has provedtroublesome (Productivity Commission, 2002,p. 9.3-9.13). A survey of providers in 2001 foundthat 21 % of job search training providers and4 % of intensive assistance providers thoughtthat over half the job seekers referred to themwere inappropriately classified (DEWR, 2002c,p. 106). In qualitative research some providersindicated that many of the job seekers referred tothem should have been referred to either a moreintensive service or to services outside JobNetwork. One of the problems mentioned is thatjob seekers do not disclose personal informationbecause they do not understand the purpose ofthe questions in the instrument. A review of thejob seeker classification instrument interviewprocess also identified some measurement errorsand inconsistencies in its application. Severalfactors were found to either involve a degree ofambiguity or make it difficult for job seekers toanswer accurately.

Overall, we can conclude that the job seekerclassification instrument is welcomed with muchcriticism. In the implementation it is perceived asan instrument for rationing to detect those jobseekers who have the ‘capacity to benefit’, whileprevious employment and training programmeswere accessible for everyone (Struyven et al.,2002). For a quasi-market, it is very important tohave a confident classification instrument, as thelevel of fees to providers is dependent on theclassification of the job seeker. If the job seeker isclassified too low, the provider will have lessincentive to put in effort (parking).

In the Netherlands, an analogous instrument isused, called the ‘chance-meter’. Job seekerswho are capable of finding a job on their ownwith a minimum of support are allocated tostream 1, while those who are not at all ready fora paid job in the open labour market are allocatedto stream 4. Job seekers allocated to streams 2and 3 are expected to need a ‘trajectory’ (aperiod of individual case management, perhapsseen as a pathway through distinct phases such

(13) According to the Productivity Commission, the cost reductions in terms of cost per net impact could also be the result of theimprecision with which the small impacts of labour market programmes are measured (Productivity Commission, 2002,p. 5.24).



as training, work experience and job search)lasting less than one year, or one to two years,respectively. The chance-meter does not sayanything about the content of the trajectory that aperson needs. It is not yet clear how far thisinstrument succeeds in predicting the cases inthe right way.

7.2.2. Choice of provider In normal markets, choices by consumers have amajor effect on company performance and incen-tives for provision of quality services. For thisincentive effect to work, consumer choices mustbe informed and must systematically favourbetter over worse enterprises. However, theauto-referral system, quantity caps and poorinformation provision to job seekers mean thatmost job seekers are assigned randomly to JobNetwork providers. On the basis of administrativedata, Centrelink estimated that only about 20 %of job seekers chose their own provider, with theremainder being allocated a provider through theautomated referral system. In addition, wherechoice is exercised, it does not appear to behighly informed. From DEWR’s job seeker satis-faction survey, it appears that of those making achoice, convenient location seems to be the mostimportant criterion for judgement. This raises theissue of whether and how to increase the scopefor choice in a way that rewards betterperformers with a greater flow of referrals. Manyobservers question (rhetorically) whether jobseekers, even with information, are able to makea real choice (Struyven et al., 2002).

In the Netherlands, job seekers have beengiven more options in the second tender. Fortarget groups of a specific minimum size,contracts are signed with multiple reintegrationcompanies. In many cases a job seeker can thenchoose from a number of different companies,together with his or her contact person from theUWV. But it remains unclear how this will work inpractice due to the lack of informed choice.

7.2.3. Client satisfactionAccording to the client satisfaction surveys of theAustralian Department, the majority of jobseekers appear satisfied with the services theyreceive from Job Network. Among job seekerssurveyed in 2001, 74 % reported that they wereeither satisfied or very satisfied with the services.Satisfaction levels were over 80 % for job search

training and intensive assistance. Among thereasons job seekers reported for being eithersatisfied or very satisfied with Job Networkservices were that the assistance helped themstay connected with the workforce and they were‘treated like an individual’. For job matching,however, the level of satisfaction with serviceswas only 56 %. The Productivity Commission isquite sceptical about these figures: ‘It is commonfor satisfaction surveys to reveal satisfactionlevels of around 80 %, irrespective of the topic’(Productivity Commission, 2002, p. J.1). Similarly,the Productivity Commission found out that in the2001 survey, satisfaction is inversely related tothe star rating of providers.

Similar results are found in the perceptions ofemployers. According to the department,employers who use Job Network rate the servicehighly. Among employers surveyed in 2001 whohad used Job Network in the previous12 months, 90 % of them were satisfied or verysatisfied (increasing from 84 % in 1999) (DEWR,2002c, p. 113). But the Productivity Commission,commenting on the 1999 survey results, notesthat the corresponding satisfaction levelsreported for other recruitment agencies was90 %, despite the fact that only 15 % ofemployers reported having to pay for JobNetwork services compared with 62 % for otheragencies (Productivity Commission, 2002,p. 6.24). The most often cited benefits from usingthe Job Network related to screening of appli-cants and saving time interviewing and findingapplicants. The most often cited poor aspectswere poorly trained/incompetent staff and poorservice. Nevertheless, the Australian Chamber ofCommerce and Industry reported that its 1999survey of 1 200 members found ‘that employerswere overwhelmingly satisfied with the perfor-mance of the providers, particularly in compar-ison with the former Commonwealth EmploymentService’ (Productivity Commission, 2002, p. 6.23).

In the Dutch case, for the time being no satis-faction data are available.

7.3. Quality

Using market forces is demonstrably more effi-cient than past models, then, but is it possiblethat this is achieved through a lower service level

or poorer quality? In what follows, we discuss thekind of services offered to job seekers within JobNetwork, with special attention paid to theamount and type of training provided.

7.3.1. Services of the Australian Job NetworkProviders within Job Network have the flexibilityto tailor the services to the needs of the jobseekers referred to them for assistance. Therationale for this is that decisions about how tohelp the unemployed to overcome their barriersto employment are best made at the local levelbetween a job seeker and their employmentconsultant. There is, for instance, no prescriptionwithin intensive assistance for the delivery ofcertain types of labour-market assistance.

Consistent with this philosophy, the governmentdoes not regularly collect data on the types ofservices delivered within intensive assistance. Thisinformation has to be obtained from surveys. The2001 Job Network participants’ survey, forexample, indicated that most job seekers (over90%) within intensive assistance received jobsearch assistance in the form of help with CVs andinterview preparation. Just over half the job seekersreported receiving some personal assistance(counselling and presentation advice). Over 40% ofjob seekers received financial assistance, includingassistance with fares and the cost of special equip-ment. Only 8% of job seekers obtained work expe-rience (DEWR, 2002c, p. 88).In the Netherlands, at the end of 2002 the firstquality labels, developed by the branch organisa-tion Borea, were awarded to 27 reintegrationcompanies. It is anticipated that 13 performanceindicators may in the near future generate dataabout throughput, success ratio and drop outratio, as well as satisfaction levels with clientsand employers.

7.3.2. Training within Job NetworkOf special interest for this paper is the impactJob Network had on the amount and type oftraining provided to job seekers.

7.3.2.1. The amount of trainingThe 2001 Job Network participants’ survey distin-guished between training in job search skills, such

as help with CVs and preparing for interviews, andmore substantial job skills training, such as ITtraining, language/literacy/numeracy training andtraining linked to a specific job (job-specifictraining). Training of this type was reported to havebeen received by 23 % of intensive assistanceparticipants surveyed in 2001. Of all the episodesof training reported in the 2001 survey, 41 % lastedless than a week, while for 32 % of episodes theduration was between two and eight weeks.

How does this compare with the period beforeJob Network? There has been a shift towards ‘jobsearch’ skills and a reduction in the amount ofshort-term training. During the first half of the1990s there were over 100 000 annual commence-ments in labour-market programmes which offeredshort-term training. Since the introduction of JobNetwork indications are (14) that around 64 000 jobseekers participate in short-term job skills trainingannually as part of intensive assistance. But overthe same period there has been an increase inparticipation in job search assistance providedthrough Job Network’s job search training service.In the early 1990s there were fewer than 30 000annual commencements in this type of assistance.The number of commencements in job searchassistance was over 85 000 in 1997 and over80 000 in 2001.

The significant role job search assistance playsin intensive assistance is further evidence of ashift towards job search activities. The evaluationof Job Network found that over 90 % of intensiveassistance participants received assistance withjob search from their providers (counselling andhelp with preparing CVs and for interviews).

The move away from short-term training isconsistent with a ‘work first’ approach and issupported by evidence on the relative effective-ness of different types of labour-market assis-tance. One of the motives for reforming thelabour-market programmes and introducing JobNetwork was the observation that training wasfailing as a labour-market instrument. Thegovernment criticised the lack of results ofexisting labour-market programmes: ‘[...] fewseemed to make any significant or lasting differ-ence in getting unemployed people back intoregular employment’ (DEWRSB, 2000a). Training


(14) On the basis of the number of job seekers starting intensive assistance in 2001, the reported level of training is equivalent toaround 64 000 participants receiving training in job skills.


programmes were criticised in particular, such asthe SkillShare programme, which offeredshort-term training opportunities. Services offeredsuch as job search assistance, job referral andcontinuing personal support contributed moresignificantly to the outcomes of the programmesthan services that provided structured training,either in combination with these and other formsof assistance or on their own (DEET, 1993).

7.3.2.2. Outcome based payment and incentivesto provide training

The fact that job seekers are offered training toonly a limited extent, (15) is attributed mainly tothe higher-risk nature of these investmentsbecause providers cannot be certain that aninvestment in training will result in a job for thejob seeker. In addition, the bias in the fee struc-ture towards employment outcomes may, insome cases, favour the pursuit of short-termemployment outcomes over training and educa-tion which may, in the longer-term, be of greaterbenefit to job seekers.

It can be concluded that, before the introduc-tion of Job Network, where job seekers were tooeasily placed into training programmes withoutthis improving their chances of finding work, atpresent too little is being invested by providerswithin Job Network in training for job seekers.The introduction of training accounts as part ofthe Australians working together welfare reform inJuly 2002 can be seen as a tacit concession inthis regard. These schemes enable eligiblemature (aged 50 and over) and indigenous jobseekers to undertake work-related training to thevalue of AUD 800. Job seekers can choose fromcourses on the Australian Training website. Inorder to participate, training must be accreditedand delivered by registered training organisations.Other reforms in the third Job Network contract inJuly 2003 also led to an increase in payments forproviding services to job seekers, without thesepayments being made dependent on a joboutcome. For example, a job seeker account willbe introduced which will be available to eligiblejob seekers who will typically enter intensivesupport services after three months in unemploy-

ment. A Job Network member may spend thismoney (on behalf of the job seeker) on training,including that supplemented by a trainingaccount.

7.4. Net impact and equity of theAustralian Job Network

Employment outcomes are the ultimate touch-stone for the summative evaluation of the intro-duction of market mechanisms in employmentand training services. A number of questions canbe identified here. The first is whether theconcentration on outcomes does not lead to avery strong focus on short-term results in theform of helping job seekers back into employ-ment, possibly at the expense of results in thelonger term (sustainability of outcomes). Thesecond question is whether the new arrange-ments succeed in achieving better employmentoutcomes than before (before-and-after compar-ison). The third question ensues from the creationof a market with different providers: to whatextent do the differences between providers leadto differences in employment outcomes (compar-ison between providers)? Finally, we can ask towhat extent the outcomes are also achieved forthe more difficult categories of job seekers, andthen specifically in relation to the incentive struc-ture for providers which focuses on outcomes(equity and impact of the fee structure on equity).

7.4.1. Gross outcomesIt is generally acknowledged that gross outcomemeasures mean very little in terms of evaluating theimpact of employment and training programmes,because many of the outcomes would haveoccurred anyway (Productivity Commission, 2002).Therefore, impact evaluations should focus on netimpacts, in terms of how many people moved offbenefits to go into paid work as a result of theassistance and training provided. As stated by theProductivity Commission, this is an exercise besetby data-related and methodological problems,

(15) It is important to note that a substitution effect may have occurred from federal level to state level, whereby although lesstraining is offered in within the Federal Job Network programme, it may be that more training programmes have been initiatedby the state governments.

requiring estimation of a variety of programmeeffects, including:(a) dead-weight losses: programme funds spent

on job seekers who would have found workanyway;

(b) compliance effects: where some job seekersreferred to a programme find work or other-wise leave benefits before the programmestarts (especially when participation iscompulsory as in Australia);

(c) attachment effects: where programme partic-ipants reduce their job search activity for theduration of a programme;

(d) displacement effects: where a programmeparticipant displaces someone out of theprogramme.

It is not the purpose of this paper to deal withthese problems in depth. Our focus is on the impactof the functioning of the market for employmentand training services. Since the case of Australia isthe best documented in terms of evaluation studies,our discussion is restricted to this one case.

7.4.2. Sustainability of outcomesAs regards the gross outcomes of Job Network,placement figures are available, measured three

months after the end of the relevant measure.The department collects these data via a writtensurvey, the post-programme monitoring survey,which has been held since the mid-1980s. Theadvantage is that these data are collectedcontinuously and provide an insight into short-term results. According to the department, thelower response rates among people in work leadto an underestimate of the positive outcomes.The findings are presented in Table 12.

The table shows that the percentage of partic-ipants in job search training who are in employ-ment three months after leaving the scheme roseover the period considered from 36.2 % to43.2 %. Half of these are in full-time employment,the other half are in part-time jobs. The percentageof participants in intensive assistance in workthree months later was 38.6 % in the period2000-01. This percentage is significantly higherthan in the preceding year, though lower than in1998-99. Roughly 40 % of those in work have afull-time job; the remaining 60 % are in part-timepositions. If outflow to training programmes isalso regarded as a positive outcome, the positiveoutflow percentages (which can not be completelyequated to employment commencement percent-


1998-99 1999-00 2000-01

Job search training

Employed 36.2 38.2 43.2

Full-time 18.3 19.9 21.5

Part-time 17.9 18.3 21.6

Further education 13.6 12.6 12.9

Positive outcomes 45.5 46.7 52.1

Intensive assistance

Employed 39.6 32.7 38.6

Full-time 18.8 14.4 16.3

Part-time 20.8 18.3 22.3

Total 100.0 100.0 100.0

Further education 7.7 8.0 7.7

Positive outcomes 45.8 39.3 44.6

NB: Outcomes related to job seekers who ceased assistance between April 1998 and March 1999, April 1999 andMarch 2000, and April 2000 and March 2001, measured three months after leaving assistance. Positive outcomesinclude employment and education/training outcomes, but are not the sum of these two outcomes because some jobseekers can achieve both an employment and education outcome.

Source:Based on DEWR (2002c)

Table 12: Gross outcomes of Job Network


ages) are slightly higher, at 52.1 % for job searchtraining and 44.6 % for intensive assistance(figures relate to 2000-01).

In the view of the department these outcomesare quite sustainable, which is one of JobNetwork’s main objectives (DEWR, 2002c).Across the three main services, about four out offive of those in jobs three months after assistancewere employed in the longer term:(a) for job matching, 83 % of job seekers who

were employed three months after placementwere employed 12 months later;

(b) for job search training and intensive assis-tance, 82 % and 76 %, respectively of thoseemployed in the short term were alsoemployed after eight months.

The evaluation results of the department alsoindicate that, for many job seekers, the quality ofoutcome (in terms of both income and skill levels)improves over time, supporting the government’sview that for some job seekers a low-quality initialjob can provide a stepping stone to a betterquality job. The sustainability of outcomes alsoappears to hold generally for most job seekers,including those who are more disadvantaged.There is, however, a group who return to incomesupport over time. Among job seekers referred tointensive assistance who left income support, forexample, about 30 % had subsequently returnedto the income support register 15 months aftertheir referral.

7.4.3. Before-and-after comparisonsDirect comparisons between programmes overtime are constrained by the fact that programmesmay have different objectives and target differentjob seeker groups, and may operate underdifferent labour-market conditions. An internalevaluation study is available in which a before-and-after comparison is made between JobNetwork and six labour-market programmes thatoperated in the mid-1990s (DEWRSB, 2001).Following the methodology of matched compar-ison groups (16), the study found a net impact ofaround 10 percentage points for intensive assis-tance and 3 percentage points for job searchtraining. This compares with the results of

previous programmes in the mid-1990s (thoughindividual programmes varied widely).

7.4.4. Provider comparisons A central feature of a quasi-market is theexistence of different providers (instead of amonopoly). The department’s evaluation reportssome findings based on a comparison betweenhigh-performing providers and low-performingproviders in terms of job seekers’ perceptions ofthe quality of the staff, and the types and qualityof assistance provided (DEWR, 2002c). The eval-uation found that the best providers seem to bemore proactive and more responsive to the needsof both job seekers and employers. Furthermore,the more successful providers are the ones whoare more likely to offer training, covering both jobsearch and job-specific skills.

7.4.5. Net impact and equityThe evaluation by the department presents esti-mates of the net impact of job search training andintensive assistance on the employmentprospects of participants. Unlike previous netimpact studies, the estimates in the third evalua-tion report use a new methodology with matchedcomparison groups that attempts to measure thecumulative effect of three elements: compliance,programme and attachment effects. The netimpact in this study was found to be modest: theJob Network programmes generate a very smallnet positive effect for participants (job seekerswho commenced employment): 8.3 percentagepoints for job search training and 0.6 percentagepoints for intensive assistance. The low result forintensive assistance can be explained by theattachment effect of the length of time jobseekers spend on the programme. Finally, theevaluation attempts to measure the compliancenet effect, which was estimated to be almost twopercentage points.

According to the department, the findings needto be interpreted very carefully because of theinherent difficulty of forming a control groupwhere there is universal entitlement to assistance(DEWR, 2002c, p. 3). The net impact measuresdo not reflect a pure comparison between inter-

(16) Comparison group of job seekers who had not been referred to or participated in the programmes in the previous six months,matched on age, gender and duration of benefits. Both programme and comparison group sample were large (26 000 jobseekers each for the intensive assistance programme and 5 000 job seekers for the job search training programme).


vention and no intervention; rather, they comparean intervention with a combination of no interven-tion and other forms of assistance. In addition,the estimates do not measure the improvedemployment prospects of participants in assis-tance who have not yet secured a job. Finally,positive net impacts are a static measure ofeffectiveness. Even if modest, these benefitscompound for successive groups of participants,producing more significant effects on the level ofdisadvantaged job seekers when considered overa longer timeframe. Nevertheless, the publicationof the results gave rise to public concern,including in Parliament, about the effectiveness ofJob Network. The department’s impact evalua-tion also prompts scepticism about the appropri-ateness of the matched control group method-ology (Productivity Commission, 2002). Noattempt is made to control for unobservablepersonal factors in influencing success in thelabour market, such as motivation, demeanourand attitude to work. This problem is known asselection bias.

Despite the tentative nature of the findings,some evidence is available concerning theequity of outcomes. The Department’s evalua-tion has found that more disadvantaged groupsbenefit most from the programme, even thoughthese job seekers have lower gross outcomesfrom assistance than other job seekers. This isthe case for the net impact of the followinggroups: mature job seekers (aged 45 and over),the low-educated (less than 10 years in educa-tion), and the long-term unemployed (12 monthsand over). This corresponds with internationalevidence of net impact measurement of labour-market interventions. However, it is found thatfinancial incentives focusing on employmentoutcomes have an adverse effect on the equaltreatment of all job seekers. Based on amodelling of the incentive effects of the currentfee structure, the evaluation concludes that thefinancial incentives in the fee structure make itrational for providers to focus assistance onthose who require only a limited level of assis-tance in order to obtain outcomes.

8.1. Assessment of evaluationstrategy

Discussion of evaluation of quasi-market reformshas shown that evaluation findings about processand impact are limited. This results from theinherent difficulties in defining and measuringoutputs, net impacts (by using control groups) andquality issues. Furthermore, an evaluation ofreforms should include before-and-after compar-isons, which suppose appropriate longitudinalresearch to distinguish rigorously between theeffect of market forces and other changes that mayhave taken place simultaneously, such as changesin the benefit system or in the labour market.

None of the countries under considerationhave a systematic evaluation strategy for the newarrangements, except in Australia, but researchto date has been monopolised by the centralgovernment. Outside researchers cannot getaccess to original data, nor they can get aninsight into its processing. Implementation of thereform by providers, job seekers and employersremains a black box due to the confidentiality ofthe data. This leads to the conclusion that officialevaluation studies are somewhat problematic and

function mainly as a justification of policy deci-sions. In this respect, the independent review bythe Productivity Commission was a breakthroughin governmental research strategy. This review isnot only based on data and findings by thedepartment, but also on the experiences of allstakeholders, and has ended in a critical debate.Evaluation of quasi-market arrangements shouldinclude all stakeholders in a more interactive wayand keep at least part of the evaluation work forexternal independent researchers.

8.2. Main findings

The experiences from the Australian and otherquasi-market reforms support the view that themarket model in providing reintegration servicesis superior to that operating previously. However,there is no hard empirical evidence for this. Themain findings of the summative evaluation aresummarised in Table 13.

The Australian case provides a very mixedpicture. The most significant gain is that themarket system leads to more cost-effective perfor-mance: services cost less, with a more or lesssimilar net impact as previous programmes and

8. Conclusions

Table 13: Empirical evidence concerning the impact of quasi-markets in Australia (1998-2002)and the Netherlands (2000-02)

Criteria Trend Main findings Main findingsAustralia the Netherlands

Gain in cost-efficiencyEfficiency + Gain in cost-effectiveness Not yet efficiency gains

(<greater efficiency)

Targeting with job seekers Screening and streaming withResponsiveness +/– Classification instrument troublesome ‘chance-meter’: predictive value?

Local needs Tends to involve regional needs

Erosion of the level of services, No empirical evidence availableQuality +/– especially job specific training Quality labels to 26 companies

Scepticism about satisfaction levels (end 2002)

Choice –No real choice for clients (<20 %) No informed choiceNo informed choice No empirical evidence available

No strong empirical evidenceEquity – Low net impact (0.6 %; 8.3 %) No empirical evidence available

More disadvantaged benefit most

adequate responsiveness and quality. Weakerpoints are the limited choice for job seekers anderosion of the level of services, especially withregard to job specific training. However, thesefindings about net impact and quality are tentativeand should be added to other findings concerningthe amount of training services, the financial incen-tives and their role for disadvantaged job seekers.It was found that the financial incentives focusingon employment outcomes have an adverse effecton the equal treatment of job seekers. The incen-tive structure favours the pursuit of short-termemployment outcomes over training and educa-

tion which may, in the longer-term, be of greaterbenefit to job seekers.

In the Netherlands, it is not yet clear how wellthe arrangements to foster responsiveness andchoice between providers may work. The Dutchreintegration market tends to be self-regulatingon quality and, at this moment, there is noevidence of increased efficiency nor equity.

Despite these mixed findings, more positiveevidence is available with regard to the formativeevaluation, i.e. the evaluation of the conditionsthat are crucial for the functioning of quasi-markets (Table 14).

Table 14: Empirical evidence concerning the conditions for quasi-markets in Australia (1998-02)and the Netherlands (2000-02)

Condition Trend Main findings Main findings Australia the Netherlands

Many providers, increasing number ofMany providers, but one central purchaser purchasers (sub-markets)Difficult industry entry Low barriers to industry entry, howeverAdministered prices and floor prices experience as a selection criterion

Market + Clear distinction between purchaser Free pricesstructure and provider Different tendering culture between

Deficiencies of the Job Network market: purchasersconcentration, large companies favoured, Deficiencies of the UWV-market: instability concentration, large companies favoured,

instability, market of organisers

More transparency of procedure, not yet transparency of the market More transparency of procedure, not yetExtensive regime for monitoring and quality transparency of the marketassurance, increasing compliance Poor monitoring information,

Information +/– burdens to providers problem of data collectionIntegrated labour-market information Development of a quality label system, but data conceived as confidential by the sector itself Star rating model lacks transparency

Tendering causes high transaction costsAnnual tendering causes high transaction

Transaction +/–Roll over of 60 % contracts under

costscosts

the third round Lowered by a uniform two-stage procedure and model agreements

Combination of input and output paymentSelection criteria based on price (25 %) ‘No cure less pay’and quality (75 %) Training separately financed

Motivation + Training not separately financed Tendency towards ‘no cure no pay’ Star rating model focussed on ‘payable leading to risk-averse behaviouroutcomes’, leading to risk-averse behaviour Tendency to give more choice to job seekerTendency to give more choice to job seeker

Providers cannot reject job seekers, Number of elements to counteract,

Cream +/– some adverse effectssome adverse effects

skimming Job seekers divided in different (broad) Number of categories;

categories; higher price for the most tendency to less target groups

difficult category



Typical problems concerning information andtransparency as well as transaction costs can beovercome by refinements in structure, tenderingmechanisms, implementation and monitoringarrangements. In this respect, the experiences arenot similar, as shown in Table 14. The differencesbetween Australia and the Netherlands representcontrasting market types, with both advantagesand disadvantages, as summarised in Table 15.

One advantage of the Australian model is thatdirection from one centre leads to a high level ofuniformity. This promotes the transparency of themarket and clarity of legislation. But single princi-palship places the Australian department in anomnipotent position. As monopsonist, it has amonopoly on the tendering policy. This is furtherreinforced by the high level of policy centralisation.

The Dutch model is characterised by more equalrelationships between buyers and providers. Moreroom is available for the various players, as well asfor the more specialised market segments. A pluri-form principalship therefore promotes the positionof the customer, who has more chance of finding aservice provider he likes in a more varied anddifferentiated players’ market.

However, the evidence discussed in this papershows that both models also have disadvan-tages. Australian principalship operates in a hier-archic/bureaucratic way as a result of its largenumber of rules and control. In the interviews,

this was characterised as a legalised market or‘lego’-market (Struyven et al., 2002; ProductivityCommission, 2002). In addition, the Job Networksystem is increasingly driven by performancemeasurement using star rating, which is disputedin the field. The star rating itself is not rejectedbut, as a result of the lack of open communica-tion about the criteria used and its method ofcalculation, it meets with much resistance. Bothfactors influence each other. The distance to thefield is further reinforced by the need to keepeverything controllable. At the time of theresearch, the market was cloaked in an atmo-sphere of mistrust concerning the Job Network.The question arises whether this encourages theplayers to take risk-avoiding measures, which isprecisely what the government wants to coun-teract by the way it acts. A second question iswhether this might prejudice the final results.

The Dutch market type suffers from a lack oftransparency. A difference in tendering culturestill prevails between the principals. Experiencesyet have to show how the transition to a busi-ness-like principalship can succeed in a modelwhere the purchasing function still remains fairlyclose to the implementation domain. The princi-pals can also be faced with undesirable side-effects in the behaviour of the providers, such asshopping and cross-subsidising betweendifferent public purchasers.

Table 15: Advantages and disadvantages of two market types based on tendering practice inAustralia and the Netherlands

Advantages Disadvantages

: Directed from one centre

Obsession with performance measurements

Steering towards outcomesHierarchic/bureaucratic

MonopsonyTransparency of the market

Dominated by rules and control (‘legalised market’)

Australian type Clear rules

Large distance from field

Legally sound Atmosphere of mistrust; lack of open

communication

More equal relationships between

buyers and providers Less transparent marketQuasi-market:

Promotes varied playing field Differences in tendering ‘culture’Dutch type

More specialisation possible Shopping for providers

More communication Cross-subsidies possible between different public

Strengthens position of customer assignments

(options)

The differences between the two types can bebrought into focus in yet another way. Bothtendering practices are founded on a mixture ofhierarchic and market relationships (Thompsonet al., 1991; Parsons, 1995; Bouckaert, 1998).Typical of the hierarchic relationship is that thedominant player directs the other actor on thebasis of rules and standards. The task imposedon the directed actor can change at any moment.In a market relationship, the mutual relationship ismanaged by the market mechanism betweendemand and supply. A market relationship ischaracterised by more balance between equalplayers. In Australia, a hierarchic relationship isexpressed in many ways: the price-regulating roleof the government, the fact that the providers ofintensive assistance and job search training alsohave to assume job matching services, etc. TheAustralian system has more in common with aholding in which various companies operate andwhere the central management controls through amix of hierarchy and market. In a less centralisedsystem with more buyers, more room is createdfor allowing the mix of hierarchy and market tointerfere through a cooperation relationship,based on agreements.

8.3. Lessons for other countries

While comparing reform in the two leading coun-tries, Australia and the Netherlands, one should

take into account the institutional andlabour-market contexts. It is clear that marketforces are not the only factor affecting theoutput of employment brokerage and reintegra-tion activities. However, the operational charac-teristics of the system do matter. Despite thedifferent context of the Australian welfare statecompared with the continental or Scandinavianmodel, on the practical level of matching supplyand demand and helping different categories ofjob seekers, all countries would appear to beconfronted with the same problems. Quasi-market experiences can help in seeking newsolutions. These elements can deal with struc-tural conditions, such as the purchaser- providersplit, the importance of a decentralisedapproach, organisational formality, the incentivestructure, and implementation conditions suchas job seeker classification and referral instru-ments, an integrated labour-market data systemor quality management. Even without imple-menting a quasi-market model, all these condi-tions play a role in the search for better practicein employment services. A central role is playedby the incentive structure. Therefore, much moreattention in evaluation research should be givento the relationship between the tendering andmarket conditions on the one hand and thebehaviour of both providers and job seekers onthe other.


List of abbreviations

CWI Centres for work and income

DEWR Department of Employment and Workplace Relations

DEWRSB Department for Employment, Workplace Relations and Small Business

RWI Council for Work and Income [Raad voor Werk en Inkomen]

SUWI Structure of Work and Income Implementation [Structuur Uitvoering Werk en Inkomen]

UWV Uitvoering Werknemersverzekeringen

WAO Wet op de arbeidsongeschikheidsverzekering

References

ANAO – Australian National Audit Office. Manage-ment of the implementation of the new employ-ment services market. Canberra: ANAO, 1998(Audit Report No 7, Performance Audit).

Bartlett, W. et al. A revolution in social policy.Quasi-market reforms in the 1990s. Bristol: ThePolicy Press, 1998.

Bouckaert, G. Sustainable development of networksin a governance context. In: Theron, F. et al.Spanning the global divide: networking forsustainabled. Stellenbosch: School of PublicManagement, 1998.

Canoy, M.; Vollaard, B. Middelen om te bemiddelen.Economisch Statistische Berichten, 1999,No 4213, p. 536-538.

Clarke, A. Evaluation research. An introduction toprinciples, methods and practice. London:Sage Publications, 1999.

Cordia, L. et al. De kunst van het aanbesteden.Onderzoek naar werking van de aanbested-ingsprocedure bij inkoop van reïntegratieac-tiviteiten. Zoetermeer: College van ToezichtSociale Verzekeringen, 2001.

Davidson, J. Unpacking the shape of the newemployment services market. Jobs Australia1998 National Seminar. Melbourne, 1998.

Deakin, N.; Walch, K. The enabling state: the role ofmarkets and contracts. Public Administration,1996, No 1, p. 33-48.

DEET – Department of Employment, Education andTraining. The SkillShare Network: an evaluation.Canberra: DEET, 1993 (EMB Report 5/93).

Derksen, W. et al. Het borgen van publiek belang.The Hague: Sdu Uitgevers, 2000.

Descy, P.; Tessaring, M. (eds). Training and learningfor competence: second report on vocationaltraining research in Europe. Synthesis report.Luxembourg: Office for Official Publications ofthe European Communities, 2001 (CedefopReference series, 6).

DEWR. Employment Services, an active participa-tion model. Canberra: DEWR, 2002a (Discus-sion Paper).

DEWR. Government response to the ProductivityCommission Independent Review of JobNetwork. Canberra: DEWR, 2002b.

DEWR. Job Network evaluation, Stage Three: effec-tiveness report. Canberra: DEWR, 2002c.

DEWRSB. Job Network evaluation. Stage One:implementation and market development.Canberra: DEWRSB, 2000a.

DEWRSB. Job Network evaluation. Stage Two:progress report. Canberra: DEWRSB, 2000b.

DEWRSB. Job Network: a net impact study.Canberra: DEWRSB, 2001 (EPPB ReportNo 1/2001).

Dijk, S. et al. Een markt in ontwikkeling? Eersteinventarisatie van de stand van zaken op demarkt voor reïntegratiediensten. The Hague:Elsevier, 2000.

Domberger, S. The contracting organisation – astrategic guide to outsourcing. Oxford: OxfordUniversity Press, 1998.

Eerste Kamer. Wet Structuur Uitvoeringsorgan-isatie Werk en Inkomen. Memorie vanAntwoord, 9 oktober 2001, Eerste Kamer derStaten-Generaal Verga-derjaar 2001-2002. TheHague: Eerste Kamer, 2001.

European Commission. Modernising Public Employ-mentServices to support the European employ-ment strategy. Communication from theCommission. Luxembourg: Office for OfficialPublications of the European Communities,1998 (COM(98) 641 final).

Fay, R. G. Making the Public Employment Servicemore effective through the introduction ofmarket signals. Paris: OECD, 1997 (LabourMarket and Social Policy Occasional Papers,No 25).

Fisher, C. M. Resource allocation in the publicsector. Values, priorities and markets in themanagement of public services. London/NewYork: Routledge, 1998.

Flynn, N. Public sector management. New York:Harvester Wheatsheaf, 1993.

Green, A. et al. Financing vocational education andtraining. In: Descy, P.; Tessaring, M. (eds).Training in Europe: second report on vocationaltraining research in Europe 2000. Backgroundreport. Luxembourg: Office for Official Publica-tions of the European Communities, 2001, Vol. 1,p. 71-113 (Cedefop Reference series, 3008).


Groot, I. et al. Lessen uit de privatisering van reïnte-gratriediensten. Amsterdam: SEO, University ofAmsterdam, 2002.

Hodge, G. A. Contracting out government services:a review of international evidence. Melbourne:Mony Tech Pty, 1996.

Le Grand, J. From knight to knave? Public policy andmarket incentives. In: Taylor-Gooby, P. (ed.) Trustand welfare. Houndmills: Macmillan, 2000.

Le Grand, J. The quasi-market experiments inpublic service delivery: did they work? Paper forpresentation at Pontignano conference6-8 April 2001. London: London School ofEconomics, 2001. Available from internet:http://www.econ-pol.unisi.it/welfare/legrand.pdf[Cited 29.9.2003].

Le Grand, J.; Bartlett, W. (eds). Quasi-markets andsocial policy. London: Macmillan Press, 1993.

LISV. Evaluatie aanbestedingsprocedure. Deel 2eindrapport. Amsterdam: Landelijk InstituutSociale Verzekeringen, 2001.

Mevissen, L. et al. Op weg naar volwassenheid?Tweede inventarisatie van de stand van zakenop de markt van reïntegratiediensten. Doet-inchem: Reed Business Information bv, 2002.

Näringsdepartementet. En effektivare arbetsmark-nadsutbildning. Stockholm: Näringsdepar-tementet, 2000 (Ds 2000:038).

OECD. Innovations in labour market policies. TheAustralian way. Paris: OECD, 2001.

Parsons, W. Public policy: An introduction to thetheory and practice of policy analysis. Chel-tenham: Edward Elgar Publishing, 1995.

Patton, M. Q. Practical evaluation. London: SagePublications, 1982.

Productivity Commission. Independent review ofthe job network. Canberra: Commonwealth ofAustralia, 2002 (Inquiry Report No 21).

Rehfuss, J. A. Contracting out in government: aguide to working with outside contractors tosupply public services. San Francisco:Jossey-Bass, 1989.

Revenboer, A.; Lantinga, H. Lessen voor de reïnte-gratiemarkt. Reïntegratie, 2002, Vol.2, No6, p.4-7.

Savas, E. S. Privatization and public-private part-nerships. New York: Chatham HousePublishers, 2000.

Struyven, L. et al. Van aanbieden naar aanbesteden.Marktwerking bij arbeidsbemiddeling en –reïn-tegratie in Australië, Nederland, VerenigdKoninkrijk en Zweden. Leuven: Acco, 2002.

Struyven, L.; Steurs, G. The competitive market foremployment services in the Netherlands. Paris:OECD, 2002 (Labour Market and Social PolicyOccasional Papers).

Struyven, L.; Steurs, G. Towards a quasi-market inreintegration services: first assessment of theDutch experience. Australian Journal of LabourEconomics, 2003, Vol. 6, No 2.

SZW. Naar een werkende reïntegratiemarkt.Regeringsnota, 28 November 2002. The Hague:Ministerie van Sociale Zaken en Werkgelegen-heid, 2002.

Thompson, G. et al. Markets, hierarchies andnetworks: the Coordination of social life.London: Sage, 1991.

Thuy, P.; Hansen, E.; Price, D. The employmentservice in a changing labour market. Geneva:International Labour Office, 2001.

Trommel, W.; Van Heffen, O.; van der Veen, R. Mark-twerking in de publieke sector. Beleid enMaatschappij, 2001, Vol. 28, No 3, p. 130-138.

UWV. Evaluatie aanbestedingsprocedure 2002.UWV, 2002. Available from internet: http://www.uwvnet.nl [cited 30.9.2003].

van der Veen, R. Restructuring a corporatist welfarestate. Managed liberalisation and the role ofthe state, the social partners and the market inDutch social security and labour marketpolicy. Paper for the conference ‘The transi-tional labour market. Outlines of an activelabour market policy’ held in Brussels,2.3.2000.

Walch, K. Public services and market mecha-nisms. Competition, contracting and the newpublic management. Basingstoke: Macmillan,1995.

Walch, K. et al. Contracting for change: contracts inhealth, social care and other local governmentservices. Oxford: Oxford University Press,1997.

Webster, E.; Harding, G. Outsourcing Public Employ-ment Services: the Australian experience.Melbourne: Melbourne Institute, 2000 (WorkingPaper, No 4/00).

Williamson, O. Markets and hierarchies: analysisand antitrust implications. New York: The FreePress, 1975.

Williamson, O. The economic institutions of capi-talism. New York: The Free Press, 1985.

quasi-market reforms in employment and training services - cedefop

Documents