Is evaluation of public policy in Is evaluation of public policy in Ireland dying?Ireland dying?

And if it is should we care?And if it is should we care?Dr. Richard BoyleHead of Research

Institute of Public AdministrationDublin

Presentation to Irish Evaluation Network 20 November 2009

OverviewOverview

The good times? 1980s and 1990sWhere are we now?Problems but green shoots?What should happen?

The good timesThe good times

Well developed evaluation system based around EU requirements for evaluation of structural funds

Public service modernisation – Strategic Management Initiative and Expenditure Review Initiative (later Value for Money and Policy Review initiative)

Where are we now?Where are we now?

EU pressure no longer a key driver National Development Plan evaluation:

– NDP evaluation unit focused on capital expenditure appraisal

– Last evaluation report was 2005 Value for Money and Policy Review

– Patchy and slow– Resourcing reviews a big challenge– Little evidence of impact

The problem is more of demand than of supply

Does the Oireachtas engage with Does the Oireachtas engage with evaluation reports?evaluation reports? 2002-2004 round of Value for Money and

Policy Reviews:– 43 reports produced– Survey evidence from Select Committees about

37 of these reports– Select Committees has received only 2 of these

reports, and one noted by the Committee but not discussed

(Source McMahon, 2007)

Now getting reports to Committees but engagement still an issue

What do VFMPR reports What do VFMPR reports recommend?recommend? 13 reports produced by Department of Agriculture and Food (one

of the leaders) reviewed:– 99 conclusions and recommendations– No direct recommendations for programme termination– 14 supported the status quo– Remainder largely focus on minor programme alterations or changes

in management processes ‘It is quite obvious that these reviews look at areas of efficiency

and effectiveness that must be addressed within the programmes themselves rather than a more fundamental process of evaluation that points to the need for a continuation of the scheme or programme or the need to radically redistribute resources between various programmes’

(Source: Smyth, 2007)

An Bord Snip NuaAn Bord Snip Nua

‘…the Group noted a general deficiency of information regarding the public service impacts associated with particular items of expenditure. The management focus across departments generally still seems to be on securing and retaining the maximum volume of expenditure for particular areas, and on accounting for departmental activities in financial terms; details on outputs and actual performance seem secondary.’

An Bord Snip NuaAn Bord Snip Nua

‘The Value for Money and Policy Reviews, in which various expenditure programmes are critically reviewed by the responsible department, have up to now had limited success in redirecting scarce resources away from lower-priority, lower performing areas to areas where they can be put to more productive use (although recent reforms are designed to address these shortcomings)’

Green shoots?Green shoots?

Philanthropies requirements for evaluation acting as an alternative external driver to the EU?

Changes to VFMPR: more focused and targeting areas of significant spending

Kinds of evaluation information Kinds of evaluation information coming from different anchorscoming from different anchors

Operational issues Impact issues Continued relevance issues

Programme manager

?

Corporate group in organisation

? ?

Corporate government group

? ?

Legislative audit office

?

Legislative bodies

Outside government

Source: Mayne, Divorski and Lemaire, 1999

Issues and what should happen now?Issues and what should happen now?

Evaluation model/approach – mix between developing internal culture and external scrutiny?

Political demand – a lost cause? Periodic scrutiny of the evaluation system Structured publication of evaluation

results and links with expenditure decisions