driving better project evaluation in queensland - the role of the revised project evaluation...
Post on 18-Jan-2018
217 Views
Preview:
DESCRIPTION
TRANSCRIPT
Driving better Project Evaluation in Queensland - the role of the Revised
Project Evaluation Guidelines
Presentation to the Australasian Evaluation Societyby Scott Hook and Mark Rice
May 2006
What is today about?• Understand the reasons for having the Project
Evaluation Guidelines (PEG).• The PEG framework and key concepts.• Importance of ex-ante and ex-post evaluation.• Assess the changes in the guidelines since the
previous 1997 version.• Using the PEG in your analysis.• Key lessons.
Objective – section 1• Key concepts.• Rationale for evaluation at the beginning
of a project.• Why it is important to use the Project
Evaluation Guidelines.• When to use the PEG?
Key Concepts – what to understand
• Cost-benefit analysis• Cost-effectiveness analysis• Discount rates• Externality• Multiplier• Opportunity cost• Shadow pricing
Mistakes happen!
However, planning can help!
Things can go wrong!• Projects can become more expensive, take
longer to complete and do not achieve the initial objective.
• Consider– Victoria’s Federation square– Commonwealth’s Seasprite acquisition
• UK government experience with planning for infrastructure projects suggests that:– Timing for non-standard engineering projects be
revised in the range of 3-25 per cent;– Costs be revised in the range of 6-66 per cent.
Fail to plan, plan to fail• Consistent, transparent and accurate
information is needed to– Align agencies’ policies, projects, programs
and activities to the Government’s priorities;– Prioritise individual projects within programs;– Ensure that their project procurement and
resource allocation decisions achieve maximum value and benefit for the community.
What are the Project Evaluation Guidelines?• The development of the PEG reflected a
concern that proposals for new projects, services or policy require guidelines.
• The PEG provides the rationale, process and requirements for project evaluation in Queensland Government Agencies.
• There is also an increasing need to use cost-benefit analysis in the evaluation process.
PEG Background• Previous Project Evaluation Guidelines –
February 1997.• Requirement for more detailed guidelines,
with inclusion of templates. Inclusion of additional material on risk management, governance arrangements and project management.
• The revised Project Evaluation Guidelines have been circulated to all agencies for comment and use.
PEG background (cont)• Charter of Social and Fiscal
Responsibility (April 2004) requires– The direction of resources and activity
towards the Government’s highest priority areas; and
– The provision of strategies to achieve value for money in delivering infrastructure and services to the community.
• PEG seeks to assist rigorous and robust analysis across Queensland Government Agencies.
Why do we need the PEG?• To provide a standard methodology and whole-
of-Government approach to the preparation of project evaluations and the analyses they contain.
• Increased recognition of the need to improve rigour and clarity for decision making purposes by increasing the use of cost-benefit analysis.
• Establishment of a whole-of-Government framework to clarify uses and linkages between other guidelines, such as PPP.
Part of Good Governance
• Thinking about options and then ranking them can improve the quality of Government decision making.
• Cabinet seeks the maximum value benefit for the whole community.
What PEG is not• A one-stop project management tool. • A method to
– micro-manage all projects;– slow agency new policy development; and– only evaluate physical infrastructure.
• Instead, PEG is– Part of the policy development toolkit for
agencies. – PEG can used for infrastructure, service delivery
and new policy initiatives.
Ex-what?• Traditional evaluation has been
conducted at the end of a project.• Increasingly, risks of a project in terms of
time, cost and quality require evaluation upfront.
• PEG and business cases are an example of ex-ante evaluation.
• Ex-post evaluation still has an important role.
Changes in 2005 PEG• 2005 PEG applies to all projects (involving
capital or recurrent expenditure or regulation), rather than capital projects.
• Increased detail on how to undertake each of the analyses.
• Social and environmental analyses integrated into CBA.
• Budget analysis on accrual basis.• New section on regulatory analysis.• Stronger statement discouraging use of
multipliers.
Objectives – Part 2• PEG contents.• Project Evaluation framework.• Understanding the cost-benefit analysis
methodology.• Role of agencies.
PEG Contents• Introduction.• Framework for Evaluating Projects.• Project Evaluation Pro forma.• Common Project Evaluation Techniques.
What is Cost-benefit analysis?• Identifies values and compares the costs
and benefits derived from investing scarce resources, such as land, labour and capital, in project options to achieve a particular objective. Costs and benefits are valued in dollar terms and adjusted for market distortions or imputed where the market does not exist.
Project Life Cycle
PEG applicationPEG application
Set objective
Generate options
Evaluate Options
Select preferred option
Project ImplementationProject Implementation
Building the case for a preferred option
Transitioning to the preferred option
Framework for Evaluating ProjectsFive Steps• Objective, priority and justification of project.• Project options identified.• Preliminary assessment.• Evaluation of project options (explained further
shortly).• Selection of preferred option.
Evaluation of Project Options
• For each of the project options, agencies prepare a:– Risk Analysis– Financial Analysis– Cost Benefit or Cost Effectiveness Analysis– Budget Analysis– Regulatory Analysis (where appropriate)
Risk Analysis• Risk Identification
– Determining what, why, where, when and how events could prevent, degrade, delay or enhance the project.
• Risk Assessment– Sources of project risk, their positive and negative consequences and
probabilities that they will occur.
• Allocation and Mitigation– Can it be prevented and/or what to do with it once it has materialised.
Financial Analysis
• Measures costs and revenues for the sponsoring agency or agencies.
• Involves cash expenditure and revenue, including capital.
• Measures the ability of the agency to cover expenses of the project option through a revenue stream.
Cost Benefit Analysis• Adopts economy wide perspective (usually for the
State as a whole). A financial analysis is the starting point.
• Includes a wider range of social and environmental costs and benefits than the financial and budget analyses, eg pollution, land degradation, traffic congestion, health outcomes, etc.
• Measures true economic worth of resources. • Project option with the greatest net present value –
most economic use of resources.
Identify and estimate the expected economic benefits and costs
We are measuring ‘social’ costs and benefits• Not calculating the distribution of those costs
and benefits.• Valued in dollar terms at their ‘true’ economic
worth.• Not confined to market transactions
– Non-market costs and benefits can be valued by imputation or by appropriate adjustment of the market price.
Calculate the net present economic value• Discounting allows benefits and costs in different
options occurring at different times to be compared on the same basis, and recognises that a dollar now is valued more highly than a dollar in the future.
• The difference between the discounted streams of benefits and costs of each project option is the Net Present Economic Value of the project option.
• A positive or negative value helps determine the economic viability of a project.
• Choice of the discount rate will be an important determinant.
Expected economic benefits (continued)
Consideration should be given to estimating:• Avoided costs;• Cost savings;• Revenues;• Benefits to consumers and to the community
as a whole; and• Residual value of assets.
Key Points – the NPV analysis• Capital expenditure estimates are
reasonable and accurate• All legitimate costs and benefits have been
included. • No invalid costs and benefits. • All costs and benefits have been valued at
their market value or economic value.• A reasonable timeframe for the project and
costs and benefits have been forecast reasonably and with transparency.
Budget Analysis• Covers the budget impact for the relevant
agency or agencies.• Involves accrual expenses and revenue
and includes non cash items, such as depreciation.
• Measures impact on Operating Statement and Statement of Financial Position.
• Measures impact on the State Government’s fiscal position.
Regulatory Analysis
• Projects are assessed to ensure they:– Do not restrict competition under the TPA;– Produce a net benefit for State where
regulations are changed; and– A Regulatory Impact Statement and /or
Public Benefit Test may be required for changes to legislation and/or subordinate legislation.
Don’t forget we need to…• Discount the value of money to account for
different flows occurring over different time periods.
• Use sensitivity analysis for the Financial Analysis and the Cost Benefit Analysis (eg, changes in prices, construction costs, usage of services or discount rates).
• Include project implementation and post-implementation review.
Garbage in – Garbage out
The analysis should contain• Textually and numerically explicit
explanations of the assumptions underlying all capital and recurrent estimates regarding labour costs, energy costs, demand growth, charges etc.
• Clear and referenced data sources for validation purposes.
When to use the PEG• The PEG should guide the assessment of
any project under consideration by an agency. Can be used for infrastructure, service delivery and new policy initiatives
• However, judgement is needed. A detailed financial and economic analysis may be impractical for small projects or expensive and complex.
• Assessment should be made according to scope, cost, complexity, riskiness and sensitivity of the project.
What are some alternatives• Economic impact analysis, which estimates flow-on economic
impacts of a project, can supplement cost-benefit analysis.– Input-output modelling measures a gross flow-on from project
expenditure (a multiplier). Applying input-output multipliers usually overstates project impacts.
• General equilibrium modelling uses a mathematical model of the economy to estimate the whole of economy impact of changes in policies or new projects.
• Why are they different to CBA?– Focuses on economic impacts (not valuing social impacts)– Includes flow-on impacts to all sectors (possibility of
overestimating benefits).
What is the role of Agencies?• The agency must make a decision about the
level of analysis and the allocation of suitable resources.
• Identify and quantify expenses, costs, revenues and benefits and project affordability.
• Undertake a rigorous cost-benefit analysis.• Liaise with the relevant business branch and
OESR for guidance on specific technical issues.
What is Treasury’s role?
• Provide training, advice and guidance to an agency to support the preparation of a cost-benefit analysis.
• Advise the Treasurer and CBRC on the information quality of project evaluations and the merits of output/investment proposals put forward by Government departments and agencies.
• Establishment of a data base that agencies can access when preparing project evaluations
The process• Several stages
– Consultation on the PEG.– Refinement and finalisation of the PEG.– Integrate PEG within the Project
Assurance Framework.
Relationship with other frameworks• Other guidelines that are related include the
Public-Private Partnership Policy Guidelines, Public Benefits Test Guidelines and Regulatory Impact Statement Guidelines.
• A Project Assurance Framework is being developed by FMB– The PAF will include guidance on a gateway process to
ensure success management of project processes.
• The PEG is being integrated into the PAF.
25 October 2005
Business Case Development
Supply Strategy Development
Source Supplier/s
Establish Service Capability
Deliver ServiceManagement of Project Agreements
Preliminary Evaluation
Strategic Assessment of Service Requirement
PPP Business Case Development
Expressions of Interest
Bidding Process
Progress as PPPProgress as traditional delivery
Key Learnings• That any project evaluation should be using the
PEG as a template and all data and assumptions are documented, and costs and benefits are explained.
• Cost-benefit analysis allows choices to be recommended to Government.
• Include the status quo as an option but exclude estimates of costs and benefits do not follow logically from assumptions.
• Never include invalid costs and benefits and uses a multiplier.
Questions?
• Views?
• Opinions?
• Comments?
top related