1
Session 5. Improving the Use of Performance Information for
Resource Allocation in R&D: Funding and Priority Setting
From the Central Budget Authority’s Perspective
Nowook ParkCenter for Performance Evaluation & Management
Korea Institute of Public Finance (KIPF)
2
Contents1. Performance information of R&D in Korea2. Issues on performance information of R&D3. Use of PI for resource allocation in R&D in Korea4. Issues on the use of PI for resource allocation in
R&D
3
1. Performance information of R&D in Korea
4
R&D Performance Evaluation System in Korea Law for R&D program performance evaluation and
management was enacted in 2005 Three evaluation systems
Program review process (Self-Assessment) 164 programs are evaluated for 2007’s performance
Evaluation process for selected programs Long-term program, big program, duplicative programs etc 27 programs are evaluated for 2007’s performance
Evaluation on public research institute 32 institutes are evaluated for 2007’s performance
Feasibility study on big R&D programs Programs with more than $ 50 million budget
5
Performance Information of R&D Programs Information from program reviews
Based on checklist Contents of checklist
Program purpose Program design Program management Performance plan Results (mostly based on performance indicators) Feedback
6
Performance Indicators of R&D For the sake of line ministries, pools of PI for 28
program types are provided 28 programs are based on program purpose and utilization
of program outputs Program purposes are classified into 4 categories
R&D, HR development, international cooperation and infrastructure development
Utilization of program outputs is divided into 7 categories Knowledge promotion (basic and applied) Industrial development (short-term, medium/long-term
development) Public and social welfare (public service, public health and
social welfare, region)
7
2. Issues on performance information of R&D
8
Issues on Performance Information of R&D Hard to develop relevant performance information
Results are realized in long term Sometimes even establishing milestone indicator is not
easy, because the process is not certain ex ante Hard to evaluate R&D programs based on indicators
Peer reviews are widely used method Expertise, balance and independence are required
Need to find optimal mix of expert judgment and alternative method
9
Issues on Performance Information of R&D in Korea PI from evaluation process is based on checklist
Evaluation on program results heavily relies on performance indicators
Cost-benefit indicators are rarely adopted in the evaluation process, even though they suggested in the PI pool Hard to come up with relevant information
Establishing reliable peer review process is hard Existence of strong social ties among scientists tends to
hinder objective evaluation process Influences from line ministries may exist
10
3. Use of PI for R&D resource allocation in Korea
11
Overview of R&D Resource Allocation in Korea Korea becomes 8th country whose R&D spending exceeds $10
billion in 2008 Ratio of R&D spending to total government spending is 4.7% in
2008 Based on “National R&D Program Total-Road Map”, strategic
resource allocation occurred Increased spending on small scale basic research by individuals Increased ratio of spending on the areas where spillover effects
are big Biotechnology, Energy, Basic research
Decreased ratios of spending on the area where private sector’s capability is mature Machinery, IT and electronics
12
Changes of R&D Budget Process in Korea (1) National Science and Technology Council (NSTC)
was established in 1998 Centralized budget process for R&D Previously there was no systematic budget process for
R&D programs Science and Technology Innovation Bureau within
Ministry of Science and Technology was established in 2004 Has authority of budget allocation of R&D programs Central budget authority usually accepted the budget
proposal from Science and Technology Innovation Bureau with little modification
13
Changes of R&D Budget Process in Korea (2)
Ministry of Strategy and Finance takes over the authority of budget allocation for R&D programs in 2008 Science and Technology Innovation Bureau was abolished
with the inception of New Administration Implies strengthening the role of the central budget
authority in R&D budget allocation Expect more active use of performance information for
R&D budget allocation
14
Strategic Resource Allocation of R&D R&D is one of 14 areas in National Fiscal
Management Plan (Medium Term Expenditure Framework)
For FY 2008, strategic resource allocation is based on “National R&D Program Total-Road Map”
Role of PI in strategic resource allocation appears to be limited
15
Resource allocation among R&D Programs PI from existing R&D evaluation systems are
not actively utilized in budget allocation so far However, as the authority of resource
allocation is transferred to the central budget authority, more active use of PI is expected Evaluated as lower 10% of programs in each line
ministry will be in danger of 10% budget cut
16
4. Issues on Using PI for Resource Allocation in R&D
17
Use of PI for R&D Resource Allocation Strategic resource allocation
Resource allocation among different areas, such as education, defense, R&D and so on
Resource allocation among R&D programs pursuing different goals
Resource allocation among R&D programs pursuing similar goals
Resource allocation among R&D programs within spending ministries/agencies
18
Issues on the Use of PI for Strategic Resource Allocation in R&D
Strategic plan and PI should be consistently integrated PI at the national level may give rough signals for
strategic resource allocation when it is combined with other considerations
It is not easy to use PI to set the priority among different areas, such as education, defense, R&D and so on Other factors may play a bigger role
19
Issues on the Use of PI for Resource Allocation for R&D Programs Selecting appropriate units for evaluation is
important to use PI for setting the priority among R&D programs pursuing similar goals Identifying programs pursuing similar goals is the first step Program review process may facilitate the process Crosscut analysis can be used for resource allocation if
relevant PI is available for each program Using PI for setting priority among programs
pursuing different goals is not easy Use of PI for management improvement is feasible Use of PI for resource allocation is more complicated
20
Issues on the Use of PI for R&D Resource Allocation in Korea PI has not been systematically used in strategic
resource allocation Use of PI in the national fiscal management plan need to
be improved More active use of PI for resource allocation is
expected but it remains to be seen how effective it will be So far, the focus is given to resource allocation within each
spending ministry Uniform criteria may be adopted to strengthen use of PI
21
Thank You!