1

Session 5. Improving the Use of Performance Information for

Resource Allocation in R&D: Funding and Priority Setting

From the Central Budget Authority’s Perspective

Nowook ParkCenter for Performance Evaluation & Management

Korea Institute of Public Finance (KIPF)

2

Contents1. Performance information of R&D in Korea2. Issues on performance information of R&D3. Use of PI for resource allocation in R&D in Korea4. Issues on the use of PI for resource allocation in

R&D

3

1. Performance information of R&D in Korea

4

R&D Performance Evaluation System in Korea Law for R&D program performance evaluation and

management was enacted in 2005 Three evaluation systems

Program review process (Self-Assessment) 164 programs are evaluated for 2007’s performance

Evaluation process for selected programs Long-term program, big program, duplicative programs etc 27 programs are evaluated for 2007’s performance

Evaluation on public research institute 32 institutes are evaluated for 2007’s performance

Feasibility study on big R&D programs Programs with more than $ 50 million budget

5

Performance Information of R&D Programs Information from program reviews

Based on checklist Contents of checklist

Program purpose Program design Program management Performance plan Results (mostly based on performance indicators) Feedback

6

Performance Indicators of R&D For the sake of line ministries, pools of PI for 28

program types are provided 28 programs are based on program purpose and utilization

of program outputs Program purposes are classified into 4 categories

R&D, HR development, international cooperation and infrastructure development

Utilization of program outputs is divided into 7 categories Knowledge promotion (basic and applied) Industrial development (short-term, medium/long-term

development) Public and social welfare (public service, public health and

social welfare, region)

7

2. Issues on performance information of R&D

8

Issues on Performance Information of R&D Hard to develop relevant performance information

Results are realized in long term Sometimes even establishing milestone indicator is not

easy, because the process is not certain ex ante Hard to evaluate R&D programs based on indicators

Peer reviews are widely used method Expertise, balance and independence are required

Need to find optimal mix of expert judgment and alternative method

9

Issues on Performance Information of R&D in Korea PI from evaluation process is based on checklist

Evaluation on program results heavily relies on performance indicators

Cost-benefit indicators are rarely adopted in the evaluation process, even though they suggested in the PI pool Hard to come up with relevant information

Establishing reliable peer review process is hard Existence of strong social ties among scientists tends to

hinder objective evaluation process Influences from line ministries may exist

10

3. Use of PI for R&D resource allocation in Korea

11

Overview of R&D Resource Allocation in Korea Korea becomes 8th country whose R&D spending exceeds $10

billion in 2008 Ratio of R&D spending to total government spending is 4.7% in

2008 Based on “National R&D Program Total-Road Map”, strategic

resource allocation occurred Increased spending on small scale basic research by individuals Increased ratio of spending on the areas where spillover effects

are big Biotechnology, Energy, Basic research

Decreased ratios of spending on the area where private sector’s capability is mature Machinery, IT and electronics

12

Changes of R&D Budget Process in Korea (1) National Science and Technology Council (NSTC)

was established in 1998 Centralized budget process for R&D Previously there was no systematic budget process for

R&D programs Science and Technology Innovation Bureau within

Ministry of Science and Technology was established in 2004 Has authority of budget allocation of R&D programs Central budget authority usually accepted the budget

proposal from Science and Technology Innovation Bureau with little modification

13

Changes of R&D Budget Process in Korea (2)

Ministry of Strategy and Finance takes over the authority of budget allocation for R&D programs in 2008 Science and Technology Innovation Bureau was abolished

with the inception of New Administration Implies strengthening the role of the central budget

authority in R&D budget allocation Expect more active use of performance information for

R&D budget allocation

14

Strategic Resource Allocation of R&D R&D is one of 14 areas in National Fiscal

Management Plan (Medium Term Expenditure Framework)

For FY 2008, strategic resource allocation is based on “National R&D Program Total-Road Map”

Role of PI in strategic resource allocation appears to be limited

15

Resource allocation among R&D Programs PI from existing R&D evaluation systems are

not actively utilized in budget allocation so far However, as the authority of resource

allocation is transferred to the central budget authority, more active use of PI is expected Evaluated as lower 10% of programs in each line

ministry will be in danger of 10% budget cut

16

4. Issues on Using PI for Resource Allocation in R&D

17

Use of PI for R&D Resource Allocation Strategic resource allocation

Resource allocation among different areas, such as education, defense, R&D and so on

Resource allocation among R&D programs pursuing different goals

Resource allocation among R&D programs pursuing similar goals

Resource allocation among R&D programs within spending ministries/agencies

18

Issues on the Use of PI for Strategic Resource Allocation in R&D

Strategic plan and PI should be consistently integrated PI at the national level may give rough signals for

strategic resource allocation when it is combined with other considerations

It is not easy to use PI to set the priority among different areas, such as education, defense, R&D and so on Other factors may play a bigger role

19

Issues on the Use of PI for Resource Allocation for R&D Programs Selecting appropriate units for evaluation is

important to use PI for setting the priority among R&D programs pursuing similar goals Identifying programs pursuing similar goals is the first step Program review process may facilitate the process Crosscut analysis can be used for resource allocation if

relevant PI is available for each program Using PI for setting priority among programs

pursuing different goals is not easy Use of PI for management improvement is feasible Use of PI for resource allocation is more complicated

20

Issues on the Use of PI for R&D Resource Allocation in Korea PI has not been systematically used in strategic

resource allocation Use of PI in the national fiscal management plan need to

be improved More active use of PI for resource allocation is

expected but it remains to be seen how effective it will be So far, the focus is given to resource allocation within each

spending ministry Uniform criteria may be adopted to strengthen use of PI

21

Thank You!