WORKING PAPER No 7

PPIORITY-SETING MECHANISMS FOR NATIONAL

AGPRCULTURAL RESEARCH SYSTEMS

PRESENT EXPERIENCE AND FUTURE NEEDS

I a rA1

International Service for National Agr~cultural Research

The International Service for National Agricultural Research (iSNAR) began operating at its headquarters in The Hague Netherlands on September 1 1980 It was estabished by the Consultative Group on International Agricultural Research (CGIAR) on the oasis of recommendations from an international task force for the purposeof assisting governments of developing countries to strengthen their agricultural research It is a non-profit autonomous agency international in character and is ncwi-politcal inmanagement staffing and ooerations

Of the 13 centers inthe CGIAR network ISNAR is the only one that focuses primarily un national agriculturalresearch issues Itprovides advice to governments upon request on resedrch policy orqnization and marageshyment issues thus complementing the activities of other sistance agencies

ISNAR has active advisory service research and tri ining programs

ISNAR is supported by a number of the members of CGIAR an informal group of approximately 43 donors in iucing courries developmeni banks internationai organizations and foundations

Citaticn Norton GW and PG Pardey Priority-Setting Mechaoisms for National Agricultural Research Systems Present Experience and Future Needs 1987 International Service for National Agricultural Research Tne Hague Netherlands

WORKING PAPER No 7

PRiORITY-SETTING MECHANISMS FOR NATIONAL

AGRICULTURAL RESEARCH SYSTEMS

PRESENT EXPERIENCE AND FUTURE NEEDS

George W Norton and Philip G Pardey

November 1987

en

International Service for National Agricultural Research

ISNAR WORKING PAPERS

The ISNAR working papers series is a flexible instrument for sharing

analysis and information about relevant organization and management

problems of the agricultural research systems in developing countries

In the course of its activities - direct assistance to national

agricultural research systems training and research ISAAR genarates a-

broad range of information and materials which eventually become the

formal products of its publications program The working papers series

enhances this program in several important ways

1 These papers are intended to be a rapid means of presenting the

results of work and experiences that are still in progress but are

already producing cesults that could be of use to others

2 They are intended to be an effective vehicle for widening the

discussion of continuing work thereby increasing the quality of the

final products critical conment is welcomed

3 The series provides an outlet for diffusing materials and information

that because of their limited coverage do not meet the requirements

of general audience publication

The series is intended mainly for the diffusion of materials produced by

ISNAR itaff but it is also available for the publication of documents

produced by other institutions should they wish to take advantage of the

opportunity

George W Norton is associate professor Virginia Polytechnic Institute and StateUniversity Blacksburg Virginia and research 1llow International Service for National Agricultural Research (ISNAR) The Ilague Netherlands Philip G Pardey is joiltly a research associate University of Minnesota St Paul Minnesota and reseach officer ISNAR The authors would like to thank Iloward Elliott Alexander Von der Osten Paul Berinell and Carlos Valverde for their helpful comments and suggestions on an earlier draft of this paper without implicating thern in any remaining errors and omissions

TABLE OF CONTENTS

I Context of Research Priority-Setting I 1I Previous Priority-Setting Models 2

Weighted Criteria Models 3Benefit-cost (Expected Economic Surplus) Analysis 3Mathematical Programming 3Simulation 3Comparative Assessment 4

III The Use of Weighted Criteria Models in the Dominican RepublicEcuaodm and Uruguay 6

Goals 7Criteria 7

Coinmoditv Criteria 7Research Area Criteria 9

Data Collection and Model Implementation 9An Assessment of the Weighted Criteria Models used in theDominican Republic Ecuador and Uruguay 12

IV The Use of Expected Economic Surplus Analysis in Peru 15

An Assessment of the Expected Economic Surplus Procedure in Peru 15 V Implications for Development of Improved Research Priority-Setting Procedures 16 VI Conclusions 18 Footnotes 19

References 20

Appendix 1 23

Appendix 2 26

Priority-Setting Mechanisms for National

Agricultural Research Systems

Present Experience and Future Needs

Recent trends towards shrinking real research budgets or many national agriculturalresearch systems (NRS) have stimulated several attempts to evaluate tileeconomic benefits ofaricultural reserch Manv of these quantitatis e economic evaluations haveemployed historical data to provide ex post assessments and a few hase used subjectivejudgments to project future research benefits [he results kthese analyses have been used at the national or inter-ministerial level to justilf the need for continued political support for agricultural research

Tighter rese rch budgets not only heighten the iced to justifV agricultural research vis a vis alternative public investments but they alVo make it important for NARS to improve procedures for setting priorities among competing research programs A varietyof strategic and project-level decisions are made within NARS The objectives of national policvmakers the needs of agricultural producers and scientific opinion must each be brotight to bear on strategic decisions to emphasize particular commodities and research program areas uld on research project selection

The purpoe of this paiper is to discuss kev elements for inclusion in improved research priority-setting procedures for NRS Ihe fLcus is on methods for strategic as opposedto more tactical projcct-le el decision m kingAfter discussing the context of research priority setting a brief summary is provided of tile iain types of research resource alloL-ition mnetlods Recciit cxperiences with priority setting in four Latin American NAR the l)ominican Republic lcuador lUVUav and Peru are compared These experiences can proside sonc guidance for the development of an improved research resource allocation procedure

I TilE CONTEXT OF RESEARCH PRIORITY SETTING

NARS decision mnkers often operating within the ministry of agriculture an agricultural research institute or as a research council make allocative choices based on their knowledge atnd prior cxperience That knowledge and experience frequentlyincludes an understar one of national and regional oals and objectives as voiced through the political process a sene o lie severity of particular types of research problems and a general feel tor vlat is achievable through research It also includes an awareness of the desires ( te-ressures) of producer groups Decision makers must decide hoW much cmpha sisto jte onparticular commodities (eg rice maize beanssheep) and types of research c v varietal improvenmnt improved livestock nutritionplant protection farming svwtes Implicit in those decisions are the locational emphasis of research the focus 0rt paJticullar factors of production (eg land laborwater) the emphasis placed Ol lone-r-term haisic versus shorter-term applied and adaptive research and the distrihotianal elfects of research on farm size on producers versus coniumners and on people atidiherent income lec

In many NA RS research resoce alocation decisions are heavily influenced by the previous years budget2 (hanges of~tn result from requests by scientists which areevaluated reLatively informally and aggregatcd into an overall plan The resulting plan

upon close scrutiny may contribute little to tire attainment of stated national goals and objectives

Judgment based upon prior knowledge and information provided by scientists is crucial for research resource alincation decisions In addition rapid or radical changes in research programs -an be costly Particularly as research budgets tighten howeverforcing changes in tile systein tile increased use of quantitative methods may be nocessary to imprwe thL objectivity of those judgments The aim is to foster consitencv of research priorities with goAs and objectives and to improve the efliciency of the research system in mser ing producer and cortsumer needs The idea is not to replacejudgment but to increase and organize the inflormation available for updating priorknowledge and beliefs The hope also is to inject continuity in the priority-settingprocess and to proide NARS decision makers x ith methods the find lielplib when rationalizing their decisions to scientists producer Erotps and politicians Docunentingthe decision proces through a strctured procedure can help minimie the possinilitvof sudden and large shilts in researh priorities in those systems characterized by a flairlv rapid turnover of research administrators

Many quantitative priority-setting procedures are avaiia ble but few have beeninstitutionalized into the decision-making practices of NA RS manaers A cruciZl factor in their non- use of these procedures urtdoubtcdlv as been the lack of ri orous vetcost-eflectie approaches which can irtcorporate tile large number of comtodities andresearch areas as well as the multiple go1ls a ld criteria found in most strateoicdecision-muakirt situations inl NRS FurtIhcrrtore the cortmputer capabiliv available through microcomputers in most developing-co ntry NRS today was not gcenerallyavailable even two or three velars ago

In this paper procedure amc ce crihed which were recently applied in four latinAmerican countries First however previously emploved priority-setting mechanisms are briefly reviewed id critiqued One of rite points that will Ie made at the end of this paper is tiLit tise methods are not nitually excilsive There is no siTele magic huletbut there appears to be real potential for developing a Ilexible computer-assisted priority-setting framework that will allow NARS administrators to call ott theappropriate tectiiques tot aSsist withit a particular decision depending ott tie time and financial resources a ailahle and the relative itiportance of the allocation decision Asecond major point is that priority setting is a process not simply a procedure and must be instit utiortliied as part of the overall process of research managementPriority-setting technritiuetes applied only once by people outside of the decision-making process have little valtie

If SUMMARY OF PRIORITY-SETTING METHODS

Methods reported itt previous studies for research priority setting include

I ) establishtert td weighting of multiple criteria for ranking commodities and research ares with a final aggregate ranking based on implicit or explicit weights

2) use of bertefit-cost analysis including expected economic surplus techniques to select cotitritodities and research areas

3) application of mathematical programming to choose an optimal research portfolioincorporatirng multiple goals md constraits

4) developrert and tise of simulation models

These four approaches are discussed below followed by a brief assessment of their advantaiges atil dis idsittages

2

Weighted Criteria Models

Several studies hae established multiple criteria for ranking priorities because of tiledesire to explicitly consider a wide variety of factors that do or perhaps should influenceresearch selection The relative weights attached to each criterion to arrive at a final listof research prioriries are sometimes left implicit or unstated A recent example is tileTAC review of priorities and future strategies for the Consultative Group onInternational Agricultural Research (CGIAR) In assessing priorities by commoditythey etablished a principal goal research objective to obtain the goal and a series ofcriteria organized into three groups relevance research productivity and efliciency ofhe CGIAR system in undertaking research Detailed tables with quantitative andqualitative information and rankings of commodities for each criterion were presentedIn developing the recommendations for hort- to medium-term funding alternatives atthe conmmodity level the weights used to aggregate across criteria were left implicit

There are examples of tudies that have incorporated multiple criteria but also explicitlyspecified and utilized a set of weights to aggregate across criteria and obtain a finalranking of research priogrities Appications of this method often called the scoringmodel approach are found in studies by Williamson lahlstede Paulsen and KaldorShumway and McCrack en and Von (Oppen and Ryan A flew studies have used congruence analysis which can be thought of as a special case of a sci-ring or weightedcriteria model with all the weight placed on the criterion of vaue of production

Benefit-cost (Expected Economic Surplus) Analysis

The benefit-cost approach to welecting research priorities has been used in differentforms (Fishel Arai Sire and Gardner l)avis Oram and Ryan Norton Ganoza andPomareda) Most studies have employed consumer-producer surplus analysis and havebuilt upon a large body of literature on ex post research evaluation Ex ante analysesusually incorporate expert opinion to determine projected research impacts adoptionrates and probabilities of research success and provide estimates of the economiceffliciencv and distributional implications of agricultural research resource allocationBenefit-cost studies typically calculate benefit-cost ratios internal rates of return andnet present values for alternative types of research or research onfor differentcommodities Ilse analyses may include andmav or not regional internationalresearch spillovers and the eflects of domestic pricing policies on research benefits

Mathematical Programming

Mathematical programming is another alternative for research selection It relies onmathematical Optimization to choose a research portfolio through maximizing amultiple-goal objective function given the resource constraints of the rescaich systemA good example of the use of this method is a study by Russell in the United Kingdomlie used goal programming to maximize the contributions of the research program toseveral goals given the con taints of budget human resources state of knowledge andcertain policies This procs Ue uses similar information to the weighted-criteria modelbut selects an optimal research portfolio rather than simply ranking research areas

Simulation

Finally the simulation method has been used to identify and select research prioritiesSimulation models vary in their construction but a good example is the model byPinstrup-Anderscn and Franklin They built a model to project the contributions andcosts of alternative research activities They established gals and then identifiedchanges in supply demand for inputs and demand for output needed to meet those

3

goals They identified neeled technologies time and financial costs and the probability of research success and adoption Finally they specified the scientists workingobjectives This model is very thorough but did require extensive amounts of data and estimation of several mathematical relationships

Comparative Assessment

The above discussion is a brie summary of the major structu al approaches that have been used to asist decision makers in research priority setting All of them attempt to go beyond the usual procedure of individual or group decision makin e without structural analyses Each of the approacles has its advantages and disadvantages (See Fable I)

The procedure of establishing criteria and using weights to arrive at a final set of research priorities has the advantage of forcing decision makers to consciously trade off multiple goals It can illcorporae both quantitative and qualhtative information and can be applied to a long list of con mnodities or research areas in a relatively liort period of time The procedure is relati cx ca sY fbr administrators to understand but does requiretheir time in obtaining the explicit w5eights for criteria Furthermore these weights are inevitably subjective and their elicitatlion must be carefully structured [lie nActhod also requires scientists time in collecting inlormation on qualitative criteria As a result the approach k better suited for periodic or major priority-setting efforts than or situations where frequent marginal changes are anticipated

The expected economic surplus approach his the major advantage of incorporatingseverml criteria related to economic elficiency and distribution into one or two measures It also can be ucd to examine the gencral equilibrium ellects ofresearch (Ramailho de Castro and Sclhh nAid tile benefits of research under alternative possibly distortionar domestic pticing and international trade policies tEdwarts and Freebairn Alston Edwards and rcecbairn Norton and (ianoza 1985) These alictors are pervasive in developing countries and affect the efficiency and distributional consequences of researh Ile procedure ie(liCe ir lel - tndcrstandw ofeconomic analysisand more anal-t s time than the weighted criteria model but less aduinistrators time It can be diflicult to apply to at large number of commodities or research areas because certain typcs of data necessary for the ainalysis often do not exist for all commodities With adequaIL tL d ita Cxpected economic surplus analysis can be incorporated into the weighted criteria model or could be used on the set of commodities which the weightedcriteria model imdicaMes to have the highest piority lhe latter approach would allow for the calculation of income foregone as a reult of placing weights on non-economic ef1ciency criteria Expected economic surplus analysis has the major advantage of calculating a rate of return which can be compared to alternative public investments

The mathematical programming approach is similar to the weighted criteria model approach becaue weights are placed on a set of goals or criteria The procedure has the advantage of explicitly considering the budget huLman resource aad other constraints on the research system tnless the constraints are well specified including possiblechanges over time liowever there is a risk of nonsense solutions The model is more intensive of an ecolonic analysts time and abilit than tie simpler weighted criteria approach and decision makers may be less willing to accept what appears to be a black box solution lrade-offs among goals are easily quantified ssith this approach

The advantage of simulation models is their tlexibility They can be constructed is relatively simple or complex tools can incorporate optimizing or ranking proceduresand can readily include probabilistic information Their major disadvantage is that to be useful they must be relatively complex and typically reqtuire extensive amounts of both data and time of skilled analysts

4

TABLE 1 COMPARISON AMONG MAJOR AGRICULTURAL RESEARCH PRIORITY-SETTING METHODS

PRIORITY-SETTING METHOD

HEIGHTED EXPECTED MATH CHARACTERISTIC CRITERIA ECON SURPLUS PROGRAM SIMULATION

OPERATIONAL CONSIDERATIONS

1 Relative cost in researchers time mudium mJium medium high2 Relative cost in priority setting analysts ime medium medium high high3 Relative cost in administrators time medium mediummedium medium 4 Relative overall data requirement medium medium medium variable 5 Relative ease of comprehonsion by decision maker high medium low low 6 Ease of incorporating subjective informaticn high high high high7 Ease of incorporating non-quantitativa inforiation high low low medium

GOAL-RELATED ISSUES

a Requires explicit elicitation of goals yes usually yes uiually9 Can determine distributional affects on consumers

and producers at various income levels no yes no yes10 Can handle uncertainty yes yes yes yes11 Can consider tradeoff among multiple goals yes sometimes yes yes

CRITERIA-RELATED ISSUES

12 Can consider private-sector research incentives yes difficult difficult yes13 Can consider economic policy and trade affects yes yes yes yes

EVALUATION-RELATED ISSUES

14 Can be used to sot priorities for research at the aggregate level no yes no es

15 Can be used to set research priorities at the commodity level yes yes yes yes

16 Can be used to set priorities for non-production or non-commodity oriented research yes difficuli yes yes

17 Can be used to set priorities for basic research yes difficult no sometimes 18 Can evaluate secondary impacts of rosuarch on

employment environment nutrition yes sometimes so etimes yes19 Usually estimates a rata of return to research yes sometimesno no 20 Can quantify geographic spillover effects no yes no yes 21 Can consider the lags involved in research

and adoption yes yesyes yes22 Facilitates priority setting when the number

of commodities is large yes difficultdifficult difficult

Source Adapted and modified from Norton and Davis

5

The above is a very brief summary of the major advantages and disadvantages of the most common formally structured priority-setting models that could be used for strategicplanning for research resource allocation Other approaches exist for research projectselection with perhaps models that attempt to identify and analyze key vield constraints being the most prevalent approach employed in NARS of less-deeloped countries

The results of ex post analysis also can provide useful guidance for ex ante research resource allocation decisions if appropriately incorporated into a systematic ex ante procedure The most common ex post approach in addition to cx post benefit-cost analysis is the econometric estimation of production or supply functions incorporating research variables To be most uteful for ex ante analysis econometric approaches mLlSt be applied with a high degree of disaggrega tion Unfortunateiy these models require a substantial amount of high-quality historical data on production farm inputs and research expenditures

Perhaps the two approaches with the most potential given additional refinements for application by NARS in less-developed countries are the weighted criteria models and the benefit-cost or expected economic surplus procedures In the next section reent experiences with applying weighted criteria models in three Latin American Countries )ominican Republic Ecuador and Uruguay are discussed and evaluated The purpose

it to piovide more iniormation about the procedures their degree of acceptance and possible improvements in the models Ihen application of expected economic surplus models are considered drawing cn a recent cx ante research evaluation studly in Peru Finally the relationship between weighted criteria and expected economic surpluts methods is described along with possible extensions

IIl THE USE OF WEIGHiTED CRITERIA MODELS IN TIlE DOMINICAN REPUBLIC ECUADOR AND URUGUAY

in 1986 iif I87 weighted criteria models were developed and implemented in the lDomin lRepublic (ISA) Ecuador (Espinosa et al) and Trugua (CIAAB)to assist with priority setting for agricultwral research In the I)ominican Republic the study was carried out at the Instituto Superior de Agnicultura ISA) by consultants for ISNAR In Ecuador the study was conducted by the plmnintg ollice oY the Instituto Nacional de Investigacion Agropecuaria (INIAP) with the assistance of consultants for the US Agency for International Development (AID) In Uruguay the study was conducted by the Ollice of the Director of the Centro de Investigacion Agricola Alberto Boergcr(CIAAB) with the assistance of ISNAR The purpose of all three studies was to applyprocedures for prioritizing agricultural research by commodity and by major research area Inthe Dominican Republic the client was the newly created natonal agriculturalresearch agency (Il)EA) In Ecuador the client was INIAP and the newly created foundation for agricultural research (IEl)IA) In Urugua the client was C1IAB

The procedures employed in the three studies were similar but with some importantdilfereices as the Ecuador study built on lessons learned inthe I)ominican Republic and the Uruguay study built on other lessons learmed in Ecuador National goals for the research system iere elicited in each country and a series of criteria establishd which relate to those goals Separate riteria were developed for conmitodities and or research areas and weights were elicited from decision makers to establish the relative importanceof the criteria Commodities and research areas were ranked according to each criterion and these rankings were multiplied by the elicited weights to arrive at research priorities

6

Goals

Three goals were identified in each of the three countries (1) to raise the average levelof income in the country (2) to increase the well-b(Aing of low-income groups in societyand (3) to reduce yearto-year income fluctuations in the country especially onl tiledownside These goals were worded slightly diffrcntly in each country but in each casethey essentially relate to the three goals referred to in previous studies as efficncy equity and security In the l)ominican Republic the local consultants identifiedthese as the goals while in lcuador and Uruguay it was done by the directors of national research agencies

Criteria

The next step was to establish criteria to as measures of whetheruse particularcommodities or research areas contribute to the attainment of the above goals A largeset of criteria was discussed in each country and a total of 15 criteria were eventuallyused in the Dominican Republic 14 in lEcuador and 10 in Uruguay to determine priorities by commodity Five criteria were used in each case to determine priorities byresearch area Refinements were made in successive studies - beginning with theDominican Republic then Ecuador followed by Iruguay - to increase the independenceof criteria and to remove criteria that were questionable measures of whether researchcontributed to tile tated goals ( Reason for dropping particular ci eria are discussedbelow) Refinements also were made with respect to the grouping of criteria to facilitatethe weighting of goals and criteria

Commodity Criteria omioditv criteria were grou l - J into four conceptual groupsproduct importance probability O success elliciencv in use of research resources anddistribution of impacts The first three of these groups relate to the efficiency or incomelevel goal and the last group relates to the equity or distributional goal In none of thethree studies were criteria included which represnted the third goal (reduced incomelluctuation [lie feeling was that the entire set of criteria as a group would serve toreduce the enlhasis placed on single or 1ew (often export) crops While this appearedto be the case in the results of thee particular studies ft ture weighted criteria studies may want to add criteria which explicitly rank commodities from lowest to hiehest withrespect to annual gross income variability (or price and yield variability separately)

The group of criteria referred to as jrodct importap contained Four criteria in theUruguay study value of production generation or saving of foreign exchange expectedfuture demand change and comparative advantage

The value of productionl is an important criterion because the cost of rseaich isbasically independent of the number of units alkected by the research results at least forthose commodities with relatively homogeneous production conditions Foreignexchange was used as a criterion because of a perception in each country that the lackof foreign exchange was impeding growtb A criterion was set up to allow more weightto be paced on those commodities for which demand is expected to increase in thefuture because the importance of commodities will change as diets are altered withincome growth aod a world markets change Comparative advantage was includedbecause income in the country will be higher if the country focuses its efforts oncommodities for which its resource base is best suited while recognizing that research itself can alter resource constraints

In the Dominican Republic and in Ecuador nutritional measurcs (calorie and proteincontributions to the diet) were included as criteria for product importance They wereincluded because the well-being of low-income groups may be affected by the availability

7

of foods which are important in their diets Nutritional criteria were dropped in Uruguay however because numerous studies have shown that income levels are tile primary determinants of nutrition levels rather than the availability of particular foods Certainly caloric and protein content are debatable criteria To the extent that increased domestic production lowers the prices of certain food commodities therebyincreasing food consumption by the poor nutritional criteria can contribute to the distributional goal Land area also was included as a criterion in the Dominican Republic study but was dropped in Ecuador and Uruguly because of the overlap with value of production

Two criteria were used in each study to measure 1robahility of reseatch success The first measure was tilegap between own-country yields and neighboing country yields The second was the potential for success indicated by the researchers themselves The rationale for the gap criterion was that the larger the yield gap between the home country and similar countries tle greater the potential for yield gains The criterion is rather crude because yield gaps also can indicate a relative resource disadvantace forproducing the commodity even if the countries appear to be geoclimatically sinilar Consequently the gap criterion was eventually dropped in the tUruguav study The potential for success as indicated by the researchers is subjective but a very necessary measure to obtain

The gro up of criteria reficrred to as e1)cienc in use otresearch resources contained three criteria the relationship to research in the international centers the degree of emphasis on tilecommodity in le current research p ogram and the incentive for the privatesector to conduct the research Some research duplicates other research croplementsand still other research bears no relationship to research at tileinternational agriculturalresearch centers For certain comniodities all three relationships may hold depending on the type of research If national research is complementary to international researchthe cost of the rational research is lower Consequently a criterion was established which places greater weight on commodities with a high degree of complementaritybetween ntin Il d international research The criterion ofdelegree of emphasis in tile current iesercli program was established because there is a cost to rapidly adjustingcurrent h uiman and physical research resources from one commodity to another The criterion of private-sector incentives was used because the social benefits from the total research program in the country will be greatest if scarce public research funds are devoted to research on commodities for which the private sector does not have an incentive to support the research itself Of coui se the private-sector producer group can tax itself and pay the public sector to conduct the research for those commodities for which it has an incentive to privately support the research These commodities tend to be export commodities because producers can capture a higher proportion of the gainsfrom productivity increases for export commodities than for commodities primarily consumed domestically

The group of criteria related to the distribution ojresarchimpacts contained two criteria in the Uruguay study number of producers and the effect of increased productivity on the price of product The larger the number of farms producing a commodity the largerthe number of farmers afficted by the research and perhaps the higher the probabilitythat smaller farmers are being affeicted The distribution of benefits depends to a significant degree on price changes resulting from increased productivity Producers benefit more if prices decline very little and consumers benefit more ifprices decline substatitially for a given production increase For export er import commodities priceswill tend to decline less than for commodities primarily produced and consumed domestically For commodities which are not traded and are very important in the diets of the poor prices will tend to decline relatively sharply when productivity increases The division of benefits between producers and consumers also is affected by the fact

that producers are also consumers Consequently in the Dominican Republic and Ecuador studies an additional criterion importance of the commodity in home-consumption on the farm where produced also was included

In summary 10-15 criteria were used in tie three studies to identify and select research priorities by commodity The linkages anong the goals and criteria described above are summarized in Figure 1

Research Area Criteria The five criteria uicel to select research priorities by research aieas were (1 ) whether the research causes an increase in the use of relatively abundant resources and a saving of relatively scarce resources (2) the number and seventy of research problems (3) non-duplication with transferable research from outside the country (4) tne extent of private-sector incentives to conduct the research and (5) current emphases in the research program

The rate of economic growth will be more rapid if new technologies are developed that utilize relatively abundant resources and save relatively scarce resources The resource base varies by region consequently information on resource abundance criteria was collected by region in tileEcuador and Uruguay studies The same was true for the quantity and severity of research problems Information on the relationship of domestic to inernational research private-sector incentives and the current emphasis in the research program was collected on a naticnal basis These criteria all relate to the income growth (efficiency) goal It is difficult to identif research area criteria that measurc whether particular types of research aifect income distribution or variability

Data Collection and Model Implementation

Information used in the analysis included both quantitative data on value of productionnumber of farms value of exports and imports person-years devoted to research on different commodities and tile of protein per day in thenumber of calories and gramsdiet (for tileDominican Republic and Ecuador studies) as well as qualitative or subjective information on such factors as probability of success private sector incentives and severity of research problems Furthermore weights had to be elicited from decision makers to place relative emphasis on the various criteria

Step 1 Develop Cotninodity and Research Area Lists -- The first step was to determine the list ofcommadities to consider and the list of research areas In the Dominican Republic information was collected on 74 commodities In Ecuador an initial list of 109 commodities on which INIAP was conducting research was discussed Throughdecisions to eliminate some and to group others this list was reduced to 44 commedities for the analysis These decisions were made by the Technical Director for Research In Uruguay a smaller list of commodities was reduced to 21 commodities or commodity groups These decisions were made in consultation with the National Research and Fxtension Directors The list of research program areas contained nine areas in the Dominican Republic 16 in Ecuador and 16 in Uruguay

Step 2 Collect Quantitative and Qualitative Data on Chosen Criteria -- Information o-i quantitative criteria were gathered from local and FAO secondary data sources and one table was constructed For each criterion Commodities were ranked for each criterion Information needed for the qualitative riteria as well as the weights to place on criteria were obtained through interviews with sc rtists and administrators at boh the national and regional experiment stations In Euador and Uruguay substantial attention was devoted to identifying the appropriate respondents to each question as this is a crucial aspect of the procedure In Ecuador those interviewed were notified one week before the interviews and 72 people were interviewed by the Planning Director of INIAP and

9

FIJR 1 CONCEPTUAL ELEHNTS IN THE HEIGHTED CRITERIA MODEL FOR PAWING RESEARCH PRIORITIES BY COtlDITY

GOALS fficiency ]

CONCEPTUAL( )CRITERIA Pri7] Probabi i+y Research Distriutjon Iincome

ii J of Succes Resource Eff o L=Acplusmn a rilbility

CRITERIA 1 Value of 1 Scientist 1Rl ofto toNumberPrcduction Estimates 1 AnnualInternational Producers Variability2 Foreign Centers 2 Price in Gro

Exchange 2 Current Effect3 Future Raturn3)Emphasis 3 HomeDemand 3 Private Consumpt ion

4 Comparative Sector Advantage Incentives

Notes

(-n future studies an additional conceptural criteria under the security goal might be health (J4tri ional criteria such as grams of potein or kilo=alories could be included as additional easures of Distributional ikipacts

4ot used in Dominican Republic Ecuador and Uruojuay studies

his assistant In Uruguay the directors of all the regional stations were called into CIIAB headquarters for a aweting to discuss the study and this was followed up byon-site interviews and meetings with scientists and directors at the regional experimentstations as well as with economists in the Ministry of Agriculture These interviews and meetings were conducted by a person assigned to work wvith the Director of Cl LAB on the priority-setting study In the Dominican Republic a survey of scientists extensionworkers and administrators was conducted by the ISNAR consultant from the Instituto Superior de Agricultura who visited the various experiment stations Less care wastaken in the Dominican Repubic than in Ecuador and Uruguay to match the questionspertaining -o individual criteria to particular individuals This probably reduced theusefulness of the responses because those most familiar with the probability of research success For example may not be the same -s those most Familiar with comparativeadvantage

Step 3 Elicit 11eihts on Critria -- Relative weights to place on the different criteria were obtained Fronm national and regional research system administrators In Ecuador34 peopic verc used to determine the weights These wveights were established separatelyIor the commoditv and the research area criteria In 1iuguay fewer people (sevensystem and station directors) were used to determine the weights and more attention was devoted to grouping the criteria pertaining to particular goals A Delphi procedure was used in which the s7even directors were shown the average of the group and provided with an opportunity to adjust their weight

Step 4 )erive Rankinqs hv Commodity and By Research Area -- Once the basic information was collected axd organized into a seies of tables one procedure was followed to arrive at a final ranking of research priorities by commodity A second procedure resulted in lists of priorities by research area In the Dominican Republicmost of the calculations were made by hand in Ecuador and Uruguay microcomputerspreadsheet programs were used to assist with the calculations and to conduct sensitivity analysis

The steps in the procedure for ranking research priorities by commodity and research area are sumnmaried below and are illustrated with a simple example in Appendix 1 Brieily the steps for ranking commodities were

a) Th conniodities were ranked for each criterion for which quantitative data were available

b) Tihese rankings were multiplied by the weights assigned to each criterion and then the weighted criteria were added across to arrive at one sub-ranking for the

Luantitative criteria c) Each commodity was given a high low or none designation for each qualitative

(subjective) criterion The response which implied a need for greater research priority was assigned the number 2 the intermediate response was assigned I and the response which implied lower research priority was assigned 0 1herefore each comniodity received a 2 1 or 0 for each criterion

d) The weights assigned to each qualitative criterion were then multiplied by these numbers and the results were added across all criteria to arrive at a subtotal for each commoditv These subtotals were ranked from highest to lowest to provide a second sub-ranking

e) The sub-rankings for the quantitative and qualitative criteria were then given their corresponding weights and added together to arrive at a final ranking by commodityThe steps for ranking research priorities by research area were the same as steps (c)and (d) above because the criteria were all qualitative

Step 5 Analsis and Interpretationof Resuls -- In the Dominican Republic the resuLts of this weighted criteria analysis were uscd to determine a small set of cormodities and

II

research areas with the highest priority Further assessment was then made of humanphysical administrative and other resources needed to structure research programsfocused on these commodities and resea-ch topics In Ecuador and Uruguay the prioritized list of commodities was split into a high-priority group an intermediate-priority group and a low-priority group Research area priorities were identified for each region of tile country A more detailed analysis was then made of the human physical and other resources needed to support the high and intermediate priorities

In summary five major -rens were involved in operationalizing the weighted criteria models A complete set of rsults for each study included more than 20 tables and are not reproduced here Ilowever the final rankings for commodities and for research areas are shown in Tables I and 2 in Appendix 2 In the Dominican Republic a team of consultants followed up the initial priority-setting exercise with additional analysisand discussions at the experiment station level and eventually recommended tile establishment of five national commodity programs (rice cornsorghum beans coffeeand livestock) and one additional major research program (natural resources) In Ecuador the prioritized lists were widely distributed and discussed in the Ministry oi Agriculture the Board of FEIDIA the Commission on Science and Technology arid in AID There was concern in Ecuador with the weights placed on the individual criteria and therefore additional sensitivity analysis was conducted by the Planning Office of the Ministry There seemed to be a recognition in Ecuador that priority setting with a weighted criteria model is an iterative process and that much 4t the models value stems from tile discussion of criteria among the decision makers and from the sensitivityanalysis The high-priority list of comniodities in Ecuador included thL ten highest-priority commodities listed in Table 2b A project paper has been written forAI) which recommends funding For Fli)IAlbr three commodities (coffee soft corn for the Sierra and dairy) Coffee and soft coitn were the number 2 and 3 ranked prioritiesin tile weighted criteria model and dairy was number 9 although it was the top-rankedlivestock commodity

In Urugu ay the results from the first run with tile model were discussed with the research directors Ilie directors made small changes to the weights placed on criteriaand the mode was rerun resulting in the prioritized list in Appendix 2 Personnel in CIAAB are CUrrcwtv undertaking additional sensitivity analysis and with tie assistance of ISNA R are developing a plan for implementation

An Assessment of the Weighted Criteria Models Used in the Dominican RepublicEcuador and Uruguay

The Dominican Republic Ecuador and Uruguay studies were perhaps the first majorattempts to implement weighted criteria models with weights explicitly elicited from decision makers in NARS in less-developed countries A number of strengths andweaknesses in the procedures were identified during the analyses Some of thL weaknesses were corrected fron one study to the next some can be corrected in future studies and others are inherent in the approach

Trhe first strength mentioned earlier is the ability of the procedure to systematicallyincorporate both quantitative and qualitative information related to a set of multiplegoals and criteria in order to prioritize a lorng ist of commodities and research areas in a relatively short period of time Second the procedure proved relatively easy for both research administrators and the loc- -nalysts to understand with the exception of the )ominican Republic where the research administrators were not directly involved7

Third the analysis as applied in Ecuador and ULuguay with the direct involvement of research system administrators at various stages forced those decision makers to

12

consciously identifv and trade off goals and criteria Fourth the use of spreadsheet programs in EcUadfor and Uruguay facilitated sensitivity analysis after the initial set of priorities was deermimd Fifth the procedure provided a relatively objectiveassessment of prioriies because individuals were not allowed to rank commodities orresearch areas directly but had to weight criteria

The first weakness inherent in ex ante research priority setting and thus in theprocedure is that there is a large amount of subjectiveness Although tileapproach isless subjective than unstructured judgment there is subjectiveness in the responses toquestiorns related to some of the criteria and also ini the weights placed by decision makers on the criteria Second it proved diflicult to specify independent criteria with no overlap Third while the procedure required less time than an expected economic surplus analysis by commodity it required more time than asking respondents to directlyrank commodities and research areas In E-cuador the consiultant worked withpersonnel in INIAP and FEI)IA for one week at the start of the analysis to determinegoals and criteria to arrive at atlist of commodities and researh areas to list secondarydata needs to formulate questions to be asked of administrators and scientists to decide who should be asked particular questions and to explain the procedure itself includingthe spreadsheet program The required iidormation was collected over the Followingmonth and the consultant returned and worked with the Planning Director in INIAP for one week to conduct the analysis and write tip the results Revisions to the initial draftof the report we-e then made over the lollowing weeks A similar procedure to the onefollowed in Ecuador was used in l_ruguay althaugli more time was spent discussing the wording of questions to be asked of scientists Furthermore n Uruguay more care was taken to match the criteria with the goas

Certain of the criteria wil always be diflicult to eplain to tileinterviewees Thecriterion on comparative advantage is difficult for some respondents because of a lack of knowledge about neighboring countries If suflicient data time and resources are available this criterion may be rnea4ured more precisely using comparative advantage or domestic resource cost procedures suggested by longmire and Winkelnann Anotherdiflicult criterion to estiniate is the potential foture demand for the product The respondents in Fcuador were asked to consider how the demands for different foodswould change as income and populatiot grow Many of them had a hard timeconceiving that the total or per capita demand for certain Foods could actually decline as income grew In Uiruguay (ibe cmiparative advantage and future dema nd questions were answered by a small group of economists from the ministries of agriculture and trade In the Ecuador study more attention was devoted to assessing the most appropriate people for answering different questions than was the case in the Dominican Republic Ilowever some criticism surfaced about the weights placed on criteria InUruguay decision makers were given the opportunity to change their weights afterviewing the initial results This was very useftl because it demonstrated the implicationsof placing different weights on the various goals and criteria

Computer spreadsheet programs were a major help in Ecuador and Uruguay Additional work needs to be devoted to developing a more mentnu-driven prcgram but the people inthe planning office of INIAP were able to use the SuperCalcR program without too tmucht difliculty despite ialack of previous experience Ihe analyst in Uruguay was veryfamiliar with lotusl which proved most helpfil

It became clear from working inall three countries that it is preferable to work directlywith the final decision makers or their designees Inthe l)ominican Republic the study was conducted by a very competent set of consultants from the local agricultural universi ty but the procedure was not institutionalized to allow for additional sensitivityanalysis or future priority-setting efforts InEcuador and especially in Uruguay the

13

decision makers were directly involved conducted sensitivity analyses and were able to assess why certain commodities or research areas received high priority as a result of the analyses In both Ecuador and Uruguay they have a basic understanding of the strengths and weaknesses of tileprocedure and therelre can explain their subsequentdecisions to groups who might disagree with tilepriorities they establish

The corntrast between the Dominican Republic and tie Uruguay studies illustrates a veryimportant point No matter which technique is used to structure the analysispriority-setting exercises must integrate the decision makers (users) into the processPriority-setting procedures must be institutionalited the results of initial analysisdiscussed thoroughly with decision makers and sensitivity anilsis conlducteU alone the lines suggested by that discussion Tecinological change generates a surplus which can be appropriated by dilkircnt sectors or groups within the economy Who benelitdepends in part on the cotaniodities selected for research and the types of technoloLies generated One of the purposes of institutionalizing a formal prioritN- setting procedureis to help mediate conflicts that may arise (for example between producer groups and consumers) as a result of research-induced improved technologies

Other issues arose during the analyses First the initial grouping and elimination of certain commodities and research areas before lhe irialvis is an important step but one which must be taken with carelul consideration of possible coniplmentarities in research on particular productS or types of rearch Second some of the commodities such as forages are inputs into other commodities such as milk (Corseqtentlv these commodities llay have to he considered itscinps because it is the value-added and not tilegross value of output that is relevant when one coniiodity requires other commodities sinputs and when criteria such as value of production are used This was an issue in I ruguav where livestock are quite important liird little attention was devoted in atv of the studies to defining appropriate criteria Ior the increased inoome stability eoal Future studies if this be an ideltJiiLd LOal shOUld define risk criteria

In sutImary the weighted criteria model as applied in Lcuador and iuguav which had the advantage of bit ildin Oillessons lCarICd illdie l)ominican RCpublic appears to have been a uselul priority-setting tool It ha its ereatest advartage ill situations ill which there are rulItiple goals to consider a relatively major reassesnieiit of prioritiesis contenplated and the list of commodities dd resen rch areas to consider is largeWeighted criteria models described in this paper were applied in small countries with limited resources but with direct involvement (except in the Dominican Republic) ofthC major decision makers These models itav be most useful in the small country situationwhere decision mAkers are more likely to be involved in tile priority-settirt process and are not just using tables of results Perhaps in larger systet7 where decisioii makine is more decentralited tileweighted criteria model can he applied at the regional level before relining research priorities at the national level

Another factor that can be uniortant in large svvtern is tle complerientarity between resea-h aid educat ion [hie larger the svstcnt the clser the linkages tend to be between universities and tileresearchi system Where universities are important tor agriclltural research an aidditional criterion may hc rteedCd to place value on the educationlal benefits of certain types of reseirch

Ihere are several additional iniplications with respect to how weighted criteria models relate to alterniative priori t-setting Inethod parIicularl the expected economic surplusapproach Befoce exploring these relationships kcx aspects of the surplus approacliare described based on aii application in Peru

14

The International Service for National Agricultural Research (iSNAR) began operating at its headquarters in The Hague Netherlands on September 1 1980 It was estabished by the Consultative Group on International Agricultural Research (CGIAR) on the oasis of recommendations from an international task force for the purposeof assisting governments of developing countries to strengthen their agricultural research It is a non-profit autonomous agency international in character and is ncwi-politcal inmanagement staffing and ooerations

Of the 13 centers inthe CGIAR network ISNAR is the only one that focuses primarily un national agriculturalresearch issues Itprovides advice to governments upon request on resedrch policy orqnization and marageshyment issues thus complementing the activities of other sistance agencies

ISNAR has active advisory service research and tri ining programs

ISNAR is supported by a number of the members of CGIAR an informal group of approximately 43 donors in iucing courries developmeni banks internationai organizations and foundations

Citaticn Norton GW and PG Pardey Priority-Setting Mechaoisms for National Agricultural Research Systems Present Experience and Future Needs 1987 International Service for National Agricultural Research Tne Hague Netherlands

WORKING PAPER No 7

PRiORITY-SETTING MECHANISMS FOR NATIONAL

AGRICULTURAL RESEARCH SYSTEMS

PRESENT EXPERIENCE AND FUTURE NEEDS

George W Norton and Philip G Pardey

November 1987

en

International Service for National Agricultural Research

ISNAR WORKING PAPERS

The ISNAR working papers series is a flexible instrument for sharing

analysis and information about relevant organization and management

problems of the agricultural research systems in developing countries

In the course of its activities - direct assistance to national

agricultural research systems training and research ISAAR genarates a-

broad range of information and materials which eventually become the

formal products of its publications program The working papers series

enhances this program in several important ways

1 These papers are intended to be a rapid means of presenting the

results of work and experiences that are still in progress but are

already producing cesults that could be of use to others

2 They are intended to be an effective vehicle for widening the

discussion of continuing work thereby increasing the quality of the

final products critical conment is welcomed

3 The series provides an outlet for diffusing materials and information

that because of their limited coverage do not meet the requirements

of general audience publication

The series is intended mainly for the diffusion of materials produced by

ISNAR itaff but it is also available for the publication of documents

produced by other institutions should they wish to take advantage of the

opportunity

George W Norton is associate professor Virginia Polytechnic Institute and StateUniversity Blacksburg Virginia and research 1llow International Service for National Agricultural Research (ISNAR) The Ilague Netherlands Philip G Pardey is joiltly a research associate University of Minnesota St Paul Minnesota and reseach officer ISNAR The authors would like to thank Iloward Elliott Alexander Von der Osten Paul Berinell and Carlos Valverde for their helpful comments and suggestions on an earlier draft of this paper without implicating thern in any remaining errors and omissions

TABLE OF CONTENTS

I Context of Research Priority-Setting I 1I Previous Priority-Setting Models 2

Weighted Criteria Models 3Benefit-cost (Expected Economic Surplus) Analysis 3Mathematical Programming 3Simulation 3Comparative Assessment 4

III The Use of Weighted Criteria Models in the Dominican RepublicEcuaodm and Uruguay 6

Goals 7Criteria 7

Coinmoditv Criteria 7Research Area Criteria 9

Data Collection and Model Implementation 9An Assessment of the Weighted Criteria Models used in theDominican Republic Ecuador and Uruguay 12

IV The Use of Expected Economic Surplus Analysis in Peru 15

An Assessment of the Expected Economic Surplus Procedure in Peru 15 V Implications for Development of Improved Research Priority-Setting Procedures 16 VI Conclusions 18 Footnotes 19

References 20

Appendix 1 23

Appendix 2 26

Priority-Setting Mechanisms for National

Agricultural Research Systems

Present Experience and Future Needs

Recent trends towards shrinking real research budgets or many national agriculturalresearch systems (NRS) have stimulated several attempts to evaluate tileeconomic benefits ofaricultural reserch Manv of these quantitatis e economic evaluations haveemployed historical data to provide ex post assessments and a few hase used subjectivejudgments to project future research benefits [he results kthese analyses have been used at the national or inter-ministerial level to justilf the need for continued political support for agricultural research

Tighter rese rch budgets not only heighten the iced to justifV agricultural research vis a vis alternative public investments but they alVo make it important for NARS to improve procedures for setting priorities among competing research programs A varietyof strategic and project-level decisions are made within NARS The objectives of national policvmakers the needs of agricultural producers and scientific opinion must each be brotight to bear on strategic decisions to emphasize particular commodities and research program areas uld on research project selection

The purpoe of this paiper is to discuss kev elements for inclusion in improved research priority-setting procedures for NRS Ihe fLcus is on methods for strategic as opposedto more tactical projcct-le el decision m kingAfter discussing the context of research priority setting a brief summary is provided of tile iain types of research resource alloL-ition mnetlods Recciit cxperiences with priority setting in four Latin American NAR the l)ominican Republic lcuador lUVUav and Peru are compared These experiences can proside sonc guidance for the development of an improved research resource allocation procedure

I TilE CONTEXT OF RESEARCH PRIORITY SETTING

NARS decision mnkers often operating within the ministry of agriculture an agricultural research institute or as a research council make allocative choices based on their knowledge atnd prior cxperience That knowledge and experience frequentlyincludes an understar one of national and regional oals and objectives as voiced through the political process a sene o lie severity of particular types of research problems and a general feel tor vlat is achievable through research It also includes an awareness of the desires ( te-ressures) of producer groups Decision makers must decide hoW much cmpha sisto jte onparticular commodities (eg rice maize beanssheep) and types of research c v varietal improvenmnt improved livestock nutritionplant protection farming svwtes Implicit in those decisions are the locational emphasis of research the focus 0rt paJticullar factors of production (eg land laborwater) the emphasis placed Ol lone-r-term haisic versus shorter-term applied and adaptive research and the distrihotianal elfects of research on farm size on producers versus coniumners and on people atidiherent income lec

In many NA RS research resoce alocation decisions are heavily influenced by the previous years budget2 (hanges of~tn result from requests by scientists which areevaluated reLatively informally and aggregatcd into an overall plan The resulting plan

upon close scrutiny may contribute little to tire attainment of stated national goals and objectives

Judgment based upon prior knowledge and information provided by scientists is crucial for research resource alincation decisions In addition rapid or radical changes in research programs -an be costly Particularly as research budgets tighten howeverforcing changes in tile systein tile increased use of quantitative methods may be nocessary to imprwe thL objectivity of those judgments The aim is to foster consitencv of research priorities with goAs and objectives and to improve the efliciency of the research system in mser ing producer and cortsumer needs The idea is not to replacejudgment but to increase and organize the inflormation available for updating priorknowledge and beliefs The hope also is to inject continuity in the priority-settingprocess and to proide NARS decision makers x ith methods the find lielplib when rationalizing their decisions to scientists producer Erotps and politicians Docunentingthe decision proces through a strctured procedure can help minimie the possinilitvof sudden and large shilts in researh priorities in those systems characterized by a flairlv rapid turnover of research administrators

Many quantitative priority-setting procedures are avaiia ble but few have beeninstitutionalized into the decision-making practices of NA RS manaers A cruciZl factor in their non- use of these procedures urtdoubtcdlv as been the lack of ri orous vetcost-eflectie approaches which can irtcorporate tile large number of comtodities andresearch areas as well as the multiple go1ls a ld criteria found in most strateoicdecision-muakirt situations inl NRS FurtIhcrrtore the cortmputer capabiliv available through microcomputers in most developing-co ntry NRS today was not gcenerallyavailable even two or three velars ago

In this paper procedure amc ce crihed which were recently applied in four latinAmerican countries First however previously emploved priority-setting mechanisms are briefly reviewed id critiqued One of rite points that will Ie made at the end of this paper is tiLit tise methods are not nitually excilsive There is no siTele magic huletbut there appears to be real potential for developing a Ilexible computer-assisted priority-setting framework that will allow NARS administrators to call ott theappropriate tectiiques tot aSsist withit a particular decision depending ott tie time and financial resources a ailahle and the relative itiportance of the allocation decision Asecond major point is that priority setting is a process not simply a procedure and must be instit utiortliied as part of the overall process of research managementPriority-setting technritiuetes applied only once by people outside of the decision-making process have little valtie

If SUMMARY OF PRIORITY-SETTING METHODS

Methods reported itt previous studies for research priority setting include

I ) establishtert td weighting of multiple criteria for ranking commodities and research ares with a final aggregate ranking based on implicit or explicit weights

2) use of bertefit-cost analysis including expected economic surplus techniques to select cotitritodities and research areas

3) application of mathematical programming to choose an optimal research portfolioincorporatirng multiple goals md constraits

4) developrert and tise of simulation models

These four approaches are discussed below followed by a brief assessment of their advantaiges atil dis idsittages

2

Weighted Criteria Models

Several studies hae established multiple criteria for ranking priorities because of tiledesire to explicitly consider a wide variety of factors that do or perhaps should influenceresearch selection The relative weights attached to each criterion to arrive at a final listof research prioriries are sometimes left implicit or unstated A recent example is tileTAC review of priorities and future strategies for the Consultative Group onInternational Agricultural Research (CGIAR) In assessing priorities by commoditythey etablished a principal goal research objective to obtain the goal and a series ofcriteria organized into three groups relevance research productivity and efliciency ofhe CGIAR system in undertaking research Detailed tables with quantitative andqualitative information and rankings of commodities for each criterion were presentedIn developing the recommendations for hort- to medium-term funding alternatives atthe conmmodity level the weights used to aggregate across criteria were left implicit

There are examples of tudies that have incorporated multiple criteria but also explicitlyspecified and utilized a set of weights to aggregate across criteria and obtain a finalranking of research priogrities Appications of this method often called the scoringmodel approach are found in studies by Williamson lahlstede Paulsen and KaldorShumway and McCrack en and Von (Oppen and Ryan A flew studies have used congruence analysis which can be thought of as a special case of a sci-ring or weightedcriteria model with all the weight placed on the criterion of vaue of production

Benefit-cost (Expected Economic Surplus) Analysis

The benefit-cost approach to welecting research priorities has been used in differentforms (Fishel Arai Sire and Gardner l)avis Oram and Ryan Norton Ganoza andPomareda) Most studies have employed consumer-producer surplus analysis and havebuilt upon a large body of literature on ex post research evaluation Ex ante analysesusually incorporate expert opinion to determine projected research impacts adoptionrates and probabilities of research success and provide estimates of the economiceffliciencv and distributional implications of agricultural research resource allocationBenefit-cost studies typically calculate benefit-cost ratios internal rates of return andnet present values for alternative types of research or research onfor differentcommodities Ilse analyses may include andmav or not regional internationalresearch spillovers and the eflects of domestic pricing policies on research benefits

Mathematical Programming

Mathematical programming is another alternative for research selection It relies onmathematical Optimization to choose a research portfolio through maximizing amultiple-goal objective function given the resource constraints of the rescaich systemA good example of the use of this method is a study by Russell in the United Kingdomlie used goal programming to maximize the contributions of the research program toseveral goals given the con taints of budget human resources state of knowledge andcertain policies This procs Ue uses similar information to the weighted-criteria modelbut selects an optimal research portfolio rather than simply ranking research areas

Simulation

Finally the simulation method has been used to identify and select research prioritiesSimulation models vary in their construction but a good example is the model byPinstrup-Anderscn and Franklin They built a model to project the contributions andcosts of alternative research activities They established gals and then identifiedchanges in supply demand for inputs and demand for output needed to meet those

3

goals They identified neeled technologies time and financial costs and the probability of research success and adoption Finally they specified the scientists workingobjectives This model is very thorough but did require extensive amounts of data and estimation of several mathematical relationships

Comparative Assessment

The above discussion is a brie summary of the major structu al approaches that have been used to asist decision makers in research priority setting All of them attempt to go beyond the usual procedure of individual or group decision makin e without structural analyses Each of the approacles has its advantages and disadvantages (See Fable I)

The procedure of establishing criteria and using weights to arrive at a final set of research priorities has the advantage of forcing decision makers to consciously trade off multiple goals It can illcorporae both quantitative and qualhtative information and can be applied to a long list of con mnodities or research areas in a relatively liort period of time The procedure is relati cx ca sY fbr administrators to understand but does requiretheir time in obtaining the explicit w5eights for criteria Furthermore these weights are inevitably subjective and their elicitatlion must be carefully structured [lie nActhod also requires scientists time in collecting inlormation on qualitative criteria As a result the approach k better suited for periodic or major priority-setting efforts than or situations where frequent marginal changes are anticipated

The expected economic surplus approach his the major advantage of incorporatingseverml criteria related to economic elficiency and distribution into one or two measures It also can be ucd to examine the gencral equilibrium ellects ofresearch (Ramailho de Castro and Sclhh nAid tile benefits of research under alternative possibly distortionar domestic pticing and international trade policies tEdwarts and Freebairn Alston Edwards and rcecbairn Norton and (ianoza 1985) These alictors are pervasive in developing countries and affect the efficiency and distributional consequences of researh Ile procedure ie(liCe ir lel - tndcrstandw ofeconomic analysisand more anal-t s time than the weighted criteria model but less aduinistrators time It can be diflicult to apply to at large number of commodities or research areas because certain typcs of data necessary for the ainalysis often do not exist for all commodities With adequaIL tL d ita Cxpected economic surplus analysis can be incorporated into the weighted criteria model or could be used on the set of commodities which the weightedcriteria model imdicaMes to have the highest piority lhe latter approach would allow for the calculation of income foregone as a reult of placing weights on non-economic ef1ciency criteria Expected economic surplus analysis has the major advantage of calculating a rate of return which can be compared to alternative public investments

The mathematical programming approach is similar to the weighted criteria model approach becaue weights are placed on a set of goals or criteria The procedure has the advantage of explicitly considering the budget huLman resource aad other constraints on the research system tnless the constraints are well specified including possiblechanges over time liowever there is a risk of nonsense solutions The model is more intensive of an ecolonic analysts time and abilit than tie simpler weighted criteria approach and decision makers may be less willing to accept what appears to be a black box solution lrade-offs among goals are easily quantified ssith this approach

The advantage of simulation models is their tlexibility They can be constructed is relatively simple or complex tools can incorporate optimizing or ranking proceduresand can readily include probabilistic information Their major disadvantage is that to be useful they must be relatively complex and typically reqtuire extensive amounts of both data and time of skilled analysts

4

TABLE 1 COMPARISON AMONG MAJOR AGRICULTURAL RESEARCH PRIORITY-SETTING METHODS

PRIORITY-SETTING METHOD

HEIGHTED EXPECTED MATH CHARACTERISTIC CRITERIA ECON SURPLUS PROGRAM SIMULATION

OPERATIONAL CONSIDERATIONS

1 Relative cost in researchers time mudium mJium medium high2 Relative cost in priority setting analysts ime medium medium high high3 Relative cost in administrators time medium mediummedium medium 4 Relative overall data requirement medium medium medium variable 5 Relative ease of comprehonsion by decision maker high medium low low 6 Ease of incorporating subjective informaticn high high high high7 Ease of incorporating non-quantitativa inforiation high low low medium

GOAL-RELATED ISSUES

a Requires explicit elicitation of goals yes usually yes uiually9 Can determine distributional affects on consumers

and producers at various income levels no yes no yes10 Can handle uncertainty yes yes yes yes11 Can consider tradeoff among multiple goals yes sometimes yes yes

CRITERIA-RELATED ISSUES

12 Can consider private-sector research incentives yes difficult difficult yes13 Can consider economic policy and trade affects yes yes yes yes

EVALUATION-RELATED ISSUES

14 Can be used to sot priorities for research at the aggregate level no yes no es

15 Can be used to set research priorities at the commodity level yes yes yes yes

16 Can be used to set priorities for non-production or non-commodity oriented research yes difficuli yes yes

17 Can be used to set priorities for basic research yes difficult no sometimes 18 Can evaluate secondary impacts of rosuarch on

employment environment nutrition yes sometimes so etimes yes19 Usually estimates a rata of return to research yes sometimesno no 20 Can quantify geographic spillover effects no yes no yes 21 Can consider the lags involved in research

and adoption yes yesyes yes22 Facilitates priority setting when the number

of commodities is large yes difficultdifficult difficult

Source Adapted and modified from Norton and Davis

5

The above is a very brief summary of the major advantages and disadvantages of the most common formally structured priority-setting models that could be used for strategicplanning for research resource allocation Other approaches exist for research projectselection with perhaps models that attempt to identify and analyze key vield constraints being the most prevalent approach employed in NARS of less-deeloped countries

The results of ex post analysis also can provide useful guidance for ex ante research resource allocation decisions if appropriately incorporated into a systematic ex ante procedure The most common ex post approach in addition to cx post benefit-cost analysis is the econometric estimation of production or supply functions incorporating research variables To be most uteful for ex ante analysis econometric approaches mLlSt be applied with a high degree of disaggrega tion Unfortunateiy these models require a substantial amount of high-quality historical data on production farm inputs and research expenditures

Perhaps the two approaches with the most potential given additional refinements for application by NARS in less-developed countries are the weighted criteria models and the benefit-cost or expected economic surplus procedures In the next section reent experiences with applying weighted criteria models in three Latin American Countries )ominican Republic Ecuador and Uruguay are discussed and evaluated The purpose

it to piovide more iniormation about the procedures their degree of acceptance and possible improvements in the models Ihen application of expected economic surplus models are considered drawing cn a recent cx ante research evaluation studly in Peru Finally the relationship between weighted criteria and expected economic surpluts methods is described along with possible extensions

IIl THE USE OF WEIGHiTED CRITERIA MODELS IN TIlE DOMINICAN REPUBLIC ECUADOR AND URUGUAY

in 1986 iif I87 weighted criteria models were developed and implemented in the lDomin lRepublic (ISA) Ecuador (Espinosa et al) and Trugua (CIAAB)to assist with priority setting for agricultwral research In the I)ominican Republic the study was carried out at the Instituto Superior de Agnicultura ISA) by consultants for ISNAR In Ecuador the study was conducted by the plmnintg ollice oY the Instituto Nacional de Investigacion Agropecuaria (INIAP) with the assistance of consultants for the US Agency for International Development (AID) In Uruguay the study was conducted by the Ollice of the Director of the Centro de Investigacion Agricola Alberto Boergcr(CIAAB) with the assistance of ISNAR The purpose of all three studies was to applyprocedures for prioritizing agricultural research by commodity and by major research area Inthe Dominican Republic the client was the newly created natonal agriculturalresearch agency (Il)EA) In Ecuador the client was INIAP and the newly created foundation for agricultural research (IEl)IA) In Urugua the client was C1IAB

The procedures employed in the three studies were similar but with some importantdilfereices as the Ecuador study built on lessons learned inthe I)ominican Republic and the Uruguay study built on other lessons learmed in Ecuador National goals for the research system iere elicited in each country and a series of criteria establishd which relate to those goals Separate riteria were developed for conmitodities and or research areas and weights were elicited from decision makers to establish the relative importanceof the criteria Commodities and research areas were ranked according to each criterion and these rankings were multiplied by the elicited weights to arrive at research priorities

6

Goals

Three goals were identified in each of the three countries (1) to raise the average levelof income in the country (2) to increase the well-b(Aing of low-income groups in societyand (3) to reduce yearto-year income fluctuations in the country especially onl tiledownside These goals were worded slightly diffrcntly in each country but in each casethey essentially relate to the three goals referred to in previous studies as efficncy equity and security In the l)ominican Republic the local consultants identifiedthese as the goals while in lcuador and Uruguay it was done by the directors of national research agencies

Criteria

The next step was to establish criteria to as measures of whetheruse particularcommodities or research areas contribute to the attainment of the above goals A largeset of criteria was discussed in each country and a total of 15 criteria were eventuallyused in the Dominican Republic 14 in lEcuador and 10 in Uruguay to determine priorities by commodity Five criteria were used in each case to determine priorities byresearch area Refinements were made in successive studies - beginning with theDominican Republic then Ecuador followed by Iruguay - to increase the independenceof criteria and to remove criteria that were questionable measures of whether researchcontributed to tile tated goals ( Reason for dropping particular ci eria are discussedbelow) Refinements also were made with respect to the grouping of criteria to facilitatethe weighting of goals and criteria

Commodity Criteria omioditv criteria were grou l - J into four conceptual groupsproduct importance probability O success elliciencv in use of research resources anddistribution of impacts The first three of these groups relate to the efficiency or incomelevel goal and the last group relates to the equity or distributional goal In none of thethree studies were criteria included which represnted the third goal (reduced incomelluctuation [lie feeling was that the entire set of criteria as a group would serve toreduce the enlhasis placed on single or 1ew (often export) crops While this appearedto be the case in the results of thee particular studies ft ture weighted criteria studies may want to add criteria which explicitly rank commodities from lowest to hiehest withrespect to annual gross income variability (or price and yield variability separately)

The group of criteria referred to as jrodct importap contained Four criteria in theUruguay study value of production generation or saving of foreign exchange expectedfuture demand change and comparative advantage

The value of productionl is an important criterion because the cost of rseaich isbasically independent of the number of units alkected by the research results at least forthose commodities with relatively homogeneous production conditions Foreignexchange was used as a criterion because of a perception in each country that the lackof foreign exchange was impeding growtb A criterion was set up to allow more weightto be paced on those commodities for which demand is expected to increase in thefuture because the importance of commodities will change as diets are altered withincome growth aod a world markets change Comparative advantage was includedbecause income in the country will be higher if the country focuses its efforts oncommodities for which its resource base is best suited while recognizing that research itself can alter resource constraints

In the Dominican Republic and in Ecuador nutritional measurcs (calorie and proteincontributions to the diet) were included as criteria for product importance They wereincluded because the well-being of low-income groups may be affected by the availability

7

of foods which are important in their diets Nutritional criteria were dropped in Uruguay however because numerous studies have shown that income levels are tile primary determinants of nutrition levels rather than the availability of particular foods Certainly caloric and protein content are debatable criteria To the extent that increased domestic production lowers the prices of certain food commodities therebyincreasing food consumption by the poor nutritional criteria can contribute to the distributional goal Land area also was included as a criterion in the Dominican Republic study but was dropped in Ecuador and Uruguly because of the overlap with value of production

Two criteria were used in each study to measure 1robahility of reseatch success The first measure was tilegap between own-country yields and neighboing country yields The second was the potential for success indicated by the researchers themselves The rationale for the gap criterion was that the larger the yield gap between the home country and similar countries tle greater the potential for yield gains The criterion is rather crude because yield gaps also can indicate a relative resource disadvantace forproducing the commodity even if the countries appear to be geoclimatically sinilar Consequently the gap criterion was eventually dropped in the tUruguav study The potential for success as indicated by the researchers is subjective but a very necessary measure to obtain

The gro up of criteria reficrred to as e1)cienc in use otresearch resources contained three criteria the relationship to research in the international centers the degree of emphasis on tilecommodity in le current research p ogram and the incentive for the privatesector to conduct the research Some research duplicates other research croplementsand still other research bears no relationship to research at tileinternational agriculturalresearch centers For certain comniodities all three relationships may hold depending on the type of research If national research is complementary to international researchthe cost of the rational research is lower Consequently a criterion was established which places greater weight on commodities with a high degree of complementaritybetween ntin Il d international research The criterion ofdelegree of emphasis in tile current iesercli program was established because there is a cost to rapidly adjustingcurrent h uiman and physical research resources from one commodity to another The criterion of private-sector incentives was used because the social benefits from the total research program in the country will be greatest if scarce public research funds are devoted to research on commodities for which the private sector does not have an incentive to support the research itself Of coui se the private-sector producer group can tax itself and pay the public sector to conduct the research for those commodities for which it has an incentive to privately support the research These commodities tend to be export commodities because producers can capture a higher proportion of the gainsfrom productivity increases for export commodities than for commodities primarily consumed domestically

The group of criteria related to the distribution ojresarchimpacts contained two criteria in the Uruguay study number of producers and the effect of increased productivity on the price of product The larger the number of farms producing a commodity the largerthe number of farmers afficted by the research and perhaps the higher the probabilitythat smaller farmers are being affeicted The distribution of benefits depends to a significant degree on price changes resulting from increased productivity Producers benefit more if prices decline very little and consumers benefit more ifprices decline substatitially for a given production increase For export er import commodities priceswill tend to decline less than for commodities primarily produced and consumed domestically For commodities which are not traded and are very important in the diets of the poor prices will tend to decline relatively sharply when productivity increases The division of benefits between producers and consumers also is affected by the fact

that producers are also consumers Consequently in the Dominican Republic and Ecuador studies an additional criterion importance of the commodity in home-consumption on the farm where produced also was included

In summary 10-15 criteria were used in tie three studies to identify and select research priorities by commodity The linkages anong the goals and criteria described above are summarized in Figure 1

Research Area Criteria The five criteria uicel to select research priorities by research aieas were (1 ) whether the research causes an increase in the use of relatively abundant resources and a saving of relatively scarce resources (2) the number and seventy of research problems (3) non-duplication with transferable research from outside the country (4) tne extent of private-sector incentives to conduct the research and (5) current emphases in the research program

The rate of economic growth will be more rapid if new technologies are developed that utilize relatively abundant resources and save relatively scarce resources The resource base varies by region consequently information on resource abundance criteria was collected by region in tileEcuador and Uruguay studies The same was true for the quantity and severity of research problems Information on the relationship of domestic to inernational research private-sector incentives and the current emphasis in the research program was collected on a naticnal basis These criteria all relate to the income growth (efficiency) goal It is difficult to identif research area criteria that measurc whether particular types of research aifect income distribution or variability

Data Collection and Model Implementation

Information used in the analysis included both quantitative data on value of productionnumber of farms value of exports and imports person-years devoted to research on different commodities and tile of protein per day in thenumber of calories and gramsdiet (for tileDominican Republic and Ecuador studies) as well as qualitative or subjective information on such factors as probability of success private sector incentives and severity of research problems Furthermore weights had to be elicited from decision makers to place relative emphasis on the various criteria

Step 1 Develop Cotninodity and Research Area Lists -- The first step was to determine the list ofcommadities to consider and the list of research areas In the Dominican Republic information was collected on 74 commodities In Ecuador an initial list of 109 commodities on which INIAP was conducting research was discussed Throughdecisions to eliminate some and to group others this list was reduced to 44 commedities for the analysis These decisions were made by the Technical Director for Research In Uruguay a smaller list of commodities was reduced to 21 commodities or commodity groups These decisions were made in consultation with the National Research and Fxtension Directors The list of research program areas contained nine areas in the Dominican Republic 16 in Ecuador and 16 in Uruguay

Step 2 Collect Quantitative and Qualitative Data on Chosen Criteria -- Information o-i quantitative criteria were gathered from local and FAO secondary data sources and one table was constructed For each criterion Commodities were ranked for each criterion Information needed for the qualitative riteria as well as the weights to place on criteria were obtained through interviews with sc rtists and administrators at boh the national and regional experiment stations In Euador and Uruguay substantial attention was devoted to identifying the appropriate respondents to each question as this is a crucial aspect of the procedure In Ecuador those interviewed were notified one week before the interviews and 72 people were interviewed by the Planning Director of INIAP and

9

FIJR 1 CONCEPTUAL ELEHNTS IN THE HEIGHTED CRITERIA MODEL FOR PAWING RESEARCH PRIORITIES BY COtlDITY

GOALS fficiency ]

CONCEPTUAL( )CRITERIA Pri7] Probabi i+y Research Distriutjon Iincome

ii J of Succes Resource Eff o L=Acplusmn a rilbility

CRITERIA 1 Value of 1 Scientist 1Rl ofto toNumberPrcduction Estimates 1 AnnualInternational Producers Variability2 Foreign Centers 2 Price in Gro

Exchange 2 Current Effect3 Future Raturn3)Emphasis 3 HomeDemand 3 Private Consumpt ion

4 Comparative Sector Advantage Incentives

Notes

(-n future studies an additional conceptural criteria under the security goal might be health (J4tri ional criteria such as grams of potein or kilo=alories could be included as additional easures of Distributional ikipacts

4ot used in Dominican Republic Ecuador and Uruojuay studies

his assistant In Uruguay the directors of all the regional stations were called into CIIAB headquarters for a aweting to discuss the study and this was followed up byon-site interviews and meetings with scientists and directors at the regional experimentstations as well as with economists in the Ministry of Agriculture These interviews and meetings were conducted by a person assigned to work wvith the Director of Cl LAB on the priority-setting study In the Dominican Republic a survey of scientists extensionworkers and administrators was conducted by the ISNAR consultant from the Instituto Superior de Agricultura who visited the various experiment stations Less care wastaken in the Dominican Repubic than in Ecuador and Uruguay to match the questionspertaining -o individual criteria to particular individuals This probably reduced theusefulness of the responses because those most familiar with the probability of research success For example may not be the same -s those most Familiar with comparativeadvantage

Step 3 Elicit 11eihts on Critria -- Relative weights to place on the different criteria were obtained Fronm national and regional research system administrators In Ecuador34 peopic verc used to determine the weights These wveights were established separatelyIor the commoditv and the research area criteria In 1iuguay fewer people (sevensystem and station directors) were used to determine the weights and more attention was devoted to grouping the criteria pertaining to particular goals A Delphi procedure was used in which the s7even directors were shown the average of the group and provided with an opportunity to adjust their weight

Step 4 )erive Rankinqs hv Commodity and By Research Area -- Once the basic information was collected axd organized into a seies of tables one procedure was followed to arrive at a final ranking of research priorities by commodity A second procedure resulted in lists of priorities by research area In the Dominican Republicmost of the calculations were made by hand in Ecuador and Uruguay microcomputerspreadsheet programs were used to assist with the calculations and to conduct sensitivity analysis

The steps in the procedure for ranking research priorities by commodity and research area are sumnmaried below and are illustrated with a simple example in Appendix 1 Brieily the steps for ranking commodities were

a) Th conniodities were ranked for each criterion for which quantitative data were available

b) Tihese rankings were multiplied by the weights assigned to each criterion and then the weighted criteria were added across to arrive at one sub-ranking for the

Luantitative criteria c) Each commodity was given a high low or none designation for each qualitative

(subjective) criterion The response which implied a need for greater research priority was assigned the number 2 the intermediate response was assigned I and the response which implied lower research priority was assigned 0 1herefore each comniodity received a 2 1 or 0 for each criterion

d) The weights assigned to each qualitative criterion were then multiplied by these numbers and the results were added across all criteria to arrive at a subtotal for each commoditv These subtotals were ranked from highest to lowest to provide a second sub-ranking

e) The sub-rankings for the quantitative and qualitative criteria were then given their corresponding weights and added together to arrive at a final ranking by commodityThe steps for ranking research priorities by research area were the same as steps (c)and (d) above because the criteria were all qualitative

Step 5 Analsis and Interpretationof Resuls -- In the Dominican Republic the resuLts of this weighted criteria analysis were uscd to determine a small set of cormodities and

II

research areas with the highest priority Further assessment was then made of humanphysical administrative and other resources needed to structure research programsfocused on these commodities and resea-ch topics In Ecuador and Uruguay the prioritized list of commodities was split into a high-priority group an intermediate-priority group and a low-priority group Research area priorities were identified for each region of tile country A more detailed analysis was then made of the human physical and other resources needed to support the high and intermediate priorities

In summary five major -rens were involved in operationalizing the weighted criteria models A complete set of rsults for each study included more than 20 tables and are not reproduced here Ilowever the final rankings for commodities and for research areas are shown in Tables I and 2 in Appendix 2 In the Dominican Republic a team of consultants followed up the initial priority-setting exercise with additional analysisand discussions at the experiment station level and eventually recommended tile establishment of five national commodity programs (rice cornsorghum beans coffeeand livestock) and one additional major research program (natural resources) In Ecuador the prioritized lists were widely distributed and discussed in the Ministry oi Agriculture the Board of FEIDIA the Commission on Science and Technology arid in AID There was concern in Ecuador with the weights placed on the individual criteria and therefore additional sensitivity analysis was conducted by the Planning Office of the Ministry There seemed to be a recognition in Ecuador that priority setting with a weighted criteria model is an iterative process and that much 4t the models value stems from tile discussion of criteria among the decision makers and from the sensitivityanalysis The high-priority list of comniodities in Ecuador included thL ten highest-priority commodities listed in Table 2b A project paper has been written forAI) which recommends funding For Fli)IAlbr three commodities (coffee soft corn for the Sierra and dairy) Coffee and soft coitn were the number 2 and 3 ranked prioritiesin tile weighted criteria model and dairy was number 9 although it was the top-rankedlivestock commodity

In Urugu ay the results from the first run with tile model were discussed with the research directors Ilie directors made small changes to the weights placed on criteriaand the mode was rerun resulting in the prioritized list in Appendix 2 Personnel in CIAAB are CUrrcwtv undertaking additional sensitivity analysis and with tie assistance of ISNA R are developing a plan for implementation

An Assessment of the Weighted Criteria Models Used in the Dominican RepublicEcuador and Uruguay

The Dominican Republic Ecuador and Uruguay studies were perhaps the first majorattempts to implement weighted criteria models with weights explicitly elicited from decision makers in NARS in less-developed countries A number of strengths andweaknesses in the procedures were identified during the analyses Some of thL weaknesses were corrected fron one study to the next some can be corrected in future studies and others are inherent in the approach

Trhe first strength mentioned earlier is the ability of the procedure to systematicallyincorporate both quantitative and qualitative information related to a set of multiplegoals and criteria in order to prioritize a lorng ist of commodities and research areas in a relatively short period of time Second the procedure proved relatively easy for both research administrators and the loc- -nalysts to understand with the exception of the )ominican Republic where the research administrators were not directly involved7

Third the analysis as applied in Ecuador and ULuguay with the direct involvement of research system administrators at various stages forced those decision makers to

12

consciously identifv and trade off goals and criteria Fourth the use of spreadsheet programs in EcUadfor and Uruguay facilitated sensitivity analysis after the initial set of priorities was deermimd Fifth the procedure provided a relatively objectiveassessment of prioriies because individuals were not allowed to rank commodities orresearch areas directly but had to weight criteria

The first weakness inherent in ex ante research priority setting and thus in theprocedure is that there is a large amount of subjectiveness Although tileapproach isless subjective than unstructured judgment there is subjectiveness in the responses toquestiorns related to some of the criteria and also ini the weights placed by decision makers on the criteria Second it proved diflicult to specify independent criteria with no overlap Third while the procedure required less time than an expected economic surplus analysis by commodity it required more time than asking respondents to directlyrank commodities and research areas In E-cuador the consiultant worked withpersonnel in INIAP and FEI)IA for one week at the start of the analysis to determinegoals and criteria to arrive at atlist of commodities and researh areas to list secondarydata needs to formulate questions to be asked of administrators and scientists to decide who should be asked particular questions and to explain the procedure itself includingthe spreadsheet program The required iidormation was collected over the Followingmonth and the consultant returned and worked with the Planning Director in INIAP for one week to conduct the analysis and write tip the results Revisions to the initial draftof the report we-e then made over the lollowing weeks A similar procedure to the onefollowed in Ecuador was used in l_ruguay althaugli more time was spent discussing the wording of questions to be asked of scientists Furthermore n Uruguay more care was taken to match the criteria with the goas

Certain of the criteria wil always be diflicult to eplain to tileinterviewees Thecriterion on comparative advantage is difficult for some respondents because of a lack of knowledge about neighboring countries If suflicient data time and resources are available this criterion may be rnea4ured more precisely using comparative advantage or domestic resource cost procedures suggested by longmire and Winkelnann Anotherdiflicult criterion to estiniate is the potential foture demand for the product The respondents in Fcuador were asked to consider how the demands for different foodswould change as income and populatiot grow Many of them had a hard timeconceiving that the total or per capita demand for certain Foods could actually decline as income grew In Uiruguay (ibe cmiparative advantage and future dema nd questions were answered by a small group of economists from the ministries of agriculture and trade In the Ecuador study more attention was devoted to assessing the most appropriate people for answering different questions than was the case in the Dominican Republic Ilowever some criticism surfaced about the weights placed on criteria InUruguay decision makers were given the opportunity to change their weights afterviewing the initial results This was very useftl because it demonstrated the implicationsof placing different weights on the various goals and criteria

Computer spreadsheet programs were a major help in Ecuador and Uruguay Additional work needs to be devoted to developing a more mentnu-driven prcgram but the people inthe planning office of INIAP were able to use the SuperCalcR program without too tmucht difliculty despite ialack of previous experience Ihe analyst in Uruguay was veryfamiliar with lotusl which proved most helpfil

It became clear from working inall three countries that it is preferable to work directlywith the final decision makers or their designees Inthe l)ominican Republic the study was conducted by a very competent set of consultants from the local agricultural universi ty but the procedure was not institutionalized to allow for additional sensitivityanalysis or future priority-setting efforts InEcuador and especially in Uruguay the

13

decision makers were directly involved conducted sensitivity analyses and were able to assess why certain commodities or research areas received high priority as a result of the analyses In both Ecuador and Uruguay they have a basic understanding of the strengths and weaknesses of tileprocedure and therelre can explain their subsequentdecisions to groups who might disagree with tilepriorities they establish

The corntrast between the Dominican Republic and tie Uruguay studies illustrates a veryimportant point No matter which technique is used to structure the analysispriority-setting exercises must integrate the decision makers (users) into the processPriority-setting procedures must be institutionalited the results of initial analysisdiscussed thoroughly with decision makers and sensitivity anilsis conlducteU alone the lines suggested by that discussion Tecinological change generates a surplus which can be appropriated by dilkircnt sectors or groups within the economy Who benelitdepends in part on the cotaniodities selected for research and the types of technoloLies generated One of the purposes of institutionalizing a formal prioritN- setting procedureis to help mediate conflicts that may arise (for example between producer groups and consumers) as a result of research-induced improved technologies

Other issues arose during the analyses First the initial grouping and elimination of certain commodities and research areas before lhe irialvis is an important step but one which must be taken with carelul consideration of possible coniplmentarities in research on particular productS or types of rearch Second some of the commodities such as forages are inputs into other commodities such as milk (Corseqtentlv these commodities llay have to he considered itscinps because it is the value-added and not tilegross value of output that is relevant when one coniiodity requires other commodities sinputs and when criteria such as value of production are used This was an issue in I ruguav where livestock are quite important liird little attention was devoted in atv of the studies to defining appropriate criteria Ior the increased inoome stability eoal Future studies if this be an ideltJiiLd LOal shOUld define risk criteria

In sutImary the weighted criteria model as applied in Lcuador and iuguav which had the advantage of bit ildin Oillessons lCarICd illdie l)ominican RCpublic appears to have been a uselul priority-setting tool It ha its ereatest advartage ill situations ill which there are rulItiple goals to consider a relatively major reassesnieiit of prioritiesis contenplated and the list of commodities dd resen rch areas to consider is largeWeighted criteria models described in this paper were applied in small countries with limited resources but with direct involvement (except in the Dominican Republic) ofthC major decision makers These models itav be most useful in the small country situationwhere decision mAkers are more likely to be involved in tile priority-settirt process and are not just using tables of results Perhaps in larger systet7 where decisioii makine is more decentralited tileweighted criteria model can he applied at the regional level before relining research priorities at the national level

Another factor that can be uniortant in large svvtern is tle complerientarity between resea-h aid educat ion [hie larger the svstcnt the clser the linkages tend to be between universities and tileresearchi system Where universities are important tor agriclltural research an aidditional criterion may hc rteedCd to place value on the educationlal benefits of certain types of reseirch

Ihere are several additional iniplications with respect to how weighted criteria models relate to alterniative priori t-setting Inethod parIicularl the expected economic surplusapproach Befoce exploring these relationships kcx aspects of the surplus approacliare described based on aii application in Peru

14

WORKING PAPER No 7

PRiORITY-SETTING MECHANISMS FOR NATIONAL

AGRICULTURAL RESEARCH SYSTEMS

PRESENT EXPERIENCE AND FUTURE NEEDS

George W Norton and Philip G Pardey

November 1987

en

International Service for National Agricultural Research

ISNAR WORKING PAPERS

The ISNAR working papers series is a flexible instrument for sharing

analysis and information about relevant organization and management

problems of the agricultural research systems in developing countries

In the course of its activities - direct assistance to national

agricultural research systems training and research ISAAR genarates a-

broad range of information and materials which eventually become the

formal products of its publications program The working papers series

enhances this program in several important ways

1 These papers are intended to be a rapid means of presenting the

results of work and experiences that are still in progress but are

already producing cesults that could be of use to others

2 They are intended to be an effective vehicle for widening the

discussion of continuing work thereby increasing the quality of the

final products critical conment is welcomed

3 The series provides an outlet for diffusing materials and information

that because of their limited coverage do not meet the requirements

of general audience publication

The series is intended mainly for the diffusion of materials produced by

ISNAR itaff but it is also available for the publication of documents

produced by other institutions should they wish to take advantage of the

opportunity

George W Norton is associate professor Virginia Polytechnic Institute and StateUniversity Blacksburg Virginia and research 1llow International Service for National Agricultural Research (ISNAR) The Ilague Netherlands Philip G Pardey is joiltly a research associate University of Minnesota St Paul Minnesota and reseach officer ISNAR The authors would like to thank Iloward Elliott Alexander Von der Osten Paul Berinell and Carlos Valverde for their helpful comments and suggestions on an earlier draft of this paper without implicating thern in any remaining errors and omissions

TABLE OF CONTENTS

I Context of Research Priority-Setting I 1I Previous Priority-Setting Models 2

Weighted Criteria Models 3Benefit-cost (Expected Economic Surplus) Analysis 3Mathematical Programming 3Simulation 3Comparative Assessment 4

III The Use of Weighted Criteria Models in the Dominican RepublicEcuaodm and Uruguay 6

Goals 7Criteria 7

Coinmoditv Criteria 7Research Area Criteria 9

Data Collection and Model Implementation 9An Assessment of the Weighted Criteria Models used in theDominican Republic Ecuador and Uruguay 12

IV The Use of Expected Economic Surplus Analysis in Peru 15

An Assessment of the Expected Economic Surplus Procedure in Peru 15 V Implications for Development of Improved Research Priority-Setting Procedures 16 VI Conclusions 18 Footnotes 19

References 20

Appendix 1 23

Appendix 2 26

Priority-Setting Mechanisms for National

Agricultural Research Systems

Present Experience and Future Needs

Recent trends towards shrinking real research budgets or many national agriculturalresearch systems (NRS) have stimulated several attempts to evaluate tileeconomic benefits ofaricultural reserch Manv of these quantitatis e economic evaluations haveemployed historical data to provide ex post assessments and a few hase used subjectivejudgments to project future research benefits [he results kthese analyses have been used at the national or inter-ministerial level to justilf the need for continued political support for agricultural research

Tighter rese rch budgets not only heighten the iced to justifV agricultural research vis a vis alternative public investments but they alVo make it important for NARS to improve procedures for setting priorities among competing research programs A varietyof strategic and project-level decisions are made within NARS The objectives of national policvmakers the needs of agricultural producers and scientific opinion must each be brotight to bear on strategic decisions to emphasize particular commodities and research program areas uld on research project selection

The purpoe of this paiper is to discuss kev elements for inclusion in improved research priority-setting procedures for NRS Ihe fLcus is on methods for strategic as opposedto more tactical projcct-le el decision m kingAfter discussing the context of research priority setting a brief summary is provided of tile iain types of research resource alloL-ition mnetlods Recciit cxperiences with priority setting in four Latin American NAR the l)ominican Republic lcuador lUVUav and Peru are compared These experiences can proside sonc guidance for the development of an improved research resource allocation procedure

I TilE CONTEXT OF RESEARCH PRIORITY SETTING

NARS decision mnkers often operating within the ministry of agriculture an agricultural research institute or as a research council make allocative choices based on their knowledge atnd prior cxperience That knowledge and experience frequentlyincludes an understar one of national and regional oals and objectives as voiced through the political process a sene o lie severity of particular types of research problems and a general feel tor vlat is achievable through research It also includes an awareness of the desires ( te-ressures) of producer groups Decision makers must decide hoW much cmpha sisto jte onparticular commodities (eg rice maize beanssheep) and types of research c v varietal improvenmnt improved livestock nutritionplant protection farming svwtes Implicit in those decisions are the locational emphasis of research the focus 0rt paJticullar factors of production (eg land laborwater) the emphasis placed Ol lone-r-term haisic versus shorter-term applied and adaptive research and the distrihotianal elfects of research on farm size on producers versus coniumners and on people atidiherent income lec

In many NA RS research resoce alocation decisions are heavily influenced by the previous years budget2 (hanges of~tn result from requests by scientists which areevaluated reLatively informally and aggregatcd into an overall plan The resulting plan

upon close scrutiny may contribute little to tire attainment of stated national goals and objectives

Judgment based upon prior knowledge and information provided by scientists is crucial for research resource alincation decisions In addition rapid or radical changes in research programs -an be costly Particularly as research budgets tighten howeverforcing changes in tile systein tile increased use of quantitative methods may be nocessary to imprwe thL objectivity of those judgments The aim is to foster consitencv of research priorities with goAs and objectives and to improve the efliciency of the research system in mser ing producer and cortsumer needs The idea is not to replacejudgment but to increase and organize the inflormation available for updating priorknowledge and beliefs The hope also is to inject continuity in the priority-settingprocess and to proide NARS decision makers x ith methods the find lielplib when rationalizing their decisions to scientists producer Erotps and politicians Docunentingthe decision proces through a strctured procedure can help minimie the possinilitvof sudden and large shilts in researh priorities in those systems characterized by a flairlv rapid turnover of research administrators

Many quantitative priority-setting procedures are avaiia ble but few have beeninstitutionalized into the decision-making practices of NA RS manaers A cruciZl factor in their non- use of these procedures urtdoubtcdlv as been the lack of ri orous vetcost-eflectie approaches which can irtcorporate tile large number of comtodities andresearch areas as well as the multiple go1ls a ld criteria found in most strateoicdecision-muakirt situations inl NRS FurtIhcrrtore the cortmputer capabiliv available through microcomputers in most developing-co ntry NRS today was not gcenerallyavailable even two or three velars ago

In this paper procedure amc ce crihed which were recently applied in four latinAmerican countries First however previously emploved priority-setting mechanisms are briefly reviewed id critiqued One of rite points that will Ie made at the end of this paper is tiLit tise methods are not nitually excilsive There is no siTele magic huletbut there appears to be real potential for developing a Ilexible computer-assisted priority-setting framework that will allow NARS administrators to call ott theappropriate tectiiques tot aSsist withit a particular decision depending ott tie time and financial resources a ailahle and the relative itiportance of the allocation decision Asecond major point is that priority setting is a process not simply a procedure and must be instit utiortliied as part of the overall process of research managementPriority-setting technritiuetes applied only once by people outside of the decision-making process have little valtie

If SUMMARY OF PRIORITY-SETTING METHODS

Methods reported itt previous studies for research priority setting include

I ) establishtert td weighting of multiple criteria for ranking commodities and research ares with a final aggregate ranking based on implicit or explicit weights

2) use of bertefit-cost analysis including expected economic surplus techniques to select cotitritodities and research areas

3) application of mathematical programming to choose an optimal research portfolioincorporatirng multiple goals md constraits

4) developrert and tise of simulation models

These four approaches are discussed below followed by a brief assessment of their advantaiges atil dis idsittages

2

Weighted Criteria Models

Several studies hae established multiple criteria for ranking priorities because of tiledesire to explicitly consider a wide variety of factors that do or perhaps should influenceresearch selection The relative weights attached to each criterion to arrive at a final listof research prioriries are sometimes left implicit or unstated A recent example is tileTAC review of priorities and future strategies for the Consultative Group onInternational Agricultural Research (CGIAR) In assessing priorities by commoditythey etablished a principal goal research objective to obtain the goal and a series ofcriteria organized into three groups relevance research productivity and efliciency ofhe CGIAR system in undertaking research Detailed tables with quantitative andqualitative information and rankings of commodities for each criterion were presentedIn developing the recommendations for hort- to medium-term funding alternatives atthe conmmodity level the weights used to aggregate across criteria were left implicit

There are examples of tudies that have incorporated multiple criteria but also explicitlyspecified and utilized a set of weights to aggregate across criteria and obtain a finalranking of research priogrities Appications of this method often called the scoringmodel approach are found in studies by Williamson lahlstede Paulsen and KaldorShumway and McCrack en and Von (Oppen and Ryan A flew studies have used congruence analysis which can be thought of as a special case of a sci-ring or weightedcriteria model with all the weight placed on the criterion of vaue of production

Benefit-cost (Expected Economic Surplus) Analysis

The benefit-cost approach to welecting research priorities has been used in differentforms (Fishel Arai Sire and Gardner l)avis Oram and Ryan Norton Ganoza andPomareda) Most studies have employed consumer-producer surplus analysis and havebuilt upon a large body of literature on ex post research evaluation Ex ante analysesusually incorporate expert opinion to determine projected research impacts adoptionrates and probabilities of research success and provide estimates of the economiceffliciencv and distributional implications of agricultural research resource allocationBenefit-cost studies typically calculate benefit-cost ratios internal rates of return andnet present values for alternative types of research or research onfor differentcommodities Ilse analyses may include andmav or not regional internationalresearch spillovers and the eflects of domestic pricing policies on research benefits

Mathematical Programming

Mathematical programming is another alternative for research selection It relies onmathematical Optimization to choose a research portfolio through maximizing amultiple-goal objective function given the resource constraints of the rescaich systemA good example of the use of this method is a study by Russell in the United Kingdomlie used goal programming to maximize the contributions of the research program toseveral goals given the con taints of budget human resources state of knowledge andcertain policies This procs Ue uses similar information to the weighted-criteria modelbut selects an optimal research portfolio rather than simply ranking research areas

Simulation

Finally the simulation method has been used to identify and select research prioritiesSimulation models vary in their construction but a good example is the model byPinstrup-Anderscn and Franklin They built a model to project the contributions andcosts of alternative research activities They established gals and then identifiedchanges in supply demand for inputs and demand for output needed to meet those

3

goals They identified neeled technologies time and financial costs and the probability of research success and adoption Finally they specified the scientists workingobjectives This model is very thorough but did require extensive amounts of data and estimation of several mathematical relationships

Comparative Assessment

The above discussion is a brie summary of the major structu al approaches that have been used to asist decision makers in research priority setting All of them attempt to go beyond the usual procedure of individual or group decision makin e without structural analyses Each of the approacles has its advantages and disadvantages (See Fable I)

The procedure of establishing criteria and using weights to arrive at a final set of research priorities has the advantage of forcing decision makers to consciously trade off multiple goals It can illcorporae both quantitative and qualhtative information and can be applied to a long list of con mnodities or research areas in a relatively liort period of time The procedure is relati cx ca sY fbr administrators to understand but does requiretheir time in obtaining the explicit w5eights for criteria Furthermore these weights are inevitably subjective and their elicitatlion must be carefully structured [lie nActhod also requires scientists time in collecting inlormation on qualitative criteria As a result the approach k better suited for periodic or major priority-setting efforts than or situations where frequent marginal changes are anticipated

The expected economic surplus approach his the major advantage of incorporatingseverml criteria related to economic elficiency and distribution into one or two measures It also can be ucd to examine the gencral equilibrium ellects ofresearch (Ramailho de Castro and Sclhh nAid tile benefits of research under alternative possibly distortionar domestic pticing and international trade policies tEdwarts and Freebairn Alston Edwards and rcecbairn Norton and (ianoza 1985) These alictors are pervasive in developing countries and affect the efficiency and distributional consequences of researh Ile procedure ie(liCe ir lel - tndcrstandw ofeconomic analysisand more anal-t s time than the weighted criteria model but less aduinistrators time It can be diflicult to apply to at large number of commodities or research areas because certain typcs of data necessary for the ainalysis often do not exist for all commodities With adequaIL tL d ita Cxpected economic surplus analysis can be incorporated into the weighted criteria model or could be used on the set of commodities which the weightedcriteria model imdicaMes to have the highest piority lhe latter approach would allow for the calculation of income foregone as a reult of placing weights on non-economic ef1ciency criteria Expected economic surplus analysis has the major advantage of calculating a rate of return which can be compared to alternative public investments

The mathematical programming approach is similar to the weighted criteria model approach becaue weights are placed on a set of goals or criteria The procedure has the advantage of explicitly considering the budget huLman resource aad other constraints on the research system tnless the constraints are well specified including possiblechanges over time liowever there is a risk of nonsense solutions The model is more intensive of an ecolonic analysts time and abilit than tie simpler weighted criteria approach and decision makers may be less willing to accept what appears to be a black box solution lrade-offs among goals are easily quantified ssith this approach

The advantage of simulation models is their tlexibility They can be constructed is relatively simple or complex tools can incorporate optimizing or ranking proceduresand can readily include probabilistic information Their major disadvantage is that to be useful they must be relatively complex and typically reqtuire extensive amounts of both data and time of skilled analysts

4

TABLE 1 COMPARISON AMONG MAJOR AGRICULTURAL RESEARCH PRIORITY-SETTING METHODS

PRIORITY-SETTING METHOD

HEIGHTED EXPECTED MATH CHARACTERISTIC CRITERIA ECON SURPLUS PROGRAM SIMULATION

OPERATIONAL CONSIDERATIONS

1 Relative cost in researchers time mudium mJium medium high2 Relative cost in priority setting analysts ime medium medium high high3 Relative cost in administrators time medium mediummedium medium 4 Relative overall data requirement medium medium medium variable 5 Relative ease of comprehonsion by decision maker high medium low low 6 Ease of incorporating subjective informaticn high high high high7 Ease of incorporating non-quantitativa inforiation high low low medium

GOAL-RELATED ISSUES

a Requires explicit elicitation of goals yes usually yes uiually9 Can determine distributional affects on consumers

and producers at various income levels no yes no yes10 Can handle uncertainty yes yes yes yes11 Can consider tradeoff among multiple goals yes sometimes yes yes

CRITERIA-RELATED ISSUES

12 Can consider private-sector research incentives yes difficult difficult yes13 Can consider economic policy and trade affects yes yes yes yes

EVALUATION-RELATED ISSUES

14 Can be used to sot priorities for research at the aggregate level no yes no es

15 Can be used to set research priorities at the commodity level yes yes yes yes

16 Can be used to set priorities for non-production or non-commodity oriented research yes difficuli yes yes

17 Can be used to set priorities for basic research yes difficult no sometimes 18 Can evaluate secondary impacts of rosuarch on

employment environment nutrition yes sometimes so etimes yes19 Usually estimates a rata of return to research yes sometimesno no 20 Can quantify geographic spillover effects no yes no yes 21 Can consider the lags involved in research

and adoption yes yesyes yes22 Facilitates priority setting when the number

of commodities is large yes difficultdifficult difficult

Source Adapted and modified from Norton and Davis

5

The above is a very brief summary of the major advantages and disadvantages of the most common formally structured priority-setting models that could be used for strategicplanning for research resource allocation Other approaches exist for research projectselection with perhaps models that attempt to identify and analyze key vield constraints being the most prevalent approach employed in NARS of less-deeloped countries

The results of ex post analysis also can provide useful guidance for ex ante research resource allocation decisions if appropriately incorporated into a systematic ex ante procedure The most common ex post approach in addition to cx post benefit-cost analysis is the econometric estimation of production or supply functions incorporating research variables To be most uteful for ex ante analysis econometric approaches mLlSt be applied with a high degree of disaggrega tion Unfortunateiy these models require a substantial amount of high-quality historical data on production farm inputs and research expenditures

Perhaps the two approaches with the most potential given additional refinements for application by NARS in less-developed countries are the weighted criteria models and the benefit-cost or expected economic surplus procedures In the next section reent experiences with applying weighted criteria models in three Latin American Countries )ominican Republic Ecuador and Uruguay are discussed and evaluated The purpose

it to piovide more iniormation about the procedures their degree of acceptance and possible improvements in the models Ihen application of expected economic surplus models are considered drawing cn a recent cx ante research evaluation studly in Peru Finally the relationship between weighted criteria and expected economic surpluts methods is described along with possible extensions

IIl THE USE OF WEIGHiTED CRITERIA MODELS IN TIlE DOMINICAN REPUBLIC ECUADOR AND URUGUAY

in 1986 iif I87 weighted criteria models were developed and implemented in the lDomin lRepublic (ISA) Ecuador (Espinosa et al) and Trugua (CIAAB)to assist with priority setting for agricultwral research In the I)ominican Republic the study was carried out at the Instituto Superior de Agnicultura ISA) by consultants for ISNAR In Ecuador the study was conducted by the plmnintg ollice oY the Instituto Nacional de Investigacion Agropecuaria (INIAP) with the assistance of consultants for the US Agency for International Development (AID) In Uruguay the study was conducted by the Ollice of the Director of the Centro de Investigacion Agricola Alberto Boergcr(CIAAB) with the assistance of ISNAR The purpose of all three studies was to applyprocedures for prioritizing agricultural research by commodity and by major research area Inthe Dominican Republic the client was the newly created natonal agriculturalresearch agency (Il)EA) In Ecuador the client was INIAP and the newly created foundation for agricultural research (IEl)IA) In Urugua the client was C1IAB

The procedures employed in the three studies were similar but with some importantdilfereices as the Ecuador study built on lessons learned inthe I)ominican Republic and the Uruguay study built on other lessons learmed in Ecuador National goals for the research system iere elicited in each country and a series of criteria establishd which relate to those goals Separate riteria were developed for conmitodities and or research areas and weights were elicited from decision makers to establish the relative importanceof the criteria Commodities and research areas were ranked according to each criterion and these rankings were multiplied by the elicited weights to arrive at research priorities

6

Goals

Three goals were identified in each of the three countries (1) to raise the average levelof income in the country (2) to increase the well-b(Aing of low-income groups in societyand (3) to reduce yearto-year income fluctuations in the country especially onl tiledownside These goals were worded slightly diffrcntly in each country but in each casethey essentially relate to the three goals referred to in previous studies as efficncy equity and security In the l)ominican Republic the local consultants identifiedthese as the goals while in lcuador and Uruguay it was done by the directors of national research agencies

Criteria

The next step was to establish criteria to as measures of whetheruse particularcommodities or research areas contribute to the attainment of the above goals A largeset of criteria was discussed in each country and a total of 15 criteria were eventuallyused in the Dominican Republic 14 in lEcuador and 10 in Uruguay to determine priorities by commodity Five criteria were used in each case to determine priorities byresearch area Refinements were made in successive studies - beginning with theDominican Republic then Ecuador followed by Iruguay - to increase the independenceof criteria and to remove criteria that were questionable measures of whether researchcontributed to tile tated goals ( Reason for dropping particular ci eria are discussedbelow) Refinements also were made with respect to the grouping of criteria to facilitatethe weighting of goals and criteria

Commodity Criteria omioditv criteria were grou l - J into four conceptual groupsproduct importance probability O success elliciencv in use of research resources anddistribution of impacts The first three of these groups relate to the efficiency or incomelevel goal and the last group relates to the equity or distributional goal In none of thethree studies were criteria included which represnted the third goal (reduced incomelluctuation [lie feeling was that the entire set of criteria as a group would serve toreduce the enlhasis placed on single or 1ew (often export) crops While this appearedto be the case in the results of thee particular studies ft ture weighted criteria studies may want to add criteria which explicitly rank commodities from lowest to hiehest withrespect to annual gross income variability (or price and yield variability separately)

The group of criteria referred to as jrodct importap contained Four criteria in theUruguay study value of production generation or saving of foreign exchange expectedfuture demand change and comparative advantage

The value of productionl is an important criterion because the cost of rseaich isbasically independent of the number of units alkected by the research results at least forthose commodities with relatively homogeneous production conditions Foreignexchange was used as a criterion because of a perception in each country that the lackof foreign exchange was impeding growtb A criterion was set up to allow more weightto be paced on those commodities for which demand is expected to increase in thefuture because the importance of commodities will change as diets are altered withincome growth aod a world markets change Comparative advantage was includedbecause income in the country will be higher if the country focuses its efforts oncommodities for which its resource base is best suited while recognizing that research itself can alter resource constraints

In the Dominican Republic and in Ecuador nutritional measurcs (calorie and proteincontributions to the diet) were included as criteria for product importance They wereincluded because the well-being of low-income groups may be affected by the availability

7

of foods which are important in their diets Nutritional criteria were dropped in Uruguay however because numerous studies have shown that income levels are tile primary determinants of nutrition levels rather than the availability of particular foods Certainly caloric and protein content are debatable criteria To the extent that increased domestic production lowers the prices of certain food commodities therebyincreasing food consumption by the poor nutritional criteria can contribute to the distributional goal Land area also was included as a criterion in the Dominican Republic study but was dropped in Ecuador and Uruguly because of the overlap with value of production

Two criteria were used in each study to measure 1robahility of reseatch success The first measure was tilegap between own-country yields and neighboing country yields The second was the potential for success indicated by the researchers themselves The rationale for the gap criterion was that the larger the yield gap between the home country and similar countries tle greater the potential for yield gains The criterion is rather crude because yield gaps also can indicate a relative resource disadvantace forproducing the commodity even if the countries appear to be geoclimatically sinilar Consequently the gap criterion was eventually dropped in the tUruguav study The potential for success as indicated by the researchers is subjective but a very necessary measure to obtain

The gro up of criteria reficrred to as e1)cienc in use otresearch resources contained three criteria the relationship to research in the international centers the degree of emphasis on tilecommodity in le current research p ogram and the incentive for the privatesector to conduct the research Some research duplicates other research croplementsand still other research bears no relationship to research at tileinternational agriculturalresearch centers For certain comniodities all three relationships may hold depending on the type of research If national research is complementary to international researchthe cost of the rational research is lower Consequently a criterion was established which places greater weight on commodities with a high degree of complementaritybetween ntin Il d international research The criterion ofdelegree of emphasis in tile current iesercli program was established because there is a cost to rapidly adjustingcurrent h uiman and physical research resources from one commodity to another The criterion of private-sector incentives was used because the social benefits from the total research program in the country will be greatest if scarce public research funds are devoted to research on commodities for which the private sector does not have an incentive to support the research itself Of coui se the private-sector producer group can tax itself and pay the public sector to conduct the research for those commodities for which it has an incentive to privately support the research These commodities tend to be export commodities because producers can capture a higher proportion of the gainsfrom productivity increases for export commodities than for commodities primarily consumed domestically

The group of criteria related to the distribution ojresarchimpacts contained two criteria in the Uruguay study number of producers and the effect of increased productivity on the price of product The larger the number of farms producing a commodity the largerthe number of farmers afficted by the research and perhaps the higher the probabilitythat smaller farmers are being affeicted The distribution of benefits depends to a significant degree on price changes resulting from increased productivity Producers benefit more if prices decline very little and consumers benefit more ifprices decline substatitially for a given production increase For export er import commodities priceswill tend to decline less than for commodities primarily produced and consumed domestically For commodities which are not traded and are very important in the diets of the poor prices will tend to decline relatively sharply when productivity increases The division of benefits between producers and consumers also is affected by the fact

that producers are also consumers Consequently in the Dominican Republic and Ecuador studies an additional criterion importance of the commodity in home-consumption on the farm where produced also was included

In summary 10-15 criteria were used in tie three studies to identify and select research priorities by commodity The linkages anong the goals and criteria described above are summarized in Figure 1

Research Area Criteria The five criteria uicel to select research priorities by research aieas were (1 ) whether the research causes an increase in the use of relatively abundant resources and a saving of relatively scarce resources (2) the number and seventy of research problems (3) non-duplication with transferable research from outside the country (4) tne extent of private-sector incentives to conduct the research and (5) current emphases in the research program

The rate of economic growth will be more rapid if new technologies are developed that utilize relatively abundant resources and save relatively scarce resources The resource base varies by region consequently information on resource abundance criteria was collected by region in tileEcuador and Uruguay studies The same was true for the quantity and severity of research problems Information on the relationship of domestic to inernational research private-sector incentives and the current emphasis in the research program was collected on a naticnal basis These criteria all relate to the income growth (efficiency) goal It is difficult to identif research area criteria that measurc whether particular types of research aifect income distribution or variability

Data Collection and Model Implementation

Information used in the analysis included both quantitative data on value of productionnumber of farms value of exports and imports person-years devoted to research on different commodities and tile of protein per day in thenumber of calories and gramsdiet (for tileDominican Republic and Ecuador studies) as well as qualitative or subjective information on such factors as probability of success private sector incentives and severity of research problems Furthermore weights had to be elicited from decision makers to place relative emphasis on the various criteria

Step 1 Develop Cotninodity and Research Area Lists -- The first step was to determine the list ofcommadities to consider and the list of research areas In the Dominican Republic information was collected on 74 commodities In Ecuador an initial list of 109 commodities on which INIAP was conducting research was discussed Throughdecisions to eliminate some and to group others this list was reduced to 44 commedities for the analysis These decisions were made by the Technical Director for Research In Uruguay a smaller list of commodities was reduced to 21 commodities or commodity groups These decisions were made in consultation with the National Research and Fxtension Directors The list of research program areas contained nine areas in the Dominican Republic 16 in Ecuador and 16 in Uruguay

Step 2 Collect Quantitative and Qualitative Data on Chosen Criteria -- Information o-i quantitative criteria were gathered from local and FAO secondary data sources and one table was constructed For each criterion Commodities were ranked for each criterion Information needed for the qualitative riteria as well as the weights to place on criteria were obtained through interviews with sc rtists and administrators at boh the national and regional experiment stations In Euador and Uruguay substantial attention was devoted to identifying the appropriate respondents to each question as this is a crucial aspect of the procedure In Ecuador those interviewed were notified one week before the interviews and 72 people were interviewed by the Planning Director of INIAP and

9

FIJR 1 CONCEPTUAL ELEHNTS IN THE HEIGHTED CRITERIA MODEL FOR PAWING RESEARCH PRIORITIES BY COtlDITY

GOALS fficiency ]

CONCEPTUAL( )CRITERIA Pri7] Probabi i+y Research Distriutjon Iincome

ii J of Succes Resource Eff o L=Acplusmn a rilbility

CRITERIA 1 Value of 1 Scientist 1Rl ofto toNumberPrcduction Estimates 1 AnnualInternational Producers Variability2 Foreign Centers 2 Price in Gro

Exchange 2 Current Effect3 Future Raturn3)Emphasis 3 HomeDemand 3 Private Consumpt ion

4 Comparative Sector Advantage Incentives

Notes

(-n future studies an additional conceptural criteria under the security goal might be health (J4tri ional criteria such as grams of potein or kilo=alories could be included as additional easures of Distributional ikipacts

4ot used in Dominican Republic Ecuador and Uruojuay studies

his assistant In Uruguay the directors of all the regional stations were called into CIIAB headquarters for a aweting to discuss the study and this was followed up byon-site interviews and meetings with scientists and directors at the regional experimentstations as well as with economists in the Ministry of Agriculture These interviews and meetings were conducted by a person assigned to work wvith the Director of Cl LAB on the priority-setting study In the Dominican Republic a survey of scientists extensionworkers and administrators was conducted by the ISNAR consultant from the Instituto Superior de Agricultura who visited the various experiment stations Less care wastaken in the Dominican Repubic than in Ecuador and Uruguay to match the questionspertaining -o individual criteria to particular individuals This probably reduced theusefulness of the responses because those most familiar with the probability of research success For example may not be the same -s those most Familiar with comparativeadvantage

Step 3 Elicit 11eihts on Critria -- Relative weights to place on the different criteria were obtained Fronm national and regional research system administrators In Ecuador34 peopic verc used to determine the weights These wveights were established separatelyIor the commoditv and the research area criteria In 1iuguay fewer people (sevensystem and station directors) were used to determine the weights and more attention was devoted to grouping the criteria pertaining to particular goals A Delphi procedure was used in which the s7even directors were shown the average of the group and provided with an opportunity to adjust their weight

Step 4 )erive Rankinqs hv Commodity and By Research Area -- Once the basic information was collected axd organized into a seies of tables one procedure was followed to arrive at a final ranking of research priorities by commodity A second procedure resulted in lists of priorities by research area In the Dominican Republicmost of the calculations were made by hand in Ecuador and Uruguay microcomputerspreadsheet programs were used to assist with the calculations and to conduct sensitivity analysis

The steps in the procedure for ranking research priorities by commodity and research area are sumnmaried below and are illustrated with a simple example in Appendix 1 Brieily the steps for ranking commodities were

a) Th conniodities were ranked for each criterion for which quantitative data were available

b) Tihese rankings were multiplied by the weights assigned to each criterion and then the weighted criteria were added across to arrive at one sub-ranking for the

Luantitative criteria c) Each commodity was given a high low or none designation for each qualitative

(subjective) criterion The response which implied a need for greater research priority was assigned the number 2 the intermediate response was assigned I and the response which implied lower research priority was assigned 0 1herefore each comniodity received a 2 1 or 0 for each criterion

d) The weights assigned to each qualitative criterion were then multiplied by these numbers and the results were added across all criteria to arrive at a subtotal for each commoditv These subtotals were ranked from highest to lowest to provide a second sub-ranking

e) The sub-rankings for the quantitative and qualitative criteria were then given their corresponding weights and added together to arrive at a final ranking by commodityThe steps for ranking research priorities by research area were the same as steps (c)and (d) above because the criteria were all qualitative

Step 5 Analsis and Interpretationof Resuls -- In the Dominican Republic the resuLts of this weighted criteria analysis were uscd to determine a small set of cormodities and

II

research areas with the highest priority Further assessment was then made of humanphysical administrative and other resources needed to structure research programsfocused on these commodities and resea-ch topics In Ecuador and Uruguay the prioritized list of commodities was split into a high-priority group an intermediate-priority group and a low-priority group Research area priorities were identified for each region of tile country A more detailed analysis was then made of the human physical and other resources needed to support the high and intermediate priorities

In summary five major -rens were involved in operationalizing the weighted criteria models A complete set of rsults for each study included more than 20 tables and are not reproduced here Ilowever the final rankings for commodities and for research areas are shown in Tables I and 2 in Appendix 2 In the Dominican Republic a team of consultants followed up the initial priority-setting exercise with additional analysisand discussions at the experiment station level and eventually recommended tile establishment of five national commodity programs (rice cornsorghum beans coffeeand livestock) and one additional major research program (natural resources) In Ecuador the prioritized lists were widely distributed and discussed in the Ministry oi Agriculture the Board of FEIDIA the Commission on Science and Technology arid in AID There was concern in Ecuador with the weights placed on the individual criteria and therefore additional sensitivity analysis was conducted by the Planning Office of the Ministry There seemed to be a recognition in Ecuador that priority setting with a weighted criteria model is an iterative process and that much 4t the models value stems from tile discussion of criteria among the decision makers and from the sensitivityanalysis The high-priority list of comniodities in Ecuador included thL ten highest-priority commodities listed in Table 2b A project paper has been written forAI) which recommends funding For Fli)IAlbr three commodities (coffee soft corn for the Sierra and dairy) Coffee and soft coitn were the number 2 and 3 ranked prioritiesin tile weighted criteria model and dairy was number 9 although it was the top-rankedlivestock commodity

In Urugu ay the results from the first run with tile model were discussed with the research directors Ilie directors made small changes to the weights placed on criteriaand the mode was rerun resulting in the prioritized list in Appendix 2 Personnel in CIAAB are CUrrcwtv undertaking additional sensitivity analysis and with tie assistance of ISNA R are developing a plan for implementation

An Assessment of the Weighted Criteria Models Used in the Dominican RepublicEcuador and Uruguay

The Dominican Republic Ecuador and Uruguay studies were perhaps the first majorattempts to implement weighted criteria models with weights explicitly elicited from decision makers in NARS in less-developed countries A number of strengths andweaknesses in the procedures were identified during the analyses Some of thL weaknesses were corrected fron one study to the next some can be corrected in future studies and others are inherent in the approach

Trhe first strength mentioned earlier is the ability of the procedure to systematicallyincorporate both quantitative and qualitative information related to a set of multiplegoals and criteria in order to prioritize a lorng ist of commodities and research areas in a relatively short period of time Second the procedure proved relatively easy for both research administrators and the loc- -nalysts to understand with the exception of the )ominican Republic where the research administrators were not directly involved7

Third the analysis as applied in Ecuador and ULuguay with the direct involvement of research system administrators at various stages forced those decision makers to

12

consciously identifv and trade off goals and criteria Fourth the use of spreadsheet programs in EcUadfor and Uruguay facilitated sensitivity analysis after the initial set of priorities was deermimd Fifth the procedure provided a relatively objectiveassessment of prioriies because individuals were not allowed to rank commodities orresearch areas directly but had to weight criteria

The first weakness inherent in ex ante research priority setting and thus in theprocedure is that there is a large amount of subjectiveness Although tileapproach isless subjective than unstructured judgment there is subjectiveness in the responses toquestiorns related to some of the criteria and also ini the weights placed by decision makers on the criteria Second it proved diflicult to specify independent criteria with no overlap Third while the procedure required less time than an expected economic surplus analysis by commodity it required more time than asking respondents to directlyrank commodities and research areas In E-cuador the consiultant worked withpersonnel in INIAP and FEI)IA for one week at the start of the analysis to determinegoals and criteria to arrive at atlist of commodities and researh areas to list secondarydata needs to formulate questions to be asked of administrators and scientists to decide who should be asked particular questions and to explain the procedure itself includingthe spreadsheet program The required iidormation was collected over the Followingmonth and the consultant returned and worked with the Planning Director in INIAP for one week to conduct the analysis and write tip the results Revisions to the initial draftof the report we-e then made over the lollowing weeks A similar procedure to the onefollowed in Ecuador was used in l_ruguay althaugli more time was spent discussing the wording of questions to be asked of scientists Furthermore n Uruguay more care was taken to match the criteria with the goas

Certain of the criteria wil always be diflicult to eplain to tileinterviewees Thecriterion on comparative advantage is difficult for some respondents because of a lack of knowledge about neighboring countries If suflicient data time and resources are available this criterion may be rnea4ured more precisely using comparative advantage or domestic resource cost procedures suggested by longmire and Winkelnann Anotherdiflicult criterion to estiniate is the potential foture demand for the product The respondents in Fcuador were asked to consider how the demands for different foodswould change as income and populatiot grow Many of them had a hard timeconceiving that the total or per capita demand for certain Foods could actually decline as income grew In Uiruguay (ibe cmiparative advantage and future dema nd questions were answered by a small group of economists from the ministries of agriculture and trade In the Ecuador study more attention was devoted to assessing the most appropriate people for answering different questions than was the case in the Dominican Republic Ilowever some criticism surfaced about the weights placed on criteria InUruguay decision makers were given the opportunity to change their weights afterviewing the initial results This was very useftl because it demonstrated the implicationsof placing different weights on the various goals and criteria

Computer spreadsheet programs were a major help in Ecuador and Uruguay Additional work needs to be devoted to developing a more mentnu-driven prcgram but the people inthe planning office of INIAP were able to use the SuperCalcR program without too tmucht difliculty despite ialack of previous experience Ihe analyst in Uruguay was veryfamiliar with lotusl which proved most helpfil

It became clear from working inall three countries that it is preferable to work directlywith the final decision makers or their designees Inthe l)ominican Republic the study was conducted by a very competent set of consultants from the local agricultural universi ty but the procedure was not institutionalized to allow for additional sensitivityanalysis or future priority-setting efforts InEcuador and especially in Uruguay the

13

decision makers were directly involved conducted sensitivity analyses and were able to assess why certain commodities or research areas received high priority as a result of the analyses In both Ecuador and Uruguay they have a basic understanding of the strengths and weaknesses of tileprocedure and therelre can explain their subsequentdecisions to groups who might disagree with tilepriorities they establish

The corntrast between the Dominican Republic and tie Uruguay studies illustrates a veryimportant point No matter which technique is used to structure the analysispriority-setting exercises must integrate the decision makers (users) into the processPriority-setting procedures must be institutionalited the results of initial analysisdiscussed thoroughly with decision makers and sensitivity anilsis conlducteU alone the lines suggested by that discussion Tecinological change generates a surplus which can be appropriated by dilkircnt sectors or groups within the economy Who benelitdepends in part on the cotaniodities selected for research and the types of technoloLies generated One of the purposes of institutionalizing a formal prioritN- setting procedureis to help mediate conflicts that may arise (for example between producer groups and consumers) as a result of research-induced improved technologies

Other issues arose during the analyses First the initial grouping and elimination of certain commodities and research areas before lhe irialvis is an important step but one which must be taken with carelul consideration of possible coniplmentarities in research on particular productS or types of rearch Second some of the commodities such as forages are inputs into other commodities such as milk (Corseqtentlv these commodities llay have to he considered itscinps because it is the value-added and not tilegross value of output that is relevant when one coniiodity requires other commodities sinputs and when criteria such as value of production are used This was an issue in I ruguav where livestock are quite important liird little attention was devoted in atv of the studies to defining appropriate criteria Ior the increased inoome stability eoal Future studies if this be an ideltJiiLd LOal shOUld define risk criteria

In sutImary the weighted criteria model as applied in Lcuador and iuguav which had the advantage of bit ildin Oillessons lCarICd illdie l)ominican RCpublic appears to have been a uselul priority-setting tool It ha its ereatest advartage ill situations ill which there are rulItiple goals to consider a relatively major reassesnieiit of prioritiesis contenplated and the list of commodities dd resen rch areas to consider is largeWeighted criteria models described in this paper were applied in small countries with limited resources but with direct involvement (except in the Dominican Republic) ofthC major decision makers These models itav be most useful in the small country situationwhere decision mAkers are more likely to be involved in tile priority-settirt process and are not just using tables of results Perhaps in larger systet7 where decisioii makine is more decentralited tileweighted criteria model can he applied at the regional level before relining research priorities at the national level

Another factor that can be uniortant in large svvtern is tle complerientarity between resea-h aid educat ion [hie larger the svstcnt the clser the linkages tend to be between universities and tileresearchi system Where universities are important tor agriclltural research an aidditional criterion may hc rteedCd to place value on the educationlal benefits of certain types of reseirch

Ihere are several additional iniplications with respect to how weighted criteria models relate to alterniative priori t-setting Inethod parIicularl the expected economic surplusapproach Befoce exploring these relationships kcx aspects of the surplus approacliare described based on aii application in Peru

14

ISNAR WORKING PAPERS

The ISNAR working papers series is a flexible instrument for sharing

analysis and information about relevant organization and management

problems of the agricultural research systems in developing countries

In the course of its activities - direct assistance to national

agricultural research systems training and research ISAAR genarates a-

broad range of information and materials which eventually become the

formal products of its publications program The working papers series

enhances this program in several important ways

1 These papers are intended to be a rapid means of presenting the

results of work and experiences that are still in progress but are

already producing cesults that could be of use to others

2 They are intended to be an effective vehicle for widening the

discussion of continuing work thereby increasing the quality of the

final products critical conment is welcomed

3 The series provides an outlet for diffusing materials and information

that because of their limited coverage do not meet the requirements

of general audience publication

The series is intended mainly for the diffusion of materials produced by

ISNAR itaff but it is also available for the publication of documents

produced by other institutions should they wish to take advantage of the

opportunity

George W Norton is associate professor Virginia Polytechnic Institute and StateUniversity Blacksburg Virginia and research 1llow International Service for National Agricultural Research (ISNAR) The Ilague Netherlands Philip G Pardey is joiltly a research associate University of Minnesota St Paul Minnesota and reseach officer ISNAR The authors would like to thank Iloward Elliott Alexander Von der Osten Paul Berinell and Carlos Valverde for their helpful comments and suggestions on an earlier draft of this paper without implicating thern in any remaining errors and omissions

TABLE OF CONTENTS

I Context of Research Priority-Setting I 1I Previous Priority-Setting Models 2

Weighted Criteria Models 3Benefit-cost (Expected Economic Surplus) Analysis 3Mathematical Programming 3Simulation 3Comparative Assessment 4

III The Use of Weighted Criteria Models in the Dominican RepublicEcuaodm and Uruguay 6

Goals 7Criteria 7

Coinmoditv Criteria 7Research Area Criteria 9

Data Collection and Model Implementation 9An Assessment of the Weighted Criteria Models used in theDominican Republic Ecuador and Uruguay 12

IV The Use of Expected Economic Surplus Analysis in Peru 15

An Assessment of the Expected Economic Surplus Procedure in Peru 15 V Implications for Development of Improved Research Priority-Setting Procedures 16 VI Conclusions 18 Footnotes 19

References 20

Appendix 1 23

Appendix 2 26

Priority-Setting Mechanisms for National

Agricultural Research Systems

Present Experience and Future Needs

Recent trends towards shrinking real research budgets or many national agriculturalresearch systems (NRS) have stimulated several attempts to evaluate tileeconomic benefits ofaricultural reserch Manv of these quantitatis e economic evaluations haveemployed historical data to provide ex post assessments and a few hase used subjectivejudgments to project future research benefits [he results kthese analyses have been used at the national or inter-ministerial level to justilf the need for continued political support for agricultural research

Tighter rese rch budgets not only heighten the iced to justifV agricultural research vis a vis alternative public investments but they alVo make it important for NARS to improve procedures for setting priorities among competing research programs A varietyof strategic and project-level decisions are made within NARS The objectives of national policvmakers the needs of agricultural producers and scientific opinion must each be brotight to bear on strategic decisions to emphasize particular commodities and research program areas uld on research project selection

The purpoe of this paiper is to discuss kev elements for inclusion in improved research priority-setting procedures for NRS Ihe fLcus is on methods for strategic as opposedto more tactical projcct-le el decision m kingAfter discussing the context of research priority setting a brief summary is provided of tile iain types of research resource alloL-ition mnetlods Recciit cxperiences with priority setting in four Latin American NAR the l)ominican Republic lcuador lUVUav and Peru are compared These experiences can proside sonc guidance for the development of an improved research resource allocation procedure

I TilE CONTEXT OF RESEARCH PRIORITY SETTING

NARS decision mnkers often operating within the ministry of agriculture an agricultural research institute or as a research council make allocative choices based on their knowledge atnd prior cxperience That knowledge and experience frequentlyincludes an understar one of national and regional oals and objectives as voiced through the political process a sene o lie severity of particular types of research problems and a general feel tor vlat is achievable through research It also includes an awareness of the desires ( te-ressures) of producer groups Decision makers must decide hoW much cmpha sisto jte onparticular commodities (eg rice maize beanssheep) and types of research c v varietal improvenmnt improved livestock nutritionplant protection farming svwtes Implicit in those decisions are the locational emphasis of research the focus 0rt paJticullar factors of production (eg land laborwater) the emphasis placed Ol lone-r-term haisic versus shorter-term applied and adaptive research and the distrihotianal elfects of research on farm size on producers versus coniumners and on people atidiherent income lec

In many NA RS research resoce alocation decisions are heavily influenced by the previous years budget2 (hanges of~tn result from requests by scientists which areevaluated reLatively informally and aggregatcd into an overall plan The resulting plan

upon close scrutiny may contribute little to tire attainment of stated national goals and objectives

Judgment based upon prior knowledge and information provided by scientists is crucial for research resource alincation decisions In addition rapid or radical changes in research programs -an be costly Particularly as research budgets tighten howeverforcing changes in tile systein tile increased use of quantitative methods may be nocessary to imprwe thL objectivity of those judgments The aim is to foster consitencv of research priorities with goAs and objectives and to improve the efliciency of the research system in mser ing producer and cortsumer needs The idea is not to replacejudgment but to increase and organize the inflormation available for updating priorknowledge and beliefs The hope also is to inject continuity in the priority-settingprocess and to proide NARS decision makers x ith methods the find lielplib when rationalizing their decisions to scientists producer Erotps and politicians Docunentingthe decision proces through a strctured procedure can help minimie the possinilitvof sudden and large shilts in researh priorities in those systems characterized by a flairlv rapid turnover of research administrators

Many quantitative priority-setting procedures are avaiia ble but few have beeninstitutionalized into the decision-making practices of NA RS manaers A cruciZl factor in their non- use of these procedures urtdoubtcdlv as been the lack of ri orous vetcost-eflectie approaches which can irtcorporate tile large number of comtodities andresearch areas as well as the multiple go1ls a ld criteria found in most strateoicdecision-muakirt situations inl NRS FurtIhcrrtore the cortmputer capabiliv available through microcomputers in most developing-co ntry NRS today was not gcenerallyavailable even two or three velars ago

In this paper procedure amc ce crihed which were recently applied in four latinAmerican countries First however previously emploved priority-setting mechanisms are briefly reviewed id critiqued One of rite points that will Ie made at the end of this paper is tiLit tise methods are not nitually excilsive There is no siTele magic huletbut there appears to be real potential for developing a Ilexible computer-assisted priority-setting framework that will allow NARS administrators to call ott theappropriate tectiiques tot aSsist withit a particular decision depending ott tie time and financial resources a ailahle and the relative itiportance of the allocation decision Asecond major point is that priority setting is a process not simply a procedure and must be instit utiortliied as part of the overall process of research managementPriority-setting technritiuetes applied only once by people outside of the decision-making process have little valtie

If SUMMARY OF PRIORITY-SETTING METHODS

Methods reported itt previous studies for research priority setting include

I ) establishtert td weighting of multiple criteria for ranking commodities and research ares with a final aggregate ranking based on implicit or explicit weights

2) use of bertefit-cost analysis including expected economic surplus techniques to select cotitritodities and research areas

3) application of mathematical programming to choose an optimal research portfolioincorporatirng multiple goals md constraits

4) developrert and tise of simulation models

These four approaches are discussed below followed by a brief assessment of their advantaiges atil dis idsittages

2

Weighted Criteria Models

Several studies hae established multiple criteria for ranking priorities because of tiledesire to explicitly consider a wide variety of factors that do or perhaps should influenceresearch selection The relative weights attached to each criterion to arrive at a final listof research prioriries are sometimes left implicit or unstated A recent example is tileTAC review of priorities and future strategies for the Consultative Group onInternational Agricultural Research (CGIAR) In assessing priorities by commoditythey etablished a principal goal research objective to obtain the goal and a series ofcriteria organized into three groups relevance research productivity and efliciency ofhe CGIAR system in undertaking research Detailed tables with quantitative andqualitative information and rankings of commodities for each criterion were presentedIn developing the recommendations for hort- to medium-term funding alternatives atthe conmmodity level the weights used to aggregate across criteria were left implicit

There are examples of tudies that have incorporated multiple criteria but also explicitlyspecified and utilized a set of weights to aggregate across criteria and obtain a finalranking of research priogrities Appications of this method often called the scoringmodel approach are found in studies by Williamson lahlstede Paulsen and KaldorShumway and McCrack en and Von (Oppen and Ryan A flew studies have used congruence analysis which can be thought of as a special case of a sci-ring or weightedcriteria model with all the weight placed on the criterion of vaue of production

Benefit-cost (Expected Economic Surplus) Analysis

The benefit-cost approach to welecting research priorities has been used in differentforms (Fishel Arai Sire and Gardner l)avis Oram and Ryan Norton Ganoza andPomareda) Most studies have employed consumer-producer surplus analysis and havebuilt upon a large body of literature on ex post research evaluation Ex ante analysesusually incorporate expert opinion to determine projected research impacts adoptionrates and probabilities of research success and provide estimates of the economiceffliciencv and distributional implications of agricultural research resource allocationBenefit-cost studies typically calculate benefit-cost ratios internal rates of return andnet present values for alternative types of research or research onfor differentcommodities Ilse analyses may include andmav or not regional internationalresearch spillovers and the eflects of domestic pricing policies on research benefits

Mathematical Programming

Mathematical programming is another alternative for research selection It relies onmathematical Optimization to choose a research portfolio through maximizing amultiple-goal objective function given the resource constraints of the rescaich systemA good example of the use of this method is a study by Russell in the United Kingdomlie used goal programming to maximize the contributions of the research program toseveral goals given the con taints of budget human resources state of knowledge andcertain policies This procs Ue uses similar information to the weighted-criteria modelbut selects an optimal research portfolio rather than simply ranking research areas

Simulation

Finally the simulation method has been used to identify and select research prioritiesSimulation models vary in their construction but a good example is the model byPinstrup-Anderscn and Franklin They built a model to project the contributions andcosts of alternative research activities They established gals and then identifiedchanges in supply demand for inputs and demand for output needed to meet those

3

goals They identified neeled technologies time and financial costs and the probability of research success and adoption Finally they specified the scientists workingobjectives This model is very thorough but did require extensive amounts of data and estimation of several mathematical relationships

Comparative Assessment

The above discussion is a brie summary of the major structu al approaches that have been used to asist decision makers in research priority setting All of them attempt to go beyond the usual procedure of individual or group decision makin e without structural analyses Each of the approacles has its advantages and disadvantages (See Fable I)

The procedure of establishing criteria and using weights to arrive at a final set of research priorities has the advantage of forcing decision makers to consciously trade off multiple goals It can illcorporae both quantitative and qualhtative information and can be applied to a long list of con mnodities or research areas in a relatively liort period of time The procedure is relati cx ca sY fbr administrators to understand but does requiretheir time in obtaining the explicit w5eights for criteria Furthermore these weights are inevitably subjective and their elicitatlion must be carefully structured [lie nActhod also requires scientists time in collecting inlormation on qualitative criteria As a result the approach k better suited for periodic or major priority-setting efforts than or situations where frequent marginal changes are anticipated

The expected economic surplus approach his the major advantage of incorporatingseverml criteria related to economic elficiency and distribution into one or two measures It also can be ucd to examine the gencral equilibrium ellects ofresearch (Ramailho de Castro and Sclhh nAid tile benefits of research under alternative possibly distortionar domestic pticing and international trade policies tEdwarts and Freebairn Alston Edwards and rcecbairn Norton and (ianoza 1985) These alictors are pervasive in developing countries and affect the efficiency and distributional consequences of researh Ile procedure ie(liCe ir lel - tndcrstandw ofeconomic analysisand more anal-t s time than the weighted criteria model but less aduinistrators time It can be diflicult to apply to at large number of commodities or research areas because certain typcs of data necessary for the ainalysis often do not exist for all commodities With adequaIL tL d ita Cxpected economic surplus analysis can be incorporated into the weighted criteria model or could be used on the set of commodities which the weightedcriteria model imdicaMes to have the highest piority lhe latter approach would allow for the calculation of income foregone as a reult of placing weights on non-economic ef1ciency criteria Expected economic surplus analysis has the major advantage of calculating a rate of return which can be compared to alternative public investments

The mathematical programming approach is similar to the weighted criteria model approach becaue weights are placed on a set of goals or criteria The procedure has the advantage of explicitly considering the budget huLman resource aad other constraints on the research system tnless the constraints are well specified including possiblechanges over time liowever there is a risk of nonsense solutions The model is more intensive of an ecolonic analysts time and abilit than tie simpler weighted criteria approach and decision makers may be less willing to accept what appears to be a black box solution lrade-offs among goals are easily quantified ssith this approach

The advantage of simulation models is their tlexibility They can be constructed is relatively simple or complex tools can incorporate optimizing or ranking proceduresand can readily include probabilistic information Their major disadvantage is that to be useful they must be relatively complex and typically reqtuire extensive amounts of both data and time of skilled analysts

4

TABLE 1 COMPARISON AMONG MAJOR AGRICULTURAL RESEARCH PRIORITY-SETTING METHODS

PRIORITY-SETTING METHOD

HEIGHTED EXPECTED MATH CHARACTERISTIC CRITERIA ECON SURPLUS PROGRAM SIMULATION

OPERATIONAL CONSIDERATIONS

1 Relative cost in researchers time mudium mJium medium high2 Relative cost in priority setting analysts ime medium medium high high3 Relative cost in administrators time medium mediummedium medium 4 Relative overall data requirement medium medium medium variable 5 Relative ease of comprehonsion by decision maker high medium low low 6 Ease of incorporating subjective informaticn high high high high7 Ease of incorporating non-quantitativa inforiation high low low medium

GOAL-RELATED ISSUES

a Requires explicit elicitation of goals yes usually yes uiually9 Can determine distributional affects on consumers

and producers at various income levels no yes no yes10 Can handle uncertainty yes yes yes yes11 Can consider tradeoff among multiple goals yes sometimes yes yes

CRITERIA-RELATED ISSUES

12 Can consider private-sector research incentives yes difficult difficult yes13 Can consider economic policy and trade affects yes yes yes yes

EVALUATION-RELATED ISSUES

14 Can be used to sot priorities for research at the aggregate level no yes no es

15 Can be used to set research priorities at the commodity level yes yes yes yes

16 Can be used to set priorities for non-production or non-commodity oriented research yes difficuli yes yes

17 Can be used to set priorities for basic research yes difficult no sometimes 18 Can evaluate secondary impacts of rosuarch on

employment environment nutrition yes sometimes so etimes yes19 Usually estimates a rata of return to research yes sometimesno no 20 Can quantify geographic spillover effects no yes no yes 21 Can consider the lags involved in research

and adoption yes yesyes yes22 Facilitates priority setting when the number

of commodities is large yes difficultdifficult difficult

Source Adapted and modified from Norton and Davis

5

The above is a very brief summary of the major advantages and disadvantages of the most common formally structured priority-setting models that could be used for strategicplanning for research resource allocation Other approaches exist for research projectselection with perhaps models that attempt to identify and analyze key vield constraints being the most prevalent approach employed in NARS of less-deeloped countries

The results of ex post analysis also can provide useful guidance for ex ante research resource allocation decisions if appropriately incorporated into a systematic ex ante procedure The most common ex post approach in addition to cx post benefit-cost analysis is the econometric estimation of production or supply functions incorporating research variables To be most uteful for ex ante analysis econometric approaches mLlSt be applied with a high degree of disaggrega tion Unfortunateiy these models require a substantial amount of high-quality historical data on production farm inputs and research expenditures

Perhaps the two approaches with the most potential given additional refinements for application by NARS in less-developed countries are the weighted criteria models and the benefit-cost or expected economic surplus procedures In the next section reent experiences with applying weighted criteria models in three Latin American Countries )ominican Republic Ecuador and Uruguay are discussed and evaluated The purpose

it to piovide more iniormation about the procedures their degree of acceptance and possible improvements in the models Ihen application of expected economic surplus models are considered drawing cn a recent cx ante research evaluation studly in Peru Finally the relationship between weighted criteria and expected economic surpluts methods is described along with possible extensions

IIl THE USE OF WEIGHiTED CRITERIA MODELS IN TIlE DOMINICAN REPUBLIC ECUADOR AND URUGUAY

in 1986 iif I87 weighted criteria models were developed and implemented in the lDomin lRepublic (ISA) Ecuador (Espinosa et al) and Trugua (CIAAB)to assist with priority setting for agricultwral research In the I)ominican Republic the study was carried out at the Instituto Superior de Agnicultura ISA) by consultants for ISNAR In Ecuador the study was conducted by the plmnintg ollice oY the Instituto Nacional de Investigacion Agropecuaria (INIAP) with the assistance of consultants for the US Agency for International Development (AID) In Uruguay the study was conducted by the Ollice of the Director of the Centro de Investigacion Agricola Alberto Boergcr(CIAAB) with the assistance of ISNAR The purpose of all three studies was to applyprocedures for prioritizing agricultural research by commodity and by major research area Inthe Dominican Republic the client was the newly created natonal agriculturalresearch agency (Il)EA) In Ecuador the client was INIAP and the newly created foundation for agricultural research (IEl)IA) In Urugua the client was C1IAB

The procedures employed in the three studies were similar but with some importantdilfereices as the Ecuador study built on lessons learned inthe I)ominican Republic and the Uruguay study built on other lessons learmed in Ecuador National goals for the research system iere elicited in each country and a series of criteria establishd which relate to those goals Separate riteria were developed for conmitodities and or research areas and weights were elicited from decision makers to establish the relative importanceof the criteria Commodities and research areas were ranked according to each criterion and these rankings were multiplied by the elicited weights to arrive at research priorities

6

Goals

Three goals were identified in each of the three countries (1) to raise the average levelof income in the country (2) to increase the well-b(Aing of low-income groups in societyand (3) to reduce yearto-year income fluctuations in the country especially onl tiledownside These goals were worded slightly diffrcntly in each country but in each casethey essentially relate to the three goals referred to in previous studies as efficncy equity and security In the l)ominican Republic the local consultants identifiedthese as the goals while in lcuador and Uruguay it was done by the directors of national research agencies

Criteria

The next step was to establish criteria to as measures of whetheruse particularcommodities or research areas contribute to the attainment of the above goals A largeset of criteria was discussed in each country and a total of 15 criteria were eventuallyused in the Dominican Republic 14 in lEcuador and 10 in Uruguay to determine priorities by commodity Five criteria were used in each case to determine priorities byresearch area Refinements were made in successive studies - beginning with theDominican Republic then Ecuador followed by Iruguay - to increase the independenceof criteria and to remove criteria that were questionable measures of whether researchcontributed to tile tated goals ( Reason for dropping particular ci eria are discussedbelow) Refinements also were made with respect to the grouping of criteria to facilitatethe weighting of goals and criteria

Commodity Criteria omioditv criteria were grou l - J into four conceptual groupsproduct importance probability O success elliciencv in use of research resources anddistribution of impacts The first three of these groups relate to the efficiency or incomelevel goal and the last group relates to the equity or distributional goal In none of thethree studies were criteria included which represnted the third goal (reduced incomelluctuation [lie feeling was that the entire set of criteria as a group would serve toreduce the enlhasis placed on single or 1ew (often export) crops While this appearedto be the case in the results of thee particular studies ft ture weighted criteria studies may want to add criteria which explicitly rank commodities from lowest to hiehest withrespect to annual gross income variability (or price and yield variability separately)

The group of criteria referred to as jrodct importap contained Four criteria in theUruguay study value of production generation or saving of foreign exchange expectedfuture demand change and comparative advantage

The value of productionl is an important criterion because the cost of rseaich isbasically independent of the number of units alkected by the research results at least forthose commodities with relatively homogeneous production conditions Foreignexchange was used as a criterion because of a perception in each country that the lackof foreign exchange was impeding growtb A criterion was set up to allow more weightto be paced on those commodities for which demand is expected to increase in thefuture because the importance of commodities will change as diets are altered withincome growth aod a world markets change Comparative advantage was includedbecause income in the country will be higher if the country focuses its efforts oncommodities for which its resource base is best suited while recognizing that research itself can alter resource constraints

In the Dominican Republic and in Ecuador nutritional measurcs (calorie and proteincontributions to the diet) were included as criteria for product importance They wereincluded because the well-being of low-income groups may be affected by the availability

7

of foods which are important in their diets Nutritional criteria were dropped in Uruguay however because numerous studies have shown that income levels are tile primary determinants of nutrition levels rather than the availability of particular foods Certainly caloric and protein content are debatable criteria To the extent that increased domestic production lowers the prices of certain food commodities therebyincreasing food consumption by the poor nutritional criteria can contribute to the distributional goal Land area also was included as a criterion in the Dominican Republic study but was dropped in Ecuador and Uruguly because of the overlap with value of production

Two criteria were used in each study to measure 1robahility of reseatch success The first measure was tilegap between own-country yields and neighboing country yields The second was the potential for success indicated by the researchers themselves The rationale for the gap criterion was that the larger the yield gap between the home country and similar countries tle greater the potential for yield gains The criterion is rather crude because yield gaps also can indicate a relative resource disadvantace forproducing the commodity even if the countries appear to be geoclimatically sinilar Consequently the gap criterion was eventually dropped in the tUruguav study The potential for success as indicated by the researchers is subjective but a very necessary measure to obtain

The gro up of criteria reficrred to as e1)cienc in use otresearch resources contained three criteria the relationship to research in the international centers the degree of emphasis on tilecommodity in le current research p ogram and the incentive for the privatesector to conduct the research Some research duplicates other research croplementsand still other research bears no relationship to research at tileinternational agriculturalresearch centers For certain comniodities all three relationships may hold depending on the type of research If national research is complementary to international researchthe cost of the rational research is lower Consequently a criterion was established which places greater weight on commodities with a high degree of complementaritybetween ntin Il d international research The criterion ofdelegree of emphasis in tile current iesercli program was established because there is a cost to rapidly adjustingcurrent h uiman and physical research resources from one commodity to another The criterion of private-sector incentives was used because the social benefits from the total research program in the country will be greatest if scarce public research funds are devoted to research on commodities for which the private sector does not have an incentive to support the research itself Of coui se the private-sector producer group can tax itself and pay the public sector to conduct the research for those commodities for which it has an incentive to privately support the research These commodities tend to be export commodities because producers can capture a higher proportion of the gainsfrom productivity increases for export commodities than for commodities primarily consumed domestically

The group of criteria related to the distribution ojresarchimpacts contained two criteria in the Uruguay study number of producers and the effect of increased productivity on the price of product The larger the number of farms producing a commodity the largerthe number of farmers afficted by the research and perhaps the higher the probabilitythat smaller farmers are being affeicted The distribution of benefits depends to a significant degree on price changes resulting from increased productivity Producers benefit more if prices decline very little and consumers benefit more ifprices decline substatitially for a given production increase For export er import commodities priceswill tend to decline less than for commodities primarily produced and consumed domestically For commodities which are not traded and are very important in the diets of the poor prices will tend to decline relatively sharply when productivity increases The division of benefits between producers and consumers also is affected by the fact

that producers are also consumers Consequently in the Dominican Republic and Ecuador studies an additional criterion importance of the commodity in home-consumption on the farm where produced also was included

In summary 10-15 criteria were used in tie three studies to identify and select research priorities by commodity The linkages anong the goals and criteria described above are summarized in Figure 1

Research Area Criteria The five criteria uicel to select research priorities by research aieas were (1 ) whether the research causes an increase in the use of relatively abundant resources and a saving of relatively scarce resources (2) the number and seventy of research problems (3) non-duplication with transferable research from outside the country (4) tne extent of private-sector incentives to conduct the research and (5) current emphases in the research program

The rate of economic growth will be more rapid if new technologies are developed that utilize relatively abundant resources and save relatively scarce resources The resource base varies by region consequently information on resource abundance criteria was collected by region in tileEcuador and Uruguay studies The same was true for the quantity and severity of research problems Information on the relationship of domestic to inernational research private-sector incentives and the current emphasis in the research program was collected on a naticnal basis These criteria all relate to the income growth (efficiency) goal It is difficult to identif research area criteria that measurc whether particular types of research aifect income distribution or variability

Data Collection and Model Implementation

Information used in the analysis included both quantitative data on value of productionnumber of farms value of exports and imports person-years devoted to research on different commodities and tile of protein per day in thenumber of calories and gramsdiet (for tileDominican Republic and Ecuador studies) as well as qualitative or subjective information on such factors as probability of success private sector incentives and severity of research problems Furthermore weights had to be elicited from decision makers to place relative emphasis on the various criteria

Step 1 Develop Cotninodity and Research Area Lists -- The first step was to determine the list ofcommadities to consider and the list of research areas In the Dominican Republic information was collected on 74 commodities In Ecuador an initial list of 109 commodities on which INIAP was conducting research was discussed Throughdecisions to eliminate some and to group others this list was reduced to 44 commedities for the analysis These decisions were made by the Technical Director for Research In Uruguay a smaller list of commodities was reduced to 21 commodities or commodity groups These decisions were made in consultation with the National Research and Fxtension Directors The list of research program areas contained nine areas in the Dominican Republic 16 in Ecuador and 16 in Uruguay

Step 2 Collect Quantitative and Qualitative Data on Chosen Criteria -- Information o-i quantitative criteria were gathered from local and FAO secondary data sources and one table was constructed For each criterion Commodities were ranked for each criterion Information needed for the qualitative riteria as well as the weights to place on criteria were obtained through interviews with sc rtists and administrators at boh the national and regional experiment stations In Euador and Uruguay substantial attention was devoted to identifying the appropriate respondents to each question as this is a crucial aspect of the procedure In Ecuador those interviewed were notified one week before the interviews and 72 people were interviewed by the Planning Director of INIAP and

9

FIJR 1 CONCEPTUAL ELEHNTS IN THE HEIGHTED CRITERIA MODEL FOR PAWING RESEARCH PRIORITIES BY COtlDITY

GOALS fficiency ]

CONCEPTUAL( )CRITERIA Pri7] Probabi i+y Research Distriutjon Iincome

ii J of Succes Resource Eff o L=Acplusmn a rilbility

CRITERIA 1 Value of 1 Scientist 1Rl ofto toNumberPrcduction Estimates 1 AnnualInternational Producers Variability2 Foreign Centers 2 Price in Gro

Exchange 2 Current Effect3 Future Raturn3)Emphasis 3 HomeDemand 3 Private Consumpt ion

4 Comparative Sector Advantage Incentives

Notes

(-n future studies an additional conceptural criteria under the security goal might be health (J4tri ional criteria such as grams of potein or kilo=alories could be included as additional easures of Distributional ikipacts

4ot used in Dominican Republic Ecuador and Uruojuay studies

his assistant In Uruguay the directors of all the regional stations were called into CIIAB headquarters for a aweting to discuss the study and this was followed up byon-site interviews and meetings with scientists and directors at the regional experimentstations as well as with economists in the Ministry of Agriculture These interviews and meetings were conducted by a person assigned to work wvith the Director of Cl LAB on the priority-setting study In the Dominican Republic a survey of scientists extensionworkers and administrators was conducted by the ISNAR consultant from the Instituto Superior de Agricultura who visited the various experiment stations Less care wastaken in the Dominican Repubic than in Ecuador and Uruguay to match the questionspertaining -o individual criteria to particular individuals This probably reduced theusefulness of the responses because those most familiar with the probability of research success For example may not be the same -s those most Familiar with comparativeadvantage

Step 3 Elicit 11eihts on Critria -- Relative weights to place on the different criteria were obtained Fronm national and regional research system administrators In Ecuador34 peopic verc used to determine the weights These wveights were established separatelyIor the commoditv and the research area criteria In 1iuguay fewer people (sevensystem and station directors) were used to determine the weights and more attention was devoted to grouping the criteria pertaining to particular goals A Delphi procedure was used in which the s7even directors were shown the average of the group and provided with an opportunity to adjust their weight

Step 4 )erive Rankinqs hv Commodity and By Research Area -- Once the basic information was collected axd organized into a seies of tables one procedure was followed to arrive at a final ranking of research priorities by commodity A second procedure resulted in lists of priorities by research area In the Dominican Republicmost of the calculations were made by hand in Ecuador and Uruguay microcomputerspreadsheet programs were used to assist with the calculations and to conduct sensitivity analysis

The steps in the procedure for ranking research priorities by commodity and research area are sumnmaried below and are illustrated with a simple example in Appendix 1 Brieily the steps for ranking commodities were

a) Th conniodities were ranked for each criterion for which quantitative data were available

b) Tihese rankings were multiplied by the weights assigned to each criterion and then the weighted criteria were added across to arrive at one sub-ranking for the

Luantitative criteria c) Each commodity was given a high low or none designation for each qualitative

(subjective) criterion The response which implied a need for greater research priority was assigned the number 2 the intermediate response was assigned I and the response which implied lower research priority was assigned 0 1herefore each comniodity received a 2 1 or 0 for each criterion

d) The weights assigned to each qualitative criterion were then multiplied by these numbers and the results were added across all criteria to arrive at a subtotal for each commoditv These subtotals were ranked from highest to lowest to provide a second sub-ranking

e) The sub-rankings for the quantitative and qualitative criteria were then given their corresponding weights and added together to arrive at a final ranking by commodityThe steps for ranking research priorities by research area were the same as steps (c)and (d) above because the criteria were all qualitative

Step 5 Analsis and Interpretationof Resuls -- In the Dominican Republic the resuLts of this weighted criteria analysis were uscd to determine a small set of cormodities and

II

research areas with the highest priority Further assessment was then made of humanphysical administrative and other resources needed to structure research programsfocused on these commodities and resea-ch topics In Ecuador and Uruguay the prioritized list of commodities was split into a high-priority group an intermediate-priority group and a low-priority group Research area priorities were identified for each region of tile country A more detailed analysis was then made of the human physical and other resources needed to support the high and intermediate priorities

In summary five major -rens were involved in operationalizing the weighted criteria models A complete set of rsults for each study included more than 20 tables and are not reproduced here Ilowever the final rankings for commodities and for research areas are shown in Tables I and 2 in Appendix 2 In the Dominican Republic a team of consultants followed up the initial priority-setting exercise with additional analysisand discussions at the experiment station level and eventually recommended tile establishment of five national commodity programs (rice cornsorghum beans coffeeand livestock) and one additional major research program (natural resources) In Ecuador the prioritized lists were widely distributed and discussed in the Ministry oi Agriculture the Board of FEIDIA the Commission on Science and Technology arid in AID There was concern in Ecuador with the weights placed on the individual criteria and therefore additional sensitivity analysis was conducted by the Planning Office of the Ministry There seemed to be a recognition in Ecuador that priority setting with a weighted criteria model is an iterative process and that much 4t the models value stems from tile discussion of criteria among the decision makers and from the sensitivityanalysis The high-priority list of comniodities in Ecuador included thL ten highest-priority commodities listed in Table 2b A project paper has been written forAI) which recommends funding For Fli)IAlbr three commodities (coffee soft corn for the Sierra and dairy) Coffee and soft coitn were the number 2 and 3 ranked prioritiesin tile weighted criteria model and dairy was number 9 although it was the top-rankedlivestock commodity

In Urugu ay the results from the first run with tile model were discussed with the research directors Ilie directors made small changes to the weights placed on criteriaand the mode was rerun resulting in the prioritized list in Appendix 2 Personnel in CIAAB are CUrrcwtv undertaking additional sensitivity analysis and with tie assistance of ISNA R are developing a plan for implementation

An Assessment of the Weighted Criteria Models Used in the Dominican RepublicEcuador and Uruguay

The Dominican Republic Ecuador and Uruguay studies were perhaps the first majorattempts to implement weighted criteria models with weights explicitly elicited from decision makers in NARS in less-developed countries A number of strengths andweaknesses in the procedures were identified during the analyses Some of thL weaknesses were corrected fron one study to the next some can be corrected in future studies and others are inherent in the approach

Trhe first strength mentioned earlier is the ability of the procedure to systematicallyincorporate both quantitative and qualitative information related to a set of multiplegoals and criteria in order to prioritize a lorng ist of commodities and research areas in a relatively short period of time Second the procedure proved relatively easy for both research administrators and the loc- -nalysts to understand with the exception of the )ominican Republic where the research administrators were not directly involved7

Third the analysis as applied in Ecuador and ULuguay with the direct involvement of research system administrators at various stages forced those decision makers to

12

consciously identifv and trade off goals and criteria Fourth the use of spreadsheet programs in EcUadfor and Uruguay facilitated sensitivity analysis after the initial set of priorities was deermimd Fifth the procedure provided a relatively objectiveassessment of prioriies because individuals were not allowed to rank commodities orresearch areas directly but had to weight criteria

The first weakness inherent in ex ante research priority setting and thus in theprocedure is that there is a large amount of subjectiveness Although tileapproach isless subjective than unstructured judgment there is subjectiveness in the responses toquestiorns related to some of the criteria and also ini the weights placed by decision makers on the criteria Second it proved diflicult to specify independent criteria with no overlap Third while the procedure required less time than an expected economic surplus analysis by commodity it required more time than asking respondents to directlyrank commodities and research areas In E-cuador the consiultant worked withpersonnel in INIAP and FEI)IA for one week at the start of the analysis to determinegoals and criteria to arrive at atlist of commodities and researh areas to list secondarydata needs to formulate questions to be asked of administrators and scientists to decide who should be asked particular questions and to explain the procedure itself includingthe spreadsheet program The required iidormation was collected over the Followingmonth and the consultant returned and worked with the Planning Director in INIAP for one week to conduct the analysis and write tip the results Revisions to the initial draftof the report we-e then made over the lollowing weeks A similar procedure to the onefollowed in Ecuador was used in l_ruguay althaugli more time was spent discussing the wording of questions to be asked of scientists Furthermore n Uruguay more care was taken to match the criteria with the goas

Certain of the criteria wil always be diflicult to eplain to tileinterviewees Thecriterion on comparative advantage is difficult for some respondents because of a lack of knowledge about neighboring countries If suflicient data time and resources are available this criterion may be rnea4ured more precisely using comparative advantage or domestic resource cost procedures suggested by longmire and Winkelnann Anotherdiflicult criterion to estiniate is the potential foture demand for the product The respondents in Fcuador were asked to consider how the demands for different foodswould change as income and populatiot grow Many of them had a hard timeconceiving that the total or per capita demand for certain Foods could actually decline as income grew In Uiruguay (ibe cmiparative advantage and future dema nd questions were answered by a small group of economists from the ministries of agriculture and trade In the Ecuador study more attention was devoted to assessing the most appropriate people for answering different questions than was the case in the Dominican Republic Ilowever some criticism surfaced about the weights placed on criteria InUruguay decision makers were given the opportunity to change their weights afterviewing the initial results This was very useftl because it demonstrated the implicationsof placing different weights on the various goals and criteria

Computer spreadsheet programs were a major help in Ecuador and Uruguay Additional work needs to be devoted to developing a more mentnu-driven prcgram but the people inthe planning office of INIAP were able to use the SuperCalcR program without too tmucht difliculty despite ialack of previous experience Ihe analyst in Uruguay was veryfamiliar with lotusl which proved most helpfil

It became clear from working inall three countries that it is preferable to work directlywith the final decision makers or their designees Inthe l)ominican Republic the study was conducted by a very competent set of consultants from the local agricultural universi ty but the procedure was not institutionalized to allow for additional sensitivityanalysis or future priority-setting efforts InEcuador and especially in Uruguay the

13

decision makers were directly involved conducted sensitivity analyses and were able to assess why certain commodities or research areas received high priority as a result of the analyses In both Ecuador and Uruguay they have a basic understanding of the strengths and weaknesses of tileprocedure and therelre can explain their subsequentdecisions to groups who might disagree with tilepriorities they establish

The corntrast between the Dominican Republic and tie Uruguay studies illustrates a veryimportant point No matter which technique is used to structure the analysispriority-setting exercises must integrate the decision makers (users) into the processPriority-setting procedures must be institutionalited the results of initial analysisdiscussed thoroughly with decision makers and sensitivity anilsis conlducteU alone the lines suggested by that discussion Tecinological change generates a surplus which can be appropriated by dilkircnt sectors or groups within the economy Who benelitdepends in part on the cotaniodities selected for research and the types of technoloLies generated One of the purposes of institutionalizing a formal prioritN- setting procedureis to help mediate conflicts that may arise (for example between producer groups and consumers) as a result of research-induced improved technologies

Other issues arose during the analyses First the initial grouping and elimination of certain commodities and research areas before lhe irialvis is an important step but one which must be taken with carelul consideration of possible coniplmentarities in research on particular productS or types of rearch Second some of the commodities such as forages are inputs into other commodities such as milk (Corseqtentlv these commodities llay have to he considered itscinps because it is the value-added and not tilegross value of output that is relevant when one coniiodity requires other commodities sinputs and when criteria such as value of production are used This was an issue in I ruguav where livestock are quite important liird little attention was devoted in atv of the studies to defining appropriate criteria Ior the increased inoome stability eoal Future studies if this be an ideltJiiLd LOal shOUld define risk criteria

In sutImary the weighted criteria model as applied in Lcuador and iuguav which had the advantage of bit ildin Oillessons lCarICd illdie l)ominican RCpublic appears to have been a uselul priority-setting tool It ha its ereatest advartage ill situations ill which there are rulItiple goals to consider a relatively major reassesnieiit of prioritiesis contenplated and the list of commodities dd resen rch areas to consider is largeWeighted criteria models described in this paper were applied in small countries with limited resources but with direct involvement (except in the Dominican Republic) ofthC major decision makers These models itav be most useful in the small country situationwhere decision mAkers are more likely to be involved in tile priority-settirt process and are not just using tables of results Perhaps in larger systet7 where decisioii makine is more decentralited tileweighted criteria model can he applied at the regional level before relining research priorities at the national level

Another factor that can be uniortant in large svvtern is tle complerientarity between resea-h aid educat ion [hie larger the svstcnt the clser the linkages tend to be between universities and tileresearchi system Where universities are important tor agriclltural research an aidditional criterion may hc rteedCd to place value on the educationlal benefits of certain types of reseirch

Ihere are several additional iniplications with respect to how weighted criteria models relate to alterniative priori t-setting Inethod parIicularl the expected economic surplusapproach Befoce exploring these relationships kcx aspects of the surplus approacliare described based on aii application in Peru

14

George W Norton is associate professor Virginia Polytechnic Institute and StateUniversity Blacksburg Virginia and research 1llow International Service for National Agricultural Research (ISNAR) The Ilague Netherlands Philip G Pardey is joiltly a research associate University of Minnesota St Paul Minnesota and reseach officer ISNAR The authors would like to thank Iloward Elliott Alexander Von der Osten Paul Berinell and Carlos Valverde for their helpful comments and suggestions on an earlier draft of this paper without implicating thern in any remaining errors and omissions

TABLE OF CONTENTS

I Context of Research Priority-Setting I 1I Previous Priority-Setting Models 2

Weighted Criteria Models 3Benefit-cost (Expected Economic Surplus) Analysis 3Mathematical Programming 3Simulation 3Comparative Assessment 4

III The Use of Weighted Criteria Models in the Dominican RepublicEcuaodm and Uruguay 6

Goals 7Criteria 7

Coinmoditv Criteria 7Research Area Criteria 9

Data Collection and Model Implementation 9An Assessment of the Weighted Criteria Models used in theDominican Republic Ecuador and Uruguay 12

IV The Use of Expected Economic Surplus Analysis in Peru 15

An Assessment of the Expected Economic Surplus Procedure in Peru 15 V Implications for Development of Improved Research Priority-Setting Procedures 16 VI Conclusions 18 Footnotes 19

References 20

Appendix 1 23

Appendix 2 26

Priority-Setting Mechanisms for National

Agricultural Research Systems

Present Experience and Future Needs

Recent trends towards shrinking real research budgets or many national agriculturalresearch systems (NRS) have stimulated several attempts to evaluate tileeconomic benefits ofaricultural reserch Manv of these quantitatis e economic evaluations haveemployed historical data to provide ex post assessments and a few hase used subjectivejudgments to project future research benefits [he results kthese analyses have been used at the national or inter-ministerial level to justilf the need for continued political support for agricultural research

Tighter rese rch budgets not only heighten the iced to justifV agricultural research vis a vis alternative public investments but they alVo make it important for NARS to improve procedures for setting priorities among competing research programs A varietyof strategic and project-level decisions are made within NARS The objectives of national policvmakers the needs of agricultural producers and scientific opinion must each be brotight to bear on strategic decisions to emphasize particular commodities and research program areas uld on research project selection

The purpoe of this paiper is to discuss kev elements for inclusion in improved research priority-setting procedures for NRS Ihe fLcus is on methods for strategic as opposedto more tactical projcct-le el decision m kingAfter discussing the context of research priority setting a brief summary is provided of tile iain types of research resource alloL-ition mnetlods Recciit cxperiences with priority setting in four Latin American NAR the l)ominican Republic lcuador lUVUav and Peru are compared These experiences can proside sonc guidance for the development of an improved research resource allocation procedure

I TilE CONTEXT OF RESEARCH PRIORITY SETTING

NARS decision mnkers often operating within the ministry of agriculture an agricultural research institute or as a research council make allocative choices based on their knowledge atnd prior cxperience That knowledge and experience frequentlyincludes an understar one of national and regional oals and objectives as voiced through the political process a sene o lie severity of particular types of research problems and a general feel tor vlat is achievable through research It also includes an awareness of the desires ( te-ressures) of producer groups Decision makers must decide hoW much cmpha sisto jte onparticular commodities (eg rice maize beanssheep) and types of research c v varietal improvenmnt improved livestock nutritionplant protection farming svwtes Implicit in those decisions are the locational emphasis of research the focus 0rt paJticullar factors of production (eg land laborwater) the emphasis placed Ol lone-r-term haisic versus shorter-term applied and adaptive research and the distrihotianal elfects of research on farm size on producers versus coniumners and on people atidiherent income lec

In many NA RS research resoce alocation decisions are heavily influenced by the previous years budget2 (hanges of~tn result from requests by scientists which areevaluated reLatively informally and aggregatcd into an overall plan The resulting plan

upon close scrutiny may contribute little to tire attainment of stated national goals and objectives

Judgment based upon prior knowledge and information provided by scientists is crucial for research resource alincation decisions In addition rapid or radical changes in research programs -an be costly Particularly as research budgets tighten howeverforcing changes in tile systein tile increased use of quantitative methods may be nocessary to imprwe thL objectivity of those judgments The aim is to foster consitencv of research priorities with goAs and objectives and to improve the efliciency of the research system in mser ing producer and cortsumer needs The idea is not to replacejudgment but to increase and organize the inflormation available for updating priorknowledge and beliefs The hope also is to inject continuity in the priority-settingprocess and to proide NARS decision makers x ith methods the find lielplib when rationalizing their decisions to scientists producer Erotps and politicians Docunentingthe decision proces through a strctured procedure can help minimie the possinilitvof sudden and large shilts in researh priorities in those systems characterized by a flairlv rapid turnover of research administrators

Many quantitative priority-setting procedures are avaiia ble but few have beeninstitutionalized into the decision-making practices of NA RS manaers A cruciZl factor in their non- use of these procedures urtdoubtcdlv as been the lack of ri orous vetcost-eflectie approaches which can irtcorporate tile large number of comtodities andresearch areas as well as the multiple go1ls a ld criteria found in most strateoicdecision-muakirt situations inl NRS FurtIhcrrtore the cortmputer capabiliv available through microcomputers in most developing-co ntry NRS today was not gcenerallyavailable even two or three velars ago

In this paper procedure amc ce crihed which were recently applied in four latinAmerican countries First however previously emploved priority-setting mechanisms are briefly reviewed id critiqued One of rite points that will Ie made at the end of this paper is tiLit tise methods are not nitually excilsive There is no siTele magic huletbut there appears to be real potential for developing a Ilexible computer-assisted priority-setting framework that will allow NARS administrators to call ott theappropriate tectiiques tot aSsist withit a particular decision depending ott tie time and financial resources a ailahle and the relative itiportance of the allocation decision Asecond major point is that priority setting is a process not simply a procedure and must be instit utiortliied as part of the overall process of research managementPriority-setting technritiuetes applied only once by people outside of the decision-making process have little valtie

If SUMMARY OF PRIORITY-SETTING METHODS

Methods reported itt previous studies for research priority setting include

I ) establishtert td weighting of multiple criteria for ranking commodities and research ares with a final aggregate ranking based on implicit or explicit weights

2) use of bertefit-cost analysis including expected economic surplus techniques to select cotitritodities and research areas

3) application of mathematical programming to choose an optimal research portfolioincorporatirng multiple goals md constraits

4) developrert and tise of simulation models

These four approaches are discussed below followed by a brief assessment of their advantaiges atil dis idsittages

2

Weighted Criteria Models

Several studies hae established multiple criteria for ranking priorities because of tiledesire to explicitly consider a wide variety of factors that do or perhaps should influenceresearch selection The relative weights attached to each criterion to arrive at a final listof research prioriries are sometimes left implicit or unstated A recent example is tileTAC review of priorities and future strategies for the Consultative Group onInternational Agricultural Research (CGIAR) In assessing priorities by commoditythey etablished a principal goal research objective to obtain the goal and a series ofcriteria organized into three groups relevance research productivity and efliciency ofhe CGIAR system in undertaking research Detailed tables with quantitative andqualitative information and rankings of commodities for each criterion were presentedIn developing the recommendations for hort- to medium-term funding alternatives atthe conmmodity level the weights used to aggregate across criteria were left implicit

There are examples of tudies that have incorporated multiple criteria but also explicitlyspecified and utilized a set of weights to aggregate across criteria and obtain a finalranking of research priogrities Appications of this method often called the scoringmodel approach are found in studies by Williamson lahlstede Paulsen and KaldorShumway and McCrack en and Von (Oppen and Ryan A flew studies have used congruence analysis which can be thought of as a special case of a sci-ring or weightedcriteria model with all the weight placed on the criterion of vaue of production

Benefit-cost (Expected Economic Surplus) Analysis

The benefit-cost approach to welecting research priorities has been used in differentforms (Fishel Arai Sire and Gardner l)avis Oram and Ryan Norton Ganoza andPomareda) Most studies have employed consumer-producer surplus analysis and havebuilt upon a large body of literature on ex post research evaluation Ex ante analysesusually incorporate expert opinion to determine projected research impacts adoptionrates and probabilities of research success and provide estimates of the economiceffliciencv and distributional implications of agricultural research resource allocationBenefit-cost studies typically calculate benefit-cost ratios internal rates of return andnet present values for alternative types of research or research onfor differentcommodities Ilse analyses may include andmav or not regional internationalresearch spillovers and the eflects of domestic pricing policies on research benefits

Mathematical Programming

Mathematical programming is another alternative for research selection It relies onmathematical Optimization to choose a research portfolio through maximizing amultiple-goal objective function given the resource constraints of the rescaich systemA good example of the use of this method is a study by Russell in the United Kingdomlie used goal programming to maximize the contributions of the research program toseveral goals given the con taints of budget human resources state of knowledge andcertain policies This procs Ue uses similar information to the weighted-criteria modelbut selects an optimal research portfolio rather than simply ranking research areas

Simulation

Finally the simulation method has been used to identify and select research prioritiesSimulation models vary in their construction but a good example is the model byPinstrup-Anderscn and Franklin They built a model to project the contributions andcosts of alternative research activities They established gals and then identifiedchanges in supply demand for inputs and demand for output needed to meet those

3

goals They identified neeled technologies time and financial costs and the probability of research success and adoption Finally they specified the scientists workingobjectives This model is very thorough but did require extensive amounts of data and estimation of several mathematical relationships

Comparative Assessment

The above discussion is a brie summary of the major structu al approaches that have been used to asist decision makers in research priority setting All of them attempt to go beyond the usual procedure of individual or group decision makin e without structural analyses Each of the approacles has its advantages and disadvantages (See Fable I)

The procedure of establishing criteria and using weights to arrive at a final set of research priorities has the advantage of forcing decision makers to consciously trade off multiple goals It can illcorporae both quantitative and qualhtative information and can be applied to a long list of con mnodities or research areas in a relatively liort period of time The procedure is relati cx ca sY fbr administrators to understand but does requiretheir time in obtaining the explicit w5eights for criteria Furthermore these weights are inevitably subjective and their elicitatlion must be carefully structured [lie nActhod also requires scientists time in collecting inlormation on qualitative criteria As a result the approach k better suited for periodic or major priority-setting efforts than or situations where frequent marginal changes are anticipated

The expected economic surplus approach his the major advantage of incorporatingseverml criteria related to economic elficiency and distribution into one or two measures It also can be ucd to examine the gencral equilibrium ellects ofresearch (Ramailho de Castro and Sclhh nAid tile benefits of research under alternative possibly distortionar domestic pticing and international trade policies tEdwarts and Freebairn Alston Edwards and rcecbairn Norton and (ianoza 1985) These alictors are pervasive in developing countries and affect the efficiency and distributional consequences of researh Ile procedure ie(liCe ir lel - tndcrstandw ofeconomic analysisand more anal-t s time than the weighted criteria model but less aduinistrators time It can be diflicult to apply to at large number of commodities or research areas because certain typcs of data necessary for the ainalysis often do not exist for all commodities With adequaIL tL d ita Cxpected economic surplus analysis can be incorporated into the weighted criteria model or could be used on the set of commodities which the weightedcriteria model imdicaMes to have the highest piority lhe latter approach would allow for the calculation of income foregone as a reult of placing weights on non-economic ef1ciency criteria Expected economic surplus analysis has the major advantage of calculating a rate of return which can be compared to alternative public investments

The mathematical programming approach is similar to the weighted criteria model approach becaue weights are placed on a set of goals or criteria The procedure has the advantage of explicitly considering the budget huLman resource aad other constraints on the research system tnless the constraints are well specified including possiblechanges over time liowever there is a risk of nonsense solutions The model is more intensive of an ecolonic analysts time and abilit than tie simpler weighted criteria approach and decision makers may be less willing to accept what appears to be a black box solution lrade-offs among goals are easily quantified ssith this approach

The advantage of simulation models is their tlexibility They can be constructed is relatively simple or complex tools can incorporate optimizing or ranking proceduresand can readily include probabilistic information Their major disadvantage is that to be useful they must be relatively complex and typically reqtuire extensive amounts of both data and time of skilled analysts

4

TABLE 1 COMPARISON AMONG MAJOR AGRICULTURAL RESEARCH PRIORITY-SETTING METHODS

PRIORITY-SETTING METHOD

HEIGHTED EXPECTED MATH CHARACTERISTIC CRITERIA ECON SURPLUS PROGRAM SIMULATION

OPERATIONAL CONSIDERATIONS

1 Relative cost in researchers time mudium mJium medium high2 Relative cost in priority setting analysts ime medium medium high high3 Relative cost in administrators time medium mediummedium medium 4 Relative overall data requirement medium medium medium variable 5 Relative ease of comprehonsion by decision maker high medium low low 6 Ease of incorporating subjective informaticn high high high high7 Ease of incorporating non-quantitativa inforiation high low low medium

GOAL-RELATED ISSUES

a Requires explicit elicitation of goals yes usually yes uiually9 Can determine distributional affects on consumers

and producers at various income levels no yes no yes10 Can handle uncertainty yes yes yes yes11 Can consider tradeoff among multiple goals yes sometimes yes yes

CRITERIA-RELATED ISSUES

12 Can consider private-sector research incentives yes difficult difficult yes13 Can consider economic policy and trade affects yes yes yes yes

EVALUATION-RELATED ISSUES

14 Can be used to sot priorities for research at the aggregate level no yes no es

15 Can be used to set research priorities at the commodity level yes yes yes yes

16 Can be used to set priorities for non-production or non-commodity oriented research yes difficuli yes yes

17 Can be used to set priorities for basic research yes difficult no sometimes 18 Can evaluate secondary impacts of rosuarch on

employment environment nutrition yes sometimes so etimes yes19 Usually estimates a rata of return to research yes sometimesno no 20 Can quantify geographic spillover effects no yes no yes 21 Can consider the lags involved in research

and adoption yes yesyes yes22 Facilitates priority setting when the number

of commodities is large yes difficultdifficult difficult

Source Adapted and modified from Norton and Davis

5

The above is a very brief summary of the major advantages and disadvantages of the most common formally structured priority-setting models that could be used for strategicplanning for research resource allocation Other approaches exist for research projectselection with perhaps models that attempt to identify and analyze key vield constraints being the most prevalent approach employed in NARS of less-deeloped countries

The results of ex post analysis also can provide useful guidance for ex ante research resource allocation decisions if appropriately incorporated into a systematic ex ante procedure The most common ex post approach in addition to cx post benefit-cost analysis is the econometric estimation of production or supply functions incorporating research variables To be most uteful for ex ante analysis econometric approaches mLlSt be applied with a high degree of disaggrega tion Unfortunateiy these models require a substantial amount of high-quality historical data on production farm inputs and research expenditures

Perhaps the two approaches with the most potential given additional refinements for application by NARS in less-developed countries are the weighted criteria models and the benefit-cost or expected economic surplus procedures In the next section reent experiences with applying weighted criteria models in three Latin American Countries )ominican Republic Ecuador and Uruguay are discussed and evaluated The purpose

it to piovide more iniormation about the procedures their degree of acceptance and possible improvements in the models Ihen application of expected economic surplus models are considered drawing cn a recent cx ante research evaluation studly in Peru Finally the relationship between weighted criteria and expected economic surpluts methods is described along with possible extensions

IIl THE USE OF WEIGHiTED CRITERIA MODELS IN TIlE DOMINICAN REPUBLIC ECUADOR AND URUGUAY

in 1986 iif I87 weighted criteria models were developed and implemented in the lDomin lRepublic (ISA) Ecuador (Espinosa et al) and Trugua (CIAAB)to assist with priority setting for agricultwral research In the I)ominican Republic the study was carried out at the Instituto Superior de Agnicultura ISA) by consultants for ISNAR In Ecuador the study was conducted by the plmnintg ollice oY the Instituto Nacional de Investigacion Agropecuaria (INIAP) with the assistance of consultants for the US Agency for International Development (AID) In Uruguay the study was conducted by the Ollice of the Director of the Centro de Investigacion Agricola Alberto Boergcr(CIAAB) with the assistance of ISNAR The purpose of all three studies was to applyprocedures for prioritizing agricultural research by commodity and by major research area Inthe Dominican Republic the client was the newly created natonal agriculturalresearch agency (Il)EA) In Ecuador the client was INIAP and the newly created foundation for agricultural research (IEl)IA) In Urugua the client was C1IAB

The procedures employed in the three studies were similar but with some importantdilfereices as the Ecuador study built on lessons learned inthe I)ominican Republic and the Uruguay study built on other lessons learmed in Ecuador National goals for the research system iere elicited in each country and a series of criteria establishd which relate to those goals Separate riteria were developed for conmitodities and or research areas and weights were elicited from decision makers to establish the relative importanceof the criteria Commodities and research areas were ranked according to each criterion and these rankings were multiplied by the elicited weights to arrive at research priorities

6

Goals

Three goals were identified in each of the three countries (1) to raise the average levelof income in the country (2) to increase the well-b(Aing of low-income groups in societyand (3) to reduce yearto-year income fluctuations in the country especially onl tiledownside These goals were worded slightly diffrcntly in each country but in each casethey essentially relate to the three goals referred to in previous studies as efficncy equity and security In the l)ominican Republic the local consultants identifiedthese as the goals while in lcuador and Uruguay it was done by the directors of national research agencies

Criteria

The next step was to establish criteria to as measures of whetheruse particularcommodities or research areas contribute to the attainment of the above goals A largeset of criteria was discussed in each country and a total of 15 criteria were eventuallyused in the Dominican Republic 14 in lEcuador and 10 in Uruguay to determine priorities by commodity Five criteria were used in each case to determine priorities byresearch area Refinements were made in successive studies - beginning with theDominican Republic then Ecuador followed by Iruguay - to increase the independenceof criteria and to remove criteria that were questionable measures of whether researchcontributed to tile tated goals ( Reason for dropping particular ci eria are discussedbelow) Refinements also were made with respect to the grouping of criteria to facilitatethe weighting of goals and criteria

Commodity Criteria omioditv criteria were grou l - J into four conceptual groupsproduct importance probability O success elliciencv in use of research resources anddistribution of impacts The first three of these groups relate to the efficiency or incomelevel goal and the last group relates to the equity or distributional goal In none of thethree studies were criteria included which represnted the third goal (reduced incomelluctuation [lie feeling was that the entire set of criteria as a group would serve toreduce the enlhasis placed on single or 1ew (often export) crops While this appearedto be the case in the results of thee particular studies ft ture weighted criteria studies may want to add criteria which explicitly rank commodities from lowest to hiehest withrespect to annual gross income variability (or price and yield variability separately)

The group of criteria referred to as jrodct importap contained Four criteria in theUruguay study value of production generation or saving of foreign exchange expectedfuture demand change and comparative advantage

The value of productionl is an important criterion because the cost of rseaich isbasically independent of the number of units alkected by the research results at least forthose commodities with relatively homogeneous production conditions Foreignexchange was used as a criterion because of a perception in each country that the lackof foreign exchange was impeding growtb A criterion was set up to allow more weightto be paced on those commodities for which demand is expected to increase in thefuture because the importance of commodities will change as diets are altered withincome growth aod a world markets change Comparative advantage was includedbecause income in the country will be higher if the country focuses its efforts oncommodities for which its resource base is best suited while recognizing that research itself can alter resource constraints

In the Dominican Republic and in Ecuador nutritional measurcs (calorie and proteincontributions to the diet) were included as criteria for product importance They wereincluded because the well-being of low-income groups may be affected by the availability

7

of foods which are important in their diets Nutritional criteria were dropped in Uruguay however because numerous studies have shown that income levels are tile primary determinants of nutrition levels rather than the availability of particular foods Certainly caloric and protein content are debatable criteria To the extent that increased domestic production lowers the prices of certain food commodities therebyincreasing food consumption by the poor nutritional criteria can contribute to the distributional goal Land area also was included as a criterion in the Dominican Republic study but was dropped in Ecuador and Uruguly because of the overlap with value of production

Two criteria were used in each study to measure 1robahility of reseatch success The first measure was tilegap between own-country yields and neighboing country yields The second was the potential for success indicated by the researchers themselves The rationale for the gap criterion was that the larger the yield gap between the home country and similar countries tle greater the potential for yield gains The criterion is rather crude because yield gaps also can indicate a relative resource disadvantace forproducing the commodity even if the countries appear to be geoclimatically sinilar Consequently the gap criterion was eventually dropped in the tUruguav study The potential for success as indicated by the researchers is subjective but a very necessary measure to obtain

The gro up of criteria reficrred to as e1)cienc in use otresearch resources contained three criteria the relationship to research in the international centers the degree of emphasis on tilecommodity in le current research p ogram and the incentive for the privatesector to conduct the research Some research duplicates other research croplementsand still other research bears no relationship to research at tileinternational agriculturalresearch centers For certain comniodities all three relationships may hold depending on the type of research If national research is complementary to international researchthe cost of the rational research is lower Consequently a criterion was established which places greater weight on commodities with a high degree of complementaritybetween ntin Il d international research The criterion ofdelegree of emphasis in tile current iesercli program was established because there is a cost to rapidly adjustingcurrent h uiman and physical research resources from one commodity to another The criterion of private-sector incentives was used because the social benefits from the total research program in the country will be greatest if scarce public research funds are devoted to research on commodities for which the private sector does not have an incentive to support the research itself Of coui se the private-sector producer group can tax itself and pay the public sector to conduct the research for those commodities for which it has an incentive to privately support the research These commodities tend to be export commodities because producers can capture a higher proportion of the gainsfrom productivity increases for export commodities than for commodities primarily consumed domestically

The group of criteria related to the distribution ojresarchimpacts contained two criteria in the Uruguay study number of producers and the effect of increased productivity on the price of product The larger the number of farms producing a commodity the largerthe number of farmers afficted by the research and perhaps the higher the probabilitythat smaller farmers are being affeicted The distribution of benefits depends to a significant degree on price changes resulting from increased productivity Producers benefit more if prices decline very little and consumers benefit more ifprices decline substatitially for a given production increase For export er import commodities priceswill tend to decline less than for commodities primarily produced and consumed domestically For commodities which are not traded and are very important in the diets of the poor prices will tend to decline relatively sharply when productivity increases The division of benefits between producers and consumers also is affected by the fact

that producers are also consumers Consequently in the Dominican Republic and Ecuador studies an additional criterion importance of the commodity in home-consumption on the farm where produced also was included

In summary 10-15 criteria were used in tie three studies to identify and select research priorities by commodity The linkages anong the goals and criteria described above are summarized in Figure 1

Research Area Criteria The five criteria uicel to select research priorities by research aieas were (1 ) whether the research causes an increase in the use of relatively abundant resources and a saving of relatively scarce resources (2) the number and seventy of research problems (3) non-duplication with transferable research from outside the country (4) tne extent of private-sector incentives to conduct the research and (5) current emphases in the research program

The rate of economic growth will be more rapid if new technologies are developed that utilize relatively abundant resources and save relatively scarce resources The resource base varies by region consequently information on resource abundance criteria was collected by region in tileEcuador and Uruguay studies The same was true for the quantity and severity of research problems Information on the relationship of domestic to inernational research private-sector incentives and the current emphasis in the research program was collected on a naticnal basis These criteria all relate to the income growth (efficiency) goal It is difficult to identif research area criteria that measurc whether particular types of research aifect income distribution or variability

Data Collection and Model Implementation

Information used in the analysis included both quantitative data on value of productionnumber of farms value of exports and imports person-years devoted to research on different commodities and tile of protein per day in thenumber of calories and gramsdiet (for tileDominican Republic and Ecuador studies) as well as qualitative or subjective information on such factors as probability of success private sector incentives and severity of research problems Furthermore weights had to be elicited from decision makers to place relative emphasis on the various criteria

Step 1 Develop Cotninodity and Research Area Lists -- The first step was to determine the list ofcommadities to consider and the list of research areas In the Dominican Republic information was collected on 74 commodities In Ecuador an initial list of 109 commodities on which INIAP was conducting research was discussed Throughdecisions to eliminate some and to group others this list was reduced to 44 commedities for the analysis These decisions were made by the Technical Director for Research In Uruguay a smaller list of commodities was reduced to 21 commodities or commodity groups These decisions were made in consultation with the National Research and Fxtension Directors The list of research program areas contained nine areas in the Dominican Republic 16 in Ecuador and 16 in Uruguay

Step 2 Collect Quantitative and Qualitative Data on Chosen Criteria -- Information o-i quantitative criteria were gathered from local and FAO secondary data sources and one table was constructed For each criterion Commodities were ranked for each criterion Information needed for the qualitative riteria as well as the weights to place on criteria were obtained through interviews with sc rtists and administrators at boh the national and regional experiment stations In Euador and Uruguay substantial attention was devoted to identifying the appropriate respondents to each question as this is a crucial aspect of the procedure In Ecuador those interviewed were notified one week before the interviews and 72 people were interviewed by the Planning Director of INIAP and

9

FIJR 1 CONCEPTUAL ELEHNTS IN THE HEIGHTED CRITERIA MODEL FOR PAWING RESEARCH PRIORITIES BY COtlDITY

GOALS fficiency ]

CONCEPTUAL( )CRITERIA Pri7] Probabi i+y Research Distriutjon Iincome

ii J of Succes Resource Eff o L=Acplusmn a rilbility

CRITERIA 1 Value of 1 Scientist 1Rl ofto toNumberPrcduction Estimates 1 AnnualInternational Producers Variability2 Foreign Centers 2 Price in Gro

Exchange 2 Current Effect3 Future Raturn3)Emphasis 3 HomeDemand 3 Private Consumpt ion

4 Comparative Sector Advantage Incentives

Notes

(-n future studies an additional conceptural criteria under the security goal might be health (J4tri ional criteria such as grams of potein or kilo=alories could be included as additional easures of Distributional ikipacts

4ot used in Dominican Republic Ecuador and Uruojuay studies

his assistant In Uruguay the directors of all the regional stations were called into CIIAB headquarters for a aweting to discuss the study and this was followed up byon-site interviews and meetings with scientists and directors at the regional experimentstations as well as with economists in the Ministry of Agriculture These interviews and meetings were conducted by a person assigned to work wvith the Director of Cl LAB on the priority-setting study In the Dominican Republic a survey of scientists extensionworkers and administrators was conducted by the ISNAR consultant from the Instituto Superior de Agricultura who visited the various experiment stations Less care wastaken in the Dominican Repubic than in Ecuador and Uruguay to match the questionspertaining -o individual criteria to particular individuals This probably reduced theusefulness of the responses because those most familiar with the probability of research success For example may not be the same -s those most Familiar with comparativeadvantage

Step 3 Elicit 11eihts on Critria -- Relative weights to place on the different criteria were obtained Fronm national and regional research system administrators In Ecuador34 peopic verc used to determine the weights These wveights were established separatelyIor the commoditv and the research area criteria In 1iuguay fewer people (sevensystem and station directors) were used to determine the weights and more attention was devoted to grouping the criteria pertaining to particular goals A Delphi procedure was used in which the s7even directors were shown the average of the group and provided with an opportunity to adjust their weight

Step 4 )erive Rankinqs hv Commodity and By Research Area -- Once the basic information was collected axd organized into a seies of tables one procedure was followed to arrive at a final ranking of research priorities by commodity A second procedure resulted in lists of priorities by research area In the Dominican Republicmost of the calculations were made by hand in Ecuador and Uruguay microcomputerspreadsheet programs were used to assist with the calculations and to conduct sensitivity analysis

The steps in the procedure for ranking research priorities by commodity and research area are sumnmaried below and are illustrated with a simple example in Appendix 1 Brieily the steps for ranking commodities were

a) Th conniodities were ranked for each criterion for which quantitative data were available

b) Tihese rankings were multiplied by the weights assigned to each criterion and then the weighted criteria were added across to arrive at one sub-ranking for the

Luantitative criteria c) Each commodity was given a high low or none designation for each qualitative

(subjective) criterion The response which implied a need for greater research priority was assigned the number 2 the intermediate response was assigned I and the response which implied lower research priority was assigned 0 1herefore each comniodity received a 2 1 or 0 for each criterion

d) The weights assigned to each qualitative criterion were then multiplied by these numbers and the results were added across all criteria to arrive at a subtotal for each commoditv These subtotals were ranked from highest to lowest to provide a second sub-ranking

e) The sub-rankings for the quantitative and qualitative criteria were then given their corresponding weights and added together to arrive at a final ranking by commodityThe steps for ranking research priorities by research area were the same as steps (c)and (d) above because the criteria were all qualitative

Step 5 Analsis and Interpretationof Resuls -- In the Dominican Republic the resuLts of this weighted criteria analysis were uscd to determine a small set of cormodities and

II

research areas with the highest priority Further assessment was then made of humanphysical administrative and other resources needed to structure research programsfocused on these commodities and resea-ch topics In Ecuador and Uruguay the prioritized list of commodities was split into a high-priority group an intermediate-priority group and a low-priority group Research area priorities were identified for each region of tile country A more detailed analysis was then made of the human physical and other resources needed to support the high and intermediate priorities

In summary five major -rens were involved in operationalizing the weighted criteria models A complete set of rsults for each study included more than 20 tables and are not reproduced here Ilowever the final rankings for commodities and for research areas are shown in Tables I and 2 in Appendix 2 In the Dominican Republic a team of consultants followed up the initial priority-setting exercise with additional analysisand discussions at the experiment station level and eventually recommended tile establishment of five national commodity programs (rice cornsorghum beans coffeeand livestock) and one additional major research program (natural resources) In Ecuador the prioritized lists were widely distributed and discussed in the Ministry oi Agriculture the Board of FEIDIA the Commission on Science and Technology arid in AID There was concern in Ecuador with the weights placed on the individual criteria and therefore additional sensitivity analysis was conducted by the Planning Office of the Ministry There seemed to be a recognition in Ecuador that priority setting with a weighted criteria model is an iterative process and that much 4t the models value stems from tile discussion of criteria among the decision makers and from the sensitivityanalysis The high-priority list of comniodities in Ecuador included thL ten highest-priority commodities listed in Table 2b A project paper has been written forAI) which recommends funding For Fli)IAlbr three commodities (coffee soft corn for the Sierra and dairy) Coffee and soft coitn were the number 2 and 3 ranked prioritiesin tile weighted criteria model and dairy was number 9 although it was the top-rankedlivestock commodity

In Urugu ay the results from the first run with tile model were discussed with the research directors Ilie directors made small changes to the weights placed on criteriaand the mode was rerun resulting in the prioritized list in Appendix 2 Personnel in CIAAB are CUrrcwtv undertaking additional sensitivity analysis and with tie assistance of ISNA R are developing a plan for implementation

An Assessment of the Weighted Criteria Models Used in the Dominican RepublicEcuador and Uruguay

The Dominican Republic Ecuador and Uruguay studies were perhaps the first majorattempts to implement weighted criteria models with weights explicitly elicited from decision makers in NARS in less-developed countries A number of strengths andweaknesses in the procedures were identified during the analyses Some of thL weaknesses were corrected fron one study to the next some can be corrected in future studies and others are inherent in the approach

Trhe first strength mentioned earlier is the ability of the procedure to systematicallyincorporate both quantitative and qualitative information related to a set of multiplegoals and criteria in order to prioritize a lorng ist of commodities and research areas in a relatively short period of time Second the procedure proved relatively easy for both research administrators and the loc- -nalysts to understand with the exception of the )ominican Republic where the research administrators were not directly involved7

Third the analysis as applied in Ecuador and ULuguay with the direct involvement of research system administrators at various stages forced those decision makers to

12

consciously identifv and trade off goals and criteria Fourth the use of spreadsheet programs in EcUadfor and Uruguay facilitated sensitivity analysis after the initial set of priorities was deermimd Fifth the procedure provided a relatively objectiveassessment of prioriies because individuals were not allowed to rank commodities orresearch areas directly but had to weight criteria

The first weakness inherent in ex ante research priority setting and thus in theprocedure is that there is a large amount of subjectiveness Although tileapproach isless subjective than unstructured judgment there is subjectiveness in the responses toquestiorns related to some of the criteria and also ini the weights placed by decision makers on the criteria Second it proved diflicult to specify independent criteria with no overlap Third while the procedure required less time than an expected economic surplus analysis by commodity it required more time than asking respondents to directlyrank commodities and research areas In E-cuador the consiultant worked withpersonnel in INIAP and FEI)IA for one week at the start of the analysis to determinegoals and criteria to arrive at atlist of commodities and researh areas to list secondarydata needs to formulate questions to be asked of administrators and scientists to decide who should be asked particular questions and to explain the procedure itself includingthe spreadsheet program The required iidormation was collected over the Followingmonth and the consultant returned and worked with the Planning Director in INIAP for one week to conduct the analysis and write tip the results Revisions to the initial draftof the report we-e then made over the lollowing weeks A similar procedure to the onefollowed in Ecuador was used in l_ruguay althaugli more time was spent discussing the wording of questions to be asked of scientists Furthermore n Uruguay more care was taken to match the criteria with the goas

Certain of the criteria wil always be diflicult to eplain to tileinterviewees Thecriterion on comparative advantage is difficult for some respondents because of a lack of knowledge about neighboring countries If suflicient data time and resources are available this criterion may be rnea4ured more precisely using comparative advantage or domestic resource cost procedures suggested by longmire and Winkelnann Anotherdiflicult criterion to estiniate is the potential foture demand for the product The respondents in Fcuador were asked to consider how the demands for different foodswould change as income and populatiot grow Many of them had a hard timeconceiving that the total or per capita demand for certain Foods could actually decline as income grew In Uiruguay (ibe cmiparative advantage and future dema nd questions were answered by a small group of economists from the ministries of agriculture and trade In the Ecuador study more attention was devoted to assessing the most appropriate people for answering different questions than was the case in the Dominican Republic Ilowever some criticism surfaced about the weights placed on criteria InUruguay decision makers were given the opportunity to change their weights afterviewing the initial results This was very useftl because it demonstrated the implicationsof placing different weights on the various goals and criteria

Computer spreadsheet programs were a major help in Ecuador and Uruguay Additional work needs to be devoted to developing a more mentnu-driven prcgram but the people inthe planning office of INIAP were able to use the SuperCalcR program without too tmucht difliculty despite ialack of previous experience Ihe analyst in Uruguay was veryfamiliar with lotusl which proved most helpfil

It became clear from working inall three countries that it is preferable to work directlywith the final decision makers or their designees Inthe l)ominican Republic the study was conducted by a very competent set of consultants from the local agricultural universi ty but the procedure was not institutionalized to allow for additional sensitivityanalysis or future priority-setting efforts InEcuador and especially in Uruguay the

13

decision makers were directly involved conducted sensitivity analyses and were able to assess why certain commodities or research areas received high priority as a result of the analyses In both Ecuador and Uruguay they have a basic understanding of the strengths and weaknesses of tileprocedure and therelre can explain their subsequentdecisions to groups who might disagree with tilepriorities they establish

The corntrast between the Dominican Republic and tie Uruguay studies illustrates a veryimportant point No matter which technique is used to structure the analysispriority-setting exercises must integrate the decision makers (users) into the processPriority-setting procedures must be institutionalited the results of initial analysisdiscussed thoroughly with decision makers and sensitivity anilsis conlducteU alone the lines suggested by that discussion Tecinological change generates a surplus which can be appropriated by dilkircnt sectors or groups within the economy Who benelitdepends in part on the cotaniodities selected for research and the types of technoloLies generated One of the purposes of institutionalizing a formal prioritN- setting procedureis to help mediate conflicts that may arise (for example between producer groups and consumers) as a result of research-induced improved technologies

Other issues arose during the analyses First the initial grouping and elimination of certain commodities and research areas before lhe irialvis is an important step but one which must be taken with carelul consideration of possible coniplmentarities in research on particular productS or types of rearch Second some of the commodities such as forages are inputs into other commodities such as milk (Corseqtentlv these commodities llay have to he considered itscinps because it is the value-added and not tilegross value of output that is relevant when one coniiodity requires other commodities sinputs and when criteria such as value of production are used This was an issue in I ruguav where livestock are quite important liird little attention was devoted in atv of the studies to defining appropriate criteria Ior the increased inoome stability eoal Future studies if this be an ideltJiiLd LOal shOUld define risk criteria

In sutImary the weighted criteria model as applied in Lcuador and iuguav which had the advantage of bit ildin Oillessons lCarICd illdie l)ominican RCpublic appears to have been a uselul priority-setting tool It ha its ereatest advartage ill situations ill which there are rulItiple goals to consider a relatively major reassesnieiit of prioritiesis contenplated and the list of commodities dd resen rch areas to consider is largeWeighted criteria models described in this paper were applied in small countries with limited resources but with direct involvement (except in the Dominican Republic) ofthC major decision makers These models itav be most useful in the small country situationwhere decision mAkers are more likely to be involved in tile priority-settirt process and are not just using tables of results Perhaps in larger systet7 where decisioii makine is more decentralited tileweighted criteria model can he applied at the regional level before relining research priorities at the national level

Another factor that can be uniortant in large svvtern is tle complerientarity between resea-h aid educat ion [hie larger the svstcnt the clser the linkages tend to be between universities and tileresearchi system Where universities are important tor agriclltural research an aidditional criterion may hc rteedCd to place value on the educationlal benefits of certain types of reseirch

Ihere are several additional iniplications with respect to how weighted criteria models relate to alterniative priori t-setting Inethod parIicularl the expected economic surplusapproach Befoce exploring these relationships kcx aspects of the surplus approacliare described based on aii application in Peru

14

TABLE OF CONTENTS

I Context of Research Priority-Setting I 1I Previous Priority-Setting Models 2

Weighted Criteria Models 3Benefit-cost (Expected Economic Surplus) Analysis 3Mathematical Programming 3Simulation 3Comparative Assessment 4

III The Use of Weighted Criteria Models in the Dominican RepublicEcuaodm and Uruguay 6

Goals 7Criteria 7

Coinmoditv Criteria 7Research Area Criteria 9

Data Collection and Model Implementation 9An Assessment of the Weighted Criteria Models used in theDominican Republic Ecuador and Uruguay 12

IV The Use of Expected Economic Surplus Analysis in Peru 15

An Assessment of the Expected Economic Surplus Procedure in Peru 15 V Implications for Development of Improved Research Priority-Setting Procedures 16 VI Conclusions 18 Footnotes 19

References 20

Appendix 1 23

Appendix 2 26

Priority-Setting Mechanisms for National

Agricultural Research Systems

Present Experience and Future Needs

Recent trends towards shrinking real research budgets or many national agriculturalresearch systems (NRS) have stimulated several attempts to evaluate tileeconomic benefits ofaricultural reserch Manv of these quantitatis e economic evaluations haveemployed historical data to provide ex post assessments and a few hase used subjectivejudgments to project future research benefits [he results kthese analyses have been used at the national or inter-ministerial level to justilf the need for continued political support for agricultural research

Tighter rese rch budgets not only heighten the iced to justifV agricultural research vis a vis alternative public investments but they alVo make it important for NARS to improve procedures for setting priorities among competing research programs A varietyof strategic and project-level decisions are made within NARS The objectives of national policvmakers the needs of agricultural producers and scientific opinion must each be brotight to bear on strategic decisions to emphasize particular commodities and research program areas uld on research project selection

The purpoe of this paiper is to discuss kev elements for inclusion in improved research priority-setting procedures for NRS Ihe fLcus is on methods for strategic as opposedto more tactical projcct-le el decision m kingAfter discussing the context of research priority setting a brief summary is provided of tile iain types of research resource alloL-ition mnetlods Recciit cxperiences with priority setting in four Latin American NAR the l)ominican Republic lcuador lUVUav and Peru are compared These experiences can proside sonc guidance for the development of an improved research resource allocation procedure

I TilE CONTEXT OF RESEARCH PRIORITY SETTING

NARS decision mnkers often operating within the ministry of agriculture an agricultural research institute or as a research council make allocative choices based on their knowledge atnd prior cxperience That knowledge and experience frequentlyincludes an understar one of national and regional oals and objectives as voiced through the political process a sene o lie severity of particular types of research problems and a general feel tor vlat is achievable through research It also includes an awareness of the desires ( te-ressures) of producer groups Decision makers must decide hoW much cmpha sisto jte onparticular commodities (eg rice maize beanssheep) and types of research c v varietal improvenmnt improved livestock nutritionplant protection farming svwtes Implicit in those decisions are the locational emphasis of research the focus 0rt paJticullar factors of production (eg land laborwater) the emphasis placed Ol lone-r-term haisic versus shorter-term applied and adaptive research and the distrihotianal elfects of research on farm size on producers versus coniumners and on people atidiherent income lec

In many NA RS research resoce alocation decisions are heavily influenced by the previous years budget2 (hanges of~tn result from requests by scientists which areevaluated reLatively informally and aggregatcd into an overall plan The resulting plan

upon close scrutiny may contribute little to tire attainment of stated national goals and objectives

Judgment based upon prior knowledge and information provided by scientists is crucial for research resource alincation decisions In addition rapid or radical changes in research programs -an be costly Particularly as research budgets tighten howeverforcing changes in tile systein tile increased use of quantitative methods may be nocessary to imprwe thL objectivity of those judgments The aim is to foster consitencv of research priorities with goAs and objectives and to improve the efliciency of the research system in mser ing producer and cortsumer needs The idea is not to replacejudgment but to increase and organize the inflormation available for updating priorknowledge and beliefs The hope also is to inject continuity in the priority-settingprocess and to proide NARS decision makers x ith methods the find lielplib when rationalizing their decisions to scientists producer Erotps and politicians Docunentingthe decision proces through a strctured procedure can help minimie the possinilitvof sudden and large shilts in researh priorities in those systems characterized by a flairlv rapid turnover of research administrators

Many quantitative priority-setting procedures are avaiia ble but few have beeninstitutionalized into the decision-making practices of NA RS manaers A cruciZl factor in their non- use of these procedures urtdoubtcdlv as been the lack of ri orous vetcost-eflectie approaches which can irtcorporate tile large number of comtodities andresearch areas as well as the multiple go1ls a ld criteria found in most strateoicdecision-muakirt situations inl NRS FurtIhcrrtore the cortmputer capabiliv available through microcomputers in most developing-co ntry NRS today was not gcenerallyavailable even two or three velars ago

In this paper procedure amc ce crihed which were recently applied in four latinAmerican countries First however previously emploved priority-setting mechanisms are briefly reviewed id critiqued One of rite points that will Ie made at the end of this paper is tiLit tise methods are not nitually excilsive There is no siTele magic huletbut there appears to be real potential for developing a Ilexible computer-assisted priority-setting framework that will allow NARS administrators to call ott theappropriate tectiiques tot aSsist withit a particular decision depending ott tie time and financial resources a ailahle and the relative itiportance of the allocation decision Asecond major point is that priority setting is a process not simply a procedure and must be instit utiortliied as part of the overall process of research managementPriority-setting technritiuetes applied only once by people outside of the decision-making process have little valtie

If SUMMARY OF PRIORITY-SETTING METHODS

Methods reported itt previous studies for research priority setting include

I ) establishtert td weighting of multiple criteria for ranking commodities and research ares with a final aggregate ranking based on implicit or explicit weights

2) use of bertefit-cost analysis including expected economic surplus techniques to select cotitritodities and research areas

3) application of mathematical programming to choose an optimal research portfolioincorporatirng multiple goals md constraits

4) developrert and tise of simulation models

These four approaches are discussed below followed by a brief assessment of their advantaiges atil dis idsittages

2

Weighted Criteria Models

Several studies hae established multiple criteria for ranking priorities because of tiledesire to explicitly consider a wide variety of factors that do or perhaps should influenceresearch selection The relative weights attached to each criterion to arrive at a final listof research prioriries are sometimes left implicit or unstated A recent example is tileTAC review of priorities and future strategies for the Consultative Group onInternational Agricultural Research (CGIAR) In assessing priorities by commoditythey etablished a principal goal research objective to obtain the goal and a series ofcriteria organized into three groups relevance research productivity and efliciency ofhe CGIAR system in undertaking research Detailed tables with quantitative andqualitative information and rankings of commodities for each criterion were presentedIn developing the recommendations for hort- to medium-term funding alternatives atthe conmmodity level the weights used to aggregate across criteria were left implicit

There are examples of tudies that have incorporated multiple criteria but also explicitlyspecified and utilized a set of weights to aggregate across criteria and obtain a finalranking of research priogrities Appications of this method often called the scoringmodel approach are found in studies by Williamson lahlstede Paulsen and KaldorShumway and McCrack en and Von (Oppen and Ryan A flew studies have used congruence analysis which can be thought of as a special case of a sci-ring or weightedcriteria model with all the weight placed on the criterion of vaue of production

Benefit-cost (Expected Economic Surplus) Analysis

The benefit-cost approach to welecting research priorities has been used in differentforms (Fishel Arai Sire and Gardner l)avis Oram and Ryan Norton Ganoza andPomareda) Most studies have employed consumer-producer surplus analysis and havebuilt upon a large body of literature on ex post research evaluation Ex ante analysesusually incorporate expert opinion to determine projected research impacts adoptionrates and probabilities of research success and provide estimates of the economiceffliciencv and distributional implications of agricultural research resource allocationBenefit-cost studies typically calculate benefit-cost ratios internal rates of return andnet present values for alternative types of research or research onfor differentcommodities Ilse analyses may include andmav or not regional internationalresearch spillovers and the eflects of domestic pricing policies on research benefits

Mathematical Programming

Mathematical programming is another alternative for research selection It relies onmathematical Optimization to choose a research portfolio through maximizing amultiple-goal objective function given the resource constraints of the rescaich systemA good example of the use of this method is a study by Russell in the United Kingdomlie used goal programming to maximize the contributions of the research program toseveral goals given the con taints of budget human resources state of knowledge andcertain policies This procs Ue uses similar information to the weighted-criteria modelbut selects an optimal research portfolio rather than simply ranking research areas

Simulation

Finally the simulation method has been used to identify and select research prioritiesSimulation models vary in their construction but a good example is the model byPinstrup-Anderscn and Franklin They built a model to project the contributions andcosts of alternative research activities They established gals and then identifiedchanges in supply demand for inputs and demand for output needed to meet those

3

goals They identified neeled technologies time and financial costs and the probability of research success and adoption Finally they specified the scientists workingobjectives This model is very thorough but did require extensive amounts of data and estimation of several mathematical relationships

Comparative Assessment

The above discussion is a brie summary of the major structu al approaches that have been used to asist decision makers in research priority setting All of them attempt to go beyond the usual procedure of individual or group decision makin e without structural analyses Each of the approacles has its advantages and disadvantages (See Fable I)

The procedure of establishing criteria and using weights to arrive at a final set of research priorities has the advantage of forcing decision makers to consciously trade off multiple goals It can illcorporae both quantitative and qualhtative information and can be applied to a long list of con mnodities or research areas in a relatively liort period of time The procedure is relati cx ca sY fbr administrators to understand but does requiretheir time in obtaining the explicit w5eights for criteria Furthermore these weights are inevitably subjective and their elicitatlion must be carefully structured [lie nActhod also requires scientists time in collecting inlormation on qualitative criteria As a result the approach k better suited for periodic or major priority-setting efforts than or situations where frequent marginal changes are anticipated

The expected economic surplus approach his the major advantage of incorporatingseverml criteria related to economic elficiency and distribution into one or two measures It also can be ucd to examine the gencral equilibrium ellects ofresearch (Ramailho de Castro and Sclhh nAid tile benefits of research under alternative possibly distortionar domestic pticing and international trade policies tEdwarts and Freebairn Alston Edwards and rcecbairn Norton and (ianoza 1985) These alictors are pervasive in developing countries and affect the efficiency and distributional consequences of researh Ile procedure ie(liCe ir lel - tndcrstandw ofeconomic analysisand more anal-t s time than the weighted criteria model but less aduinistrators time It can be diflicult to apply to at large number of commodities or research areas because certain typcs of data necessary for the ainalysis often do not exist for all commodities With adequaIL tL d ita Cxpected economic surplus analysis can be incorporated into the weighted criteria model or could be used on the set of commodities which the weightedcriteria model imdicaMes to have the highest piority lhe latter approach would allow for the calculation of income foregone as a reult of placing weights on non-economic ef1ciency criteria Expected economic surplus analysis has the major advantage of calculating a rate of return which can be compared to alternative public investments

The mathematical programming approach is similar to the weighted criteria model approach becaue weights are placed on a set of goals or criteria The procedure has the advantage of explicitly considering the budget huLman resource aad other constraints on the research system tnless the constraints are well specified including possiblechanges over time liowever there is a risk of nonsense solutions The model is more intensive of an ecolonic analysts time and abilit than tie simpler weighted criteria approach and decision makers may be less willing to accept what appears to be a black box solution lrade-offs among goals are easily quantified ssith this approach

The advantage of simulation models is their tlexibility They can be constructed is relatively simple or complex tools can incorporate optimizing or ranking proceduresand can readily include probabilistic information Their major disadvantage is that to be useful they must be relatively complex and typically reqtuire extensive amounts of both data and time of skilled analysts

4

TABLE 1 COMPARISON AMONG MAJOR AGRICULTURAL RESEARCH PRIORITY-SETTING METHODS

PRIORITY-SETTING METHOD

HEIGHTED EXPECTED MATH CHARACTERISTIC CRITERIA ECON SURPLUS PROGRAM SIMULATION

OPERATIONAL CONSIDERATIONS

1 Relative cost in researchers time mudium mJium medium high2 Relative cost in priority setting analysts ime medium medium high high3 Relative cost in administrators time medium mediummedium medium 4 Relative overall data requirement medium medium medium variable 5 Relative ease of comprehonsion by decision maker high medium low low 6 Ease of incorporating subjective informaticn high high high high7 Ease of incorporating non-quantitativa inforiation high low low medium

GOAL-RELATED ISSUES

a Requires explicit elicitation of goals yes usually yes uiually9 Can determine distributional affects on consumers

and producers at various income levels no yes no yes10 Can handle uncertainty yes yes yes yes11 Can consider tradeoff among multiple goals yes sometimes yes yes

CRITERIA-RELATED ISSUES

12 Can consider private-sector research incentives yes difficult difficult yes13 Can consider economic policy and trade affects yes yes yes yes

EVALUATION-RELATED ISSUES

14 Can be used to sot priorities for research at the aggregate level no yes no es

15 Can be used to set research priorities at the commodity level yes yes yes yes

16 Can be used to set priorities for non-production or non-commodity oriented research yes difficuli yes yes

17 Can be used to set priorities for basic research yes difficult no sometimes 18 Can evaluate secondary impacts of rosuarch on

employment environment nutrition yes sometimes so etimes yes19 Usually estimates a rata of return to research yes sometimesno no 20 Can quantify geographic spillover effects no yes no yes 21 Can consider the lags involved in research

and adoption yes yesyes yes22 Facilitates priority setting when the number

of commodities is large yes difficultdifficult difficult

Source Adapted and modified from Norton and Davis

5

The above is a very brief summary of the major advantages and disadvantages of the most common formally structured priority-setting models that could be used for strategicplanning for research resource allocation Other approaches exist for research projectselection with perhaps models that attempt to identify and analyze key vield constraints being the most prevalent approach employed in NARS of less-deeloped countries

The results of ex post analysis also can provide useful guidance for ex ante research resource allocation decisions if appropriately incorporated into a systematic ex ante procedure The most common ex post approach in addition to cx post benefit-cost analysis is the econometric estimation of production or supply functions incorporating research variables To be most uteful for ex ante analysis econometric approaches mLlSt be applied with a high degree of disaggrega tion Unfortunateiy these models require a substantial amount of high-quality historical data on production farm inputs and research expenditures

Perhaps the two approaches with the most potential given additional refinements for application by NARS in less-developed countries are the weighted criteria models and the benefit-cost or expected economic surplus procedures In the next section reent experiences with applying weighted criteria models in three Latin American Countries )ominican Republic Ecuador and Uruguay are discussed and evaluated The purpose

it to piovide more iniormation about the procedures their degree of acceptance and possible improvements in the models Ihen application of expected economic surplus models are considered drawing cn a recent cx ante research evaluation studly in Peru Finally the relationship between weighted criteria and expected economic surpluts methods is described along with possible extensions

IIl THE USE OF WEIGHiTED CRITERIA MODELS IN TIlE DOMINICAN REPUBLIC ECUADOR AND URUGUAY

in 1986 iif I87 weighted criteria models were developed and implemented in the lDomin lRepublic (ISA) Ecuador (Espinosa et al) and Trugua (CIAAB)to assist with priority setting for agricultwral research In the I)ominican Republic the study was carried out at the Instituto Superior de Agnicultura ISA) by consultants for ISNAR In Ecuador the study was conducted by the plmnintg ollice oY the Instituto Nacional de Investigacion Agropecuaria (INIAP) with the assistance of consultants for the US Agency for International Development (AID) In Uruguay the study was conducted by the Ollice of the Director of the Centro de Investigacion Agricola Alberto Boergcr(CIAAB) with the assistance of ISNAR The purpose of all three studies was to applyprocedures for prioritizing agricultural research by commodity and by major research area Inthe Dominican Republic the client was the newly created natonal agriculturalresearch agency (Il)EA) In Ecuador the client was INIAP and the newly created foundation for agricultural research (IEl)IA) In Urugua the client was C1IAB

The procedures employed in the three studies were similar but with some importantdilfereices as the Ecuador study built on lessons learned inthe I)ominican Republic and the Uruguay study built on other lessons learmed in Ecuador National goals for the research system iere elicited in each country and a series of criteria establishd which relate to those goals Separate riteria were developed for conmitodities and or research areas and weights were elicited from decision makers to establish the relative importanceof the criteria Commodities and research areas were ranked according to each criterion and these rankings were multiplied by the elicited weights to arrive at research priorities

6

Goals

Three goals were identified in each of the three countries (1) to raise the average levelof income in the country (2) to increase the well-b(Aing of low-income groups in societyand (3) to reduce yearto-year income fluctuations in the country especially onl tiledownside These goals were worded slightly diffrcntly in each country but in each casethey essentially relate to the three goals referred to in previous studies as efficncy equity and security In the l)ominican Republic the local consultants identifiedthese as the goals while in lcuador and Uruguay it was done by the directors of national research agencies

Criteria

The next step was to establish criteria to as measures of whetheruse particularcommodities or research areas contribute to the attainment of the above goals A largeset of criteria was discussed in each country and a total of 15 criteria were eventuallyused in the Dominican Republic 14 in lEcuador and 10 in Uruguay to determine priorities by commodity Five criteria were used in each case to determine priorities byresearch area Refinements were made in successive studies - beginning with theDominican Republic then Ecuador followed by Iruguay - to increase the independenceof criteria and to remove criteria that were questionable measures of whether researchcontributed to tile tated goals ( Reason for dropping particular ci eria are discussedbelow) Refinements also were made with respect to the grouping of criteria to facilitatethe weighting of goals and criteria

Commodity Criteria omioditv criteria were grou l - J into four conceptual groupsproduct importance probability O success elliciencv in use of research resources anddistribution of impacts The first three of these groups relate to the efficiency or incomelevel goal and the last group relates to the equity or distributional goal In none of thethree studies were criteria included which represnted the third goal (reduced incomelluctuation [lie feeling was that the entire set of criteria as a group would serve toreduce the enlhasis placed on single or 1ew (often export) crops While this appearedto be the case in the results of thee particular studies ft ture weighted criteria studies may want to add criteria which explicitly rank commodities from lowest to hiehest withrespect to annual gross income variability (or price and yield variability separately)

The group of criteria referred to as jrodct importap contained Four criteria in theUruguay study value of production generation or saving of foreign exchange expectedfuture demand change and comparative advantage

The value of productionl is an important criterion because the cost of rseaich isbasically independent of the number of units alkected by the research results at least forthose commodities with relatively homogeneous production conditions Foreignexchange was used as a criterion because of a perception in each country that the lackof foreign exchange was impeding growtb A criterion was set up to allow more weightto be paced on those commodities for which demand is expected to increase in thefuture because the importance of commodities will change as diets are altered withincome growth aod a world markets change Comparative advantage was includedbecause income in the country will be higher if the country focuses its efforts oncommodities for which its resource base is best suited while recognizing that research itself can alter resource constraints

In the Dominican Republic and in Ecuador nutritional measurcs (calorie and proteincontributions to the diet) were included as criteria for product importance They wereincluded because the well-being of low-income groups may be affected by the availability

7

of foods which are important in their diets Nutritional criteria were dropped in Uruguay however because numerous studies have shown that income levels are tile primary determinants of nutrition levels rather than the availability of particular foods Certainly caloric and protein content are debatable criteria To the extent that increased domestic production lowers the prices of certain food commodities therebyincreasing food consumption by the poor nutritional criteria can contribute to the distributional goal Land area also was included as a criterion in the Dominican Republic study but was dropped in Ecuador and Uruguly because of the overlap with value of production

Two criteria were used in each study to measure 1robahility of reseatch success The first measure was tilegap between own-country yields and neighboing country yields The second was the potential for success indicated by the researchers themselves The rationale for the gap criterion was that the larger the yield gap between the home country and similar countries tle greater the potential for yield gains The criterion is rather crude because yield gaps also can indicate a relative resource disadvantace forproducing the commodity even if the countries appear to be geoclimatically sinilar Consequently the gap criterion was eventually dropped in the tUruguav study The potential for success as indicated by the researchers is subjective but a very necessary measure to obtain

The gro up of criteria reficrred to as e1)cienc in use otresearch resources contained three criteria the relationship to research in the international centers the degree of emphasis on tilecommodity in le current research p ogram and the incentive for the privatesector to conduct the research Some research duplicates other research croplementsand still other research bears no relationship to research at tileinternational agriculturalresearch centers For certain comniodities all three relationships may hold depending on the type of research If national research is complementary to international researchthe cost of the rational research is lower Consequently a criterion was established which places greater weight on commodities with a high degree of complementaritybetween ntin Il d international research The criterion ofdelegree of emphasis in tile current iesercli program was established because there is a cost to rapidly adjustingcurrent h uiman and physical research resources from one commodity to another The criterion of private-sector incentives was used because the social benefits from the total research program in the country will be greatest if scarce public research funds are devoted to research on commodities for which the private sector does not have an incentive to support the research itself Of coui se the private-sector producer group can tax itself and pay the public sector to conduct the research for those commodities for which it has an incentive to privately support the research These commodities tend to be export commodities because producers can capture a higher proportion of the gainsfrom productivity increases for export commodities than for commodities primarily consumed domestically

The group of criteria related to the distribution ojresarchimpacts contained two criteria in the Uruguay study number of producers and the effect of increased productivity on the price of product The larger the number of farms producing a commodity the largerthe number of farmers afficted by the research and perhaps the higher the probabilitythat smaller farmers are being affeicted The distribution of benefits depends to a significant degree on price changes resulting from increased productivity Producers benefit more if prices decline very little and consumers benefit more ifprices decline substatitially for a given production increase For export er import commodities priceswill tend to decline less than for commodities primarily produced and consumed domestically For commodities which are not traded and are very important in the diets of the poor prices will tend to decline relatively sharply when productivity increases The division of benefits between producers and consumers also is affected by the fact

that producers are also consumers Consequently in the Dominican Republic and Ecuador studies an additional criterion importance of the commodity in home-consumption on the farm where produced also was included

In summary 10-15 criteria were used in tie three studies to identify and select research priorities by commodity The linkages anong the goals and criteria described above are summarized in Figure 1

Research Area Criteria The five criteria uicel to select research priorities by research aieas were (1 ) whether the research causes an increase in the use of relatively abundant resources and a saving of relatively scarce resources (2) the number and seventy of research problems (3) non-duplication with transferable research from outside the country (4) tne extent of private-sector incentives to conduct the research and (5) current emphases in the research program

The rate of economic growth will be more rapid if new technologies are developed that utilize relatively abundant resources and save relatively scarce resources The resource base varies by region consequently information on resource abundance criteria was collected by region in tileEcuador and Uruguay studies The same was true for the quantity and severity of research problems Information on the relationship of domestic to inernational research private-sector incentives and the current emphasis in the research program was collected on a naticnal basis These criteria all relate to the income growth (efficiency) goal It is difficult to identif research area criteria that measurc whether particular types of research aifect income distribution or variability

Data Collection and Model Implementation

Information used in the analysis included both quantitative data on value of productionnumber of farms value of exports and imports person-years devoted to research on different commodities and tile of protein per day in thenumber of calories and gramsdiet (for tileDominican Republic and Ecuador studies) as well as qualitative or subjective information on such factors as probability of success private sector incentives and severity of research problems Furthermore weights had to be elicited from decision makers to place relative emphasis on the various criteria

Step 1 Develop Cotninodity and Research Area Lists -- The first step was to determine the list ofcommadities to consider and the list of research areas In the Dominican Republic information was collected on 74 commodities In Ecuador an initial list of 109 commodities on which INIAP was conducting research was discussed Throughdecisions to eliminate some and to group others this list was reduced to 44 commedities for the analysis These decisions were made by the Technical Director for Research In Uruguay a smaller list of commodities was reduced to 21 commodities or commodity groups These decisions were made in consultation with the National Research and Fxtension Directors The list of research program areas contained nine areas in the Dominican Republic 16 in Ecuador and 16 in Uruguay

Step 2 Collect Quantitative and Qualitative Data on Chosen Criteria -- Information o-i quantitative criteria were gathered from local and FAO secondary data sources and one table was constructed For each criterion Commodities were ranked for each criterion Information needed for the qualitative riteria as well as the weights to place on criteria were obtained through interviews with sc rtists and administrators at boh the national and regional experiment stations In Euador and Uruguay substantial attention was devoted to identifying the appropriate respondents to each question as this is a crucial aspect of the procedure In Ecuador those interviewed were notified one week before the interviews and 72 people were interviewed by the Planning Director of INIAP and

9

FIJR 1 CONCEPTUAL ELEHNTS IN THE HEIGHTED CRITERIA MODEL FOR PAWING RESEARCH PRIORITIES BY COtlDITY

GOALS fficiency ]

CONCEPTUAL( )CRITERIA Pri7] Probabi i+y Research Distriutjon Iincome

ii J of Succes Resource Eff o L=Acplusmn a rilbility

CRITERIA 1 Value of 1 Scientist 1Rl ofto toNumberPrcduction Estimates 1 AnnualInternational Producers Variability2 Foreign Centers 2 Price in Gro

Exchange 2 Current Effect3 Future Raturn3)Emphasis 3 HomeDemand 3 Private Consumpt ion

4 Comparative Sector Advantage Incentives

Notes

(-n future studies an additional conceptural criteria under the security goal might be health (J4tri ional criteria such as grams of potein or kilo=alories could be included as additional easures of Distributional ikipacts

4ot used in Dominican Republic Ecuador and Uruojuay studies

his assistant In Uruguay the directors of all the regional stations were called into CIIAB headquarters for a aweting to discuss the study and this was followed up byon-site interviews and meetings with scientists and directors at the regional experimentstations as well as with economists in the Ministry of Agriculture These interviews and meetings were conducted by a person assigned to work wvith the Director of Cl LAB on the priority-setting study In the Dominican Republic a survey of scientists extensionworkers and administrators was conducted by the ISNAR consultant from the Instituto Superior de Agricultura who visited the various experiment stations Less care wastaken in the Dominican Repubic than in Ecuador and Uruguay to match the questionspertaining -o individual criteria to particular individuals This probably reduced theusefulness of the responses because those most familiar with the probability of research success For example may not be the same -s those most Familiar with comparativeadvantage

Step 3 Elicit 11eihts on Critria -- Relative weights to place on the different criteria were obtained Fronm national and regional research system administrators In Ecuador34 peopic verc used to determine the weights These wveights were established separatelyIor the commoditv and the research area criteria In 1iuguay fewer people (sevensystem and station directors) were used to determine the weights and more attention was devoted to grouping the criteria pertaining to particular goals A Delphi procedure was used in which the s7even directors were shown the average of the group and provided with an opportunity to adjust their weight

Step 4 )erive Rankinqs hv Commodity and By Research Area -- Once the basic information was collected axd organized into a seies of tables one procedure was followed to arrive at a final ranking of research priorities by commodity A second procedure resulted in lists of priorities by research area In the Dominican Republicmost of the calculations were made by hand in Ecuador and Uruguay microcomputerspreadsheet programs were used to assist with the calculations and to conduct sensitivity analysis

The steps in the procedure for ranking research priorities by commodity and research area are sumnmaried below and are illustrated with a simple example in Appendix 1 Brieily the steps for ranking commodities were

a) Th conniodities were ranked for each criterion for which quantitative data were available

b) Tihese rankings were multiplied by the weights assigned to each criterion and then the weighted criteria were added across to arrive at one sub-ranking for the

Luantitative criteria c) Each commodity was given a high low or none designation for each qualitative

(subjective) criterion The response which implied a need for greater research priority was assigned the number 2 the intermediate response was assigned I and the response which implied lower research priority was assigned 0 1herefore each comniodity received a 2 1 or 0 for each criterion

d) The weights assigned to each qualitative criterion were then multiplied by these numbers and the results were added across all criteria to arrive at a subtotal for each commoditv These subtotals were ranked from highest to lowest to provide a second sub-ranking

e) The sub-rankings for the quantitative and qualitative criteria were then given their corresponding weights and added together to arrive at a final ranking by commodityThe steps for ranking research priorities by research area were the same as steps (c)and (d) above because the criteria were all qualitative

Step 5 Analsis and Interpretationof Resuls -- In the Dominican Republic the resuLts of this weighted criteria analysis were uscd to determine a small set of cormodities and

II

research areas with the highest priority Further assessment was then made of humanphysical administrative and other resources needed to structure research programsfocused on these commodities and resea-ch topics In Ecuador and Uruguay the prioritized list of commodities was split into a high-priority group an intermediate-priority group and a low-priority group Research area priorities were identified for each region of tile country A more detailed analysis was then made of the human physical and other resources needed to support the high and intermediate priorities

In summary five major -rens were involved in operationalizing the weighted criteria models A complete set of rsults for each study included more than 20 tables and are not reproduced here Ilowever the final rankings for commodities and for research areas are shown in Tables I and 2 in Appendix 2 In the Dominican Republic a team of consultants followed up the initial priority-setting exercise with additional analysisand discussions at the experiment station level and eventually recommended tile establishment of five national commodity programs (rice cornsorghum beans coffeeand livestock) and one additional major research program (natural resources) In Ecuador the prioritized lists were widely distributed and discussed in the Ministry oi Agriculture the Board of FEIDIA the Commission on Science and Technology arid in AID There was concern in Ecuador with the weights placed on the individual criteria and therefore additional sensitivity analysis was conducted by the Planning Office of the Ministry There seemed to be a recognition in Ecuador that priority setting with a weighted criteria model is an iterative process and that much 4t the models value stems from tile discussion of criteria among the decision makers and from the sensitivityanalysis The high-priority list of comniodities in Ecuador included thL ten highest-priority commodities listed in Table 2b A project paper has been written forAI) which recommends funding For Fli)IAlbr three commodities (coffee soft corn for the Sierra and dairy) Coffee and soft coitn were the number 2 and 3 ranked prioritiesin tile weighted criteria model and dairy was number 9 although it was the top-rankedlivestock commodity

In Urugu ay the results from the first run with tile model were discussed with the research directors Ilie directors made small changes to the weights placed on criteriaand the mode was rerun resulting in the prioritized list in Appendix 2 Personnel in CIAAB are CUrrcwtv undertaking additional sensitivity analysis and with tie assistance of ISNA R are developing a plan for implementation

An Assessment of the Weighted Criteria Models Used in the Dominican RepublicEcuador and Uruguay

The Dominican Republic Ecuador and Uruguay studies were perhaps the first majorattempts to implement weighted criteria models with weights explicitly elicited from decision makers in NARS in less-developed countries A number of strengths andweaknesses in the procedures were identified during the analyses Some of thL weaknesses were corrected fron one study to the next some can be corrected in future studies and others are inherent in the approach

Trhe first strength mentioned earlier is the ability of the procedure to systematicallyincorporate both quantitative and qualitative information related to a set of multiplegoals and criteria in order to prioritize a lorng ist of commodities and research areas in a relatively short period of time Second the procedure proved relatively easy for both research administrators and the loc- -nalysts to understand with the exception of the )ominican Republic where the research administrators were not directly involved7

Third the analysis as applied in Ecuador and ULuguay with the direct involvement of research system administrators at various stages forced those decision makers to

12

consciously identifv and trade off goals and criteria Fourth the use of spreadsheet programs in EcUadfor and Uruguay facilitated sensitivity analysis after the initial set of priorities was deermimd Fifth the procedure provided a relatively objectiveassessment of prioriies because individuals were not allowed to rank commodities orresearch areas directly but had to weight criteria

The first weakness inherent in ex ante research priority setting and thus in theprocedure is that there is a large amount of subjectiveness Although tileapproach isless subjective than unstructured judgment there is subjectiveness in the responses toquestiorns related to some of the criteria and also ini the weights placed by decision makers on the criteria Second it proved diflicult to specify independent criteria with no overlap Third while the procedure required less time than an expected economic surplus analysis by commodity it required more time than asking respondents to directlyrank commodities and research areas In E-cuador the consiultant worked withpersonnel in INIAP and FEI)IA for one week at the start of the analysis to determinegoals and criteria to arrive at atlist of commodities and researh areas to list secondarydata needs to formulate questions to be asked of administrators and scientists to decide who should be asked particular questions and to explain the procedure itself includingthe spreadsheet program The required iidormation was collected over the Followingmonth and the consultant returned and worked with the Planning Director in INIAP for one week to conduct the analysis and write tip the results Revisions to the initial draftof the report we-e then made over the lollowing weeks A similar procedure to the onefollowed in Ecuador was used in l_ruguay althaugli more time was spent discussing the wording of questions to be asked of scientists Furthermore n Uruguay more care was taken to match the criteria with the goas

Certain of the criteria wil always be diflicult to eplain to tileinterviewees Thecriterion on comparative advantage is difficult for some respondents because of a lack of knowledge about neighboring countries If suflicient data time and resources are available this criterion may be rnea4ured more precisely using comparative advantage or domestic resource cost procedures suggested by longmire and Winkelnann Anotherdiflicult criterion to estiniate is the potential foture demand for the product The respondents in Fcuador were asked to consider how the demands for different foodswould change as income and populatiot grow Many of them had a hard timeconceiving that the total or per capita demand for certain Foods could actually decline as income grew In Uiruguay (ibe cmiparative advantage and future dema nd questions were answered by a small group of economists from the ministries of agriculture and trade In the Ecuador study more attention was devoted to assessing the most appropriate people for answering different questions than was the case in the Dominican Republic Ilowever some criticism surfaced about the weights placed on criteria InUruguay decision makers were given the opportunity to change their weights afterviewing the initial results This was very useftl because it demonstrated the implicationsof placing different weights on the various goals and criteria

Computer spreadsheet programs were a major help in Ecuador and Uruguay Additional work needs to be devoted to developing a more mentnu-driven prcgram but the people inthe planning office of INIAP were able to use the SuperCalcR program without too tmucht difliculty despite ialack of previous experience Ihe analyst in Uruguay was veryfamiliar with lotusl which proved most helpfil

It became clear from working inall three countries that it is preferable to work directlywith the final decision makers or their designees Inthe l)ominican Republic the study was conducted by a very competent set of consultants from the local agricultural universi ty but the procedure was not institutionalized to allow for additional sensitivityanalysis or future priority-setting efforts InEcuador and especially in Uruguay the

13

decision makers were directly involved conducted sensitivity analyses and were able to assess why certain commodities or research areas received high priority as a result of the analyses In both Ecuador and Uruguay they have a basic understanding of the strengths and weaknesses of tileprocedure and therelre can explain their subsequentdecisions to groups who might disagree with tilepriorities they establish

The corntrast between the Dominican Republic and tie Uruguay studies illustrates a veryimportant point No matter which technique is used to structure the analysispriority-setting exercises must integrate the decision makers (users) into the processPriority-setting procedures must be institutionalited the results of initial analysisdiscussed thoroughly with decision makers and sensitivity anilsis conlducteU alone the lines suggested by that discussion Tecinological change generates a surplus which can be appropriated by dilkircnt sectors or groups within the economy Who benelitdepends in part on the cotaniodities selected for research and the types of technoloLies generated One of the purposes of institutionalizing a formal prioritN- setting procedureis to help mediate conflicts that may arise (for example between producer groups and consumers) as a result of research-induced improved technologies

Other issues arose during the analyses First the initial grouping and elimination of certain commodities and research areas before lhe irialvis is an important step but one which must be taken with carelul consideration of possible coniplmentarities in research on particular productS or types of rearch Second some of the commodities such as forages are inputs into other commodities such as milk (Corseqtentlv these commodities llay have to he considered itscinps because it is the value-added and not tilegross value of output that is relevant when one coniiodity requires other commodities sinputs and when criteria such as value of production are used This was an issue in I ruguav where livestock are quite important liird little attention was devoted in atv of the studies to defining appropriate criteria Ior the increased inoome stability eoal Future studies if this be an ideltJiiLd LOal shOUld define risk criteria

In sutImary the weighted criteria model as applied in Lcuador and iuguav which had the advantage of bit ildin Oillessons lCarICd illdie l)ominican RCpublic appears to have been a uselul priority-setting tool It ha its ereatest advartage ill situations ill which there are rulItiple goals to consider a relatively major reassesnieiit of prioritiesis contenplated and the list of commodities dd resen rch areas to consider is largeWeighted criteria models described in this paper were applied in small countries with limited resources but with direct involvement (except in the Dominican Republic) ofthC major decision makers These models itav be most useful in the small country situationwhere decision mAkers are more likely to be involved in tile priority-settirt process and are not just using tables of results Perhaps in larger systet7 where decisioii makine is more decentralited tileweighted criteria model can he applied at the regional level before relining research priorities at the national level

Another factor that can be uniortant in large svvtern is tle complerientarity between resea-h aid educat ion [hie larger the svstcnt the clser the linkages tend to be between universities and tileresearchi system Where universities are important tor agriclltural research an aidditional criterion may hc rteedCd to place value on the educationlal benefits of certain types of reseirch

Ihere are several additional iniplications with respect to how weighted criteria models relate to alterniative priori t-setting Inethod parIicularl the expected economic surplusapproach Befoce exploring these relationships kcx aspects of the surplus approacliare described based on aii application in Peru

14

Priority-Setting Mechanisms for National

Agricultural Research Systems

Present Experience and Future Needs

Recent trends towards shrinking real research budgets or many national agriculturalresearch systems (NRS) have stimulated several attempts to evaluate tileeconomic benefits ofaricultural reserch Manv of these quantitatis e economic evaluations haveemployed historical data to provide ex post assessments and a few hase used subjectivejudgments to project future research benefits [he results kthese analyses have been used at the national or inter-ministerial level to justilf the need for continued political support for agricultural research

Tighter rese rch budgets not only heighten the iced to justifV agricultural research vis a vis alternative public investments but they alVo make it important for NARS to improve procedures for setting priorities among competing research programs A varietyof strategic and project-level decisions are made within NARS The objectives of national policvmakers the needs of agricultural producers and scientific opinion must each be brotight to bear on strategic decisions to emphasize particular commodities and research program areas uld on research project selection

The purpoe of this paiper is to discuss kev elements for inclusion in improved research priority-setting procedures for NRS Ihe fLcus is on methods for strategic as opposedto more tactical projcct-le el decision m kingAfter discussing the context of research priority setting a brief summary is provided of tile iain types of research resource alloL-ition mnetlods Recciit cxperiences with priority setting in four Latin American NAR the l)ominican Republic lcuador lUVUav and Peru are compared These experiences can proside sonc guidance for the development of an improved research resource allocation procedure

I TilE CONTEXT OF RESEARCH PRIORITY SETTING

NARS decision mnkers often operating within the ministry of agriculture an agricultural research institute or as a research council make allocative choices based on their knowledge atnd prior cxperience That knowledge and experience frequentlyincludes an understar one of national and regional oals and objectives as voiced through the political process a sene o lie severity of particular types of research problems and a general feel tor vlat is achievable through research It also includes an awareness of the desires ( te-ressures) of producer groups Decision makers must decide hoW much cmpha sisto jte onparticular commodities (eg rice maize beanssheep) and types of research c v varietal improvenmnt improved livestock nutritionplant protection farming svwtes Implicit in those decisions are the locational emphasis of research the focus 0rt paJticullar factors of production (eg land laborwater) the emphasis placed Ol lone-r-term haisic versus shorter-term applied and adaptive research and the distrihotianal elfects of research on farm size on producers versus coniumners and on people atidiherent income lec

In many NA RS research resoce alocation decisions are heavily influenced by the previous years budget2 (hanges of~tn result from requests by scientists which areevaluated reLatively informally and aggregatcd into an overall plan The resulting plan

upon close scrutiny may contribute little to tire attainment of stated national goals and objectives

Judgment based upon prior knowledge and information provided by scientists is crucial for research resource alincation decisions In addition rapid or radical changes in research programs -an be costly Particularly as research budgets tighten howeverforcing changes in tile systein tile increased use of quantitative methods may be nocessary to imprwe thL objectivity of those judgments The aim is to foster consitencv of research priorities with goAs and objectives and to improve the efliciency of the research system in mser ing producer and cortsumer needs The idea is not to replacejudgment but to increase and organize the inflormation available for updating priorknowledge and beliefs The hope also is to inject continuity in the priority-settingprocess and to proide NARS decision makers x ith methods the find lielplib when rationalizing their decisions to scientists producer Erotps and politicians Docunentingthe decision proces through a strctured procedure can help minimie the possinilitvof sudden and large shilts in researh priorities in those systems characterized by a flairlv rapid turnover of research administrators

Many quantitative priority-setting procedures are avaiia ble but few have beeninstitutionalized into the decision-making practices of NA RS manaers A cruciZl factor in their non- use of these procedures urtdoubtcdlv as been the lack of ri orous vetcost-eflectie approaches which can irtcorporate tile large number of comtodities andresearch areas as well as the multiple go1ls a ld criteria found in most strateoicdecision-muakirt situations inl NRS FurtIhcrrtore the cortmputer capabiliv available through microcomputers in most developing-co ntry NRS today was not gcenerallyavailable even two or three velars ago

In this paper procedure amc ce crihed which were recently applied in four latinAmerican countries First however previously emploved priority-setting mechanisms are briefly reviewed id critiqued One of rite points that will Ie made at the end of this paper is tiLit tise methods are not nitually excilsive There is no siTele magic huletbut there appears to be real potential for developing a Ilexible computer-assisted priority-setting framework that will allow NARS administrators to call ott theappropriate tectiiques tot aSsist withit a particular decision depending ott tie time and financial resources a ailahle and the relative itiportance of the allocation decision Asecond major point is that priority setting is a process not simply a procedure and must be instit utiortliied as part of the overall process of research managementPriority-setting technritiuetes applied only once by people outside of the decision-making process have little valtie

If SUMMARY OF PRIORITY-SETTING METHODS

Methods reported itt previous studies for research priority setting include

I ) establishtert td weighting of multiple criteria for ranking commodities and research ares with a final aggregate ranking based on implicit or explicit weights

2) use of bertefit-cost analysis including expected economic surplus techniques to select cotitritodities and research areas

3) application of mathematical programming to choose an optimal research portfolioincorporatirng multiple goals md constraits

4) developrert and tise of simulation models

These four approaches are discussed below followed by a brief assessment of their advantaiges atil dis idsittages

2

Weighted Criteria Models

Several studies hae established multiple criteria for ranking priorities because of tiledesire to explicitly consider a wide variety of factors that do or perhaps should influenceresearch selection The relative weights attached to each criterion to arrive at a final listof research prioriries are sometimes left implicit or unstated A recent example is tileTAC review of priorities and future strategies for the Consultative Group onInternational Agricultural Research (CGIAR) In assessing priorities by commoditythey etablished a principal goal research objective to obtain the goal and a series ofcriteria organized into three groups relevance research productivity and efliciency ofhe CGIAR system in undertaking research Detailed tables with quantitative andqualitative information and rankings of commodities for each criterion were presentedIn developing the recommendations for hort- to medium-term funding alternatives atthe conmmodity level the weights used to aggregate across criteria were left implicit

There are examples of tudies that have incorporated multiple criteria but also explicitlyspecified and utilized a set of weights to aggregate across criteria and obtain a finalranking of research priogrities Appications of this method often called the scoringmodel approach are found in studies by Williamson lahlstede Paulsen and KaldorShumway and McCrack en and Von (Oppen and Ryan A flew studies have used congruence analysis which can be thought of as a special case of a sci-ring or weightedcriteria model with all the weight placed on the criterion of vaue of production

Benefit-cost (Expected Economic Surplus) Analysis

The benefit-cost approach to welecting research priorities has been used in differentforms (Fishel Arai Sire and Gardner l)avis Oram and Ryan Norton Ganoza andPomareda) Most studies have employed consumer-producer surplus analysis and havebuilt upon a large body of literature on ex post research evaluation Ex ante analysesusually incorporate expert opinion to determine projected research impacts adoptionrates and probabilities of research success and provide estimates of the economiceffliciencv and distributional implications of agricultural research resource allocationBenefit-cost studies typically calculate benefit-cost ratios internal rates of return andnet present values for alternative types of research or research onfor differentcommodities Ilse analyses may include andmav or not regional internationalresearch spillovers and the eflects of domestic pricing policies on research benefits

Mathematical Programming

Mathematical programming is another alternative for research selection It relies onmathematical Optimization to choose a research portfolio through maximizing amultiple-goal objective function given the resource constraints of the rescaich systemA good example of the use of this method is a study by Russell in the United Kingdomlie used goal programming to maximize the contributions of the research program toseveral goals given the con taints of budget human resources state of knowledge andcertain policies This procs Ue uses similar information to the weighted-criteria modelbut selects an optimal research portfolio rather than simply ranking research areas

Simulation

Finally the simulation method has been used to identify and select research prioritiesSimulation models vary in their construction but a good example is the model byPinstrup-Anderscn and Franklin They built a model to project the contributions andcosts of alternative research activities They established gals and then identifiedchanges in supply demand for inputs and demand for output needed to meet those

3

goals They identified neeled technologies time and financial costs and the probability of research success and adoption Finally they specified the scientists workingobjectives This model is very thorough but did require extensive amounts of data and estimation of several mathematical relationships

Comparative Assessment

The above discussion is a brie summary of the major structu al approaches that have been used to asist decision makers in research priority setting All of them attempt to go beyond the usual procedure of individual or group decision makin e without structural analyses Each of the approacles has its advantages and disadvantages (See Fable I)

The procedure of establishing criteria and using weights to arrive at a final set of research priorities has the advantage of forcing decision makers to consciously trade off multiple goals It can illcorporae both quantitative and qualhtative information and can be applied to a long list of con mnodities or research areas in a relatively liort period of time The procedure is relati cx ca sY fbr administrators to understand but does requiretheir time in obtaining the explicit w5eights for criteria Furthermore these weights are inevitably subjective and their elicitatlion must be carefully structured [lie nActhod also requires scientists time in collecting inlormation on qualitative criteria As a result the approach k better suited for periodic or major priority-setting efforts than or situations where frequent marginal changes are anticipated

The expected economic surplus approach his the major advantage of incorporatingseverml criteria related to economic elficiency and distribution into one or two measures It also can be ucd to examine the gencral equilibrium ellects ofresearch (Ramailho de Castro and Sclhh nAid tile benefits of research under alternative possibly distortionar domestic pticing and international trade policies tEdwarts and Freebairn Alston Edwards and rcecbairn Norton and (ianoza 1985) These alictors are pervasive in developing countries and affect the efficiency and distributional consequences of researh Ile procedure ie(liCe ir lel - tndcrstandw ofeconomic analysisand more anal-t s time than the weighted criteria model but less aduinistrators time It can be diflicult to apply to at large number of commodities or research areas because certain typcs of data necessary for the ainalysis often do not exist for all commodities With adequaIL tL d ita Cxpected economic surplus analysis can be incorporated into the weighted criteria model or could be used on the set of commodities which the weightedcriteria model imdicaMes to have the highest piority lhe latter approach would allow for the calculation of income foregone as a reult of placing weights on non-economic ef1ciency criteria Expected economic surplus analysis has the major advantage of calculating a rate of return which can be compared to alternative public investments

The mathematical programming approach is similar to the weighted criteria model approach becaue weights are placed on a set of goals or criteria The procedure has the advantage of explicitly considering the budget huLman resource aad other constraints on the research system tnless the constraints are well specified including possiblechanges over time liowever there is a risk of nonsense solutions The model is more intensive of an ecolonic analysts time and abilit than tie simpler weighted criteria approach and decision makers may be less willing to accept what appears to be a black box solution lrade-offs among goals are easily quantified ssith this approach

The advantage of simulation models is their tlexibility They can be constructed is relatively simple or complex tools can incorporate optimizing or ranking proceduresand can readily include probabilistic information Their major disadvantage is that to be useful they must be relatively complex and typically reqtuire extensive amounts of both data and time of skilled analysts

4

TABLE 1 COMPARISON AMONG MAJOR AGRICULTURAL RESEARCH PRIORITY-SETTING METHODS

PRIORITY-SETTING METHOD

HEIGHTED EXPECTED MATH CHARACTERISTIC CRITERIA ECON SURPLUS PROGRAM SIMULATION

OPERATIONAL CONSIDERATIONS

1 Relative cost in researchers time mudium mJium medium high2 Relative cost in priority setting analysts ime medium medium high high3 Relative cost in administrators time medium mediummedium medium 4 Relative overall data requirement medium medium medium variable 5 Relative ease of comprehonsion by decision maker high medium low low 6 Ease of incorporating subjective informaticn high high high high7 Ease of incorporating non-quantitativa inforiation high low low medium

GOAL-RELATED ISSUES

a Requires explicit elicitation of goals yes usually yes uiually9 Can determine distributional affects on consumers

and producers at various income levels no yes no yes10 Can handle uncertainty yes yes yes yes11 Can consider tradeoff among multiple goals yes sometimes yes yes

CRITERIA-RELATED ISSUES

12 Can consider private-sector research incentives yes difficult difficult yes13 Can consider economic policy and trade affects yes yes yes yes

EVALUATION-RELATED ISSUES

14 Can be used to sot priorities for research at the aggregate level no yes no es

15 Can be used to set research priorities at the commodity level yes yes yes yes

16 Can be used to set priorities for non-production or non-commodity oriented research yes difficuli yes yes

17 Can be used to set priorities for basic research yes difficult no sometimes 18 Can evaluate secondary impacts of rosuarch on

employment environment nutrition yes sometimes so etimes yes19 Usually estimates a rata of return to research yes sometimesno no 20 Can quantify geographic spillover effects no yes no yes 21 Can consider the lags involved in research

and adoption yes yesyes yes22 Facilitates priority setting when the number

of commodities is large yes difficultdifficult difficult

Source Adapted and modified from Norton and Davis

5

The above is a very brief summary of the major advantages and disadvantages of the most common formally structured priority-setting models that could be used for strategicplanning for research resource allocation Other approaches exist for research projectselection with perhaps models that attempt to identify and analyze key vield constraints being the most prevalent approach employed in NARS of less-deeloped countries

The results of ex post analysis also can provide useful guidance for ex ante research resource allocation decisions if appropriately incorporated into a systematic ex ante procedure The most common ex post approach in addition to cx post benefit-cost analysis is the econometric estimation of production or supply functions incorporating research variables To be most uteful for ex ante analysis econometric approaches mLlSt be applied with a high degree of disaggrega tion Unfortunateiy these models require a substantial amount of high-quality historical data on production farm inputs and research expenditures

Perhaps the two approaches with the most potential given additional refinements for application by NARS in less-developed countries are the weighted criteria models and the benefit-cost or expected economic surplus procedures In the next section reent experiences with applying weighted criteria models in three Latin American Countries )ominican Republic Ecuador and Uruguay are discussed and evaluated The purpose

it to piovide more iniormation about the procedures their degree of acceptance and possible improvements in the models Ihen application of expected economic surplus models are considered drawing cn a recent cx ante research evaluation studly in Peru Finally the relationship between weighted criteria and expected economic surpluts methods is described along with possible extensions

IIl THE USE OF WEIGHiTED CRITERIA MODELS IN TIlE DOMINICAN REPUBLIC ECUADOR AND URUGUAY

in 1986 iif I87 weighted criteria models were developed and implemented in the lDomin lRepublic (ISA) Ecuador (Espinosa et al) and Trugua (CIAAB)to assist with priority setting for agricultwral research In the I)ominican Republic the study was carried out at the Instituto Superior de Agnicultura ISA) by consultants for ISNAR In Ecuador the study was conducted by the plmnintg ollice oY the Instituto Nacional de Investigacion Agropecuaria (INIAP) with the assistance of consultants for the US Agency for International Development (AID) In Uruguay the study was conducted by the Ollice of the Director of the Centro de Investigacion Agricola Alberto Boergcr(CIAAB) with the assistance of ISNAR The purpose of all three studies was to applyprocedures for prioritizing agricultural research by commodity and by major research area Inthe Dominican Republic the client was the newly created natonal agriculturalresearch agency (Il)EA) In Ecuador the client was INIAP and the newly created foundation for agricultural research (IEl)IA) In Urugua the client was C1IAB

The procedures employed in the three studies were similar but with some importantdilfereices as the Ecuador study built on lessons learned inthe I)ominican Republic and the Uruguay study built on other lessons learmed in Ecuador National goals for the research system iere elicited in each country and a series of criteria establishd which relate to those goals Separate riteria were developed for conmitodities and or research areas and weights were elicited from decision makers to establish the relative importanceof the criteria Commodities and research areas were ranked according to each criterion and these rankings were multiplied by the elicited weights to arrive at research priorities

6

Goals

Three goals were identified in each of the three countries (1) to raise the average levelof income in the country (2) to increase the well-b(Aing of low-income groups in societyand (3) to reduce yearto-year income fluctuations in the country especially onl tiledownside These goals were worded slightly diffrcntly in each country but in each casethey essentially relate to the three goals referred to in previous studies as efficncy equity and security In the l)ominican Republic the local consultants identifiedthese as the goals while in lcuador and Uruguay it was done by the directors of national research agencies

Criteria

The next step was to establish criteria to as measures of whetheruse particularcommodities or research areas contribute to the attainment of the above goals A largeset of criteria was discussed in each country and a total of 15 criteria were eventuallyused in the Dominican Republic 14 in lEcuador and 10 in Uruguay to determine priorities by commodity Five criteria were used in each case to determine priorities byresearch area Refinements were made in successive studies - beginning with theDominican Republic then Ecuador followed by Iruguay - to increase the independenceof criteria and to remove criteria that were questionable measures of whether researchcontributed to tile tated goals ( Reason for dropping particular ci eria are discussedbelow) Refinements also were made with respect to the grouping of criteria to facilitatethe weighting of goals and criteria

Commodity Criteria omioditv criteria were grou l - J into four conceptual groupsproduct importance probability O success elliciencv in use of research resources anddistribution of impacts The first three of these groups relate to the efficiency or incomelevel goal and the last group relates to the equity or distributional goal In none of thethree studies were criteria included which represnted the third goal (reduced incomelluctuation [lie feeling was that the entire set of criteria as a group would serve toreduce the enlhasis placed on single or 1ew (often export) crops While this appearedto be the case in the results of thee particular studies ft ture weighted criteria studies may want to add criteria which explicitly rank commodities from lowest to hiehest withrespect to annual gross income variability (or price and yield variability separately)

The group of criteria referred to as jrodct importap contained Four criteria in theUruguay study value of production generation or saving of foreign exchange expectedfuture demand change and comparative advantage

The value of productionl is an important criterion because the cost of rseaich isbasically independent of the number of units alkected by the research results at least forthose commodities with relatively homogeneous production conditions Foreignexchange was used as a criterion because of a perception in each country that the lackof foreign exchange was impeding growtb A criterion was set up to allow more weightto be paced on those commodities for which demand is expected to increase in thefuture because the importance of commodities will change as diets are altered withincome growth aod a world markets change Comparative advantage was includedbecause income in the country will be higher if the country focuses its efforts oncommodities for which its resource base is best suited while recognizing that research itself can alter resource constraints

In the Dominican Republic and in Ecuador nutritional measurcs (calorie and proteincontributions to the diet) were included as criteria for product importance They wereincluded because the well-being of low-income groups may be affected by the availability

7

of foods which are important in their diets Nutritional criteria were dropped in Uruguay however because numerous studies have shown that income levels are tile primary determinants of nutrition levels rather than the availability of particular foods Certainly caloric and protein content are debatable criteria To the extent that increased domestic production lowers the prices of certain food commodities therebyincreasing food consumption by the poor nutritional criteria can contribute to the distributional goal Land area also was included as a criterion in the Dominican Republic study but was dropped in Ecuador and Uruguly because of the overlap with value of production

Two criteria were used in each study to measure 1robahility of reseatch success The first measure was tilegap between own-country yields and neighboing country yields The second was the potential for success indicated by the researchers themselves The rationale for the gap criterion was that the larger the yield gap between the home country and similar countries tle greater the potential for yield gains The criterion is rather crude because yield gaps also can indicate a relative resource disadvantace forproducing the commodity even if the countries appear to be geoclimatically sinilar Consequently the gap criterion was eventually dropped in the tUruguav study The potential for success as indicated by the researchers is subjective but a very necessary measure to obtain

The gro up of criteria reficrred to as e1)cienc in use otresearch resources contained three criteria the relationship to research in the international centers the degree of emphasis on tilecommodity in le current research p ogram and the incentive for the privatesector to conduct the research Some research duplicates other research croplementsand still other research bears no relationship to research at tileinternational agriculturalresearch centers For certain comniodities all three relationships may hold depending on the type of research If national research is complementary to international researchthe cost of the rational research is lower Consequently a criterion was established which places greater weight on commodities with a high degree of complementaritybetween ntin Il d international research The criterion ofdelegree of emphasis in tile current iesercli program was established because there is a cost to rapidly adjustingcurrent h uiman and physical research resources from one commodity to another The criterion of private-sector incentives was used because the social benefits from the total research program in the country will be greatest if scarce public research funds are devoted to research on commodities for which the private sector does not have an incentive to support the research itself Of coui se the private-sector producer group can tax itself and pay the public sector to conduct the research for those commodities for which it has an incentive to privately support the research These commodities tend to be export commodities because producers can capture a higher proportion of the gainsfrom productivity increases for export commodities than for commodities primarily consumed domestically

The group of criteria related to the distribution ojresarchimpacts contained two criteria in the Uruguay study number of producers and the effect of increased productivity on the price of product The larger the number of farms producing a commodity the largerthe number of farmers afficted by the research and perhaps the higher the probabilitythat smaller farmers are being affeicted The distribution of benefits depends to a significant degree on price changes resulting from increased productivity Producers benefit more if prices decline very little and consumers benefit more ifprices decline substatitially for a given production increase For export er import commodities priceswill tend to decline less than for commodities primarily produced and consumed domestically For commodities which are not traded and are very important in the diets of the poor prices will tend to decline relatively sharply when productivity increases The division of benefits between producers and consumers also is affected by the fact

that producers are also consumers Consequently in the Dominican Republic and Ecuador studies an additional criterion importance of the commodity in home-consumption on the farm where produced also was included

In summary 10-15 criteria were used in tie three studies to identify and select research priorities by commodity The linkages anong the goals and criteria described above are summarized in Figure 1

Research Area Criteria The five criteria uicel to select research priorities by research aieas were (1 ) whether the research causes an increase in the use of relatively abundant resources and a saving of relatively scarce resources (2) the number and seventy of research problems (3) non-duplication with transferable research from outside the country (4) tne extent of private-sector incentives to conduct the research and (5) current emphases in the research program

The rate of economic growth will be more rapid if new technologies are developed that utilize relatively abundant resources and save relatively scarce resources The resource base varies by region consequently information on resource abundance criteria was collected by region in tileEcuador and Uruguay studies The same was true for the quantity and severity of research problems Information on the relationship of domestic to inernational research private-sector incentives and the current emphasis in the research program was collected on a naticnal basis These criteria all relate to the income growth (efficiency) goal It is difficult to identif research area criteria that measurc whether particular types of research aifect income distribution or variability

Data Collection and Model Implementation

Information used in the analysis included both quantitative data on value of productionnumber of farms value of exports and imports person-years devoted to research on different commodities and tile of protein per day in thenumber of calories and gramsdiet (for tileDominican Republic and Ecuador studies) as well as qualitative or subjective information on such factors as probability of success private sector incentives and severity of research problems Furthermore weights had to be elicited from decision makers to place relative emphasis on the various criteria

Step 1 Develop Cotninodity and Research Area Lists -- The first step was to determine the list ofcommadities to consider and the list of research areas In the Dominican Republic information was collected on 74 commodities In Ecuador an initial list of 109 commodities on which INIAP was conducting research was discussed Throughdecisions to eliminate some and to group others this list was reduced to 44 commedities for the analysis These decisions were made by the Technical Director for Research In Uruguay a smaller list of commodities was reduced to 21 commodities or commodity groups These decisions were made in consultation with the National Research and Fxtension Directors The list of research program areas contained nine areas in the Dominican Republic 16 in Ecuador and 16 in Uruguay

Step 2 Collect Quantitative and Qualitative Data on Chosen Criteria -- Information o-i quantitative criteria were gathered from local and FAO secondary data sources and one table was constructed For each criterion Commodities were ranked for each criterion Information needed for the qualitative riteria as well as the weights to place on criteria were obtained through interviews with sc rtists and administrators at boh the national and regional experiment stations In Euador and Uruguay substantial attention was devoted to identifying the appropriate respondents to each question as this is a crucial aspect of the procedure In Ecuador those interviewed were notified one week before the interviews and 72 people were interviewed by the Planning Director of INIAP and

9

FIJR 1 CONCEPTUAL ELEHNTS IN THE HEIGHTED CRITERIA MODEL FOR PAWING RESEARCH PRIORITIES BY COtlDITY

GOALS fficiency ]

CONCEPTUAL( )CRITERIA Pri7] Probabi i+y Research Distriutjon Iincome

ii J of Succes Resource Eff o L=Acplusmn a rilbility

CRITERIA 1 Value of 1 Scientist 1Rl ofto toNumberPrcduction Estimates 1 AnnualInternational Producers Variability2 Foreign Centers 2 Price in Gro

Exchange 2 Current Effect3 Future Raturn3)Emphasis 3 HomeDemand 3 Private Consumpt ion

4 Comparative Sector Advantage Incentives

Notes

(-n future studies an additional conceptural criteria under the security goal might be health (J4tri ional criteria such as grams of potein or kilo=alories could be included as additional easures of Distributional ikipacts

4ot used in Dominican Republic Ecuador and Uruojuay studies

his assistant In Uruguay the directors of all the regional stations were called into CIIAB headquarters for a aweting to discuss the study and this was followed up byon-site interviews and meetings with scientists and directors at the regional experimentstations as well as with economists in the Ministry of Agriculture These interviews and meetings were conducted by a person assigned to work wvith the Director of Cl LAB on the priority-setting study In the Dominican Republic a survey of scientists extensionworkers and administrators was conducted by the ISNAR consultant from the Instituto Superior de Agricultura who visited the various experiment stations Less care wastaken in the Dominican Repubic than in Ecuador and Uruguay to match the questionspertaining -o individual criteria to particular individuals This probably reduced theusefulness of the responses because those most familiar with the probability of research success For example may not be the same -s those most Familiar with comparativeadvantage

Step 3 Elicit 11eihts on Critria -- Relative weights to place on the different criteria were obtained Fronm national and regional research system administrators In Ecuador34 peopic verc used to determine the weights These wveights were established separatelyIor the commoditv and the research area criteria In 1iuguay fewer people (sevensystem and station directors) were used to determine the weights and more attention was devoted to grouping the criteria pertaining to particular goals A Delphi procedure was used in which the s7even directors were shown the average of the group and provided with an opportunity to adjust their weight

Step 4 )erive Rankinqs hv Commodity and By Research Area -- Once the basic information was collected axd organized into a seies of tables one procedure was followed to arrive at a final ranking of research priorities by commodity A second procedure resulted in lists of priorities by research area In the Dominican Republicmost of the calculations were made by hand in Ecuador and Uruguay microcomputerspreadsheet programs were used to assist with the calculations and to conduct sensitivity analysis

The steps in the procedure for ranking research priorities by commodity and research area are sumnmaried below and are illustrated with a simple example in Appendix 1 Brieily the steps for ranking commodities were

a) Th conniodities were ranked for each criterion for which quantitative data were available

b) Tihese rankings were multiplied by the weights assigned to each criterion and then the weighted criteria were added across to arrive at one sub-ranking for the

Luantitative criteria c) Each commodity was given a high low or none designation for each qualitative

(subjective) criterion The response which implied a need for greater research priority was assigned the number 2 the intermediate response was assigned I and the response which implied lower research priority was assigned 0 1herefore each comniodity received a 2 1 or 0 for each criterion

d) The weights assigned to each qualitative criterion were then multiplied by these numbers and the results were added across all criteria to arrive at a subtotal for each commoditv These subtotals were ranked from highest to lowest to provide a second sub-ranking

e) The sub-rankings for the quantitative and qualitative criteria were then given their corresponding weights and added together to arrive at a final ranking by commodityThe steps for ranking research priorities by research area were the same as steps (c)and (d) above because the criteria were all qualitative

Step 5 Analsis and Interpretationof Resuls -- In the Dominican Republic the resuLts of this weighted criteria analysis were uscd to determine a small set of cormodities and

II

research areas with the highest priority Further assessment was then made of humanphysical administrative and other resources needed to structure research programsfocused on these commodities and resea-ch topics In Ecuador and Uruguay the prioritized list of commodities was split into a high-priority group an intermediate-priority group and a low-priority group Research area priorities were identified for each region of tile country A more detailed analysis was then made of the human physical and other resources needed to support the high and intermediate priorities

In summary five major -rens were involved in operationalizing the weighted criteria models A complete set of rsults for each study included more than 20 tables and are not reproduced here Ilowever the final rankings for commodities and for research areas are shown in Tables I and 2 in Appendix 2 In the Dominican Republic a team of consultants followed up the initial priority-setting exercise with additional analysisand discussions at the experiment station level and eventually recommended tile establishment of five national commodity programs (rice cornsorghum beans coffeeand livestock) and one additional major research program (natural resources) In Ecuador the prioritized lists were widely distributed and discussed in the Ministry oi Agriculture the Board of FEIDIA the Commission on Science and Technology arid in AID There was concern in Ecuador with the weights placed on the individual criteria and therefore additional sensitivity analysis was conducted by the Planning Office of the Ministry There seemed to be a recognition in Ecuador that priority setting with a weighted criteria model is an iterative process and that much 4t the models value stems from tile discussion of criteria among the decision makers and from the sensitivityanalysis The high-priority list of comniodities in Ecuador included thL ten highest-priority commodities listed in Table 2b A project paper has been written forAI) which recommends funding For Fli)IAlbr three commodities (coffee soft corn for the Sierra and dairy) Coffee and soft coitn were the number 2 and 3 ranked prioritiesin tile weighted criteria model and dairy was number 9 although it was the top-rankedlivestock commodity

In Urugu ay the results from the first run with tile model were discussed with the research directors Ilie directors made small changes to the weights placed on criteriaand the mode was rerun resulting in the prioritized list in Appendix 2 Personnel in CIAAB are CUrrcwtv undertaking additional sensitivity analysis and with tie assistance of ISNA R are developing a plan for implementation

An Assessment of the Weighted Criteria Models Used in the Dominican RepublicEcuador and Uruguay

The Dominican Republic Ecuador and Uruguay studies were perhaps the first majorattempts to implement weighted criteria models with weights explicitly elicited from decision makers in NARS in less-developed countries A number of strengths andweaknesses in the procedures were identified during the analyses Some of thL weaknesses were corrected fron one study to the next some can be corrected in future studies and others are inherent in the approach

Trhe first strength mentioned earlier is the ability of the procedure to systematicallyincorporate both quantitative and qualitative information related to a set of multiplegoals and criteria in order to prioritize a lorng ist of commodities and research areas in a relatively short period of time Second the procedure proved relatively easy for both research administrators and the loc- -nalysts to understand with the exception of the )ominican Republic where the research administrators were not directly involved7

Third the analysis as applied in Ecuador and ULuguay with the direct involvement of research system administrators at various stages forced those decision makers to

12

consciously identifv and trade off goals and criteria Fourth the use of spreadsheet programs in EcUadfor and Uruguay facilitated sensitivity analysis after the initial set of priorities was deermimd Fifth the procedure provided a relatively objectiveassessment of prioriies because individuals were not allowed to rank commodities orresearch areas directly but had to weight criteria

The first weakness inherent in ex ante research priority setting and thus in theprocedure is that there is a large amount of subjectiveness Although tileapproach isless subjective than unstructured judgment there is subjectiveness in the responses toquestiorns related to some of the criteria and also ini the weights placed by decision makers on the criteria Second it proved diflicult to specify independent criteria with no overlap Third while the procedure required less time than an expected economic surplus analysis by commodity it required more time than asking respondents to directlyrank commodities and research areas In E-cuador the consiultant worked withpersonnel in INIAP and FEI)IA for one week at the start of the analysis to determinegoals and criteria to arrive at atlist of commodities and researh areas to list secondarydata needs to formulate questions to be asked of administrators and scientists to decide who should be asked particular questions and to explain the procedure itself includingthe spreadsheet program The required iidormation was collected over the Followingmonth and the consultant returned and worked with the Planning Director in INIAP for one week to conduct the analysis and write tip the results Revisions to the initial draftof the report we-e then made over the lollowing weeks A similar procedure to the onefollowed in Ecuador was used in l_ruguay althaugli more time was spent discussing the wording of questions to be asked of scientists Furthermore n Uruguay more care was taken to match the criteria with the goas

Certain of the criteria wil always be diflicult to eplain to tileinterviewees Thecriterion on comparative advantage is difficult for some respondents because of a lack of knowledge about neighboring countries If suflicient data time and resources are available this criterion may be rnea4ured more precisely using comparative advantage or domestic resource cost procedures suggested by longmire and Winkelnann Anotherdiflicult criterion to estiniate is the potential foture demand for the product The respondents in Fcuador were asked to consider how the demands for different foodswould change as income and populatiot grow Many of them had a hard timeconceiving that the total or per capita demand for certain Foods could actually decline as income grew In Uiruguay (ibe cmiparative advantage and future dema nd questions were answered by a small group of economists from the ministries of agriculture and trade In the Ecuador study more attention was devoted to assessing the most appropriate people for answering different questions than was the case in the Dominican Republic Ilowever some criticism surfaced about the weights placed on criteria InUruguay decision makers were given the opportunity to change their weights afterviewing the initial results This was very useftl because it demonstrated the implicationsof placing different weights on the various goals and criteria

Computer spreadsheet programs were a major help in Ecuador and Uruguay Additional work needs to be devoted to developing a more mentnu-driven prcgram but the people inthe planning office of INIAP were able to use the SuperCalcR program without too tmucht difliculty despite ialack of previous experience Ihe analyst in Uruguay was veryfamiliar with lotusl which proved most helpfil

It became clear from working inall three countries that it is preferable to work directlywith the final decision makers or their designees Inthe l)ominican Republic the study was conducted by a very competent set of consultants from the local agricultural universi ty but the procedure was not institutionalized to allow for additional sensitivityanalysis or future priority-setting efforts InEcuador and especially in Uruguay the

13

decision makers were directly involved conducted sensitivity analyses and were able to assess why certain commodities or research areas received high priority as a result of the analyses In both Ecuador and Uruguay they have a basic understanding of the strengths and weaknesses of tileprocedure and therelre can explain their subsequentdecisions to groups who might disagree with tilepriorities they establish

The corntrast between the Dominican Republic and tie Uruguay studies illustrates a veryimportant point No matter which technique is used to structure the analysispriority-setting exercises must integrate the decision makers (users) into the processPriority-setting procedures must be institutionalited the results of initial analysisdiscussed thoroughly with decision makers and sensitivity anilsis conlducteU alone the lines suggested by that discussion Tecinological change generates a surplus which can be appropriated by dilkircnt sectors or groups within the economy Who benelitdepends in part on the cotaniodities selected for research and the types of technoloLies generated One of the purposes of institutionalizing a formal prioritN- setting procedureis to help mediate conflicts that may arise (for example between producer groups and consumers) as a result of research-induced improved technologies

Other issues arose during the analyses First the initial grouping and elimination of certain commodities and research areas before lhe irialvis is an important step but one which must be taken with carelul consideration of possible coniplmentarities in research on particular productS or types of rearch Second some of the commodities such as forages are inputs into other commodities such as milk (Corseqtentlv these commodities llay have to he considered itscinps because it is the value-added and not tilegross value of output that is relevant when one coniiodity requires other commodities sinputs and when criteria such as value of production are used This was an issue in I ruguav where livestock are quite important liird little attention was devoted in atv of the studies to defining appropriate criteria Ior the increased inoome stability eoal Future studies if this be an ideltJiiLd LOal shOUld define risk criteria

In sutImary the weighted criteria model as applied in Lcuador and iuguav which had the advantage of bit ildin Oillessons lCarICd illdie l)ominican RCpublic appears to have been a uselul priority-setting tool It ha its ereatest advartage ill situations ill which there are rulItiple goals to consider a relatively major reassesnieiit of prioritiesis contenplated and the list of commodities dd resen rch areas to consider is largeWeighted criteria models described in this paper were applied in small countries with limited resources but with direct involvement (except in the Dominican Republic) ofthC major decision makers These models itav be most useful in the small country situationwhere decision mAkers are more likely to be involved in tile priority-settirt process and are not just using tables of results Perhaps in larger systet7 where decisioii makine is more decentralited tileweighted criteria model can he applied at the regional level before relining research priorities at the national level

Another factor that can be uniortant in large svvtern is tle complerientarity between resea-h aid educat ion [hie larger the svstcnt the clser the linkages tend to be between universities and tileresearchi system Where universities are important tor agriclltural research an aidditional criterion may hc rteedCd to place value on the educationlal benefits of certain types of reseirch

Ihere are several additional iniplications with respect to how weighted criteria models relate to alterniative priori t-setting Inethod parIicularl the expected economic surplusapproach Befoce exploring these relationships kcx aspects of the surplus approacliare described based on aii application in Peru

14

upon close scrutiny may contribute little to tire attainment of stated national goals and objectives

Judgment based upon prior knowledge and information provided by scientists is crucial for research resource alincation decisions In addition rapid or radical changes in research programs -an be costly Particularly as research budgets tighten howeverforcing changes in tile systein tile increased use of quantitative methods may be nocessary to imprwe thL objectivity of those judgments The aim is to foster consitencv of research priorities with goAs and objectives and to improve the efliciency of the research system in mser ing producer and cortsumer needs The idea is not to replacejudgment but to increase and organize the inflormation available for updating priorknowledge and beliefs The hope also is to inject continuity in the priority-settingprocess and to proide NARS decision makers x ith methods the find lielplib when rationalizing their decisions to scientists producer Erotps and politicians Docunentingthe decision proces through a strctured procedure can help minimie the possinilitvof sudden and large shilts in researh priorities in those systems characterized by a flairlv rapid turnover of research administrators

Many quantitative priority-setting procedures are avaiia ble but few have beeninstitutionalized into the decision-making practices of NA RS manaers A cruciZl factor in their non- use of these procedures urtdoubtcdlv as been the lack of ri orous vetcost-eflectie approaches which can irtcorporate tile large number of comtodities andresearch areas as well as the multiple go1ls a ld criteria found in most strateoicdecision-muakirt situations inl NRS FurtIhcrrtore the cortmputer capabiliv available through microcomputers in most developing-co ntry NRS today was not gcenerallyavailable even two or three velars ago

In this paper procedure amc ce crihed which were recently applied in four latinAmerican countries First however previously emploved priority-setting mechanisms are briefly reviewed id critiqued One of rite points that will Ie made at the end of this paper is tiLit tise methods are not nitually excilsive There is no siTele magic huletbut there appears to be real potential for developing a Ilexible computer-assisted priority-setting framework that will allow NARS administrators to call ott theappropriate tectiiques tot aSsist withit a particular decision depending ott tie time and financial resources a ailahle and the relative itiportance of the allocation decision Asecond major point is that priority setting is a process not simply a procedure and must be instit utiortliied as part of the overall process of research managementPriority-setting technritiuetes applied only once by people outside of the decision-making process have little valtie

If SUMMARY OF PRIORITY-SETTING METHODS

Methods reported itt previous studies for research priority setting include

I ) establishtert td weighting of multiple criteria for ranking commodities and research ares with a final aggregate ranking based on implicit or explicit weights

2) use of bertefit-cost analysis including expected economic surplus techniques to select cotitritodities and research areas

3) application of mathematical programming to choose an optimal research portfolioincorporatirng multiple goals md constraits

4) developrert and tise of simulation models

These four approaches are discussed below followed by a brief assessment of their advantaiges atil dis idsittages

2

Weighted Criteria Models

Several studies hae established multiple criteria for ranking priorities because of tiledesire to explicitly consider a wide variety of factors that do or perhaps should influenceresearch selection The relative weights attached to each criterion to arrive at a final listof research prioriries are sometimes left implicit or unstated A recent example is tileTAC review of priorities and future strategies for the Consultative Group onInternational Agricultural Research (CGIAR) In assessing priorities by commoditythey etablished a principal goal research objective to obtain the goal and a series ofcriteria organized into three groups relevance research productivity and efliciency ofhe CGIAR system in undertaking research Detailed tables with quantitative andqualitative information and rankings of commodities for each criterion were presentedIn developing the recommendations for hort- to medium-term funding alternatives atthe conmmodity level the weights used to aggregate across criteria were left implicit

There are examples of tudies that have incorporated multiple criteria but also explicitlyspecified and utilized a set of weights to aggregate across criteria and obtain a finalranking of research priogrities Appications of this method often called the scoringmodel approach are found in studies by Williamson lahlstede Paulsen and KaldorShumway and McCrack en and Von (Oppen and Ryan A flew studies have used congruence analysis which can be thought of as a special case of a sci-ring or weightedcriteria model with all the weight placed on the criterion of vaue of production

Benefit-cost (Expected Economic Surplus) Analysis

The benefit-cost approach to welecting research priorities has been used in differentforms (Fishel Arai Sire and Gardner l)avis Oram and Ryan Norton Ganoza andPomareda) Most studies have employed consumer-producer surplus analysis and havebuilt upon a large body of literature on ex post research evaluation Ex ante analysesusually incorporate expert opinion to determine projected research impacts adoptionrates and probabilities of research success and provide estimates of the economiceffliciencv and distributional implications of agricultural research resource allocationBenefit-cost studies typically calculate benefit-cost ratios internal rates of return andnet present values for alternative types of research or research onfor differentcommodities Ilse analyses may include andmav or not regional internationalresearch spillovers and the eflects of domestic pricing policies on research benefits

Mathematical Programming

Mathematical programming is another alternative for research selection It relies onmathematical Optimization to choose a research portfolio through maximizing amultiple-goal objective function given the resource constraints of the rescaich systemA good example of the use of this method is a study by Russell in the United Kingdomlie used goal programming to maximize the contributions of the research program toseveral goals given the con taints of budget human resources state of knowledge andcertain policies This procs Ue uses similar information to the weighted-criteria modelbut selects an optimal research portfolio rather than simply ranking research areas

Simulation

Finally the simulation method has been used to identify and select research prioritiesSimulation models vary in their construction but a good example is the model byPinstrup-Anderscn and Franklin They built a model to project the contributions andcosts of alternative research activities They established gals and then identifiedchanges in supply demand for inputs and demand for output needed to meet those

3

goals They identified neeled technologies time and financial costs and the probability of research success and adoption Finally they specified the scientists workingobjectives This model is very thorough but did require extensive amounts of data and estimation of several mathematical relationships

Comparative Assessment

The above discussion is a brie summary of the major structu al approaches that have been used to asist decision makers in research priority setting All of them attempt to go beyond the usual procedure of individual or group decision makin e without structural analyses Each of the approacles has its advantages and disadvantages (See Fable I)

The procedure of establishing criteria and using weights to arrive at a final set of research priorities has the advantage of forcing decision makers to consciously trade off multiple goals It can illcorporae both quantitative and qualhtative information and can be applied to a long list of con mnodities or research areas in a relatively liort period of time The procedure is relati cx ca sY fbr administrators to understand but does requiretheir time in obtaining the explicit w5eights for criteria Furthermore these weights are inevitably subjective and their elicitatlion must be carefully structured [lie nActhod also requires scientists time in collecting inlormation on qualitative criteria As a result the approach k better suited for periodic or major priority-setting efforts than or situations where frequent marginal changes are anticipated

The expected economic surplus approach his the major advantage of incorporatingseverml criteria related to economic elficiency and distribution into one or two measures It also can be ucd to examine the gencral equilibrium ellects ofresearch (Ramailho de Castro and Sclhh nAid tile benefits of research under alternative possibly distortionar domestic pticing and international trade policies tEdwarts and Freebairn Alston Edwards and rcecbairn Norton and (ianoza 1985) These alictors are pervasive in developing countries and affect the efficiency and distributional consequences of researh Ile procedure ie(liCe ir lel - tndcrstandw ofeconomic analysisand more anal-t s time than the weighted criteria model but less aduinistrators time It can be diflicult to apply to at large number of commodities or research areas because certain typcs of data necessary for the ainalysis often do not exist for all commodities With adequaIL tL d ita Cxpected economic surplus analysis can be incorporated into the weighted criteria model or could be used on the set of commodities which the weightedcriteria model imdicaMes to have the highest piority lhe latter approach would allow for the calculation of income foregone as a reult of placing weights on non-economic ef1ciency criteria Expected economic surplus analysis has the major advantage of calculating a rate of return which can be compared to alternative public investments

The mathematical programming approach is similar to the weighted criteria model approach becaue weights are placed on a set of goals or criteria The procedure has the advantage of explicitly considering the budget huLman resource aad other constraints on the research system tnless the constraints are well specified including possiblechanges over time liowever there is a risk of nonsense solutions The model is more intensive of an ecolonic analysts time and abilit than tie simpler weighted criteria approach and decision makers may be less willing to accept what appears to be a black box solution lrade-offs among goals are easily quantified ssith this approach

The advantage of simulation models is their tlexibility They can be constructed is relatively simple or complex tools can incorporate optimizing or ranking proceduresand can readily include probabilistic information Their major disadvantage is that to be useful they must be relatively complex and typically reqtuire extensive amounts of both data and time of skilled analysts

4

TABLE 1 COMPARISON AMONG MAJOR AGRICULTURAL RESEARCH PRIORITY-SETTING METHODS

PRIORITY-SETTING METHOD

HEIGHTED EXPECTED MATH CHARACTERISTIC CRITERIA ECON SURPLUS PROGRAM SIMULATION

OPERATIONAL CONSIDERATIONS

1 Relative cost in researchers time mudium mJium medium high2 Relative cost in priority setting analysts ime medium medium high high3 Relative cost in administrators time medium mediummedium medium 4 Relative overall data requirement medium medium medium variable 5 Relative ease of comprehonsion by decision maker high medium low low 6 Ease of incorporating subjective informaticn high high high high7 Ease of incorporating non-quantitativa inforiation high low low medium

GOAL-RELATED ISSUES

a Requires explicit elicitation of goals yes usually yes uiually9 Can determine distributional affects on consumers

and producers at various income levels no yes no yes10 Can handle uncertainty yes yes yes yes11 Can consider tradeoff among multiple goals yes sometimes yes yes

CRITERIA-RELATED ISSUES

12 Can consider private-sector research incentives yes difficult difficult yes13 Can consider economic policy and trade affects yes yes yes yes

EVALUATION-RELATED ISSUES

14 Can be used to sot priorities for research at the aggregate level no yes no es

15 Can be used to set research priorities at the commodity level yes yes yes yes

16 Can be used to set priorities for non-production or non-commodity oriented research yes difficuli yes yes

17 Can be used to set priorities for basic research yes difficult no sometimes 18 Can evaluate secondary impacts of rosuarch on

employment environment nutrition yes sometimes so etimes yes19 Usually estimates a rata of return to research yes sometimesno no 20 Can quantify geographic spillover effects no yes no yes 21 Can consider the lags involved in research

and adoption yes yesyes yes22 Facilitates priority setting when the number

of commodities is large yes difficultdifficult difficult

Source Adapted and modified from Norton and Davis

5

The above is a very brief summary of the major advantages and disadvantages of the most common formally structured priority-setting models that could be used for strategicplanning for research resource allocation Other approaches exist for research projectselection with perhaps models that attempt to identify and analyze key vield constraints being the most prevalent approach employed in NARS of less-deeloped countries

The results of ex post analysis also can provide useful guidance for ex ante research resource allocation decisions if appropriately incorporated into a systematic ex ante procedure The most common ex post approach in addition to cx post benefit-cost analysis is the econometric estimation of production or supply functions incorporating research variables To be most uteful for ex ante analysis econometric approaches mLlSt be applied with a high degree of disaggrega tion Unfortunateiy these models require a substantial amount of high-quality historical data on production farm inputs and research expenditures

Perhaps the two approaches with the most potential given additional refinements for application by NARS in less-developed countries are the weighted criteria models and the benefit-cost or expected economic surplus procedures In the next section reent experiences with applying weighted criteria models in three Latin American Countries )ominican Republic Ecuador and Uruguay are discussed and evaluated The purpose

it to piovide more iniormation about the procedures their degree of acceptance and possible improvements in the models Ihen application of expected economic surplus models are considered drawing cn a recent cx ante research evaluation studly in Peru Finally the relationship between weighted criteria and expected economic surpluts methods is described along with possible extensions

IIl THE USE OF WEIGHiTED CRITERIA MODELS IN TIlE DOMINICAN REPUBLIC ECUADOR AND URUGUAY

in 1986 iif I87 weighted criteria models were developed and implemented in the lDomin lRepublic (ISA) Ecuador (Espinosa et al) and Trugua (CIAAB)to assist with priority setting for agricultwral research In the I)ominican Republic the study was carried out at the Instituto Superior de Agnicultura ISA) by consultants for ISNAR In Ecuador the study was conducted by the plmnintg ollice oY the Instituto Nacional de Investigacion Agropecuaria (INIAP) with the assistance of consultants for the US Agency for International Development (AID) In Uruguay the study was conducted by the Ollice of the Director of the Centro de Investigacion Agricola Alberto Boergcr(CIAAB) with the assistance of ISNAR The purpose of all three studies was to applyprocedures for prioritizing agricultural research by commodity and by major research area Inthe Dominican Republic the client was the newly created natonal agriculturalresearch agency (Il)EA) In Ecuador the client was INIAP and the newly created foundation for agricultural research (IEl)IA) In Urugua the client was C1IAB

The procedures employed in the three studies were similar but with some importantdilfereices as the Ecuador study built on lessons learned inthe I)ominican Republic and the Uruguay study built on other lessons learmed in Ecuador National goals for the research system iere elicited in each country and a series of criteria establishd which relate to those goals Separate riteria were developed for conmitodities and or research areas and weights were elicited from decision makers to establish the relative importanceof the criteria Commodities and research areas were ranked according to each criterion and these rankings were multiplied by the elicited weights to arrive at research priorities

6

Goals

Three goals were identified in each of the three countries (1) to raise the average levelof income in the country (2) to increase the well-b(Aing of low-income groups in societyand (3) to reduce yearto-year income fluctuations in the country especially onl tiledownside These goals were worded slightly diffrcntly in each country but in each casethey essentially relate to the three goals referred to in previous studies as efficncy equity and security In the l)ominican Republic the local consultants identifiedthese as the goals while in lcuador and Uruguay it was done by the directors of national research agencies

Criteria

The next step was to establish criteria to as measures of whetheruse particularcommodities or research areas contribute to the attainment of the above goals A largeset of criteria was discussed in each country and a total of 15 criteria were eventuallyused in the Dominican Republic 14 in lEcuador and 10 in Uruguay to determine priorities by commodity Five criteria were used in each case to determine priorities byresearch area Refinements were made in successive studies - beginning with theDominican Republic then Ecuador followed by Iruguay - to increase the independenceof criteria and to remove criteria that were questionable measures of whether researchcontributed to tile tated goals ( Reason for dropping particular ci eria are discussedbelow) Refinements also were made with respect to the grouping of criteria to facilitatethe weighting of goals and criteria

Commodity Criteria omioditv criteria were grou l - J into four conceptual groupsproduct importance probability O success elliciencv in use of research resources anddistribution of impacts The first three of these groups relate to the efficiency or incomelevel goal and the last group relates to the equity or distributional goal In none of thethree studies were criteria included which represnted the third goal (reduced incomelluctuation [lie feeling was that the entire set of criteria as a group would serve toreduce the enlhasis placed on single or 1ew (often export) crops While this appearedto be the case in the results of thee particular studies ft ture weighted criteria studies may want to add criteria which explicitly rank commodities from lowest to hiehest withrespect to annual gross income variability (or price and yield variability separately)

The group of criteria referred to as jrodct importap contained Four criteria in theUruguay study value of production generation or saving of foreign exchange expectedfuture demand change and comparative advantage

The value of productionl is an important criterion because the cost of rseaich isbasically independent of the number of units alkected by the research results at least forthose commodities with relatively homogeneous production conditions Foreignexchange was used as a criterion because of a perception in each country that the lackof foreign exchange was impeding growtb A criterion was set up to allow more weightto be paced on those commodities for which demand is expected to increase in thefuture because the importance of commodities will change as diets are altered withincome growth aod a world markets change Comparative advantage was includedbecause income in the country will be higher if the country focuses its efforts oncommodities for which its resource base is best suited while recognizing that research itself can alter resource constraints

In the Dominican Republic and in Ecuador nutritional measurcs (calorie and proteincontributions to the diet) were included as criteria for product importance They wereincluded because the well-being of low-income groups may be affected by the availability

7

of foods which are important in their diets Nutritional criteria were dropped in Uruguay however because numerous studies have shown that income levels are tile primary determinants of nutrition levels rather than the availability of particular foods Certainly caloric and protein content are debatable criteria To the extent that increased domestic production lowers the prices of certain food commodities therebyincreasing food consumption by the poor nutritional criteria can contribute to the distributional goal Land area also was included as a criterion in the Dominican Republic study but was dropped in Ecuador and Uruguly because of the overlap with value of production

Two criteria were used in each study to measure 1robahility of reseatch success The first measure was tilegap between own-country yields and neighboing country yields The second was the potential for success indicated by the researchers themselves The rationale for the gap criterion was that the larger the yield gap between the home country and similar countries tle greater the potential for yield gains The criterion is rather crude because yield gaps also can indicate a relative resource disadvantace forproducing the commodity even if the countries appear to be geoclimatically sinilar Consequently the gap criterion was eventually dropped in the tUruguav study The potential for success as indicated by the researchers is subjective but a very necessary measure to obtain

The gro up of criteria reficrred to as e1)cienc in use otresearch resources contained three criteria the relationship to research in the international centers the degree of emphasis on tilecommodity in le current research p ogram and the incentive for the privatesector to conduct the research Some research duplicates other research croplementsand still other research bears no relationship to research at tileinternational agriculturalresearch centers For certain comniodities all three relationships may hold depending on the type of research If national research is complementary to international researchthe cost of the rational research is lower Consequently a criterion was established which places greater weight on commodities with a high degree of complementaritybetween ntin Il d international research The criterion ofdelegree of emphasis in tile current iesercli program was established because there is a cost to rapidly adjustingcurrent h uiman and physical research resources from one commodity to another The criterion of private-sector incentives was used because the social benefits from the total research program in the country will be greatest if scarce public research funds are devoted to research on commodities for which the private sector does not have an incentive to support the research itself Of coui se the private-sector producer group can tax itself and pay the public sector to conduct the research for those commodities for which it has an incentive to privately support the research These commodities tend to be export commodities because producers can capture a higher proportion of the gainsfrom productivity increases for export commodities than for commodities primarily consumed domestically

The group of criteria related to the distribution ojresarchimpacts contained two criteria in the Uruguay study number of producers and the effect of increased productivity on the price of product The larger the number of farms producing a commodity the largerthe number of farmers afficted by the research and perhaps the higher the probabilitythat smaller farmers are being affeicted The distribution of benefits depends to a significant degree on price changes resulting from increased productivity Producers benefit more if prices decline very little and consumers benefit more ifprices decline substatitially for a given production increase For export er import commodities priceswill tend to decline less than for commodities primarily produced and consumed domestically For commodities which are not traded and are very important in the diets of the poor prices will tend to decline relatively sharply when productivity increases The division of benefits between producers and consumers also is affected by the fact

that producers are also consumers Consequently in the Dominican Republic and Ecuador studies an additional criterion importance of the commodity in home-consumption on the farm where produced also was included

In summary 10-15 criteria were used in tie three studies to identify and select research priorities by commodity The linkages anong the goals and criteria described above are summarized in Figure 1

Research Area Criteria The five criteria uicel to select research priorities by research aieas were (1 ) whether the research causes an increase in the use of relatively abundant resources and a saving of relatively scarce resources (2) the number and seventy of research problems (3) non-duplication with transferable research from outside the country (4) tne extent of private-sector incentives to conduct the research and (5) current emphases in the research program

The rate of economic growth will be more rapid if new technologies are developed that utilize relatively abundant resources and save relatively scarce resources The resource base varies by region consequently information on resource abundance criteria was collected by region in tileEcuador and Uruguay studies The same was true for the quantity and severity of research problems Information on the relationship of domestic to inernational research private-sector incentives and the current emphasis in the research program was collected on a naticnal basis These criteria all relate to the income growth (efficiency) goal It is difficult to identif research area criteria that measurc whether particular types of research aifect income distribution or variability

Data Collection and Model Implementation

Information used in the analysis included both quantitative data on value of productionnumber of farms value of exports and imports person-years devoted to research on different commodities and tile of protein per day in thenumber of calories and gramsdiet (for tileDominican Republic and Ecuador studies) as well as qualitative or subjective information on such factors as probability of success private sector incentives and severity of research problems Furthermore weights had to be elicited from decision makers to place relative emphasis on the various criteria

Step 1 Develop Cotninodity and Research Area Lists -- The first step was to determine the list ofcommadities to consider and the list of research areas In the Dominican Republic information was collected on 74 commodities In Ecuador an initial list of 109 commodities on which INIAP was conducting research was discussed Throughdecisions to eliminate some and to group others this list was reduced to 44 commedities for the analysis These decisions were made by the Technical Director for Research In Uruguay a smaller list of commodities was reduced to 21 commodities or commodity groups These decisions were made in consultation with the National Research and Fxtension Directors The list of research program areas contained nine areas in the Dominican Republic 16 in Ecuador and 16 in Uruguay

Step 2 Collect Quantitative and Qualitative Data on Chosen Criteria -- Information o-i quantitative criteria were gathered from local and FAO secondary data sources and one table was constructed For each criterion Commodities were ranked for each criterion Information needed for the qualitative riteria as well as the weights to place on criteria were obtained through interviews with sc rtists and administrators at boh the national and regional experiment stations In Euador and Uruguay substantial attention was devoted to identifying the appropriate respondents to each question as this is a crucial aspect of the procedure In Ecuador those interviewed were notified one week before the interviews and 72 people were interviewed by the Planning Director of INIAP and

9

FIJR 1 CONCEPTUAL ELEHNTS IN THE HEIGHTED CRITERIA MODEL FOR PAWING RESEARCH PRIORITIES BY COtlDITY

GOALS fficiency ]

CONCEPTUAL( )CRITERIA Pri7] Probabi i+y Research Distriutjon Iincome

ii J of Succes Resource Eff o L=Acplusmn a rilbility

CRITERIA 1 Value of 1 Scientist 1Rl ofto toNumberPrcduction Estimates 1 AnnualInternational Producers Variability2 Foreign Centers 2 Price in Gro

Exchange 2 Current Effect3 Future Raturn3)Emphasis 3 HomeDemand 3 Private Consumpt ion

4 Comparative Sector Advantage Incentives

Notes

(-n future studies an additional conceptural criteria under the security goal might be health (J4tri ional criteria such as grams of potein or kilo=alories could be included as additional easures of Distributional ikipacts

4ot used in Dominican Republic Ecuador and Uruojuay studies

his assistant In Uruguay the directors of all the regional stations were called into CIIAB headquarters for a aweting to discuss the study and this was followed up byon-site interviews and meetings with scientists and directors at the regional experimentstations as well as with economists in the Ministry of Agriculture These interviews and meetings were conducted by a person assigned to work wvith the Director of Cl LAB on the priority-setting study In the Dominican Republic a survey of scientists extensionworkers and administrators was conducted by the ISNAR consultant from the Instituto Superior de Agricultura who visited the various experiment stations Less care wastaken in the Dominican Repubic than in Ecuador and Uruguay to match the questionspertaining -o individual criteria to particular individuals This probably reduced theusefulness of the responses because those most familiar with the probability of research success For example may not be the same -s those most Familiar with comparativeadvantage

Step 3 Elicit 11eihts on Critria -- Relative weights to place on the different criteria were obtained Fronm national and regional research system administrators In Ecuador34 peopic verc used to determine the weights These wveights were established separatelyIor the commoditv and the research area criteria In 1iuguay fewer people (sevensystem and station directors) were used to determine the weights and more attention was devoted to grouping the criteria pertaining to particular goals A Delphi procedure was used in which the s7even directors were shown the average of the group and provided with an opportunity to adjust their weight

Step 4 )erive Rankinqs hv Commodity and By Research Area -- Once the basic information was collected axd organized into a seies of tables one procedure was followed to arrive at a final ranking of research priorities by commodity A second procedure resulted in lists of priorities by research area In the Dominican Republicmost of the calculations were made by hand in Ecuador and Uruguay microcomputerspreadsheet programs were used to assist with the calculations and to conduct sensitivity analysis

The steps in the procedure for ranking research priorities by commodity and research area are sumnmaried below and are illustrated with a simple example in Appendix 1 Brieily the steps for ranking commodities were

a) Th conniodities were ranked for each criterion for which quantitative data were available

b) Tihese rankings were multiplied by the weights assigned to each criterion and then the weighted criteria were added across to arrive at one sub-ranking for the

Luantitative criteria c) Each commodity was given a high low or none designation for each qualitative

(subjective) criterion The response which implied a need for greater research priority was assigned the number 2 the intermediate response was assigned I and the response which implied lower research priority was assigned 0 1herefore each comniodity received a 2 1 or 0 for each criterion

d) The weights assigned to each qualitative criterion were then multiplied by these numbers and the results were added across all criteria to arrive at a subtotal for each commoditv These subtotals were ranked from highest to lowest to provide a second sub-ranking

e) The sub-rankings for the quantitative and qualitative criteria were then given their corresponding weights and added together to arrive at a final ranking by commodityThe steps for ranking research priorities by research area were the same as steps (c)and (d) above because the criteria were all qualitative

Step 5 Analsis and Interpretationof Resuls -- In the Dominican Republic the resuLts of this weighted criteria analysis were uscd to determine a small set of cormodities and

II

research areas with the highest priority Further assessment was then made of humanphysical administrative and other resources needed to structure research programsfocused on these commodities and resea-ch topics In Ecuador and Uruguay the prioritized list of commodities was split into a high-priority group an intermediate-priority group and a low-priority group Research area priorities were identified for each region of tile country A more detailed analysis was then made of the human physical and other resources needed to support the high and intermediate priorities

In summary five major -rens were involved in operationalizing the weighted criteria models A complete set of rsults for each study included more than 20 tables and are not reproduced here Ilowever the final rankings for commodities and for research areas are shown in Tables I and 2 in Appendix 2 In the Dominican Republic a team of consultants followed up the initial priority-setting exercise with additional analysisand discussions at the experiment station level and eventually recommended tile establishment of five national commodity programs (rice cornsorghum beans coffeeand livestock) and one additional major research program (natural resources) In Ecuador the prioritized lists were widely distributed and discussed in the Ministry oi Agriculture the Board of FEIDIA the Commission on Science and Technology arid in AID There was concern in Ecuador with the weights placed on the individual criteria and therefore additional sensitivity analysis was conducted by the Planning Office of the Ministry There seemed to be a recognition in Ecuador that priority setting with a weighted criteria model is an iterative process and that much 4t the models value stems from tile discussion of criteria among the decision makers and from the sensitivityanalysis The high-priority list of comniodities in Ecuador included thL ten highest-priority commodities listed in Table 2b A project paper has been written forAI) which recommends funding For Fli)IAlbr three commodities (coffee soft corn for the Sierra and dairy) Coffee and soft coitn were the number 2 and 3 ranked prioritiesin tile weighted criteria model and dairy was number 9 although it was the top-rankedlivestock commodity

In Urugu ay the results from the first run with tile model were discussed with the research directors Ilie directors made small changes to the weights placed on criteriaand the mode was rerun resulting in the prioritized list in Appendix 2 Personnel in CIAAB are CUrrcwtv undertaking additional sensitivity analysis and with tie assistance of ISNA R are developing a plan for implementation

An Assessment of the Weighted Criteria Models Used in the Dominican RepublicEcuador and Uruguay

The Dominican Republic Ecuador and Uruguay studies were perhaps the first majorattempts to implement weighted criteria models with weights explicitly elicited from decision makers in NARS in less-developed countries A number of strengths andweaknesses in the procedures were identified during the analyses Some of thL weaknesses were corrected fron one study to the next some can be corrected in future studies and others are inherent in the approach

Trhe first strength mentioned earlier is the ability of the procedure to systematicallyincorporate both quantitative and qualitative information related to a set of multiplegoals and criteria in order to prioritize a lorng ist of commodities and research areas in a relatively short period of time Second the procedure proved relatively easy for both research administrators and the loc- -nalysts to understand with the exception of the )ominican Republic where the research administrators were not directly involved7

Third the analysis as applied in Ecuador and ULuguay with the direct involvement of research system administrators at various stages forced those decision makers to

12

consciously identifv and trade off goals and criteria Fourth the use of spreadsheet programs in EcUadfor and Uruguay facilitated sensitivity analysis after the initial set of priorities was deermimd Fifth the procedure provided a relatively objectiveassessment of prioriies because individuals were not allowed to rank commodities orresearch areas directly but had to weight criteria

The first weakness inherent in ex ante research priority setting and thus in theprocedure is that there is a large amount of subjectiveness Although tileapproach isless subjective than unstructured judgment there is subjectiveness in the responses toquestiorns related to some of the criteria and also ini the weights placed by decision makers on the criteria Second it proved diflicult to specify independent criteria with no overlap Third while the procedure required less time than an expected economic surplus analysis by commodity it required more time than asking respondents to directlyrank commodities and research areas In E-cuador the consiultant worked withpersonnel in INIAP and FEI)IA for one week at the start of the analysis to determinegoals and criteria to arrive at atlist of commodities and researh areas to list secondarydata needs to formulate questions to be asked of administrators and scientists to decide who should be asked particular questions and to explain the procedure itself includingthe spreadsheet program The required iidormation was collected over the Followingmonth and the consultant returned and worked with the Planning Director in INIAP for one week to conduct the analysis and write tip the results Revisions to the initial draftof the report we-e then made over the lollowing weeks A similar procedure to the onefollowed in Ecuador was used in l_ruguay althaugli more time was spent discussing the wording of questions to be asked of scientists Furthermore n Uruguay more care was taken to match the criteria with the goas

Certain of the criteria wil always be diflicult to eplain to tileinterviewees Thecriterion on comparative advantage is difficult for some respondents because of a lack of knowledge about neighboring countries If suflicient data time and resources are available this criterion may be rnea4ured more precisely using comparative advantage or domestic resource cost procedures suggested by longmire and Winkelnann Anotherdiflicult criterion to estiniate is the potential foture demand for the product The respondents in Fcuador were asked to consider how the demands for different foodswould change as income and populatiot grow Many of them had a hard timeconceiving that the total or per capita demand for certain Foods could actually decline as income grew In Uiruguay (ibe cmiparative advantage and future dema nd questions were answered by a small group of economists from the ministries of agriculture and trade In the Ecuador study more attention was devoted to assessing the most appropriate people for answering different questions than was the case in the Dominican Republic Ilowever some criticism surfaced about the weights placed on criteria InUruguay decision makers were given the opportunity to change their weights afterviewing the initial results This was very useftl because it demonstrated the implicationsof placing different weights on the various goals and criteria

Computer spreadsheet programs were a major help in Ecuador and Uruguay Additional work needs to be devoted to developing a more mentnu-driven prcgram but the people inthe planning office of INIAP were able to use the SuperCalcR program without too tmucht difliculty despite ialack of previous experience Ihe analyst in Uruguay was veryfamiliar with lotusl which proved most helpfil

It became clear from working inall three countries that it is preferable to work directlywith the final decision makers or their designees Inthe l)ominican Republic the study was conducted by a very competent set of consultants from the local agricultural universi ty but the procedure was not institutionalized to allow for additional sensitivityanalysis or future priority-setting efforts InEcuador and especially in Uruguay the

13

decision makers were directly involved conducted sensitivity analyses and were able to assess why certain commodities or research areas received high priority as a result of the analyses In both Ecuador and Uruguay they have a basic understanding of the strengths and weaknesses of tileprocedure and therelre can explain their subsequentdecisions to groups who might disagree with tilepriorities they establish

The corntrast between the Dominican Republic and tie Uruguay studies illustrates a veryimportant point No matter which technique is used to structure the analysispriority-setting exercises must integrate the decision makers (users) into the processPriority-setting procedures must be institutionalited the results of initial analysisdiscussed thoroughly with decision makers and sensitivity anilsis conlducteU alone the lines suggested by that discussion Tecinological change generates a surplus which can be appropriated by dilkircnt sectors or groups within the economy Who benelitdepends in part on the cotaniodities selected for research and the types of technoloLies generated One of the purposes of institutionalizing a formal prioritN- setting procedureis to help mediate conflicts that may arise (for example between producer groups and consumers) as a result of research-induced improved technologies

Other issues arose during the analyses First the initial grouping and elimination of certain commodities and research areas before lhe irialvis is an important step but one which must be taken with carelul consideration of possible coniplmentarities in research on particular productS or types of rearch Second some of the commodities such as forages are inputs into other commodities such as milk (Corseqtentlv these commodities llay have to he considered itscinps because it is the value-added and not tilegross value of output that is relevant when one coniiodity requires other commodities sinputs and when criteria such as value of production are used This was an issue in I ruguav where livestock are quite important liird little attention was devoted in atv of the studies to defining appropriate criteria Ior the increased inoome stability eoal Future studies if this be an ideltJiiLd LOal shOUld define risk criteria

In sutImary the weighted criteria model as applied in Lcuador and iuguav which had the advantage of bit ildin Oillessons lCarICd illdie l)ominican RCpublic appears to have been a uselul priority-setting tool It ha its ereatest advartage ill situations ill which there are rulItiple goals to consider a relatively major reassesnieiit of prioritiesis contenplated and the list of commodities dd resen rch areas to consider is largeWeighted criteria models described in this paper were applied in small countries with limited resources but with direct involvement (except in the Dominican Republic) ofthC major decision makers These models itav be most useful in the small country situationwhere decision mAkers are more likely to be involved in tile priority-settirt process and are not just using tables of results Perhaps in larger systet7 where decisioii makine is more decentralited tileweighted criteria model can he applied at the regional level before relining research priorities at the national level

Another factor that can be uniortant in large svvtern is tle complerientarity between resea-h aid educat ion [hie larger the svstcnt the clser the linkages tend to be between universities and tileresearchi system Where universities are important tor agriclltural research an aidditional criterion may hc rteedCd to place value on the educationlal benefits of certain types of reseirch

Ihere are several additional iniplications with respect to how weighted criteria models relate to alterniative priori t-setting Inethod parIicularl the expected economic surplusapproach Befoce exploring these relationships kcx aspects of the surplus approacliare described based on aii application in Peru

14

Weighted Criteria Models

Several studies hae established multiple criteria for ranking priorities because of tiledesire to explicitly consider a wide variety of factors that do or perhaps should influenceresearch selection The relative weights attached to each criterion to arrive at a final listof research prioriries are sometimes left implicit or unstated A recent example is tileTAC review of priorities and future strategies for the Consultative Group onInternational Agricultural Research (CGIAR) In assessing priorities by commoditythey etablished a principal goal research objective to obtain the goal and a series ofcriteria organized into three groups relevance research productivity and efliciency ofhe CGIAR system in undertaking research Detailed tables with quantitative andqualitative information and rankings of commodities for each criterion were presentedIn developing the recommendations for hort- to medium-term funding alternatives atthe conmmodity level the weights used to aggregate across criteria were left implicit

There are examples of tudies that have incorporated multiple criteria but also explicitlyspecified and utilized a set of weights to aggregate across criteria and obtain a finalranking of research priogrities Appications of this method often called the scoringmodel approach are found in studies by Williamson lahlstede Paulsen and KaldorShumway and McCrack en and Von (Oppen and Ryan A flew studies have used congruence analysis which can be thought of as a special case of a sci-ring or weightedcriteria model with all the weight placed on the criterion of vaue of production

Benefit-cost (Expected Economic Surplus) Analysis

The benefit-cost approach to welecting research priorities has been used in differentforms (Fishel Arai Sire and Gardner l)avis Oram and Ryan Norton Ganoza andPomareda) Most studies have employed consumer-producer surplus analysis and havebuilt upon a large body of literature on ex post research evaluation Ex ante analysesusually incorporate expert opinion to determine projected research impacts adoptionrates and probabilities of research success and provide estimates of the economiceffliciencv and distributional implications of agricultural research resource allocationBenefit-cost studies typically calculate benefit-cost ratios internal rates of return andnet present values for alternative types of research or research onfor differentcommodities Ilse analyses may include andmav or not regional internationalresearch spillovers and the eflects of domestic pricing policies on research benefits

Mathematical Programming

Mathematical programming is another alternative for research selection It relies onmathematical Optimization to choose a research portfolio through maximizing amultiple-goal objective function given the resource constraints of the rescaich systemA good example of the use of this method is a study by Russell in the United Kingdomlie used goal programming to maximize the contributions of the research program toseveral goals given the con taints of budget human resources state of knowledge andcertain policies This procs Ue uses similar information to the weighted-criteria modelbut selects an optimal research portfolio rather than simply ranking research areas

Simulation

Finally the simulation method has been used to identify and select research prioritiesSimulation models vary in their construction but a good example is the model byPinstrup-Anderscn and Franklin They built a model to project the contributions andcosts of alternative research activities They established gals and then identifiedchanges in supply demand for inputs and demand for output needed to meet those

3

goals They identified neeled technologies time and financial costs and the probability of research success and adoption Finally they specified the scientists workingobjectives This model is very thorough but did require extensive amounts of data and estimation of several mathematical relationships

Comparative Assessment

The above discussion is a brie summary of the major structu al approaches that have been used to asist decision makers in research priority setting All of them attempt to go beyond the usual procedure of individual or group decision makin e without structural analyses Each of the approacles has its advantages and disadvantages (See Fable I)

The procedure of establishing criteria and using weights to arrive at a final set of research priorities has the advantage of forcing decision makers to consciously trade off multiple goals It can illcorporae both quantitative and qualhtative information and can be applied to a long list of con mnodities or research areas in a relatively liort period of time The procedure is relati cx ca sY fbr administrators to understand but does requiretheir time in obtaining the explicit w5eights for criteria Furthermore these weights are inevitably subjective and their elicitatlion must be carefully structured [lie nActhod also requires scientists time in collecting inlormation on qualitative criteria As a result the approach k better suited for periodic or major priority-setting efforts than or situations where frequent marginal changes are anticipated

The expected economic surplus approach his the major advantage of incorporatingseverml criteria related to economic elficiency and distribution into one or two measures It also can be ucd to examine the gencral equilibrium ellects ofresearch (Ramailho de Castro and Sclhh nAid tile benefits of research under alternative possibly distortionar domestic pticing and international trade policies tEdwarts and Freebairn Alston Edwards and rcecbairn Norton and (ianoza 1985) These alictors are pervasive in developing countries and affect the efficiency and distributional consequences of researh Ile procedure ie(liCe ir lel - tndcrstandw ofeconomic analysisand more anal-t s time than the weighted criteria model but less aduinistrators time It can be diflicult to apply to at large number of commodities or research areas because certain typcs of data necessary for the ainalysis often do not exist for all commodities With adequaIL tL d ita Cxpected economic surplus analysis can be incorporated into the weighted criteria model or could be used on the set of commodities which the weightedcriteria model imdicaMes to have the highest piority lhe latter approach would allow for the calculation of income foregone as a reult of placing weights on non-economic ef1ciency criteria Expected economic surplus analysis has the major advantage of calculating a rate of return which can be compared to alternative public investments

The mathematical programming approach is similar to the weighted criteria model approach becaue weights are placed on a set of goals or criteria The procedure has the advantage of explicitly considering the budget huLman resource aad other constraints on the research system tnless the constraints are well specified including possiblechanges over time liowever there is a risk of nonsense solutions The model is more intensive of an ecolonic analysts time and abilit than tie simpler weighted criteria approach and decision makers may be less willing to accept what appears to be a black box solution lrade-offs among goals are easily quantified ssith this approach

The advantage of simulation models is their tlexibility They can be constructed is relatively simple or complex tools can incorporate optimizing or ranking proceduresand can readily include probabilistic information Their major disadvantage is that to be useful they must be relatively complex and typically reqtuire extensive amounts of both data and time of skilled analysts

4

TABLE 1 COMPARISON AMONG MAJOR AGRICULTURAL RESEARCH PRIORITY-SETTING METHODS

PRIORITY-SETTING METHOD

HEIGHTED EXPECTED MATH CHARACTERISTIC CRITERIA ECON SURPLUS PROGRAM SIMULATION

OPERATIONAL CONSIDERATIONS

1 Relative cost in researchers time mudium mJium medium high2 Relative cost in priority setting analysts ime medium medium high high3 Relative cost in administrators time medium mediummedium medium 4 Relative overall data requirement medium medium medium variable 5 Relative ease of comprehonsion by decision maker high medium low low 6 Ease of incorporating subjective informaticn high high high high7 Ease of incorporating non-quantitativa inforiation high low low medium

GOAL-RELATED ISSUES

a Requires explicit elicitation of goals yes usually yes uiually9 Can determine distributional affects on consumers

and producers at various income levels no yes no yes10 Can handle uncertainty yes yes yes yes11 Can consider tradeoff among multiple goals yes sometimes yes yes

CRITERIA-RELATED ISSUES

12 Can consider private-sector research incentives yes difficult difficult yes13 Can consider economic policy and trade affects yes yes yes yes

EVALUATION-RELATED ISSUES

14 Can be used to sot priorities for research at the aggregate level no yes no es

15 Can be used to set research priorities at the commodity level yes yes yes yes

16 Can be used to set priorities for non-production or non-commodity oriented research yes difficuli yes yes

17 Can be used to set priorities for basic research yes difficult no sometimes 18 Can evaluate secondary impacts of rosuarch on

employment environment nutrition yes sometimes so etimes yes19 Usually estimates a rata of return to research yes sometimesno no 20 Can quantify geographic spillover effects no yes no yes 21 Can consider the lags involved in research

and adoption yes yesyes yes22 Facilitates priority setting when the number

of commodities is large yes difficultdifficult difficult

Source Adapted and modified from Norton and Davis

5

The above is a very brief summary of the major advantages and disadvantages of the most common formally structured priority-setting models that could be used for strategicplanning for research resource allocation Other approaches exist for research projectselection with perhaps models that attempt to identify and analyze key vield constraints being the most prevalent approach employed in NARS of less-deeloped countries

The results of ex post analysis also can provide useful guidance for ex ante research resource allocation decisions if appropriately incorporated into a systematic ex ante procedure The most common ex post approach in addition to cx post benefit-cost analysis is the econometric estimation of production or supply functions incorporating research variables To be most uteful for ex ante analysis econometric approaches mLlSt be applied with a high degree of disaggrega tion Unfortunateiy these models require a substantial amount of high-quality historical data on production farm inputs and research expenditures

Perhaps the two approaches with the most potential given additional refinements for application by NARS in less-developed countries are the weighted criteria models and the benefit-cost or expected economic surplus procedures In the next section reent experiences with applying weighted criteria models in three Latin American Countries )ominican Republic Ecuador and Uruguay are discussed and evaluated The purpose

it to piovide more iniormation about the procedures their degree of acceptance and possible improvements in the models Ihen application of expected economic surplus models are considered drawing cn a recent cx ante research evaluation studly in Peru Finally the relationship between weighted criteria and expected economic surpluts methods is described along with possible extensions

IIl THE USE OF WEIGHiTED CRITERIA MODELS IN TIlE DOMINICAN REPUBLIC ECUADOR AND URUGUAY

in 1986 iif I87 weighted criteria models were developed and implemented in the lDomin lRepublic (ISA) Ecuador (Espinosa et al) and Trugua (CIAAB)to assist with priority setting for agricultwral research In the I)ominican Republic the study was carried out at the Instituto Superior de Agnicultura ISA) by consultants for ISNAR In Ecuador the study was conducted by the plmnintg ollice oY the Instituto Nacional de Investigacion Agropecuaria (INIAP) with the assistance of consultants for the US Agency for International Development (AID) In Uruguay the study was conducted by the Ollice of the Director of the Centro de Investigacion Agricola Alberto Boergcr(CIAAB) with the assistance of ISNAR The purpose of all three studies was to applyprocedures for prioritizing agricultural research by commodity and by major research area Inthe Dominican Republic the client was the newly created natonal agriculturalresearch agency (Il)EA) In Ecuador the client was INIAP and the newly created foundation for agricultural research (IEl)IA) In Urugua the client was C1IAB

The procedures employed in the three studies were similar but with some importantdilfereices as the Ecuador study built on lessons learned inthe I)ominican Republic and the Uruguay study built on other lessons learmed in Ecuador National goals for the research system iere elicited in each country and a series of criteria establishd which relate to those goals Separate riteria were developed for conmitodities and or research areas and weights were elicited from decision makers to establish the relative importanceof the criteria Commodities and research areas were ranked according to each criterion and these rankings were multiplied by the elicited weights to arrive at research priorities

6

Goals

Three goals were identified in each of the three countries (1) to raise the average levelof income in the country (2) to increase the well-b(Aing of low-income groups in societyand (3) to reduce yearto-year income fluctuations in the country especially onl tiledownside These goals were worded slightly diffrcntly in each country but in each casethey essentially relate to the three goals referred to in previous studies as efficncy equity and security In the l)ominican Republic the local consultants identifiedthese as the goals while in lcuador and Uruguay it was done by the directors of national research agencies

Criteria

The next step was to establish criteria to as measures of whetheruse particularcommodities or research areas contribute to the attainment of the above goals A largeset of criteria was discussed in each country and a total of 15 criteria were eventuallyused in the Dominican Republic 14 in lEcuador and 10 in Uruguay to determine priorities by commodity Five criteria were used in each case to determine priorities byresearch area Refinements were made in successive studies - beginning with theDominican Republic then Ecuador followed by Iruguay - to increase the independenceof criteria and to remove criteria that were questionable measures of whether researchcontributed to tile tated goals ( Reason for dropping particular ci eria are discussedbelow) Refinements also were made with respect to the grouping of criteria to facilitatethe weighting of goals and criteria

Commodity Criteria omioditv criteria were grou l - J into four conceptual groupsproduct importance probability O success elliciencv in use of research resources anddistribution of impacts The first three of these groups relate to the efficiency or incomelevel goal and the last group relates to the equity or distributional goal In none of thethree studies were criteria included which represnted the third goal (reduced incomelluctuation [lie feeling was that the entire set of criteria as a group would serve toreduce the enlhasis placed on single or 1ew (often export) crops While this appearedto be the case in the results of thee particular studies ft ture weighted criteria studies may want to add criteria which explicitly rank commodities from lowest to hiehest withrespect to annual gross income variability (or price and yield variability separately)

The group of criteria referred to as jrodct importap contained Four criteria in theUruguay study value of production generation or saving of foreign exchange expectedfuture demand change and comparative advantage

The value of productionl is an important criterion because the cost of rseaich isbasically independent of the number of units alkected by the research results at least forthose commodities with relatively homogeneous production conditions Foreignexchange was used as a criterion because of a perception in each country that the lackof foreign exchange was impeding growtb A criterion was set up to allow more weightto be paced on those commodities for which demand is expected to increase in thefuture because the importance of commodities will change as diets are altered withincome growth aod a world markets change Comparative advantage was includedbecause income in the country will be higher if the country focuses its efforts oncommodities for which its resource base is best suited while recognizing that research itself can alter resource constraints

In the Dominican Republic and in Ecuador nutritional measurcs (calorie and proteincontributions to the diet) were included as criteria for product importance They wereincluded because the well-being of low-income groups may be affected by the availability

7

of foods which are important in their diets Nutritional criteria were dropped in Uruguay however because numerous studies have shown that income levels are tile primary determinants of nutrition levels rather than the availability of particular foods Certainly caloric and protein content are debatable criteria To the extent that increased domestic production lowers the prices of certain food commodities therebyincreasing food consumption by the poor nutritional criteria can contribute to the distributional goal Land area also was included as a criterion in the Dominican Republic study but was dropped in Ecuador and Uruguly because of the overlap with value of production

Two criteria were used in each study to measure 1robahility of reseatch success The first measure was tilegap between own-country yields and neighboing country yields The second was the potential for success indicated by the researchers themselves The rationale for the gap criterion was that the larger the yield gap between the home country and similar countries tle greater the potential for yield gains The criterion is rather crude because yield gaps also can indicate a relative resource disadvantace forproducing the commodity even if the countries appear to be geoclimatically sinilar Consequently the gap criterion was eventually dropped in the tUruguav study The potential for success as indicated by the researchers is subjective but a very necessary measure to obtain

The gro up of criteria reficrred to as e1)cienc in use otresearch resources contained three criteria the relationship to research in the international centers the degree of emphasis on tilecommodity in le current research p ogram and the incentive for the privatesector to conduct the research Some research duplicates other research croplementsand still other research bears no relationship to research at tileinternational agriculturalresearch centers For certain comniodities all three relationships may hold depending on the type of research If national research is complementary to international researchthe cost of the rational research is lower Consequently a criterion was established which places greater weight on commodities with a high degree of complementaritybetween ntin Il d international research The criterion ofdelegree of emphasis in tile current iesercli program was established because there is a cost to rapidly adjustingcurrent h uiman and physical research resources from one commodity to another The criterion of private-sector incentives was used because the social benefits from the total research program in the country will be greatest if scarce public research funds are devoted to research on commodities for which the private sector does not have an incentive to support the research itself Of coui se the private-sector producer group can tax itself and pay the public sector to conduct the research for those commodities for which it has an incentive to privately support the research These commodities tend to be export commodities because producers can capture a higher proportion of the gainsfrom productivity increases for export commodities than for commodities primarily consumed domestically

The group of criteria related to the distribution ojresarchimpacts contained two criteria in the Uruguay study number of producers and the effect of increased productivity on the price of product The larger the number of farms producing a commodity the largerthe number of farmers afficted by the research and perhaps the higher the probabilitythat smaller farmers are being affeicted The distribution of benefits depends to a significant degree on price changes resulting from increased productivity Producers benefit more if prices decline very little and consumers benefit more ifprices decline substatitially for a given production increase For export er import commodities priceswill tend to decline less than for commodities primarily produced and consumed domestically For commodities which are not traded and are very important in the diets of the poor prices will tend to decline relatively sharply when productivity increases The division of benefits between producers and consumers also is affected by the fact

that producers are also consumers Consequently in the Dominican Republic and Ecuador studies an additional criterion importance of the commodity in home-consumption on the farm where produced also was included

In summary 10-15 criteria were used in tie three studies to identify and select research priorities by commodity The linkages anong the goals and criteria described above are summarized in Figure 1

Research Area Criteria The five criteria uicel to select research priorities by research aieas were (1 ) whether the research causes an increase in the use of relatively abundant resources and a saving of relatively scarce resources (2) the number and seventy of research problems (3) non-duplication with transferable research from outside the country (4) tne extent of private-sector incentives to conduct the research and (5) current emphases in the research program

The rate of economic growth will be more rapid if new technologies are developed that utilize relatively abundant resources and save relatively scarce resources The resource base varies by region consequently information on resource abundance criteria was collected by region in tileEcuador and Uruguay studies The same was true for the quantity and severity of research problems Information on the relationship of domestic to inernational research private-sector incentives and the current emphasis in the research program was collected on a naticnal basis These criteria all relate to the income growth (efficiency) goal It is difficult to identif research area criteria that measurc whether particular types of research aifect income distribution or variability

Data Collection and Model Implementation

Information used in the analysis included both quantitative data on value of productionnumber of farms value of exports and imports person-years devoted to research on different commodities and tile of protein per day in thenumber of calories and gramsdiet (for tileDominican Republic and Ecuador studies) as well as qualitative or subjective information on such factors as probability of success private sector incentives and severity of research problems Furthermore weights had to be elicited from decision makers to place relative emphasis on the various criteria

Step 1 Develop Cotninodity and Research Area Lists -- The first step was to determine the list ofcommadities to consider and the list of research areas In the Dominican Republic information was collected on 74 commodities In Ecuador an initial list of 109 commodities on which INIAP was conducting research was discussed Throughdecisions to eliminate some and to group others this list was reduced to 44 commedities for the analysis These decisions were made by the Technical Director for Research In Uruguay a smaller list of commodities was reduced to 21 commodities or commodity groups These decisions were made in consultation with the National Research and Fxtension Directors The list of research program areas contained nine areas in the Dominican Republic 16 in Ecuador and 16 in Uruguay

Step 2 Collect Quantitative and Qualitative Data on Chosen Criteria -- Information o-i quantitative criteria were gathered from local and FAO secondary data sources and one table was constructed For each criterion Commodities were ranked for each criterion Information needed for the qualitative riteria as well as the weights to place on criteria were obtained through interviews with sc rtists and administrators at boh the national and regional experiment stations In Euador and Uruguay substantial attention was devoted to identifying the appropriate respondents to each question as this is a crucial aspect of the procedure In Ecuador those interviewed were notified one week before the interviews and 72 people were interviewed by the Planning Director of INIAP and

9

FIJR 1 CONCEPTUAL ELEHNTS IN THE HEIGHTED CRITERIA MODEL FOR PAWING RESEARCH PRIORITIES BY COtlDITY

GOALS fficiency ]

CONCEPTUAL( )CRITERIA Pri7] Probabi i+y Research Distriutjon Iincome

ii J of Succes Resource Eff o L=Acplusmn a rilbility

CRITERIA 1 Value of 1 Scientist 1Rl ofto toNumberPrcduction Estimates 1 AnnualInternational Producers Variability2 Foreign Centers 2 Price in Gro

Exchange 2 Current Effect3 Future Raturn3)Emphasis 3 HomeDemand 3 Private Consumpt ion

4 Comparative Sector Advantage Incentives

Notes

(-n future studies an additional conceptural criteria under the security goal might be health (J4tri ional criteria such as grams of potein or kilo=alories could be included as additional easures of Distributional ikipacts

4ot used in Dominican Republic Ecuador and Uruojuay studies

his assistant In Uruguay the directors of all the regional stations were called into CIIAB headquarters for a aweting to discuss the study and this was followed up byon-site interviews and meetings with scientists and directors at the regional experimentstations as well as with economists in the Ministry of Agriculture These interviews and meetings were conducted by a person assigned to work wvith the Director of Cl LAB on the priority-setting study In the Dominican Republic a survey of scientists extensionworkers and administrators was conducted by the ISNAR consultant from the Instituto Superior de Agricultura who visited the various experiment stations Less care wastaken in the Dominican Repubic than in Ecuador and Uruguay to match the questionspertaining -o individual criteria to particular individuals This probably reduced theusefulness of the responses because those most familiar with the probability of research success For example may not be the same -s those most Familiar with comparativeadvantage

Step 3 Elicit 11eihts on Critria -- Relative weights to place on the different criteria were obtained Fronm national and regional research system administrators In Ecuador34 peopic verc used to determine the weights These wveights were established separatelyIor the commoditv and the research area criteria In 1iuguay fewer people (sevensystem and station directors) were used to determine the weights and more attention was devoted to grouping the criteria pertaining to particular goals A Delphi procedure was used in which the s7even directors were shown the average of the group and provided with an opportunity to adjust their weight

Step 4 )erive Rankinqs hv Commodity and By Research Area -- Once the basic information was collected axd organized into a seies of tables one procedure was followed to arrive at a final ranking of research priorities by commodity A second procedure resulted in lists of priorities by research area In the Dominican Republicmost of the calculations were made by hand in Ecuador and Uruguay microcomputerspreadsheet programs were used to assist with the calculations and to conduct sensitivity analysis

The steps in the procedure for ranking research priorities by commodity and research area are sumnmaried below and are illustrated with a simple example in Appendix 1 Brieily the steps for ranking commodities were

a) Th conniodities were ranked for each criterion for which quantitative data were available

b) Tihese rankings were multiplied by the weights assigned to each criterion and then the weighted criteria were added across to arrive at one sub-ranking for the

Luantitative criteria c) Each commodity was given a high low or none designation for each qualitative

(subjective) criterion The response which implied a need for greater research priority was assigned the number 2 the intermediate response was assigned I and the response which implied lower research priority was assigned 0 1herefore each comniodity received a 2 1 or 0 for each criterion

d) The weights assigned to each qualitative criterion were then multiplied by these numbers and the results were added across all criteria to arrive at a subtotal for each commoditv These subtotals were ranked from highest to lowest to provide a second sub-ranking

e) The sub-rankings for the quantitative and qualitative criteria were then given their corresponding weights and added together to arrive at a final ranking by commodityThe steps for ranking research priorities by research area were the same as steps (c)and (d) above because the criteria were all qualitative

Step 5 Analsis and Interpretationof Resuls -- In the Dominican Republic the resuLts of this weighted criteria analysis were uscd to determine a small set of cormodities and

II

research areas with the highest priority Further assessment was then made of humanphysical administrative and other resources needed to structure research programsfocused on these commodities and resea-ch topics In Ecuador and Uruguay the prioritized list of commodities was split into a high-priority group an intermediate-priority group and a low-priority group Research area priorities were identified for each region of tile country A more detailed analysis was then made of the human physical and other resources needed to support the high and intermediate priorities

In summary five major -rens were involved in operationalizing the weighted criteria models A complete set of rsults for each study included more than 20 tables and are not reproduced here Ilowever the final rankings for commodities and for research areas are shown in Tables I and 2 in Appendix 2 In the Dominican Republic a team of consultants followed up the initial priority-setting exercise with additional analysisand discussions at the experiment station level and eventually recommended tile establishment of five national commodity programs (rice cornsorghum beans coffeeand livestock) and one additional major research program (natural resources) In Ecuador the prioritized lists were widely distributed and discussed in the Ministry oi Agriculture the Board of FEIDIA the Commission on Science and Technology arid in AID There was concern in Ecuador with the weights placed on the individual criteria and therefore additional sensitivity analysis was conducted by the Planning Office of the Ministry There seemed to be a recognition in Ecuador that priority setting with a weighted criteria model is an iterative process and that much 4t the models value stems from tile discussion of criteria among the decision makers and from the sensitivityanalysis The high-priority list of comniodities in Ecuador included thL ten highest-priority commodities listed in Table 2b A project paper has been written forAI) which recommends funding For Fli)IAlbr three commodities (coffee soft corn for the Sierra and dairy) Coffee and soft coitn were the number 2 and 3 ranked prioritiesin tile weighted criteria model and dairy was number 9 although it was the top-rankedlivestock commodity

In Urugu ay the results from the first run with tile model were discussed with the research directors Ilie directors made small changes to the weights placed on criteriaand the mode was rerun resulting in the prioritized list in Appendix 2 Personnel in CIAAB are CUrrcwtv undertaking additional sensitivity analysis and with tie assistance of ISNA R are developing a plan for implementation

An Assessment of the Weighted Criteria Models Used in the Dominican RepublicEcuador and Uruguay

The Dominican Republic Ecuador and Uruguay studies were perhaps the first majorattempts to implement weighted criteria models with weights explicitly elicited from decision makers in NARS in less-developed countries A number of strengths andweaknesses in the procedures were identified during the analyses Some of thL weaknesses were corrected fron one study to the next some can be corrected in future studies and others are inherent in the approach

Trhe first strength mentioned earlier is the ability of the procedure to systematicallyincorporate both quantitative and qualitative information related to a set of multiplegoals and criteria in order to prioritize a lorng ist of commodities and research areas in a relatively short period of time Second the procedure proved relatively easy for both research administrators and the loc- -nalysts to understand with the exception of the )ominican Republic where the research administrators were not directly involved7

Third the analysis as applied in Ecuador and ULuguay with the direct involvement of research system administrators at various stages forced those decision makers to

12

consciously identifv and trade off goals and criteria Fourth the use of spreadsheet programs in EcUadfor and Uruguay facilitated sensitivity analysis after the initial set of priorities was deermimd Fifth the procedure provided a relatively objectiveassessment of prioriies because individuals were not allowed to rank commodities orresearch areas directly but had to weight criteria

The first weakness inherent in ex ante research priority setting and thus in theprocedure is that there is a large amount of subjectiveness Although tileapproach isless subjective than unstructured judgment there is subjectiveness in the responses toquestiorns related to some of the criteria and also ini the weights placed by decision makers on the criteria Second it proved diflicult to specify independent criteria with no overlap Third while the procedure required less time than an expected economic surplus analysis by commodity it required more time than asking respondents to directlyrank commodities and research areas In E-cuador the consiultant worked withpersonnel in INIAP and FEI)IA for one week at the start of the analysis to determinegoals and criteria to arrive at atlist of commodities and researh areas to list secondarydata needs to formulate questions to be asked of administrators and scientists to decide who should be asked particular questions and to explain the procedure itself includingthe spreadsheet program The required iidormation was collected over the Followingmonth and the consultant returned and worked with the Planning Director in INIAP for one week to conduct the analysis and write tip the results Revisions to the initial draftof the report we-e then made over the lollowing weeks A similar procedure to the onefollowed in Ecuador was used in l_ruguay althaugli more time was spent discussing the wording of questions to be asked of scientists Furthermore n Uruguay more care was taken to match the criteria with the goas

Certain of the criteria wil always be diflicult to eplain to tileinterviewees Thecriterion on comparative advantage is difficult for some respondents because of a lack of knowledge about neighboring countries If suflicient data time and resources are available this criterion may be rnea4ured more precisely using comparative advantage or domestic resource cost procedures suggested by longmire and Winkelnann Anotherdiflicult criterion to estiniate is the potential foture demand for the product The respondents in Fcuador were asked to consider how the demands for different foodswould change as income and populatiot grow Many of them had a hard timeconceiving that the total or per capita demand for certain Foods could actually decline as income grew In Uiruguay (ibe cmiparative advantage and future dema nd questions were answered by a small group of economists from the ministries of agriculture and trade In the Ecuador study more attention was devoted to assessing the most appropriate people for answering different questions than was the case in the Dominican Republic Ilowever some criticism surfaced about the weights placed on criteria InUruguay decision makers were given the opportunity to change their weights afterviewing the initial results This was very useftl because it demonstrated the implicationsof placing different weights on the various goals and criteria

Computer spreadsheet programs were a major help in Ecuador and Uruguay Additional work needs to be devoted to developing a more mentnu-driven prcgram but the people inthe planning office of INIAP were able to use the SuperCalcR program without too tmucht difliculty despite ialack of previous experience Ihe analyst in Uruguay was veryfamiliar with lotusl which proved most helpfil

It became clear from working inall three countries that it is preferable to work directlywith the final decision makers or their designees Inthe l)ominican Republic the study was conducted by a very competent set of consultants from the local agricultural universi ty but the procedure was not institutionalized to allow for additional sensitivityanalysis or future priority-setting efforts InEcuador and especially in Uruguay the

13

decision makers were directly involved conducted sensitivity analyses and were able to assess why certain commodities or research areas received high priority as a result of the analyses In both Ecuador and Uruguay they have a basic understanding of the strengths and weaknesses of tileprocedure and therelre can explain their subsequentdecisions to groups who might disagree with tilepriorities they establish

The corntrast between the Dominican Republic and tie Uruguay studies illustrates a veryimportant point No matter which technique is used to structure the analysispriority-setting exercises must integrate the decision makers (users) into the processPriority-setting procedures must be institutionalited the results of initial analysisdiscussed thoroughly with decision makers and sensitivity anilsis conlducteU alone the lines suggested by that discussion Tecinological change generates a surplus which can be appropriated by dilkircnt sectors or groups within the economy Who benelitdepends in part on the cotaniodities selected for research and the types of technoloLies generated One of the purposes of institutionalizing a formal prioritN- setting procedureis to help mediate conflicts that may arise (for example between producer groups and consumers) as a result of research-induced improved technologies

Other issues arose during the analyses First the initial grouping and elimination of certain commodities and research areas before lhe irialvis is an important step but one which must be taken with carelul consideration of possible coniplmentarities in research on particular productS or types of rearch Second some of the commodities such as forages are inputs into other commodities such as milk (Corseqtentlv these commodities llay have to he considered itscinps because it is the value-added and not tilegross value of output that is relevant when one coniiodity requires other commodities sinputs and when criteria such as value of production are used This was an issue in I ruguav where livestock are quite important liird little attention was devoted in atv of the studies to defining appropriate criteria Ior the increased inoome stability eoal Future studies if this be an ideltJiiLd LOal shOUld define risk criteria

In sutImary the weighted criteria model as applied in Lcuador and iuguav which had the advantage of bit ildin Oillessons lCarICd illdie l)ominican RCpublic appears to have been a uselul priority-setting tool It ha its ereatest advartage ill situations ill which there are rulItiple goals to consider a relatively major reassesnieiit of prioritiesis contenplated and the list of commodities dd resen rch areas to consider is largeWeighted criteria models described in this paper were applied in small countries with limited resources but with direct involvement (except in the Dominican Republic) ofthC major decision makers These models itav be most useful in the small country situationwhere decision mAkers are more likely to be involved in tile priority-settirt process and are not just using tables of results Perhaps in larger systet7 where decisioii makine is more decentralited tileweighted criteria model can he applied at the regional level before relining research priorities at the national level

Another factor that can be uniortant in large svvtern is tle complerientarity between resea-h aid educat ion [hie larger the svstcnt the clser the linkages tend to be between universities and tileresearchi system Where universities are important tor agriclltural research an aidditional criterion may hc rteedCd to place value on the educationlal benefits of certain types of reseirch

Ihere are several additional iniplications with respect to how weighted criteria models relate to alterniative priori t-setting Inethod parIicularl the expected economic surplusapproach Befoce exploring these relationships kcx aspects of the surplus approacliare described based on aii application in Peru

14

goals They identified neeled technologies time and financial costs and the probability of research success and adoption Finally they specified the scientists workingobjectives This model is very thorough but did require extensive amounts of data and estimation of several mathematical relationships

Comparative Assessment

The above discussion is a brie summary of the major structu al approaches that have been used to asist decision makers in research priority setting All of them attempt to go beyond the usual procedure of individual or group decision makin e without structural analyses Each of the approacles has its advantages and disadvantages (See Fable I)

The procedure of establishing criteria and using weights to arrive at a final set of research priorities has the advantage of forcing decision makers to consciously trade off multiple goals It can illcorporae both quantitative and qualhtative information and can be applied to a long list of con mnodities or research areas in a relatively liort period of time The procedure is relati cx ca sY fbr administrators to understand but does requiretheir time in obtaining the explicit w5eights for criteria Furthermore these weights are inevitably subjective and their elicitatlion must be carefully structured [lie nActhod also requires scientists time in collecting inlormation on qualitative criteria As a result the approach k better suited for periodic or major priority-setting efforts than or situations where frequent marginal changes are anticipated

The expected economic surplus approach his the major advantage of incorporatingseverml criteria related to economic elficiency and distribution into one or two measures It also can be ucd to examine the gencral equilibrium ellects ofresearch (Ramailho de Castro and Sclhh nAid tile benefits of research under alternative possibly distortionar domestic pticing and international trade policies tEdwarts and Freebairn Alston Edwards and rcecbairn Norton and (ianoza 1985) These alictors are pervasive in developing countries and affect the efficiency and distributional consequences of researh Ile procedure ie(liCe ir lel - tndcrstandw ofeconomic analysisand more anal-t s time than the weighted criteria model but less aduinistrators time It can be diflicult to apply to at large number of commodities or research areas because certain typcs of data necessary for the ainalysis often do not exist for all commodities With adequaIL tL d ita Cxpected economic surplus analysis can be incorporated into the weighted criteria model or could be used on the set of commodities which the weightedcriteria model imdicaMes to have the highest piority lhe latter approach would allow for the calculation of income foregone as a reult of placing weights on non-economic ef1ciency criteria Expected economic surplus analysis has the major advantage of calculating a rate of return which can be compared to alternative public investments

The mathematical programming approach is similar to the weighted criteria model approach becaue weights are placed on a set of goals or criteria The procedure has the advantage of explicitly considering the budget huLman resource aad other constraints on the research system tnless the constraints are well specified including possiblechanges over time liowever there is a risk of nonsense solutions The model is more intensive of an ecolonic analysts time and abilit than tie simpler weighted criteria approach and decision makers may be less willing to accept what appears to be a black box solution lrade-offs among goals are easily quantified ssith this approach

The advantage of simulation models is their tlexibility They can be constructed is relatively simple or complex tools can incorporate optimizing or ranking proceduresand can readily include probabilistic information Their major disadvantage is that to be useful they must be relatively complex and typically reqtuire extensive amounts of both data and time of skilled analysts

4

TABLE 1 COMPARISON AMONG MAJOR AGRICULTURAL RESEARCH PRIORITY-SETTING METHODS

PRIORITY-SETTING METHOD

HEIGHTED EXPECTED MATH CHARACTERISTIC CRITERIA ECON SURPLUS PROGRAM SIMULATION

OPERATIONAL CONSIDERATIONS

1 Relative cost in researchers time mudium mJium medium high2 Relative cost in priority setting analysts ime medium medium high high3 Relative cost in administrators time medium mediummedium medium 4 Relative overall data requirement medium medium medium variable 5 Relative ease of comprehonsion by decision maker high medium low low 6 Ease of incorporating subjective informaticn high high high high7 Ease of incorporating non-quantitativa inforiation high low low medium

GOAL-RELATED ISSUES

a Requires explicit elicitation of goals yes usually yes uiually9 Can determine distributional affects on consumers

and producers at various income levels no yes no yes10 Can handle uncertainty yes yes yes yes11 Can consider tradeoff among multiple goals yes sometimes yes yes

CRITERIA-RELATED ISSUES

12 Can consider private-sector research incentives yes difficult difficult yes13 Can consider economic policy and trade affects yes yes yes yes

EVALUATION-RELATED ISSUES

14 Can be used to sot priorities for research at the aggregate level no yes no es

15 Can be used to set research priorities at the commodity level yes yes yes yes

16 Can be used to set priorities for non-production or non-commodity oriented research yes difficuli yes yes

17 Can be used to set priorities for basic research yes difficult no sometimes 18 Can evaluate secondary impacts of rosuarch on

employment environment nutrition yes sometimes so etimes yes19 Usually estimates a rata of return to research yes sometimesno no 20 Can quantify geographic spillover effects no yes no yes 21 Can consider the lags involved in research

and adoption yes yesyes yes22 Facilitates priority setting when the number

of commodities is large yes difficultdifficult difficult

Source Adapted and modified from Norton and Davis

5

The above is a very brief summary of the major advantages and disadvantages of the most common formally structured priority-setting models that could be used for strategicplanning for research resource allocation Other approaches exist for research projectselection with perhaps models that attempt to identify and analyze key vield constraints being the most prevalent approach employed in NARS of less-deeloped countries

The results of ex post analysis also can provide useful guidance for ex ante research resource allocation decisions if appropriately incorporated into a systematic ex ante procedure The most common ex post approach in addition to cx post benefit-cost analysis is the econometric estimation of production or supply functions incorporating research variables To be most uteful for ex ante analysis econometric approaches mLlSt be applied with a high degree of disaggrega tion Unfortunateiy these models require a substantial amount of high-quality historical data on production farm inputs and research expenditures

Perhaps the two approaches with the most potential given additional refinements for application by NARS in less-developed countries are the weighted criteria models and the benefit-cost or expected economic surplus procedures In the next section reent experiences with applying weighted criteria models in three Latin American Countries )ominican Republic Ecuador and Uruguay are discussed and evaluated The purpose

it to piovide more iniormation about the procedures their degree of acceptance and possible improvements in the models Ihen application of expected economic surplus models are considered drawing cn a recent cx ante research evaluation studly in Peru Finally the relationship between weighted criteria and expected economic surpluts methods is described along with possible extensions

IIl THE USE OF WEIGHiTED CRITERIA MODELS IN TIlE DOMINICAN REPUBLIC ECUADOR AND URUGUAY

in 1986 iif I87 weighted criteria models were developed and implemented in the lDomin lRepublic (ISA) Ecuador (Espinosa et al) and Trugua (CIAAB)to assist with priority setting for agricultwral research In the I)ominican Republic the study was carried out at the Instituto Superior de Agnicultura ISA) by consultants for ISNAR In Ecuador the study was conducted by the plmnintg ollice oY the Instituto Nacional de Investigacion Agropecuaria (INIAP) with the assistance of consultants for the US Agency for International Development (AID) In Uruguay the study was conducted by the Ollice of the Director of the Centro de Investigacion Agricola Alberto Boergcr(CIAAB) with the assistance of ISNAR The purpose of all three studies was to applyprocedures for prioritizing agricultural research by commodity and by major research area Inthe Dominican Republic the client was the newly created natonal agriculturalresearch agency (Il)EA) In Ecuador the client was INIAP and the newly created foundation for agricultural research (IEl)IA) In Urugua the client was C1IAB

The procedures employed in the three studies were similar but with some importantdilfereices as the Ecuador study built on lessons learned inthe I)ominican Republic and the Uruguay study built on other lessons learmed in Ecuador National goals for the research system iere elicited in each country and a series of criteria establishd which relate to those goals Separate riteria were developed for conmitodities and or research areas and weights were elicited from decision makers to establish the relative importanceof the criteria Commodities and research areas were ranked according to each criterion and these rankings were multiplied by the elicited weights to arrive at research priorities

6

Goals

Three goals were identified in each of the three countries (1) to raise the average levelof income in the country (2) to increase the well-b(Aing of low-income groups in societyand (3) to reduce yearto-year income fluctuations in the country especially onl tiledownside These goals were worded slightly diffrcntly in each country but in each casethey essentially relate to the three goals referred to in previous studies as efficncy equity and security In the l)ominican Republic the local consultants identifiedthese as the goals while in lcuador and Uruguay it was done by the directors of national research agencies

Criteria

The next step was to establish criteria to as measures of whetheruse particularcommodities or research areas contribute to the attainment of the above goals A largeset of criteria was discussed in each country and a total of 15 criteria were eventuallyused in the Dominican Republic 14 in lEcuador and 10 in Uruguay to determine priorities by commodity Five criteria were used in each case to determine priorities byresearch area Refinements were made in successive studies - beginning with theDominican Republic then Ecuador followed by Iruguay - to increase the independenceof criteria and to remove criteria that were questionable measures of whether researchcontributed to tile tated goals ( Reason for dropping particular ci eria are discussedbelow) Refinements also were made with respect to the grouping of criteria to facilitatethe weighting of goals and criteria

Commodity Criteria omioditv criteria were grou l - J into four conceptual groupsproduct importance probability O success elliciencv in use of research resources anddistribution of impacts The first three of these groups relate to the efficiency or incomelevel goal and the last group relates to the equity or distributional goal In none of thethree studies were criteria included which represnted the third goal (reduced incomelluctuation [lie feeling was that the entire set of criteria as a group would serve toreduce the enlhasis placed on single or 1ew (often export) crops While this appearedto be the case in the results of thee particular studies ft ture weighted criteria studies may want to add criteria which explicitly rank commodities from lowest to hiehest withrespect to annual gross income variability (or price and yield variability separately)

The group of criteria referred to as jrodct importap contained Four criteria in theUruguay study value of production generation or saving of foreign exchange expectedfuture demand change and comparative advantage

The value of productionl is an important criterion because the cost of rseaich isbasically independent of the number of units alkected by the research results at least forthose commodities with relatively homogeneous production conditions Foreignexchange was used as a criterion because of a perception in each country that the lackof foreign exchange was impeding growtb A criterion was set up to allow more weightto be paced on those commodities for which demand is expected to increase in thefuture because the importance of commodities will change as diets are altered withincome growth aod a world markets change Comparative advantage was includedbecause income in the country will be higher if the country focuses its efforts oncommodities for which its resource base is best suited while recognizing that research itself can alter resource constraints

In the Dominican Republic and in Ecuador nutritional measurcs (calorie and proteincontributions to the diet) were included as criteria for product importance They wereincluded because the well-being of low-income groups may be affected by the availability

7

of foods which are important in their diets Nutritional criteria were dropped in Uruguay however because numerous studies have shown that income levels are tile primary determinants of nutrition levels rather than the availability of particular foods Certainly caloric and protein content are debatable criteria To the extent that increased domestic production lowers the prices of certain food commodities therebyincreasing food consumption by the poor nutritional criteria can contribute to the distributional goal Land area also was included as a criterion in the Dominican Republic study but was dropped in Ecuador and Uruguly because of the overlap with value of production

Two criteria were used in each study to measure 1robahility of reseatch success The first measure was tilegap between own-country yields and neighboing country yields The second was the potential for success indicated by the researchers themselves The rationale for the gap criterion was that the larger the yield gap between the home country and similar countries tle greater the potential for yield gains The criterion is rather crude because yield gaps also can indicate a relative resource disadvantace forproducing the commodity even if the countries appear to be geoclimatically sinilar Consequently the gap criterion was eventually dropped in the tUruguav study The potential for success as indicated by the researchers is subjective but a very necessary measure to obtain

The gro up of criteria reficrred to as e1)cienc in use otresearch resources contained three criteria the relationship to research in the international centers the degree of emphasis on tilecommodity in le current research p ogram and the incentive for the privatesector to conduct the research Some research duplicates other research croplementsand still other research bears no relationship to research at tileinternational agriculturalresearch centers For certain comniodities all three relationships may hold depending on the type of research If national research is complementary to international researchthe cost of the rational research is lower Consequently a criterion was established which places greater weight on commodities with a high degree of complementaritybetween ntin Il d international research The criterion ofdelegree of emphasis in tile current iesercli program was established because there is a cost to rapidly adjustingcurrent h uiman and physical research resources from one commodity to another The criterion of private-sector incentives was used because the social benefits from the total research program in the country will be greatest if scarce public research funds are devoted to research on commodities for which the private sector does not have an incentive to support the research itself Of coui se the private-sector producer group can tax itself and pay the public sector to conduct the research for those commodities for which it has an incentive to privately support the research These commodities tend to be export commodities because producers can capture a higher proportion of the gainsfrom productivity increases for export commodities than for commodities primarily consumed domestically

The group of criteria related to the distribution ojresarchimpacts contained two criteria in the Uruguay study number of producers and the effect of increased productivity on the price of product The larger the number of farms producing a commodity the largerthe number of farmers afficted by the research and perhaps the higher the probabilitythat smaller farmers are being affeicted The distribution of benefits depends to a significant degree on price changes resulting from increased productivity Producers benefit more if prices decline very little and consumers benefit more ifprices decline substatitially for a given production increase For export er import commodities priceswill tend to decline less than for commodities primarily produced and consumed domestically For commodities which are not traded and are very important in the diets of the poor prices will tend to decline relatively sharply when productivity increases The division of benefits between producers and consumers also is affected by the fact

that producers are also consumers Consequently in the Dominican Republic and Ecuador studies an additional criterion importance of the commodity in home-consumption on the farm where produced also was included

In summary 10-15 criteria were used in tie three studies to identify and select research priorities by commodity The linkages anong the goals and criteria described above are summarized in Figure 1

Research Area Criteria The five criteria uicel to select research priorities by research aieas were (1 ) whether the research causes an increase in the use of relatively abundant resources and a saving of relatively scarce resources (2) the number and seventy of research problems (3) non-duplication with transferable research from outside the country (4) tne extent of private-sector incentives to conduct the research and (5) current emphases in the research program

The rate of economic growth will be more rapid if new technologies are developed that utilize relatively abundant resources and save relatively scarce resources The resource base varies by region consequently information on resource abundance criteria was collected by region in tileEcuador and Uruguay studies The same was true for the quantity and severity of research problems Information on the relationship of domestic to inernational research private-sector incentives and the current emphasis in the research program was collected on a naticnal basis These criteria all relate to the income growth (efficiency) goal It is difficult to identif research area criteria that measurc whether particular types of research aifect income distribution or variability

Data Collection and Model Implementation

Information used in the analysis included both quantitative data on value of productionnumber of farms value of exports and imports person-years devoted to research on different commodities and tile of protein per day in thenumber of calories and gramsdiet (for tileDominican Republic and Ecuador studies) as well as qualitative or subjective information on such factors as probability of success private sector incentives and severity of research problems Furthermore weights had to be elicited from decision makers to place relative emphasis on the various criteria

Step 1 Develop Cotninodity and Research Area Lists -- The first step was to determine the list ofcommadities to consider and the list of research areas In the Dominican Republic information was collected on 74 commodities In Ecuador an initial list of 109 commodities on which INIAP was conducting research was discussed Throughdecisions to eliminate some and to group others this list was reduced to 44 commedities for the analysis These decisions were made by the Technical Director for Research In Uruguay a smaller list of commodities was reduced to 21 commodities or commodity groups These decisions were made in consultation with the National Research and Fxtension Directors The list of research program areas contained nine areas in the Dominican Republic 16 in Ecuador and 16 in Uruguay

Step 2 Collect Quantitative and Qualitative Data on Chosen Criteria -- Information o-i quantitative criteria were gathered from local and FAO secondary data sources and one table was constructed For each criterion Commodities were ranked for each criterion Information needed for the qualitative riteria as well as the weights to place on criteria were obtained through interviews with sc rtists and administrators at boh the national and regional experiment stations In Euador and Uruguay substantial attention was devoted to identifying the appropriate respondents to each question as this is a crucial aspect of the procedure In Ecuador those interviewed were notified one week before the interviews and 72 people were interviewed by the Planning Director of INIAP and

9

FIJR 1 CONCEPTUAL ELEHNTS IN THE HEIGHTED CRITERIA MODEL FOR PAWING RESEARCH PRIORITIES BY COtlDITY

GOALS fficiency ]

CONCEPTUAL( )CRITERIA Pri7] Probabi i+y Research Distriutjon Iincome

ii J of Succes Resource Eff o L=Acplusmn a rilbility

CRITERIA 1 Value of 1 Scientist 1Rl ofto toNumberPrcduction Estimates 1 AnnualInternational Producers Variability2 Foreign Centers 2 Price in Gro

Exchange 2 Current Effect3 Future Raturn3)Emphasis 3 HomeDemand 3 Private Consumpt ion

4 Comparative Sector Advantage Incentives

Notes

(-n future studies an additional conceptural criteria under the security goal might be health (J4tri ional criteria such as grams of potein or kilo=alories could be included as additional easures of Distributional ikipacts

4ot used in Dominican Republic Ecuador and Uruojuay studies

his assistant In Uruguay the directors of all the regional stations were called into CIIAB headquarters for a aweting to discuss the study and this was followed up byon-site interviews and meetings with scientists and directors at the regional experimentstations as well as with economists in the Ministry of Agriculture These interviews and meetings were conducted by a person assigned to work wvith the Director of Cl LAB on the priority-setting study In the Dominican Republic a survey of scientists extensionworkers and administrators was conducted by the ISNAR consultant from the Instituto Superior de Agricultura who visited the various experiment stations Less care wastaken in the Dominican Repubic than in Ecuador and Uruguay to match the questionspertaining -o individual criteria to particular individuals This probably reduced theusefulness of the responses because those most familiar with the probability of research success For example may not be the same -s those most Familiar with comparativeadvantage

Step 3 Elicit 11eihts on Critria -- Relative weights to place on the different criteria were obtained Fronm national and regional research system administrators In Ecuador34 peopic verc used to determine the weights These wveights were established separatelyIor the commoditv and the research area criteria In 1iuguay fewer people (sevensystem and station directors) were used to determine the weights and more attention was devoted to grouping the criteria pertaining to particular goals A Delphi procedure was used in which the s7even directors were shown the average of the group and provided with an opportunity to adjust their weight

Step 4 )erive Rankinqs hv Commodity and By Research Area -- Once the basic information was collected axd organized into a seies of tables one procedure was followed to arrive at a final ranking of research priorities by commodity A second procedure resulted in lists of priorities by research area In the Dominican Republicmost of the calculations were made by hand in Ecuador and Uruguay microcomputerspreadsheet programs were used to assist with the calculations and to conduct sensitivity analysis

The steps in the procedure for ranking research priorities by commodity and research area are sumnmaried below and are illustrated with a simple example in Appendix 1 Brieily the steps for ranking commodities were

a) Th conniodities were ranked for each criterion for which quantitative data were available

b) Tihese rankings were multiplied by the weights assigned to each criterion and then the weighted criteria were added across to arrive at one sub-ranking for the

Luantitative criteria c) Each commodity was given a high low or none designation for each qualitative

(subjective) criterion The response which implied a need for greater research priority was assigned the number 2 the intermediate response was assigned I and the response which implied lower research priority was assigned 0 1herefore each comniodity received a 2 1 or 0 for each criterion

d) The weights assigned to each qualitative criterion were then multiplied by these numbers and the results were added across all criteria to arrive at a subtotal for each commoditv These subtotals were ranked from highest to lowest to provide a second sub-ranking

e) The sub-rankings for the quantitative and qualitative criteria were then given their corresponding weights and added together to arrive at a final ranking by commodityThe steps for ranking research priorities by research area were the same as steps (c)and (d) above because the criteria were all qualitative

Step 5 Analsis and Interpretationof Resuls -- In the Dominican Republic the resuLts of this weighted criteria analysis were uscd to determine a small set of cormodities and

II

research areas with the highest priority Further assessment was then made of humanphysical administrative and other resources needed to structure research programsfocused on these commodities and resea-ch topics In Ecuador and Uruguay the prioritized list of commodities was split into a high-priority group an intermediate-priority group and a low-priority group Research area priorities were identified for each region of tile country A more detailed analysis was then made of the human physical and other resources needed to support the high and intermediate priorities

In summary five major -rens were involved in operationalizing the weighted criteria models A complete set of rsults for each study included more than 20 tables and are not reproduced here Ilowever the final rankings for commodities and for research areas are shown in Tables I and 2 in Appendix 2 In the Dominican Republic a team of consultants followed up the initial priority-setting exercise with additional analysisand discussions at the experiment station level and eventually recommended tile establishment of five national commodity programs (rice cornsorghum beans coffeeand livestock) and one additional major research program (natural resources) In Ecuador the prioritized lists were widely distributed and discussed in the Ministry oi Agriculture the Board of FEIDIA the Commission on Science and Technology arid in AID There was concern in Ecuador with the weights placed on the individual criteria and therefore additional sensitivity analysis was conducted by the Planning Office of the Ministry There seemed to be a recognition in Ecuador that priority setting with a weighted criteria model is an iterative process and that much 4t the models value stems from tile discussion of criteria among the decision makers and from the sensitivityanalysis The high-priority list of comniodities in Ecuador included thL ten highest-priority commodities listed in Table 2b A project paper has been written forAI) which recommends funding For Fli)IAlbr three commodities (coffee soft corn for the Sierra and dairy) Coffee and soft coitn were the number 2 and 3 ranked prioritiesin tile weighted criteria model and dairy was number 9 although it was the top-rankedlivestock commodity

In Urugu ay the results from the first run with tile model were discussed with the research directors Ilie directors made small changes to the weights placed on criteriaand the mode was rerun resulting in the prioritized list in Appendix 2 Personnel in CIAAB are CUrrcwtv undertaking additional sensitivity analysis and with tie assistance of ISNA R are developing a plan for implementation

An Assessment of the Weighted Criteria Models Used in the Dominican RepublicEcuador and Uruguay

The Dominican Republic Ecuador and Uruguay studies were perhaps the first majorattempts to implement weighted criteria models with weights explicitly elicited from decision makers in NARS in less-developed countries A number of strengths andweaknesses in the procedures were identified during the analyses Some of thL weaknesses were corrected fron one study to the next some can be corrected in future studies and others are inherent in the approach

Trhe first strength mentioned earlier is the ability of the procedure to systematicallyincorporate both quantitative and qualitative information related to a set of multiplegoals and criteria in order to prioritize a lorng ist of commodities and research areas in a relatively short period of time Second the procedure proved relatively easy for both research administrators and the loc- -nalysts to understand with the exception of the )ominican Republic where the research administrators were not directly involved7

Third the analysis as applied in Ecuador and ULuguay with the direct involvement of research system administrators at various stages forced those decision makers to

12

consciously identifv and trade off goals and criteria Fourth the use of spreadsheet programs in EcUadfor and Uruguay facilitated sensitivity analysis after the initial set of priorities was deermimd Fifth the procedure provided a relatively objectiveassessment of prioriies because individuals were not allowed to rank commodities orresearch areas directly but had to weight criteria

The first weakness inherent in ex ante research priority setting and thus in theprocedure is that there is a large amount of subjectiveness Although tileapproach isless subjective than unstructured judgment there is subjectiveness in the responses toquestiorns related to some of the criteria and also ini the weights placed by decision makers on the criteria Second it proved diflicult to specify independent criteria with no overlap Third while the procedure required less time than an expected economic surplus analysis by commodity it required more time than asking respondents to directlyrank commodities and research areas In E-cuador the consiultant worked withpersonnel in INIAP and FEI)IA for one week at the start of the analysis to determinegoals and criteria to arrive at atlist of commodities and researh areas to list secondarydata needs to formulate questions to be asked of administrators and scientists to decide who should be asked particular questions and to explain the procedure itself includingthe spreadsheet program The required iidormation was collected over the Followingmonth and the consultant returned and worked with the Planning Director in INIAP for one week to conduct the analysis and write tip the results Revisions to the initial draftof the report we-e then made over the lollowing weeks A similar procedure to the onefollowed in Ecuador was used in l_ruguay althaugli more time was spent discussing the wording of questions to be asked of scientists Furthermore n Uruguay more care was taken to match the criteria with the goas

Certain of the criteria wil always be diflicult to eplain to tileinterviewees Thecriterion on comparative advantage is difficult for some respondents because of a lack of knowledge about neighboring countries If suflicient data time and resources are available this criterion may be rnea4ured more precisely using comparative advantage or domestic resource cost procedures suggested by longmire and Winkelnann Anotherdiflicult criterion to estiniate is the potential foture demand for the product The respondents in Fcuador were asked to consider how the demands for different foodswould change as income and populatiot grow Many of them had a hard timeconceiving that the total or per capita demand for certain Foods could actually decline as income grew In Uiruguay (ibe cmiparative advantage and future dema nd questions were answered by a small group of economists from the ministries of agriculture and trade In the Ecuador study more attention was devoted to assessing the most appropriate people for answering different questions than was the case in the Dominican Republic Ilowever some criticism surfaced about the weights placed on criteria InUruguay decision makers were given the opportunity to change their weights afterviewing the initial results This was very useftl because it demonstrated the implicationsof placing different weights on the various goals and criteria

Computer spreadsheet programs were a major help in Ecuador and Uruguay Additional work needs to be devoted to developing a more mentnu-driven prcgram but the people inthe planning office of INIAP were able to use the SuperCalcR program without too tmucht difliculty despite ialack of previous experience Ihe analyst in Uruguay was veryfamiliar with lotusl which proved most helpfil

It became clear from working inall three countries that it is preferable to work directlywith the final decision makers or their designees Inthe l)ominican Republic the study was conducted by a very competent set of consultants from the local agricultural universi ty but the procedure was not institutionalized to allow for additional sensitivityanalysis or future priority-setting efforts InEcuador and especially in Uruguay the

13

decision makers were directly involved conducted sensitivity analyses and were able to assess why certain commodities or research areas received high priority as a result of the analyses In both Ecuador and Uruguay they have a basic understanding of the strengths and weaknesses of tileprocedure and therelre can explain their subsequentdecisions to groups who might disagree with tilepriorities they establish

The corntrast between the Dominican Republic and tie Uruguay studies illustrates a veryimportant point No matter which technique is used to structure the analysispriority-setting exercises must integrate the decision makers (users) into the processPriority-setting procedures must be institutionalited the results of initial analysisdiscussed thoroughly with decision makers and sensitivity anilsis conlducteU alone the lines suggested by that discussion Tecinological change generates a surplus which can be appropriated by dilkircnt sectors or groups within the economy Who benelitdepends in part on the cotaniodities selected for research and the types of technoloLies generated One of the purposes of institutionalizing a formal prioritN- setting procedureis to help mediate conflicts that may arise (for example between producer groups and consumers) as a result of research-induced improved technologies

Other issues arose during the analyses First the initial grouping and elimination of certain commodities and research areas before lhe irialvis is an important step but one which must be taken with carelul consideration of possible coniplmentarities in research on particular productS or types of rearch Second some of the commodities such as forages are inputs into other commodities such as milk (Corseqtentlv these commodities llay have to he considered itscinps because it is the value-added and not tilegross value of output that is relevant when one coniiodity requires other commodities sinputs and when criteria such as value of production are used This was an issue in I ruguav where livestock are quite important liird little attention was devoted in atv of the studies to defining appropriate criteria Ior the increased inoome stability eoal Future studies if this be an ideltJiiLd LOal shOUld define risk criteria

In sutImary the weighted criteria model as applied in Lcuador and iuguav which had the advantage of bit ildin Oillessons lCarICd illdie l)ominican RCpublic appears to have been a uselul priority-setting tool It ha its ereatest advartage ill situations ill which there are rulItiple goals to consider a relatively major reassesnieiit of prioritiesis contenplated and the list of commodities dd resen rch areas to consider is largeWeighted criteria models described in this paper were applied in small countries with limited resources but with direct involvement (except in the Dominican Republic) ofthC major decision makers These models itav be most useful in the small country situationwhere decision mAkers are more likely to be involved in tile priority-settirt process and are not just using tables of results Perhaps in larger systet7 where decisioii makine is more decentralited tileweighted criteria model can he applied at the regional level before relining research priorities at the national level

Another factor that can be uniortant in large svvtern is tle complerientarity between resea-h aid educat ion [hie larger the svstcnt the clser the linkages tend to be between universities and tileresearchi system Where universities are important tor agriclltural research an aidditional criterion may hc rteedCd to place value on the educationlal benefits of certain types of reseirch

Ihere are several additional iniplications with respect to how weighted criteria models relate to alterniative priori t-setting Inethod parIicularl the expected economic surplusapproach Befoce exploring these relationships kcx aspects of the surplus approacliare described based on aii application in Peru

14

TABLE 1 COMPARISON AMONG MAJOR AGRICULTURAL RESEARCH PRIORITY-SETTING METHODS

PRIORITY-SETTING METHOD

HEIGHTED EXPECTED MATH CHARACTERISTIC CRITERIA ECON SURPLUS PROGRAM SIMULATION

OPERATIONAL CONSIDERATIONS

1 Relative cost in researchers time mudium mJium medium high2 Relative cost in priority setting analysts ime medium medium high high3 Relative cost in administrators time medium mediummedium medium 4 Relative overall data requirement medium medium medium variable 5 Relative ease of comprehonsion by decision maker high medium low low 6 Ease of incorporating subjective informaticn high high high high7 Ease of incorporating non-quantitativa inforiation high low low medium

GOAL-RELATED ISSUES

a Requires explicit elicitation of goals yes usually yes uiually9 Can determine distributional affects on consumers

and producers at various income levels no yes no yes10 Can handle uncertainty yes yes yes yes11 Can consider tradeoff among multiple goals yes sometimes yes yes

CRITERIA-RELATED ISSUES

12 Can consider private-sector research incentives yes difficult difficult yes13 Can consider economic policy and trade affects yes yes yes yes

EVALUATION-RELATED ISSUES

14 Can be used to sot priorities for research at the aggregate level no yes no es

15 Can be used to set research priorities at the commodity level yes yes yes yes

16 Can be used to set priorities for non-production or non-commodity oriented research yes difficuli yes yes

17 Can be used to set priorities for basic research yes difficult no sometimes 18 Can evaluate secondary impacts of rosuarch on

employment environment nutrition yes sometimes so etimes yes19 Usually estimates a rata of return to research yes sometimesno no 20 Can quantify geographic spillover effects no yes no yes 21 Can consider the lags involved in research

and adoption yes yesyes yes22 Facilitates priority setting when the number

of commodities is large yes difficultdifficult difficult

Source Adapted and modified from Norton and Davis

5

The above is a very brief summary of the major advantages and disadvantages of the most common formally structured priority-setting models that could be used for strategicplanning for research resource allocation Other approaches exist for research projectselection with perhaps models that attempt to identify and analyze key vield constraints being the most prevalent approach employed in NARS of less-deeloped countries

The results of ex post analysis also can provide useful guidance for ex ante research resource allocation decisions if appropriately incorporated into a systematic ex ante procedure The most common ex post approach in addition to cx post benefit-cost analysis is the econometric estimation of production or supply functions incorporating research variables To be most uteful for ex ante analysis econometric approaches mLlSt be applied with a high degree of disaggrega tion Unfortunateiy these models require a substantial amount of high-quality historical data on production farm inputs and research expenditures

Perhaps the two approaches with the most potential given additional refinements for application by NARS in less-developed countries are the weighted criteria models and the benefit-cost or expected economic surplus procedures In the next section reent experiences with applying weighted criteria models in three Latin American Countries )ominican Republic Ecuador and Uruguay are discussed and evaluated The purpose

it to piovide more iniormation about the procedures their degree of acceptance and possible improvements in the models Ihen application of expected economic surplus models are considered drawing cn a recent cx ante research evaluation studly in Peru Finally the relationship between weighted criteria and expected economic surpluts methods is described along with possible extensions

IIl THE USE OF WEIGHiTED CRITERIA MODELS IN TIlE DOMINICAN REPUBLIC ECUADOR AND URUGUAY

in 1986 iif I87 weighted criteria models were developed and implemented in the lDomin lRepublic (ISA) Ecuador (Espinosa et al) and Trugua (CIAAB)to assist with priority setting for agricultwral research In the I)ominican Republic the study was carried out at the Instituto Superior de Agnicultura ISA) by consultants for ISNAR In Ecuador the study was conducted by the plmnintg ollice oY the Instituto Nacional de Investigacion Agropecuaria (INIAP) with the assistance of consultants for the US Agency for International Development (AID) In Uruguay the study was conducted by the Ollice of the Director of the Centro de Investigacion Agricola Alberto Boergcr(CIAAB) with the assistance of ISNAR The purpose of all three studies was to applyprocedures for prioritizing agricultural research by commodity and by major research area Inthe Dominican Republic the client was the newly created natonal agriculturalresearch agency (Il)EA) In Ecuador the client was INIAP and the newly created foundation for agricultural research (IEl)IA) In Urugua the client was C1IAB

The procedures employed in the three studies were similar but with some importantdilfereices as the Ecuador study built on lessons learned inthe I)ominican Republic and the Uruguay study built on other lessons learmed in Ecuador National goals for the research system iere elicited in each country and a series of criteria establishd which relate to those goals Separate riteria were developed for conmitodities and or research areas and weights were elicited from decision makers to establish the relative importanceof the criteria Commodities and research areas were ranked according to each criterion and these rankings were multiplied by the elicited weights to arrive at research priorities

6

Goals

Three goals were identified in each of the three countries (1) to raise the average levelof income in the country (2) to increase the well-b(Aing of low-income groups in societyand (3) to reduce yearto-year income fluctuations in the country especially onl tiledownside These goals were worded slightly diffrcntly in each country but in each casethey essentially relate to the three goals referred to in previous studies as efficncy equity and security In the l)ominican Republic the local consultants identifiedthese as the goals while in lcuador and Uruguay it was done by the directors of national research agencies

Criteria

The next step was to establish criteria to as measures of whetheruse particularcommodities or research areas contribute to the attainment of the above goals A largeset of criteria was discussed in each country and a total of 15 criteria were eventuallyused in the Dominican Republic 14 in lEcuador and 10 in Uruguay to determine priorities by commodity Five criteria were used in each case to determine priorities byresearch area Refinements were made in successive studies - beginning with theDominican Republic then Ecuador followed by Iruguay - to increase the independenceof criteria and to remove criteria that were questionable measures of whether researchcontributed to tile tated goals ( Reason for dropping particular ci eria are discussedbelow) Refinements also were made with respect to the grouping of criteria to facilitatethe weighting of goals and criteria

Commodity Criteria omioditv criteria were grou l - J into four conceptual groupsproduct importance probability O success elliciencv in use of research resources anddistribution of impacts The first three of these groups relate to the efficiency or incomelevel goal and the last group relates to the equity or distributional goal In none of thethree studies were criteria included which represnted the third goal (reduced incomelluctuation [lie feeling was that the entire set of criteria as a group would serve toreduce the enlhasis placed on single or 1ew (often export) crops While this appearedto be the case in the results of thee particular studies ft ture weighted criteria studies may want to add criteria which explicitly rank commodities from lowest to hiehest withrespect to annual gross income variability (or price and yield variability separately)

The group of criteria referred to as jrodct importap contained Four criteria in theUruguay study value of production generation or saving of foreign exchange expectedfuture demand change and comparative advantage

The value of productionl is an important criterion because the cost of rseaich isbasically independent of the number of units alkected by the research results at least forthose commodities with relatively homogeneous production conditions Foreignexchange was used as a criterion because of a perception in each country that the lackof foreign exchange was impeding growtb A criterion was set up to allow more weightto be paced on those commodities for which demand is expected to increase in thefuture because the importance of commodities will change as diets are altered withincome growth aod a world markets change Comparative advantage was includedbecause income in the country will be higher if the country focuses its efforts oncommodities for which its resource base is best suited while recognizing that research itself can alter resource constraints

In the Dominican Republic and in Ecuador nutritional measurcs (calorie and proteincontributions to the diet) were included as criteria for product importance They wereincluded because the well-being of low-income groups may be affected by the availability

7

of foods which are important in their diets Nutritional criteria were dropped in Uruguay however because numerous studies have shown that income levels are tile primary determinants of nutrition levels rather than the availability of particular foods Certainly caloric and protein content are debatable criteria To the extent that increased domestic production lowers the prices of certain food commodities therebyincreasing food consumption by the poor nutritional criteria can contribute to the distributional goal Land area also was included as a criterion in the Dominican Republic study but was dropped in Ecuador and Uruguly because of the overlap with value of production

Two criteria were used in each study to measure 1robahility of reseatch success The first measure was tilegap between own-country yields and neighboing country yields The second was the potential for success indicated by the researchers themselves The rationale for the gap criterion was that the larger the yield gap between the home country and similar countries tle greater the potential for yield gains The criterion is rather crude because yield gaps also can indicate a relative resource disadvantace forproducing the commodity even if the countries appear to be geoclimatically sinilar Consequently the gap criterion was eventually dropped in the tUruguav study The potential for success as indicated by the researchers is subjective but a very necessary measure to obtain

The gro up of criteria reficrred to as e1)cienc in use otresearch resources contained three criteria the relationship to research in the international centers the degree of emphasis on tilecommodity in le current research p ogram and the incentive for the privatesector to conduct the research Some research duplicates other research croplementsand still other research bears no relationship to research at tileinternational agriculturalresearch centers For certain comniodities all three relationships may hold depending on the type of research If national research is complementary to international researchthe cost of the rational research is lower Consequently a criterion was established which places greater weight on commodities with a high degree of complementaritybetween ntin Il d international research The criterion ofdelegree of emphasis in tile current iesercli program was established because there is a cost to rapidly adjustingcurrent h uiman and physical research resources from one commodity to another The criterion of private-sector incentives was used because the social benefits from the total research program in the country will be greatest if scarce public research funds are devoted to research on commodities for which the private sector does not have an incentive to support the research itself Of coui se the private-sector producer group can tax itself and pay the public sector to conduct the research for those commodities for which it has an incentive to privately support the research These commodities tend to be export commodities because producers can capture a higher proportion of the gainsfrom productivity increases for export commodities than for commodities primarily consumed domestically

The group of criteria related to the distribution ojresarchimpacts contained two criteria in the Uruguay study number of producers and the effect of increased productivity on the price of product The larger the number of farms producing a commodity the largerthe number of farmers afficted by the research and perhaps the higher the probabilitythat smaller farmers are being affeicted The distribution of benefits depends to a significant degree on price changes resulting from increased productivity Producers benefit more if prices decline very little and consumers benefit more ifprices decline substatitially for a given production increase For export er import commodities priceswill tend to decline less than for commodities primarily produced and consumed domestically For commodities which are not traded and are very important in the diets of the poor prices will tend to decline relatively sharply when productivity increases The division of benefits between producers and consumers also is affected by the fact

that producers are also consumers Consequently in the Dominican Republic and Ecuador studies an additional criterion importance of the commodity in home-consumption on the farm where produced also was included

In summary 10-15 criteria were used in tie three studies to identify and select research priorities by commodity The linkages anong the goals and criteria described above are summarized in Figure 1

Research Area Criteria The five criteria uicel to select research priorities by research aieas were (1 ) whether the research causes an increase in the use of relatively abundant resources and a saving of relatively scarce resources (2) the number and seventy of research problems (3) non-duplication with transferable research from outside the country (4) tne extent of private-sector incentives to conduct the research and (5) current emphases in the research program

The rate of economic growth will be more rapid if new technologies are developed that utilize relatively abundant resources and save relatively scarce resources The resource base varies by region consequently information on resource abundance criteria was collected by region in tileEcuador and Uruguay studies The same was true for the quantity and severity of research problems Information on the relationship of domestic to inernational research private-sector incentives and the current emphasis in the research program was collected on a naticnal basis These criteria all relate to the income growth (efficiency) goal It is difficult to identif research area criteria that measurc whether particular types of research aifect income distribution or variability

Data Collection and Model Implementation

Information used in the analysis included both quantitative data on value of productionnumber of farms value of exports and imports person-years devoted to research on different commodities and tile of protein per day in thenumber of calories and gramsdiet (for tileDominican Republic and Ecuador studies) as well as qualitative or subjective information on such factors as probability of success private sector incentives and severity of research problems Furthermore weights had to be elicited from decision makers to place relative emphasis on the various criteria

Step 1 Develop Cotninodity and Research Area Lists -- The first step was to determine the list ofcommadities to consider and the list of research areas In the Dominican Republic information was collected on 74 commodities In Ecuador an initial list of 109 commodities on which INIAP was conducting research was discussed Throughdecisions to eliminate some and to group others this list was reduced to 44 commedities for the analysis These decisions were made by the Technical Director for Research In Uruguay a smaller list of commodities was reduced to 21 commodities or commodity groups These decisions were made in consultation with the National Research and Fxtension Directors The list of research program areas contained nine areas in the Dominican Republic 16 in Ecuador and 16 in Uruguay

Step 2 Collect Quantitative and Qualitative Data on Chosen Criteria -- Information o-i quantitative criteria were gathered from local and FAO secondary data sources and one table was constructed For each criterion Commodities were ranked for each criterion Information needed for the qualitative riteria as well as the weights to place on criteria were obtained through interviews with sc rtists and administrators at boh the national and regional experiment stations In Euador and Uruguay substantial attention was devoted to identifying the appropriate respondents to each question as this is a crucial aspect of the procedure In Ecuador those interviewed were notified one week before the interviews and 72 people were interviewed by the Planning Director of INIAP and

9

FIJR 1 CONCEPTUAL ELEHNTS IN THE HEIGHTED CRITERIA MODEL FOR PAWING RESEARCH PRIORITIES BY COtlDITY

GOALS fficiency ]

CONCEPTUAL( )CRITERIA Pri7] Probabi i+y Research Distriutjon Iincome

ii J of Succes Resource Eff o L=Acplusmn a rilbility

CRITERIA 1 Value of 1 Scientist 1Rl ofto toNumberPrcduction Estimates 1 AnnualInternational Producers Variability2 Foreign Centers 2 Price in Gro

Exchange 2 Current Effect3 Future Raturn3)Emphasis 3 HomeDemand 3 Private Consumpt ion

4 Comparative Sector Advantage Incentives

Notes

(-n future studies an additional conceptural criteria under the security goal might be health (J4tri ional criteria such as grams of potein or kilo=alories could be included as additional easures of Distributional ikipacts

4ot used in Dominican Republic Ecuador and Uruojuay studies

his assistant In Uruguay the directors of all the regional stations were called into CIIAB headquarters for a aweting to discuss the study and this was followed up byon-site interviews and meetings with scientists and directors at the regional experimentstations as well as with economists in the Ministry of Agriculture These interviews and meetings were conducted by a person assigned to work wvith the Director of Cl LAB on the priority-setting study In the Dominican Republic a survey of scientists extensionworkers and administrators was conducted by the ISNAR consultant from the Instituto Superior de Agricultura who visited the various experiment stations Less care wastaken in the Dominican Repubic than in Ecuador and Uruguay to match the questionspertaining -o individual criteria to particular individuals This probably reduced theusefulness of the responses because those most familiar with the probability of research success For example may not be the same -s those most Familiar with comparativeadvantage

Step 3 Elicit 11eihts on Critria -- Relative weights to place on the different criteria were obtained Fronm national and regional research system administrators In Ecuador34 peopic verc used to determine the weights These wveights were established separatelyIor the commoditv and the research area criteria In 1iuguay fewer people (sevensystem and station directors) were used to determine the weights and more attention was devoted to grouping the criteria pertaining to particular goals A Delphi procedure was used in which the s7even directors were shown the average of the group and provided with an opportunity to adjust their weight

Step 4 )erive Rankinqs hv Commodity and By Research Area -- Once the basic information was collected axd organized into a seies of tables one procedure was followed to arrive at a final ranking of research priorities by commodity A second procedure resulted in lists of priorities by research area In the Dominican Republicmost of the calculations were made by hand in Ecuador and Uruguay microcomputerspreadsheet programs were used to assist with the calculations and to conduct sensitivity analysis

The steps in the procedure for ranking research priorities by commodity and research area are sumnmaried below and are illustrated with a simple example in Appendix 1 Brieily the steps for ranking commodities were

a) Th conniodities were ranked for each criterion for which quantitative data were available

b) Tihese rankings were multiplied by the weights assigned to each criterion and then the weighted criteria were added across to arrive at one sub-ranking for the

Luantitative criteria c) Each commodity was given a high low or none designation for each qualitative

(subjective) criterion The response which implied a need for greater research priority was assigned the number 2 the intermediate response was assigned I and the response which implied lower research priority was assigned 0 1herefore each comniodity received a 2 1 or 0 for each criterion

d) The weights assigned to each qualitative criterion were then multiplied by these numbers and the results were added across all criteria to arrive at a subtotal for each commoditv These subtotals were ranked from highest to lowest to provide a second sub-ranking

e) The sub-rankings for the quantitative and qualitative criteria were then given their corresponding weights and added together to arrive at a final ranking by commodityThe steps for ranking research priorities by research area were the same as steps (c)and (d) above because the criteria were all qualitative

Step 5 Analsis and Interpretationof Resuls -- In the Dominican Republic the resuLts of this weighted criteria analysis were uscd to determine a small set of cormodities and

II

research areas with the highest priority Further assessment was then made of humanphysical administrative and other resources needed to structure research programsfocused on these commodities and resea-ch topics In Ecuador and Uruguay the prioritized list of commodities was split into a high-priority group an intermediate-priority group and a low-priority group Research area priorities were identified for each region of tile country A more detailed analysis was then made of the human physical and other resources needed to support the high and intermediate priorities

In summary five major -rens were involved in operationalizing the weighted criteria models A complete set of rsults for each study included more than 20 tables and are not reproduced here Ilowever the final rankings for commodities and for research areas are shown in Tables I and 2 in Appendix 2 In the Dominican Republic a team of consultants followed up the initial priority-setting exercise with additional analysisand discussions at the experiment station level and eventually recommended tile establishment of five national commodity programs (rice cornsorghum beans coffeeand livestock) and one additional major research program (natural resources) In Ecuador the prioritized lists were widely distributed and discussed in the Ministry oi Agriculture the Board of FEIDIA the Commission on Science and Technology arid in AID There was concern in Ecuador with the weights placed on the individual criteria and therefore additional sensitivity analysis was conducted by the Planning Office of the Ministry There seemed to be a recognition in Ecuador that priority setting with a weighted criteria model is an iterative process and that much 4t the models value stems from tile discussion of criteria among the decision makers and from the sensitivityanalysis The high-priority list of comniodities in Ecuador included thL ten highest-priority commodities listed in Table 2b A project paper has been written forAI) which recommends funding For Fli)IAlbr three commodities (coffee soft corn for the Sierra and dairy) Coffee and soft coitn were the number 2 and 3 ranked prioritiesin tile weighted criteria model and dairy was number 9 although it was the top-rankedlivestock commodity

In Urugu ay the results from the first run with tile model were discussed with the research directors Ilie directors made small changes to the weights placed on criteriaand the mode was rerun resulting in the prioritized list in Appendix 2 Personnel in CIAAB are CUrrcwtv undertaking additional sensitivity analysis and with tie assistance of ISNA R are developing a plan for implementation

An Assessment of the Weighted Criteria Models Used in the Dominican RepublicEcuador and Uruguay

The Dominican Republic Ecuador and Uruguay studies were perhaps the first majorattempts to implement weighted criteria models with weights explicitly elicited from decision makers in NARS in less-developed countries A number of strengths andweaknesses in the procedures were identified during the analyses Some of thL weaknesses were corrected fron one study to the next some can be corrected in future studies and others are inherent in the approach

Trhe first strength mentioned earlier is the ability of the procedure to systematicallyincorporate both quantitative and qualitative information related to a set of multiplegoals and criteria in order to prioritize a lorng ist of commodities and research areas in a relatively short period of time Second the procedure proved relatively easy for both research administrators and the loc- -nalysts to understand with the exception of the )ominican Republic where the research administrators were not directly involved7

Third the analysis as applied in Ecuador and ULuguay with the direct involvement of research system administrators at various stages forced those decision makers to

12

consciously identifv and trade off goals and criteria Fourth the use of spreadsheet programs in EcUadfor and Uruguay facilitated sensitivity analysis after the initial set of priorities was deermimd Fifth the procedure provided a relatively objectiveassessment of prioriies because individuals were not allowed to rank commodities orresearch areas directly but had to weight criteria

The first weakness inherent in ex ante research priority setting and thus in theprocedure is that there is a large amount of subjectiveness Although tileapproach isless subjective than unstructured judgment there is subjectiveness in the responses toquestiorns related to some of the criteria and also ini the weights placed by decision makers on the criteria Second it proved diflicult to specify independent criteria with no overlap Third while the procedure required less time than an expected economic surplus analysis by commodity it required more time than asking respondents to directlyrank commodities and research areas In E-cuador the consiultant worked withpersonnel in INIAP and FEI)IA for one week at the start of the analysis to determinegoals and criteria to arrive at atlist of commodities and researh areas to list secondarydata needs to formulate questions to be asked of administrators and scientists to decide who should be asked particular questions and to explain the procedure itself includingthe spreadsheet program The required iidormation was collected over the Followingmonth and the consultant returned and worked with the Planning Director in INIAP for one week to conduct the analysis and write tip the results Revisions to the initial draftof the report we-e then made over the lollowing weeks A similar procedure to the onefollowed in Ecuador was used in l_ruguay althaugli more time was spent discussing the wording of questions to be asked of scientists Furthermore n Uruguay more care was taken to match the criteria with the goas

Certain of the criteria wil always be diflicult to eplain to tileinterviewees Thecriterion on comparative advantage is difficult for some respondents because of a lack of knowledge about neighboring countries If suflicient data time and resources are available this criterion may be rnea4ured more precisely using comparative advantage or domestic resource cost procedures suggested by longmire and Winkelnann Anotherdiflicult criterion to estiniate is the potential foture demand for the product The respondents in Fcuador were asked to consider how the demands for different foodswould change as income and populatiot grow Many of them had a hard timeconceiving that the total or per capita demand for certain Foods could actually decline as income grew In Uiruguay (ibe cmiparative advantage and future dema nd questions were answered by a small group of economists from the ministries of agriculture and trade In the Ecuador study more attention was devoted to assessing the most appropriate people for answering different questions than was the case in the Dominican Republic Ilowever some criticism surfaced about the weights placed on criteria InUruguay decision makers were given the opportunity to change their weights afterviewing the initial results This was very useftl because it demonstrated the implicationsof placing different weights on the various goals and criteria

Computer spreadsheet programs were a major help in Ecuador and Uruguay Additional work needs to be devoted to developing a more mentnu-driven prcgram but the people inthe planning office of INIAP were able to use the SuperCalcR program without too tmucht difliculty despite ialack of previous experience Ihe analyst in Uruguay was veryfamiliar with lotusl which proved most helpfil

It became clear from working inall three countries that it is preferable to work directlywith the final decision makers or their designees Inthe l)ominican Republic the study was conducted by a very competent set of consultants from the local agricultural universi ty but the procedure was not institutionalized to allow for additional sensitivityanalysis or future priority-setting efforts InEcuador and especially in Uruguay the

13

decision makers were directly involved conducted sensitivity analyses and were able to assess why certain commodities or research areas received high priority as a result of the analyses In both Ecuador and Uruguay they have a basic understanding of the strengths and weaknesses of tileprocedure and therelre can explain their subsequentdecisions to groups who might disagree with tilepriorities they establish

The corntrast between the Dominican Republic and tie Uruguay studies illustrates a veryimportant point No matter which technique is used to structure the analysispriority-setting exercises must integrate the decision makers (users) into the processPriority-setting procedures must be institutionalited the results of initial analysisdiscussed thoroughly with decision makers and sensitivity anilsis conlducteU alone the lines suggested by that discussion Tecinological change generates a surplus which can be appropriated by dilkircnt sectors or groups within the economy Who benelitdepends in part on the cotaniodities selected for research and the types of technoloLies generated One of the purposes of institutionalizing a formal prioritN- setting procedureis to help mediate conflicts that may arise (for example between producer groups and consumers) as a result of research-induced improved technologies

Other issues arose during the analyses First the initial grouping and elimination of certain commodities and research areas before lhe irialvis is an important step but one which must be taken with carelul consideration of possible coniplmentarities in research on particular productS or types of rearch Second some of the commodities such as forages are inputs into other commodities such as milk (Corseqtentlv these commodities llay have to he considered itscinps because it is the value-added and not tilegross value of output that is relevant when one coniiodity requires other commodities sinputs and when criteria such as value of production are used This was an issue in I ruguav where livestock are quite important liird little attention was devoted in atv of the studies to defining appropriate criteria Ior the increased inoome stability eoal Future studies if this be an ideltJiiLd LOal shOUld define risk criteria

In sutImary the weighted criteria model as applied in Lcuador and iuguav which had the advantage of bit ildin Oillessons lCarICd illdie l)ominican RCpublic appears to have been a uselul priority-setting tool It ha its ereatest advartage ill situations ill which there are rulItiple goals to consider a relatively major reassesnieiit of prioritiesis contenplated and the list of commodities dd resen rch areas to consider is largeWeighted criteria models described in this paper were applied in small countries with limited resources but with direct involvement (except in the Dominican Republic) ofthC major decision makers These models itav be most useful in the small country situationwhere decision mAkers are more likely to be involved in tile priority-settirt process and are not just using tables of results Perhaps in larger systet7 where decisioii makine is more decentralited tileweighted criteria model can he applied at the regional level before relining research priorities at the national level

Another factor that can be uniortant in large svvtern is tle complerientarity between resea-h aid educat ion [hie larger the svstcnt the clser the linkages tend to be between universities and tileresearchi system Where universities are important tor agriclltural research an aidditional criterion may hc rteedCd to place value on the educationlal benefits of certain types of reseirch

Ihere are several additional iniplications with respect to how weighted criteria models relate to alterniative priori t-setting Inethod parIicularl the expected economic surplusapproach Befoce exploring these relationships kcx aspects of the surplus approacliare described based on aii application in Peru

14

The above is a very brief summary of the major advantages and disadvantages of the most common formally structured priority-setting models that could be used for strategicplanning for research resource allocation Other approaches exist for research projectselection with perhaps models that attempt to identify and analyze key vield constraints being the most prevalent approach employed in NARS of less-deeloped countries

The results of ex post analysis also can provide useful guidance for ex ante research resource allocation decisions if appropriately incorporated into a systematic ex ante procedure The most common ex post approach in addition to cx post benefit-cost analysis is the econometric estimation of production or supply functions incorporating research variables To be most uteful for ex ante analysis econometric approaches mLlSt be applied with a high degree of disaggrega tion Unfortunateiy these models require a substantial amount of high-quality historical data on production farm inputs and research expenditures

Perhaps the two approaches with the most potential given additional refinements for application by NARS in less-developed countries are the weighted criteria models and the benefit-cost or expected economic surplus procedures In the next section reent experiences with applying weighted criteria models in three Latin American Countries )ominican Republic Ecuador and Uruguay are discussed and evaluated The purpose

it to piovide more iniormation about the procedures their degree of acceptance and possible improvements in the models Ihen application of expected economic surplus models are considered drawing cn a recent cx ante research evaluation studly in Peru Finally the relationship between weighted criteria and expected economic surpluts methods is described along with possible extensions

IIl THE USE OF WEIGHiTED CRITERIA MODELS IN TIlE DOMINICAN REPUBLIC ECUADOR AND URUGUAY

in 1986 iif I87 weighted criteria models were developed and implemented in the lDomin lRepublic (ISA) Ecuador (Espinosa et al) and Trugua (CIAAB)to assist with priority setting for agricultwral research In the I)ominican Republic the study was carried out at the Instituto Superior de Agnicultura ISA) by consultants for ISNAR In Ecuador the study was conducted by the plmnintg ollice oY the Instituto Nacional de Investigacion Agropecuaria (INIAP) with the assistance of consultants for the US Agency for International Development (AID) In Uruguay the study was conducted by the Ollice of the Director of the Centro de Investigacion Agricola Alberto Boergcr(CIAAB) with the assistance of ISNAR The purpose of all three studies was to applyprocedures for prioritizing agricultural research by commodity and by major research area Inthe Dominican Republic the client was the newly created natonal agriculturalresearch agency (Il)EA) In Ecuador the client was INIAP and the newly created foundation for agricultural research (IEl)IA) In Urugua the client was C1IAB

The procedures employed in the three studies were similar but with some importantdilfereices as the Ecuador study built on lessons learned inthe I)ominican Republic and the Uruguay study built on other lessons learmed in Ecuador National goals for the research system iere elicited in each country and a series of criteria establishd which relate to those goals Separate riteria were developed for conmitodities and or research areas and weights were elicited from decision makers to establish the relative importanceof the criteria Commodities and research areas were ranked according to each criterion and these rankings were multiplied by the elicited weights to arrive at research priorities

6

Goals

Three goals were identified in each of the three countries (1) to raise the average levelof income in the country (2) to increase the well-b(Aing of low-income groups in societyand (3) to reduce yearto-year income fluctuations in the country especially onl tiledownside These goals were worded slightly diffrcntly in each country but in each casethey essentially relate to the three goals referred to in previous studies as efficncy equity and security In the l)ominican Republic the local consultants identifiedthese as the goals while in lcuador and Uruguay it was done by the directors of national research agencies

Criteria

The next step was to establish criteria to as measures of whetheruse particularcommodities or research areas contribute to the attainment of the above goals A largeset of criteria was discussed in each country and a total of 15 criteria were eventuallyused in the Dominican Republic 14 in lEcuador and 10 in Uruguay to determine priorities by commodity Five criteria were used in each case to determine priorities byresearch area Refinements were made in successive studies - beginning with theDominican Republic then Ecuador followed by Iruguay - to increase the independenceof criteria and to remove criteria that were questionable measures of whether researchcontributed to tile tated goals ( Reason for dropping particular ci eria are discussedbelow) Refinements also were made with respect to the grouping of criteria to facilitatethe weighting of goals and criteria

Commodity Criteria omioditv criteria were grou l - J into four conceptual groupsproduct importance probability O success elliciencv in use of research resources anddistribution of impacts The first three of these groups relate to the efficiency or incomelevel goal and the last group relates to the equity or distributional goal In none of thethree studies were criteria included which represnted the third goal (reduced incomelluctuation [lie feeling was that the entire set of criteria as a group would serve toreduce the enlhasis placed on single or 1ew (often export) crops While this appearedto be the case in the results of thee particular studies ft ture weighted criteria studies may want to add criteria which explicitly rank commodities from lowest to hiehest withrespect to annual gross income variability (or price and yield variability separately)

The group of criteria referred to as jrodct importap contained Four criteria in theUruguay study value of production generation or saving of foreign exchange expectedfuture demand change and comparative advantage

The value of productionl is an important criterion because the cost of rseaich isbasically independent of the number of units alkected by the research results at least forthose commodities with relatively homogeneous production conditions Foreignexchange was used as a criterion because of a perception in each country that the lackof foreign exchange was impeding growtb A criterion was set up to allow more weightto be paced on those commodities for which demand is expected to increase in thefuture because the importance of commodities will change as diets are altered withincome growth aod a world markets change Comparative advantage was includedbecause income in the country will be higher if the country focuses its efforts oncommodities for which its resource base is best suited while recognizing that research itself can alter resource constraints

In the Dominican Republic and in Ecuador nutritional measurcs (calorie and proteincontributions to the diet) were included as criteria for product importance They wereincluded because the well-being of low-income groups may be affected by the availability

7

of foods which are important in their diets Nutritional criteria were dropped in Uruguay however because numerous studies have shown that income levels are tile primary determinants of nutrition levels rather than the availability of particular foods Certainly caloric and protein content are debatable criteria To the extent that increased domestic production lowers the prices of certain food commodities therebyincreasing food consumption by the poor nutritional criteria can contribute to the distributional goal Land area also was included as a criterion in the Dominican Republic study but was dropped in Ecuador and Uruguly because of the overlap with value of production

Two criteria were used in each study to measure 1robahility of reseatch success The first measure was tilegap between own-country yields and neighboing country yields The second was the potential for success indicated by the researchers themselves The rationale for the gap criterion was that the larger the yield gap between the home country and similar countries tle greater the potential for yield gains The criterion is rather crude because yield gaps also can indicate a relative resource disadvantace forproducing the commodity even if the countries appear to be geoclimatically sinilar Consequently the gap criterion was eventually dropped in the tUruguav study The potential for success as indicated by the researchers is subjective but a very necessary measure to obtain

The gro up of criteria reficrred to as e1)cienc in use otresearch resources contained three criteria the relationship to research in the international centers the degree of emphasis on tilecommodity in le current research p ogram and the incentive for the privatesector to conduct the research Some research duplicates other research croplementsand still other research bears no relationship to research at tileinternational agriculturalresearch centers For certain comniodities all three relationships may hold depending on the type of research If national research is complementary to international researchthe cost of the rational research is lower Consequently a criterion was established which places greater weight on commodities with a high degree of complementaritybetween ntin Il d international research The criterion ofdelegree of emphasis in tile current iesercli program was established because there is a cost to rapidly adjustingcurrent h uiman and physical research resources from one commodity to another The criterion of private-sector incentives was used because the social benefits from the total research program in the country will be greatest if scarce public research funds are devoted to research on commodities for which the private sector does not have an incentive to support the research itself Of coui se the private-sector producer group can tax itself and pay the public sector to conduct the research for those commodities for which it has an incentive to privately support the research These commodities tend to be export commodities because producers can capture a higher proportion of the gainsfrom productivity increases for export commodities than for commodities primarily consumed domestically

The group of criteria related to the distribution ojresarchimpacts contained two criteria in the Uruguay study number of producers and the effect of increased productivity on the price of product The larger the number of farms producing a commodity the largerthe number of farmers afficted by the research and perhaps the higher the probabilitythat smaller farmers are being affeicted The distribution of benefits depends to a significant degree on price changes resulting from increased productivity Producers benefit more if prices decline very little and consumers benefit more ifprices decline substatitially for a given production increase For export er import commodities priceswill tend to decline less than for commodities primarily produced and consumed domestically For commodities which are not traded and are very important in the diets of the poor prices will tend to decline relatively sharply when productivity increases The division of benefits between producers and consumers also is affected by the fact

that producers are also consumers Consequently in the Dominican Republic and Ecuador studies an additional criterion importance of the commodity in home-consumption on the farm where produced also was included

In summary 10-15 criteria were used in tie three studies to identify and select research priorities by commodity The linkages anong the goals and criteria described above are summarized in Figure 1

Research Area Criteria The five criteria uicel to select research priorities by research aieas were (1 ) whether the research causes an increase in the use of relatively abundant resources and a saving of relatively scarce resources (2) the number and seventy of research problems (3) non-duplication with transferable research from outside the country (4) tne extent of private-sector incentives to conduct the research and (5) current emphases in the research program

The rate of economic growth will be more rapid if new technologies are developed that utilize relatively abundant resources and save relatively scarce resources The resource base varies by region consequently information on resource abundance criteria was collected by region in tileEcuador and Uruguay studies The same was true for the quantity and severity of research problems Information on the relationship of domestic to inernational research private-sector incentives and the current emphasis in the research program was collected on a naticnal basis These criteria all relate to the income growth (efficiency) goal It is difficult to identif research area criteria that measurc whether particular types of research aifect income distribution or variability

Data Collection and Model Implementation

Information used in the analysis included both quantitative data on value of productionnumber of farms value of exports and imports person-years devoted to research on different commodities and tile of protein per day in thenumber of calories and gramsdiet (for tileDominican Republic and Ecuador studies) as well as qualitative or subjective information on such factors as probability of success private sector incentives and severity of research problems Furthermore weights had to be elicited from decision makers to place relative emphasis on the various criteria

Step 1 Develop Cotninodity and Research Area Lists -- The first step was to determine the list ofcommadities to consider and the list of research areas In the Dominican Republic information was collected on 74 commodities In Ecuador an initial list of 109 commodities on which INIAP was conducting research was discussed Throughdecisions to eliminate some and to group others this list was reduced to 44 commedities for the analysis These decisions were made by the Technical Director for Research In Uruguay a smaller list of commodities was reduced to 21 commodities or commodity groups These decisions were made in consultation with the National Research and Fxtension Directors The list of research program areas contained nine areas in the Dominican Republic 16 in Ecuador and 16 in Uruguay

Step 2 Collect Quantitative and Qualitative Data on Chosen Criteria -- Information o-i quantitative criteria were gathered from local and FAO secondary data sources and one table was constructed For each criterion Commodities were ranked for each criterion Information needed for the qualitative riteria as well as the weights to place on criteria were obtained through interviews with sc rtists and administrators at boh the national and regional experiment stations In Euador and Uruguay substantial attention was devoted to identifying the appropriate respondents to each question as this is a crucial aspect of the procedure In Ecuador those interviewed were notified one week before the interviews and 72 people were interviewed by the Planning Director of INIAP and

9

FIJR 1 CONCEPTUAL ELEHNTS IN THE HEIGHTED CRITERIA MODEL FOR PAWING RESEARCH PRIORITIES BY COtlDITY

GOALS fficiency ]

CONCEPTUAL( )CRITERIA Pri7] Probabi i+y Research Distriutjon Iincome

ii J of Succes Resource Eff o L=Acplusmn a rilbility

CRITERIA 1 Value of 1 Scientist 1Rl ofto toNumberPrcduction Estimates 1 AnnualInternational Producers Variability2 Foreign Centers 2 Price in Gro

Exchange 2 Current Effect3 Future Raturn3)Emphasis 3 HomeDemand 3 Private Consumpt ion

4 Comparative Sector Advantage Incentives

Notes

(-n future studies an additional conceptural criteria under the security goal might be health (J4tri ional criteria such as grams of potein or kilo=alories could be included as additional easures of Distributional ikipacts

4ot used in Dominican Republic Ecuador and Uruojuay studies

his assistant In Uruguay the directors of all the regional stations were called into CIIAB headquarters for a aweting to discuss the study and this was followed up byon-site interviews and meetings with scientists and directors at the regional experimentstations as well as with economists in the Ministry of Agriculture These interviews and meetings were conducted by a person assigned to work wvith the Director of Cl LAB on the priority-setting study In the Dominican Republic a survey of scientists extensionworkers and administrators was conducted by the ISNAR consultant from the Instituto Superior de Agricultura who visited the various experiment stations Less care wastaken in the Dominican Repubic than in Ecuador and Uruguay to match the questionspertaining -o individual criteria to particular individuals This probably reduced theusefulness of the responses because those most familiar with the probability of research success For example may not be the same -s those most Familiar with comparativeadvantage

Step 3 Elicit 11eihts on Critria -- Relative weights to place on the different criteria were obtained Fronm national and regional research system administrators In Ecuador34 peopic verc used to determine the weights These wveights were established separatelyIor the commoditv and the research area criteria In 1iuguay fewer people (sevensystem and station directors) were used to determine the weights and more attention was devoted to grouping the criteria pertaining to particular goals A Delphi procedure was used in which the s7even directors were shown the average of the group and provided with an opportunity to adjust their weight

Step 4 )erive Rankinqs hv Commodity and By Research Area -- Once the basic information was collected axd organized into a seies of tables one procedure was followed to arrive at a final ranking of research priorities by commodity A second procedure resulted in lists of priorities by research area In the Dominican Republicmost of the calculations were made by hand in Ecuador and Uruguay microcomputerspreadsheet programs were used to assist with the calculations and to conduct sensitivity analysis

The steps in the procedure for ranking research priorities by commodity and research area are sumnmaried below and are illustrated with a simple example in Appendix 1 Brieily the steps for ranking commodities were

a) Th conniodities were ranked for each criterion for which quantitative data were available

b) Tihese rankings were multiplied by the weights assigned to each criterion and then the weighted criteria were added across to arrive at one sub-ranking for the

Luantitative criteria c) Each commodity was given a high low or none designation for each qualitative

(subjective) criterion The response which implied a need for greater research priority was assigned the number 2 the intermediate response was assigned I and the response which implied lower research priority was assigned 0 1herefore each comniodity received a 2 1 or 0 for each criterion

d) The weights assigned to each qualitative criterion were then multiplied by these numbers and the results were added across all criteria to arrive at a subtotal for each commoditv These subtotals were ranked from highest to lowest to provide a second sub-ranking

e) The sub-rankings for the quantitative and qualitative criteria were then given their corresponding weights and added together to arrive at a final ranking by commodityThe steps for ranking research priorities by research area were the same as steps (c)and (d) above because the criteria were all qualitative

Step 5 Analsis and Interpretationof Resuls -- In the Dominican Republic the resuLts of this weighted criteria analysis were uscd to determine a small set of cormodities and

II

research areas with the highest priority Further assessment was then made of humanphysical administrative and other resources needed to structure research programsfocused on these commodities and resea-ch topics In Ecuador and Uruguay the prioritized list of commodities was split into a high-priority group an intermediate-priority group and a low-priority group Research area priorities were identified for each region of tile country A more detailed analysis was then made of the human physical and other resources needed to support the high and intermediate priorities

In summary five major -rens were involved in operationalizing the weighted criteria models A complete set of rsults for each study included more than 20 tables and are not reproduced here Ilowever the final rankings for commodities and for research areas are shown in Tables I and 2 in Appendix 2 In the Dominican Republic a team of consultants followed up the initial priority-setting exercise with additional analysisand discussions at the experiment station level and eventually recommended tile establishment of five national commodity programs (rice cornsorghum beans coffeeand livestock) and one additional major research program (natural resources) In Ecuador the prioritized lists were widely distributed and discussed in the Ministry oi Agriculture the Board of FEIDIA the Commission on Science and Technology arid in AID There was concern in Ecuador with the weights placed on the individual criteria and therefore additional sensitivity analysis was conducted by the Planning Office of the Ministry There seemed to be a recognition in Ecuador that priority setting with a weighted criteria model is an iterative process and that much 4t the models value stems from tile discussion of criteria among the decision makers and from the sensitivityanalysis The high-priority list of comniodities in Ecuador included thL ten highest-priority commodities listed in Table 2b A project paper has been written forAI) which recommends funding For Fli)IAlbr three commodities (coffee soft corn for the Sierra and dairy) Coffee and soft coitn were the number 2 and 3 ranked prioritiesin tile weighted criteria model and dairy was number 9 although it was the top-rankedlivestock commodity

In Urugu ay the results from the first run with tile model were discussed with the research directors Ilie directors made small changes to the weights placed on criteriaand the mode was rerun resulting in the prioritized list in Appendix 2 Personnel in CIAAB are CUrrcwtv undertaking additional sensitivity analysis and with tie assistance of ISNA R are developing a plan for implementation

An Assessment of the Weighted Criteria Models Used in the Dominican RepublicEcuador and Uruguay

The Dominican Republic Ecuador and Uruguay studies were perhaps the first majorattempts to implement weighted criteria models with weights explicitly elicited from decision makers in NARS in less-developed countries A number of strengths andweaknesses in the procedures were identified during the analyses Some of thL weaknesses were corrected fron one study to the next some can be corrected in future studies and others are inherent in the approach

Trhe first strength mentioned earlier is the ability of the procedure to systematicallyincorporate both quantitative and qualitative information related to a set of multiplegoals and criteria in order to prioritize a lorng ist of commodities and research areas in a relatively short period of time Second the procedure proved relatively easy for both research administrators and the loc- -nalysts to understand with the exception of the )ominican Republic where the research administrators were not directly involved7

Third the analysis as applied in Ecuador and ULuguay with the direct involvement of research system administrators at various stages forced those decision makers to

12

consciously identifv and trade off goals and criteria Fourth the use of spreadsheet programs in EcUadfor and Uruguay facilitated sensitivity analysis after the initial set of priorities was deermimd Fifth the procedure provided a relatively objectiveassessment of prioriies because individuals were not allowed to rank commodities orresearch areas directly but had to weight criteria

The first weakness inherent in ex ante research priority setting and thus in theprocedure is that there is a large amount of subjectiveness Although tileapproach isless subjective than unstructured judgment there is subjectiveness in the responses toquestiorns related to some of the criteria and also ini the weights placed by decision makers on the criteria Second it proved diflicult to specify independent criteria with no overlap Third while the procedure required less time than an expected economic surplus analysis by commodity it required more time than asking respondents to directlyrank commodities and research areas In E-cuador the consiultant worked withpersonnel in INIAP and FEI)IA for one week at the start of the analysis to determinegoals and criteria to arrive at atlist of commodities and researh areas to list secondarydata needs to formulate questions to be asked of administrators and scientists to decide who should be asked particular questions and to explain the procedure itself includingthe spreadsheet program The required iidormation was collected over the Followingmonth and the consultant returned and worked with the Planning Director in INIAP for one week to conduct the analysis and write tip the results Revisions to the initial draftof the report we-e then made over the lollowing weeks A similar procedure to the onefollowed in Ecuador was used in l_ruguay althaugli more time was spent discussing the wording of questions to be asked of scientists Furthermore n Uruguay more care was taken to match the criteria with the goas

Certain of the criteria wil always be diflicult to eplain to tileinterviewees Thecriterion on comparative advantage is difficult for some respondents because of a lack of knowledge about neighboring countries If suflicient data time and resources are available this criterion may be rnea4ured more precisely using comparative advantage or domestic resource cost procedures suggested by longmire and Winkelnann Anotherdiflicult criterion to estiniate is the potential foture demand for the product The respondents in Fcuador were asked to consider how the demands for different foodswould change as income and populatiot grow Many of them had a hard timeconceiving that the total or per capita demand for certain Foods could actually decline as income grew In Uiruguay (ibe cmiparative advantage and future dema nd questions were answered by a small group of economists from the ministries of agriculture and trade In the Ecuador study more attention was devoted to assessing the most appropriate people for answering different questions than was the case in the Dominican Republic Ilowever some criticism surfaced about the weights placed on criteria InUruguay decision makers were given the opportunity to change their weights afterviewing the initial results This was very useftl because it demonstrated the implicationsof placing different weights on the various goals and criteria

Computer spreadsheet programs were a major help in Ecuador and Uruguay Additional work needs to be devoted to developing a more mentnu-driven prcgram but the people inthe planning office of INIAP were able to use the SuperCalcR program without too tmucht difliculty despite ialack of previous experience Ihe analyst in Uruguay was veryfamiliar with lotusl which proved most helpfil

It became clear from working inall three countries that it is preferable to work directlywith the final decision makers or their designees Inthe l)ominican Republic the study was conducted by a very competent set of consultants from the local agricultural universi ty but the procedure was not institutionalized to allow for additional sensitivityanalysis or future priority-setting efforts InEcuador and especially in Uruguay the

13

decision makers were directly involved conducted sensitivity analyses and were able to assess why certain commodities or research areas received high priority as a result of the analyses In both Ecuador and Uruguay they have a basic understanding of the strengths and weaknesses of tileprocedure and therelre can explain their subsequentdecisions to groups who might disagree with tilepriorities they establish

The corntrast between the Dominican Republic and tie Uruguay studies illustrates a veryimportant point No matter which technique is used to structure the analysispriority-setting exercises must integrate the decision makers (users) into the processPriority-setting procedures must be institutionalited the results of initial analysisdiscussed thoroughly with decision makers and sensitivity anilsis conlducteU alone the lines suggested by that discussion Tecinological change generates a surplus which can be appropriated by dilkircnt sectors or groups within the economy Who benelitdepends in part on the cotaniodities selected for research and the types of technoloLies generated One of the purposes of institutionalizing a formal prioritN- setting procedureis to help mediate conflicts that may arise (for example between producer groups and consumers) as a result of research-induced improved technologies

Other issues arose during the analyses First the initial grouping and elimination of certain commodities and research areas before lhe irialvis is an important step but one which must be taken with carelul consideration of possible coniplmentarities in research on particular productS or types of rearch Second some of the commodities such as forages are inputs into other commodities such as milk (Corseqtentlv these commodities llay have to he considered itscinps because it is the value-added and not tilegross value of output that is relevant when one coniiodity requires other commodities sinputs and when criteria such as value of production are used This was an issue in I ruguav where livestock are quite important liird little attention was devoted in atv of the studies to defining appropriate criteria Ior the increased inoome stability eoal Future studies if this be an ideltJiiLd LOal shOUld define risk criteria

In sutImary the weighted criteria model as applied in Lcuador and iuguav which had the advantage of bit ildin Oillessons lCarICd illdie l)ominican RCpublic appears to have been a uselul priority-setting tool It ha its ereatest advartage ill situations ill which there are rulItiple goals to consider a relatively major reassesnieiit of prioritiesis contenplated and the list of commodities dd resen rch areas to consider is largeWeighted criteria models described in this paper were applied in small countries with limited resources but with direct involvement (except in the Dominican Republic) ofthC major decision makers These models itav be most useful in the small country situationwhere decision mAkers are more likely to be involved in tile priority-settirt process and are not just using tables of results Perhaps in larger systet7 where decisioii makine is more decentralited tileweighted criteria model can he applied at the regional level before relining research priorities at the national level

Another factor that can be uniortant in large svvtern is tle complerientarity between resea-h aid educat ion [hie larger the svstcnt the clser the linkages tend to be between universities and tileresearchi system Where universities are important tor agriclltural research an aidditional criterion may hc rteedCd to place value on the educationlal benefits of certain types of reseirch

Ihere are several additional iniplications with respect to how weighted criteria models relate to alterniative priori t-setting Inethod parIicularl the expected economic surplusapproach Befoce exploring these relationships kcx aspects of the surplus approacliare described based on aii application in Peru

14

Goals

Three goals were identified in each of the three countries (1) to raise the average levelof income in the country (2) to increase the well-b(Aing of low-income groups in societyand (3) to reduce yearto-year income fluctuations in the country especially onl tiledownside These goals were worded slightly diffrcntly in each country but in each casethey essentially relate to the three goals referred to in previous studies as efficncy equity and security In the l)ominican Republic the local consultants identifiedthese as the goals while in lcuador and Uruguay it was done by the directors of national research agencies

Criteria

The next step was to establish criteria to as measures of whetheruse particularcommodities or research areas contribute to the attainment of the above goals A largeset of criteria was discussed in each country and a total of 15 criteria were eventuallyused in the Dominican Republic 14 in lEcuador and 10 in Uruguay to determine priorities by commodity Five criteria were used in each case to determine priorities byresearch area Refinements were made in successive studies - beginning with theDominican Republic then Ecuador followed by Iruguay - to increase the independenceof criteria and to remove criteria that were questionable measures of whether researchcontributed to tile tated goals ( Reason for dropping particular ci eria are discussedbelow) Refinements also were made with respect to the grouping of criteria to facilitatethe weighting of goals and criteria

Commodity Criteria omioditv criteria were grou l - J into four conceptual groupsproduct importance probability O success elliciencv in use of research resources anddistribution of impacts The first three of these groups relate to the efficiency or incomelevel goal and the last group relates to the equity or distributional goal In none of thethree studies were criteria included which represnted the third goal (reduced incomelluctuation [lie feeling was that the entire set of criteria as a group would serve toreduce the enlhasis placed on single or 1ew (often export) crops While this appearedto be the case in the results of thee particular studies ft ture weighted criteria studies may want to add criteria which explicitly rank commodities from lowest to hiehest withrespect to annual gross income variability (or price and yield variability separately)

The group of criteria referred to as jrodct importap contained Four criteria in theUruguay study value of production generation or saving of foreign exchange expectedfuture demand change and comparative advantage

The value of productionl is an important criterion because the cost of rseaich isbasically independent of the number of units alkected by the research results at least forthose commodities with relatively homogeneous production conditions Foreignexchange was used as a criterion because of a perception in each country that the lackof foreign exchange was impeding growtb A criterion was set up to allow more weightto be paced on those commodities for which demand is expected to increase in thefuture because the importance of commodities will change as diets are altered withincome growth aod a world markets change Comparative advantage was includedbecause income in the country will be higher if the country focuses its efforts oncommodities for which its resource base is best suited while recognizing that research itself can alter resource constraints

In the Dominican Republic and in Ecuador nutritional measurcs (calorie and proteincontributions to the diet) were included as criteria for product importance They wereincluded because the well-being of low-income groups may be affected by the availability

7

of foods which are important in their diets Nutritional criteria were dropped in Uruguay however because numerous studies have shown that income levels are tile primary determinants of nutrition levels rather than the availability of particular foods Certainly caloric and protein content are debatable criteria To the extent that increased domestic production lowers the prices of certain food commodities therebyincreasing food consumption by the poor nutritional criteria can contribute to the distributional goal Land area also was included as a criterion in the Dominican Republic study but was dropped in Ecuador and Uruguly because of the overlap with value of production

Two criteria were used in each study to measure 1robahility of reseatch success The first measure was tilegap between own-country yields and neighboing country yields The second was the potential for success indicated by the researchers themselves The rationale for the gap criterion was that the larger the yield gap between the home country and similar countries tle greater the potential for yield gains The criterion is rather crude because yield gaps also can indicate a relative resource disadvantace forproducing the commodity even if the countries appear to be geoclimatically sinilar Consequently the gap criterion was eventually dropped in the tUruguav study The potential for success as indicated by the researchers is subjective but a very necessary measure to obtain

The gro up of criteria reficrred to as e1)cienc in use otresearch resources contained three criteria the relationship to research in the international centers the degree of emphasis on tilecommodity in le current research p ogram and the incentive for the privatesector to conduct the research Some research duplicates other research croplementsand still other research bears no relationship to research at tileinternational agriculturalresearch centers For certain comniodities all three relationships may hold depending on the type of research If national research is complementary to international researchthe cost of the rational research is lower Consequently a criterion was established which places greater weight on commodities with a high degree of complementaritybetween ntin Il d international research The criterion ofdelegree of emphasis in tile current iesercli program was established because there is a cost to rapidly adjustingcurrent h uiman and physical research resources from one commodity to another The criterion of private-sector incentives was used because the social benefits from the total research program in the country will be greatest if scarce public research funds are devoted to research on commodities for which the private sector does not have an incentive to support the research itself Of coui se the private-sector producer group can tax itself and pay the public sector to conduct the research for those commodities for which it has an incentive to privately support the research These commodities tend to be export commodities because producers can capture a higher proportion of the gainsfrom productivity increases for export commodities than for commodities primarily consumed domestically

The group of criteria related to the distribution ojresarchimpacts contained two criteria in the Uruguay study number of producers and the effect of increased productivity on the price of product The larger the number of farms producing a commodity the largerthe number of farmers afficted by the research and perhaps the higher the probabilitythat smaller farmers are being affeicted The distribution of benefits depends to a significant degree on price changes resulting from increased productivity Producers benefit more if prices decline very little and consumers benefit more ifprices decline substatitially for a given production increase For export er import commodities priceswill tend to decline less than for commodities primarily produced and consumed domestically For commodities which are not traded and are very important in the diets of the poor prices will tend to decline relatively sharply when productivity increases The division of benefits between producers and consumers also is affected by the fact

that producers are also consumers Consequently in the Dominican Republic and Ecuador studies an additional criterion importance of the commodity in home-consumption on the farm where produced also was included

In summary 10-15 criteria were used in tie three studies to identify and select research priorities by commodity The linkages anong the goals and criteria described above are summarized in Figure 1

Research Area Criteria The five criteria uicel to select research priorities by research aieas were (1 ) whether the research causes an increase in the use of relatively abundant resources and a saving of relatively scarce resources (2) the number and seventy of research problems (3) non-duplication with transferable research from outside the country (4) tne extent of private-sector incentives to conduct the research and (5) current emphases in the research program

The rate of economic growth will be more rapid if new technologies are developed that utilize relatively abundant resources and save relatively scarce resources The resource base varies by region consequently information on resource abundance criteria was collected by region in tileEcuador and Uruguay studies The same was true for the quantity and severity of research problems Information on the relationship of domestic to inernational research private-sector incentives and the current emphasis in the research program was collected on a naticnal basis These criteria all relate to the income growth (efficiency) goal It is difficult to identif research area criteria that measurc whether particular types of research aifect income distribution or variability

Data Collection and Model Implementation

Information used in the analysis included both quantitative data on value of productionnumber of farms value of exports and imports person-years devoted to research on different commodities and tile of protein per day in thenumber of calories and gramsdiet (for tileDominican Republic and Ecuador studies) as well as qualitative or subjective information on such factors as probability of success private sector incentives and severity of research problems Furthermore weights had to be elicited from decision makers to place relative emphasis on the various criteria

Step 1 Develop Cotninodity and Research Area Lists -- The first step was to determine the list ofcommadities to consider and the list of research areas In the Dominican Republic information was collected on 74 commodities In Ecuador an initial list of 109 commodities on which INIAP was conducting research was discussed Throughdecisions to eliminate some and to group others this list was reduced to 44 commedities for the analysis These decisions were made by the Technical Director for Research In Uruguay a smaller list of commodities was reduced to 21 commodities or commodity groups These decisions were made in consultation with the National Research and Fxtension Directors The list of research program areas contained nine areas in the Dominican Republic 16 in Ecuador and 16 in Uruguay

Step 2 Collect Quantitative and Qualitative Data on Chosen Criteria -- Information o-i quantitative criteria were gathered from local and FAO secondary data sources and one table was constructed For each criterion Commodities were ranked for each criterion Information needed for the qualitative riteria as well as the weights to place on criteria were obtained through interviews with sc rtists and administrators at boh the national and regional experiment stations In Euador and Uruguay substantial attention was devoted to identifying the appropriate respondents to each question as this is a crucial aspect of the procedure In Ecuador those interviewed were notified one week before the interviews and 72 people were interviewed by the Planning Director of INIAP and

9

FIJR 1 CONCEPTUAL ELEHNTS IN THE HEIGHTED CRITERIA MODEL FOR PAWING RESEARCH PRIORITIES BY COtlDITY

GOALS fficiency ]

CONCEPTUAL( )CRITERIA Pri7] Probabi i+y Research Distriutjon Iincome

ii J of Succes Resource Eff o L=Acplusmn a rilbility

CRITERIA 1 Value of 1 Scientist 1Rl ofto toNumberPrcduction Estimates 1 AnnualInternational Producers Variability2 Foreign Centers 2 Price in Gro

Exchange 2 Current Effect3 Future Raturn3)Emphasis 3 HomeDemand 3 Private Consumpt ion

4 Comparative Sector Advantage Incentives

Notes

(-n future studies an additional conceptural criteria under the security goal might be health (J4tri ional criteria such as grams of potein or kilo=alories could be included as additional easures of Distributional ikipacts

4ot used in Dominican Republic Ecuador and Uruojuay studies

his assistant In Uruguay the directors of all the regional stations were called into CIIAB headquarters for a aweting to discuss the study and this was followed up byon-site interviews and meetings with scientists and directors at the regional experimentstations as well as with economists in the Ministry of Agriculture These interviews and meetings were conducted by a person assigned to work wvith the Director of Cl LAB on the priority-setting study In the Dominican Republic a survey of scientists extensionworkers and administrators was conducted by the ISNAR consultant from the Instituto Superior de Agricultura who visited the various experiment stations Less care wastaken in the Dominican Repubic than in Ecuador and Uruguay to match the questionspertaining -o individual criteria to particular individuals This probably reduced theusefulness of the responses because those most familiar with the probability of research success For example may not be the same -s those most Familiar with comparativeadvantage

Step 3 Elicit 11eihts on Critria -- Relative weights to place on the different criteria were obtained Fronm national and regional research system administrators In Ecuador34 peopic verc used to determine the weights These wveights were established separatelyIor the commoditv and the research area criteria In 1iuguay fewer people (sevensystem and station directors) were used to determine the weights and more attention was devoted to grouping the criteria pertaining to particular goals A Delphi procedure was used in which the s7even directors were shown the average of the group and provided with an opportunity to adjust their weight

Step 4 )erive Rankinqs hv Commodity and By Research Area -- Once the basic information was collected axd organized into a seies of tables one procedure was followed to arrive at a final ranking of research priorities by commodity A second procedure resulted in lists of priorities by research area In the Dominican Republicmost of the calculations were made by hand in Ecuador and Uruguay microcomputerspreadsheet programs were used to assist with the calculations and to conduct sensitivity analysis

The steps in the procedure for ranking research priorities by commodity and research area are sumnmaried below and are illustrated with a simple example in Appendix 1 Brieily the steps for ranking commodities were

a) Th conniodities were ranked for each criterion for which quantitative data were available

b) Tihese rankings were multiplied by the weights assigned to each criterion and then the weighted criteria were added across to arrive at one sub-ranking for the

Luantitative criteria c) Each commodity was given a high low or none designation for each qualitative

(subjective) criterion The response which implied a need for greater research priority was assigned the number 2 the intermediate response was assigned I and the response which implied lower research priority was assigned 0 1herefore each comniodity received a 2 1 or 0 for each criterion

d) The weights assigned to each qualitative criterion were then multiplied by these numbers and the results were added across all criteria to arrive at a subtotal for each commoditv These subtotals were ranked from highest to lowest to provide a second sub-ranking

e) The sub-rankings for the quantitative and qualitative criteria were then given their corresponding weights and added together to arrive at a final ranking by commodityThe steps for ranking research priorities by research area were the same as steps (c)and (d) above because the criteria were all qualitative

Step 5 Analsis and Interpretationof Resuls -- In the Dominican Republic the resuLts of this weighted criteria analysis were uscd to determine a small set of cormodities and

II

research areas with the highest priority Further assessment was then made of humanphysical administrative and other resources needed to structure research programsfocused on these commodities and resea-ch topics In Ecuador and Uruguay the prioritized list of commodities was split into a high-priority group an intermediate-priority group and a low-priority group Research area priorities were identified for each region of tile country A more detailed analysis was then made of the human physical and other resources needed to support the high and intermediate priorities

In summary five major -rens were involved in operationalizing the weighted criteria models A complete set of rsults for each study included more than 20 tables and are not reproduced here Ilowever the final rankings for commodities and for research areas are shown in Tables I and 2 in Appendix 2 In the Dominican Republic a team of consultants followed up the initial priority-setting exercise with additional analysisand discussions at the experiment station level and eventually recommended tile establishment of five national commodity programs (rice cornsorghum beans coffeeand livestock) and one additional major research program (natural resources) In Ecuador the prioritized lists were widely distributed and discussed in the Ministry oi Agriculture the Board of FEIDIA the Commission on Science and Technology arid in AID There was concern in Ecuador with the weights placed on the individual criteria and therefore additional sensitivity analysis was conducted by the Planning Office of the Ministry There seemed to be a recognition in Ecuador that priority setting with a weighted criteria model is an iterative process and that much 4t the models value stems from tile discussion of criteria among the decision makers and from the sensitivityanalysis The high-priority list of comniodities in Ecuador included thL ten highest-priority commodities listed in Table 2b A project paper has been written forAI) which recommends funding For Fli)IAlbr three commodities (coffee soft corn for the Sierra and dairy) Coffee and soft coitn were the number 2 and 3 ranked prioritiesin tile weighted criteria model and dairy was number 9 although it was the top-rankedlivestock commodity

In Urugu ay the results from the first run with tile model were discussed with the research directors Ilie directors made small changes to the weights placed on criteriaand the mode was rerun resulting in the prioritized list in Appendix 2 Personnel in CIAAB are CUrrcwtv undertaking additional sensitivity analysis and with tie assistance of ISNA R are developing a plan for implementation

An Assessment of the Weighted Criteria Models Used in the Dominican RepublicEcuador and Uruguay

The Dominican Republic Ecuador and Uruguay studies were perhaps the first majorattempts to implement weighted criteria models with weights explicitly elicited from decision makers in NARS in less-developed countries A number of strengths andweaknesses in the procedures were identified during the analyses Some of thL weaknesses were corrected fron one study to the next some can be corrected in future studies and others are inherent in the approach

Trhe first strength mentioned earlier is the ability of the procedure to systematicallyincorporate both quantitative and qualitative information related to a set of multiplegoals and criteria in order to prioritize a lorng ist of commodities and research areas in a relatively short period of time Second the procedure proved relatively easy for both research administrators and the loc- -nalysts to understand with the exception of the )ominican Republic where the research administrators were not directly involved7

Third the analysis as applied in Ecuador and ULuguay with the direct involvement of research system administrators at various stages forced those decision makers to

12

consciously identifv and trade off goals and criteria Fourth the use of spreadsheet programs in EcUadfor and Uruguay facilitated sensitivity analysis after the initial set of priorities was deermimd Fifth the procedure provided a relatively objectiveassessment of prioriies because individuals were not allowed to rank commodities orresearch areas directly but had to weight criteria

The first weakness inherent in ex ante research priority setting and thus in theprocedure is that there is a large amount of subjectiveness Although tileapproach isless subjective than unstructured judgment there is subjectiveness in the responses toquestiorns related to some of the criteria and also ini the weights placed by decision makers on the criteria Second it proved diflicult to specify independent criteria with no overlap Third while the procedure required less time than an expected economic surplus analysis by commodity it required more time than asking respondents to directlyrank commodities and research areas In E-cuador the consiultant worked withpersonnel in INIAP and FEI)IA for one week at the start of the analysis to determinegoals and criteria to arrive at atlist of commodities and researh areas to list secondarydata needs to formulate questions to be asked of administrators and scientists to decide who should be asked particular questions and to explain the procedure itself includingthe spreadsheet program The required iidormation was collected over the Followingmonth and the consultant returned and worked with the Planning Director in INIAP for one week to conduct the analysis and write tip the results Revisions to the initial draftof the report we-e then made over the lollowing weeks A similar procedure to the onefollowed in Ecuador was used in l_ruguay althaugli more time was spent discussing the wording of questions to be asked of scientists Furthermore n Uruguay more care was taken to match the criteria with the goas

Certain of the criteria wil always be diflicult to eplain to tileinterviewees Thecriterion on comparative advantage is difficult for some respondents because of a lack of knowledge about neighboring countries If suflicient data time and resources are available this criterion may be rnea4ured more precisely using comparative advantage or domestic resource cost procedures suggested by longmire and Winkelnann Anotherdiflicult criterion to estiniate is the potential foture demand for the product The respondents in Fcuador were asked to consider how the demands for different foodswould change as income and populatiot grow Many of them had a hard timeconceiving that the total or per capita demand for certain Foods could actually decline as income grew In Uiruguay (ibe cmiparative advantage and future dema nd questions were answered by a small group of economists from the ministries of agriculture and trade In the Ecuador study more attention was devoted to assessing the most appropriate people for answering different questions than was the case in the Dominican Republic Ilowever some criticism surfaced about the weights placed on criteria InUruguay decision makers were given the opportunity to change their weights afterviewing the initial results This was very useftl because it demonstrated the implicationsof placing different weights on the various goals and criteria

Computer spreadsheet programs were a major help in Ecuador and Uruguay Additional work needs to be devoted to developing a more mentnu-driven prcgram but the people inthe planning office of INIAP were able to use the SuperCalcR program without too tmucht difliculty despite ialack of previous experience Ihe analyst in Uruguay was veryfamiliar with lotusl which proved most helpfil

It became clear from working inall three countries that it is preferable to work directlywith the final decision makers or their designees Inthe l)ominican Republic the study was conducted by a very competent set of consultants from the local agricultural universi ty but the procedure was not institutionalized to allow for additional sensitivityanalysis or future priority-setting efforts InEcuador and especially in Uruguay the

13

decision makers were directly involved conducted sensitivity analyses and were able to assess why certain commodities or research areas received high priority as a result of the analyses In both Ecuador and Uruguay they have a basic understanding of the strengths and weaknesses of tileprocedure and therelre can explain their subsequentdecisions to groups who might disagree with tilepriorities they establish

The corntrast between the Dominican Republic and tie Uruguay studies illustrates a veryimportant point No matter which technique is used to structure the analysispriority-setting exercises must integrate the decision makers (users) into the processPriority-setting procedures must be institutionalited the results of initial analysisdiscussed thoroughly with decision makers and sensitivity anilsis conlducteU alone the lines suggested by that discussion Tecinological change generates a surplus which can be appropriated by dilkircnt sectors or groups within the economy Who benelitdepends in part on the cotaniodities selected for research and the types of technoloLies generated One of the purposes of institutionalizing a formal prioritN- setting procedureis to help mediate conflicts that may arise (for example between producer groups and consumers) as a result of research-induced improved technologies

Other issues arose during the analyses First the initial grouping and elimination of certain commodities and research areas before lhe irialvis is an important step but one which must be taken with carelul consideration of possible coniplmentarities in research on particular productS or types of rearch Second some of the commodities such as forages are inputs into other commodities such as milk (Corseqtentlv these commodities llay have to he considered itscinps because it is the value-added and not tilegross value of output that is relevant when one coniiodity requires other commodities sinputs and when criteria such as value of production are used This was an issue in I ruguav where livestock are quite important liird little attention was devoted in atv of the studies to defining appropriate criteria Ior the increased inoome stability eoal Future studies if this be an ideltJiiLd LOal shOUld define risk criteria

In sutImary the weighted criteria model as applied in Lcuador and iuguav which had the advantage of bit ildin Oillessons lCarICd illdie l)ominican RCpublic appears to have been a uselul priority-setting tool It ha its ereatest advartage ill situations ill which there are rulItiple goals to consider a relatively major reassesnieiit of prioritiesis contenplated and the list of commodities dd resen rch areas to consider is largeWeighted criteria models described in this paper were applied in small countries with limited resources but with direct involvement (except in the Dominican Republic) ofthC major decision makers These models itav be most useful in the small country situationwhere decision mAkers are more likely to be involved in tile priority-settirt process and are not just using tables of results Perhaps in larger systet7 where decisioii makine is more decentralited tileweighted criteria model can he applied at the regional level before relining research priorities at the national level

Another factor that can be uniortant in large svvtern is tle complerientarity between resea-h aid educat ion [hie larger the svstcnt the clser the linkages tend to be between universities and tileresearchi system Where universities are important tor agriclltural research an aidditional criterion may hc rteedCd to place value on the educationlal benefits of certain types of reseirch

Ihere are several additional iniplications with respect to how weighted criteria models relate to alterniative priori t-setting Inethod parIicularl the expected economic surplusapproach Befoce exploring these relationships kcx aspects of the surplus approacliare described based on aii application in Peru

14

of foods which are important in their diets Nutritional criteria were dropped in Uruguay however because numerous studies have shown that income levels are tile primary determinants of nutrition levels rather than the availability of particular foods Certainly caloric and protein content are debatable criteria To the extent that increased domestic production lowers the prices of certain food commodities therebyincreasing food consumption by the poor nutritional criteria can contribute to the distributional goal Land area also was included as a criterion in the Dominican Republic study but was dropped in Ecuador and Uruguly because of the overlap with value of production

Two criteria were used in each study to measure 1robahility of reseatch success The first measure was tilegap between own-country yields and neighboing country yields The second was the potential for success indicated by the researchers themselves The rationale for the gap criterion was that the larger the yield gap between the home country and similar countries tle greater the potential for yield gains The criterion is rather crude because yield gaps also can indicate a relative resource disadvantace forproducing the commodity even if the countries appear to be geoclimatically sinilar Consequently the gap criterion was eventually dropped in the tUruguav study The potential for success as indicated by the researchers is subjective but a very necessary measure to obtain

The gro up of criteria reficrred to as e1)cienc in use otresearch resources contained three criteria the relationship to research in the international centers the degree of emphasis on tilecommodity in le current research p ogram and the incentive for the privatesector to conduct the research Some research duplicates other research croplementsand still other research bears no relationship to research at tileinternational agriculturalresearch centers For certain comniodities all three relationships may hold depending on the type of research If national research is complementary to international researchthe cost of the rational research is lower Consequently a criterion was established which places greater weight on commodities with a high degree of complementaritybetween ntin Il d international research The criterion ofdelegree of emphasis in tile current iesercli program was established because there is a cost to rapidly adjustingcurrent h uiman and physical research resources from one commodity to another The criterion of private-sector incentives was used because the social benefits from the total research program in the country will be greatest if scarce public research funds are devoted to research on commodities for which the private sector does not have an incentive to support the research itself Of coui se the private-sector producer group can tax itself and pay the public sector to conduct the research for those commodities for which it has an incentive to privately support the research These commodities tend to be export commodities because producers can capture a higher proportion of the gainsfrom productivity increases for export commodities than for commodities primarily consumed domestically

The group of criteria related to the distribution ojresarchimpacts contained two criteria in the Uruguay study number of producers and the effect of increased productivity on the price of product The larger the number of farms producing a commodity the largerthe number of farmers afficted by the research and perhaps the higher the probabilitythat smaller farmers are being affeicted The distribution of benefits depends to a significant degree on price changes resulting from increased productivity Producers benefit more if prices decline very little and consumers benefit more ifprices decline substatitially for a given production increase For export er import commodities priceswill tend to decline less than for commodities primarily produced and consumed domestically For commodities which are not traded and are very important in the diets of the poor prices will tend to decline relatively sharply when productivity increases The division of benefits between producers and consumers also is affected by the fact

that producers are also consumers Consequently in the Dominican Republic and Ecuador studies an additional criterion importance of the commodity in home-consumption on the farm where produced also was included

In summary 10-15 criteria were used in tie three studies to identify and select research priorities by commodity The linkages anong the goals and criteria described above are summarized in Figure 1

Research Area Criteria The five criteria uicel to select research priorities by research aieas were (1 ) whether the research causes an increase in the use of relatively abundant resources and a saving of relatively scarce resources (2) the number and seventy of research problems (3) non-duplication with transferable research from outside the country (4) tne extent of private-sector incentives to conduct the research and (5) current emphases in the research program

The rate of economic growth will be more rapid if new technologies are developed that utilize relatively abundant resources and save relatively scarce resources The resource base varies by region consequently information on resource abundance criteria was collected by region in tileEcuador and Uruguay studies The same was true for the quantity and severity of research problems Information on the relationship of domestic to inernational research private-sector incentives and the current emphasis in the research program was collected on a naticnal basis These criteria all relate to the income growth (efficiency) goal It is difficult to identif research area criteria that measurc whether particular types of research aifect income distribution or variability

Data Collection and Model Implementation

Information used in the analysis included both quantitative data on value of productionnumber of farms value of exports and imports person-years devoted to research on different commodities and tile of protein per day in thenumber of calories and gramsdiet (for tileDominican Republic and Ecuador studies) as well as qualitative or subjective information on such factors as probability of success private sector incentives and severity of research problems Furthermore weights had to be elicited from decision makers to place relative emphasis on the various criteria

Step 1 Develop Cotninodity and Research Area Lists -- The first step was to determine the list ofcommadities to consider and the list of research areas In the Dominican Republic information was collected on 74 commodities In Ecuador an initial list of 109 commodities on which INIAP was conducting research was discussed Throughdecisions to eliminate some and to group others this list was reduced to 44 commedities for the analysis These decisions were made by the Technical Director for Research In Uruguay a smaller list of commodities was reduced to 21 commodities or commodity groups These decisions were made in consultation with the National Research and Fxtension Directors The list of research program areas contained nine areas in the Dominican Republic 16 in Ecuador and 16 in Uruguay

Step 2 Collect Quantitative and Qualitative Data on Chosen Criteria -- Information o-i quantitative criteria were gathered from local and FAO secondary data sources and one table was constructed For each criterion Commodities were ranked for each criterion Information needed for the qualitative riteria as well as the weights to place on criteria were obtained through interviews with sc rtists and administrators at boh the national and regional experiment stations In Euador and Uruguay substantial attention was devoted to identifying the appropriate respondents to each question as this is a crucial aspect of the procedure In Ecuador those interviewed were notified one week before the interviews and 72 people were interviewed by the Planning Director of INIAP and

9

FIJR 1 CONCEPTUAL ELEHNTS IN THE HEIGHTED CRITERIA MODEL FOR PAWING RESEARCH PRIORITIES BY COtlDITY

GOALS fficiency ]

CONCEPTUAL( )CRITERIA Pri7] Probabi i+y Research Distriutjon Iincome

ii J of Succes Resource Eff o L=Acplusmn a rilbility

CRITERIA 1 Value of 1 Scientist 1Rl ofto toNumberPrcduction Estimates 1 AnnualInternational Producers Variability2 Foreign Centers 2 Price in Gro

Exchange 2 Current Effect3 Future Raturn3)Emphasis 3 HomeDemand 3 Private Consumpt ion

4 Comparative Sector Advantage Incentives

Notes

(-n future studies an additional conceptural criteria under the security goal might be health (J4tri ional criteria such as grams of potein or kilo=alories could be included as additional easures of Distributional ikipacts

4ot used in Dominican Republic Ecuador and Uruojuay studies

his assistant In Uruguay the directors of all the regional stations were called into CIIAB headquarters for a aweting to discuss the study and this was followed up byon-site interviews and meetings with scientists and directors at the regional experimentstations as well as with economists in the Ministry of Agriculture These interviews and meetings were conducted by a person assigned to work wvith the Director of Cl LAB on the priority-setting study In the Dominican Republic a survey of scientists extensionworkers and administrators was conducted by the ISNAR consultant from the Instituto Superior de Agricultura who visited the various experiment stations Less care wastaken in the Dominican Repubic than in Ecuador and Uruguay to match the questionspertaining -o individual criteria to particular individuals This probably reduced theusefulness of the responses because those most familiar with the probability of research success For example may not be the same -s those most Familiar with comparativeadvantage

Step 3 Elicit 11eihts on Critria -- Relative weights to place on the different criteria were obtained Fronm national and regional research system administrators In Ecuador34 peopic verc used to determine the weights These wveights were established separatelyIor the commoditv and the research area criteria In 1iuguay fewer people (sevensystem and station directors) were used to determine the weights and more attention was devoted to grouping the criteria pertaining to particular goals A Delphi procedure was used in which the s7even directors were shown the average of the group and provided with an opportunity to adjust their weight

Step 4 )erive Rankinqs hv Commodity and By Research Area -- Once the basic information was collected axd organized into a seies of tables one procedure was followed to arrive at a final ranking of research priorities by commodity A second procedure resulted in lists of priorities by research area In the Dominican Republicmost of the calculations were made by hand in Ecuador and Uruguay microcomputerspreadsheet programs were used to assist with the calculations and to conduct sensitivity analysis

The steps in the procedure for ranking research priorities by commodity and research area are sumnmaried below and are illustrated with a simple example in Appendix 1 Brieily the steps for ranking commodities were

a) Th conniodities were ranked for each criterion for which quantitative data were available

b) Tihese rankings were multiplied by the weights assigned to each criterion and then the weighted criteria were added across to arrive at one sub-ranking for the

Luantitative criteria c) Each commodity was given a high low or none designation for each qualitative

(subjective) criterion The response which implied a need for greater research priority was assigned the number 2 the intermediate response was assigned I and the response which implied lower research priority was assigned 0 1herefore each comniodity received a 2 1 or 0 for each criterion

d) The weights assigned to each qualitative criterion were then multiplied by these numbers and the results were added across all criteria to arrive at a subtotal for each commoditv These subtotals were ranked from highest to lowest to provide a second sub-ranking

e) The sub-rankings for the quantitative and qualitative criteria were then given their corresponding weights and added together to arrive at a final ranking by commodityThe steps for ranking research priorities by research area were the same as steps (c)and (d) above because the criteria were all qualitative

Step 5 Analsis and Interpretationof Resuls -- In the Dominican Republic the resuLts of this weighted criteria analysis were uscd to determine a small set of cormodities and

II

research areas with the highest priority Further assessment was then made of humanphysical administrative and other resources needed to structure research programsfocused on these commodities and resea-ch topics In Ecuador and Uruguay the prioritized list of commodities was split into a high-priority group an intermediate-priority group and a low-priority group Research area priorities were identified for each region of tile country A more detailed analysis was then made of the human physical and other resources needed to support the high and intermediate priorities

In summary five major -rens were involved in operationalizing the weighted criteria models A complete set of rsults for each study included more than 20 tables and are not reproduced here Ilowever the final rankings for commodities and for research areas are shown in Tables I and 2 in Appendix 2 In the Dominican Republic a team of consultants followed up the initial priority-setting exercise with additional analysisand discussions at the experiment station level and eventually recommended tile establishment of five national commodity programs (rice cornsorghum beans coffeeand livestock) and one additional major research program (natural resources) In Ecuador the prioritized lists were widely distributed and discussed in the Ministry oi Agriculture the Board of FEIDIA the Commission on Science and Technology arid in AID There was concern in Ecuador with the weights placed on the individual criteria and therefore additional sensitivity analysis was conducted by the Planning Office of the Ministry There seemed to be a recognition in Ecuador that priority setting with a weighted criteria model is an iterative process and that much 4t the models value stems from tile discussion of criteria among the decision makers and from the sensitivityanalysis The high-priority list of comniodities in Ecuador included thL ten highest-priority commodities listed in Table 2b A project paper has been written forAI) which recommends funding For Fli)IAlbr three commodities (coffee soft corn for the Sierra and dairy) Coffee and soft coitn were the number 2 and 3 ranked prioritiesin tile weighted criteria model and dairy was number 9 although it was the top-rankedlivestock commodity

In Urugu ay the results from the first run with tile model were discussed with the research directors Ilie directors made small changes to the weights placed on criteriaand the mode was rerun resulting in the prioritized list in Appendix 2 Personnel in CIAAB are CUrrcwtv undertaking additional sensitivity analysis and with tie assistance of ISNA R are developing a plan for implementation

An Assessment of the Weighted Criteria Models Used in the Dominican RepublicEcuador and Uruguay

The Dominican Republic Ecuador and Uruguay studies were perhaps the first majorattempts to implement weighted criteria models with weights explicitly elicited from decision makers in NARS in less-developed countries A number of strengths andweaknesses in the procedures were identified during the analyses Some of thL weaknesses were corrected fron one study to the next some can be corrected in future studies and others are inherent in the approach

Trhe first strength mentioned earlier is the ability of the procedure to systematicallyincorporate both quantitative and qualitative information related to a set of multiplegoals and criteria in order to prioritize a lorng ist of commodities and research areas in a relatively short period of time Second the procedure proved relatively easy for both research administrators and the loc- -nalysts to understand with the exception of the )ominican Republic where the research administrators were not directly involved7

Third the analysis as applied in Ecuador and ULuguay with the direct involvement of research system administrators at various stages forced those decision makers to

12

consciously identifv and trade off goals and criteria Fourth the use of spreadsheet programs in EcUadfor and Uruguay facilitated sensitivity analysis after the initial set of priorities was deermimd Fifth the procedure provided a relatively objectiveassessment of prioriies because individuals were not allowed to rank commodities orresearch areas directly but had to weight criteria

The first weakness inherent in ex ante research priority setting and thus in theprocedure is that there is a large amount of subjectiveness Although tileapproach isless subjective than unstructured judgment there is subjectiveness in the responses toquestiorns related to some of the criteria and also ini the weights placed by decision makers on the criteria Second it proved diflicult to specify independent criteria with no overlap Third while the procedure required less time than an expected economic surplus analysis by commodity it required more time than asking respondents to directlyrank commodities and research areas In E-cuador the consiultant worked withpersonnel in INIAP and FEI)IA for one week at the start of the analysis to determinegoals and criteria to arrive at atlist of commodities and researh areas to list secondarydata needs to formulate questions to be asked of administrators and scientists to decide who should be asked particular questions and to explain the procedure itself includingthe spreadsheet program The required iidormation was collected over the Followingmonth and the consultant returned and worked with the Planning Director in INIAP for one week to conduct the analysis and write tip the results Revisions to the initial draftof the report we-e then made over the lollowing weeks A similar procedure to the onefollowed in Ecuador was used in l_ruguay althaugli more time was spent discussing the wording of questions to be asked of scientists Furthermore n Uruguay more care was taken to match the criteria with the goas

Certain of the criteria wil always be diflicult to eplain to tileinterviewees Thecriterion on comparative advantage is difficult for some respondents because of a lack of knowledge about neighboring countries If suflicient data time and resources are available this criterion may be rnea4ured more precisely using comparative advantage or domestic resource cost procedures suggested by longmire and Winkelnann Anotherdiflicult criterion to estiniate is the potential foture demand for the product The respondents in Fcuador were asked to consider how the demands for different foodswould change as income and populatiot grow Many of them had a hard timeconceiving that the total or per capita demand for certain Foods could actually decline as income grew In Uiruguay (ibe cmiparative advantage and future dema nd questions were answered by a small group of economists from the ministries of agriculture and trade In the Ecuador study more attention was devoted to assessing the most appropriate people for answering different questions than was the case in the Dominican Republic Ilowever some criticism surfaced about the weights placed on criteria InUruguay decision makers were given the opportunity to change their weights afterviewing the initial results This was very useftl because it demonstrated the implicationsof placing different weights on the various goals and criteria

Computer spreadsheet programs were a major help in Ecuador and Uruguay Additional work needs to be devoted to developing a more mentnu-driven prcgram but the people inthe planning office of INIAP were able to use the SuperCalcR program without too tmucht difliculty despite ialack of previous experience Ihe analyst in Uruguay was veryfamiliar with lotusl which proved most helpfil

It became clear from working inall three countries that it is preferable to work directlywith the final decision makers or their designees Inthe l)ominican Republic the study was conducted by a very competent set of consultants from the local agricultural universi ty but the procedure was not institutionalized to allow for additional sensitivityanalysis or future priority-setting efforts InEcuador and especially in Uruguay the

13

decision makers were directly involved conducted sensitivity analyses and were able to assess why certain commodities or research areas received high priority as a result of the analyses In both Ecuador and Uruguay they have a basic understanding of the strengths and weaknesses of tileprocedure and therelre can explain their subsequentdecisions to groups who might disagree with tilepriorities they establish

The corntrast between the Dominican Republic and tie Uruguay studies illustrates a veryimportant point No matter which technique is used to structure the analysispriority-setting exercises must integrate the decision makers (users) into the processPriority-setting procedures must be institutionalited the results of initial analysisdiscussed thoroughly with decision makers and sensitivity anilsis conlducteU alone the lines suggested by that discussion Tecinological change generates a surplus which can be appropriated by dilkircnt sectors or groups within the economy Who benelitdepends in part on the cotaniodities selected for research and the types of technoloLies generated One of the purposes of institutionalizing a formal prioritN- setting procedureis to help mediate conflicts that may arise (for example between producer groups and consumers) as a result of research-induced improved technologies

Other issues arose during the analyses First the initial grouping and elimination of certain commodities and research areas before lhe irialvis is an important step but one which must be taken with carelul consideration of possible coniplmentarities in research on particular productS or types of rearch Second some of the commodities such as forages are inputs into other commodities such as milk (Corseqtentlv these commodities llay have to he considered itscinps because it is the value-added and not tilegross value of output that is relevant when one coniiodity requires other commodities sinputs and when criteria such as value of production are used This was an issue in I ruguav where livestock are quite important liird little attention was devoted in atv of the studies to defining appropriate criteria Ior the increased inoome stability eoal Future studies if this be an ideltJiiLd LOal shOUld define risk criteria

In sutImary the weighted criteria model as applied in Lcuador and iuguav which had the advantage of bit ildin Oillessons lCarICd illdie l)ominican RCpublic appears to have been a uselul priority-setting tool It ha its ereatest advartage ill situations ill which there are rulItiple goals to consider a relatively major reassesnieiit of prioritiesis contenplated and the list of commodities dd resen rch areas to consider is largeWeighted criteria models described in this paper were applied in small countries with limited resources but with direct involvement (except in the Dominican Republic) ofthC major decision makers These models itav be most useful in the small country situationwhere decision mAkers are more likely to be involved in tile priority-settirt process and are not just using tables of results Perhaps in larger systet7 where decisioii makine is more decentralited tileweighted criteria model can he applied at the regional level before relining research priorities at the national level

Another factor that can be uniortant in large svvtern is tle complerientarity between resea-h aid educat ion [hie larger the svstcnt the clser the linkages tend to be between universities and tileresearchi system Where universities are important tor agriclltural research an aidditional criterion may hc rteedCd to place value on the educationlal benefits of certain types of reseirch

Ihere are several additional iniplications with respect to how weighted criteria models relate to alterniative priori t-setting Inethod parIicularl the expected economic surplusapproach Befoce exploring these relationships kcx aspects of the surplus approacliare described based on aii application in Peru

14

that producers are also consumers Consequently in the Dominican Republic and Ecuador studies an additional criterion importance of the commodity in home-consumption on the farm where produced also was included

In summary 10-15 criteria were used in tie three studies to identify and select research priorities by commodity The linkages anong the goals and criteria described above are summarized in Figure 1

Research Area Criteria The five criteria uicel to select research priorities by research aieas were (1 ) whether the research causes an increase in the use of relatively abundant resources and a saving of relatively scarce resources (2) the number and seventy of research problems (3) non-duplication with transferable research from outside the country (4) tne extent of private-sector incentives to conduct the research and (5) current emphases in the research program

The rate of economic growth will be more rapid if new technologies are developed that utilize relatively abundant resources and save relatively scarce resources The resource base varies by region consequently information on resource abundance criteria was collected by region in tileEcuador and Uruguay studies The same was true for the quantity and severity of research problems Information on the relationship of domestic to inernational research private-sector incentives and the current emphasis in the research program was collected on a naticnal basis These criteria all relate to the income growth (efficiency) goal It is difficult to identif research area criteria that measurc whether particular types of research aifect income distribution or variability

Data Collection and Model Implementation

Information used in the analysis included both quantitative data on value of productionnumber of farms value of exports and imports person-years devoted to research on different commodities and tile of protein per day in thenumber of calories and gramsdiet (for tileDominican Republic and Ecuador studies) as well as qualitative or subjective information on such factors as probability of success private sector incentives and severity of research problems Furthermore weights had to be elicited from decision makers to place relative emphasis on the various criteria

Step 1 Develop Cotninodity and Research Area Lists -- The first step was to determine the list ofcommadities to consider and the list of research areas In the Dominican Republic information was collected on 74 commodities In Ecuador an initial list of 109 commodities on which INIAP was conducting research was discussed Throughdecisions to eliminate some and to group others this list was reduced to 44 commedities for the analysis These decisions were made by the Technical Director for Research In Uruguay a smaller list of commodities was reduced to 21 commodities or commodity groups These decisions were made in consultation with the National Research and Fxtension Directors The list of research program areas contained nine areas in the Dominican Republic 16 in Ecuador and 16 in Uruguay

Step 2 Collect Quantitative and Qualitative Data on Chosen Criteria -- Information o-i quantitative criteria were gathered from local and FAO secondary data sources and one table was constructed For each criterion Commodities were ranked for each criterion Information needed for the qualitative riteria as well as the weights to place on criteria were obtained through interviews with sc rtists and administrators at boh the national and regional experiment stations In Euador and Uruguay substantial attention was devoted to identifying the appropriate respondents to each question as this is a crucial aspect of the procedure In Ecuador those interviewed were notified one week before the interviews and 72 people were interviewed by the Planning Director of INIAP and

9

FIJR 1 CONCEPTUAL ELEHNTS IN THE HEIGHTED CRITERIA MODEL FOR PAWING RESEARCH PRIORITIES BY COtlDITY

GOALS fficiency ]

CONCEPTUAL( )CRITERIA Pri7] Probabi i+y Research Distriutjon Iincome

ii J of Succes Resource Eff o L=Acplusmn a rilbility

CRITERIA 1 Value of 1 Scientist 1Rl ofto toNumberPrcduction Estimates 1 AnnualInternational Producers Variability2 Foreign Centers 2 Price in Gro

Exchange 2 Current Effect3 Future Raturn3)Emphasis 3 HomeDemand 3 Private Consumpt ion

4 Comparative Sector Advantage Incentives

Notes

(-n future studies an additional conceptural criteria under the security goal might be health (J4tri ional criteria such as grams of potein or kilo=alories could be included as additional easures of Distributional ikipacts

4ot used in Dominican Republic Ecuador and Uruojuay studies

his assistant In Uruguay the directors of all the regional stations were called into CIIAB headquarters for a aweting to discuss the study and this was followed up byon-site interviews and meetings with scientists and directors at the regional experimentstations as well as with economists in the Ministry of Agriculture These interviews and meetings were conducted by a person assigned to work wvith the Director of Cl LAB on the priority-setting study In the Dominican Republic a survey of scientists extensionworkers and administrators was conducted by the ISNAR consultant from the Instituto Superior de Agricultura who visited the various experiment stations Less care wastaken in the Dominican Repubic than in Ecuador and Uruguay to match the questionspertaining -o individual criteria to particular individuals This probably reduced theusefulness of the responses because those most familiar with the probability of research success For example may not be the same -s those most Familiar with comparativeadvantage

Step 3 Elicit 11eihts on Critria -- Relative weights to place on the different criteria were obtained Fronm national and regional research system administrators In Ecuador34 peopic verc used to determine the weights These wveights were established separatelyIor the commoditv and the research area criteria In 1iuguay fewer people (sevensystem and station directors) were used to determine the weights and more attention was devoted to grouping the criteria pertaining to particular goals A Delphi procedure was used in which the s7even directors were shown the average of the group and provided with an opportunity to adjust their weight

Step 4 )erive Rankinqs hv Commodity and By Research Area -- Once the basic information was collected axd organized into a seies of tables one procedure was followed to arrive at a final ranking of research priorities by commodity A second procedure resulted in lists of priorities by research area In the Dominican Republicmost of the calculations were made by hand in Ecuador and Uruguay microcomputerspreadsheet programs were used to assist with the calculations and to conduct sensitivity analysis

The steps in the procedure for ranking research priorities by commodity and research area are sumnmaried below and are illustrated with a simple example in Appendix 1 Brieily the steps for ranking commodities were

a) Th conniodities were ranked for each criterion for which quantitative data were available

b) Tihese rankings were multiplied by the weights assigned to each criterion and then the weighted criteria were added across to arrive at one sub-ranking for the

Luantitative criteria c) Each commodity was given a high low or none designation for each qualitative

(subjective) criterion The response which implied a need for greater research priority was assigned the number 2 the intermediate response was assigned I and the response which implied lower research priority was assigned 0 1herefore each comniodity received a 2 1 or 0 for each criterion

d) The weights assigned to each qualitative criterion were then multiplied by these numbers and the results were added across all criteria to arrive at a subtotal for each commoditv These subtotals were ranked from highest to lowest to provide a second sub-ranking

e) The sub-rankings for the quantitative and qualitative criteria were then given their corresponding weights and added together to arrive at a final ranking by commodityThe steps for ranking research priorities by research area were the same as steps (c)and (d) above because the criteria were all qualitative

Step 5 Analsis and Interpretationof Resuls -- In the Dominican Republic the resuLts of this weighted criteria analysis were uscd to determine a small set of cormodities and

II

research areas with the highest priority Further assessment was then made of humanphysical administrative and other resources needed to structure research programsfocused on these commodities and resea-ch topics In Ecuador and Uruguay the prioritized list of commodities was split into a high-priority group an intermediate-priority group and a low-priority group Research area priorities were identified for each region of tile country A more detailed analysis was then made of the human physical and other resources needed to support the high and intermediate priorities

In summary five major -rens were involved in operationalizing the weighted criteria models A complete set of rsults for each study included more than 20 tables and are not reproduced here Ilowever the final rankings for commodities and for research areas are shown in Tables I and 2 in Appendix 2 In the Dominican Republic a team of consultants followed up the initial priority-setting exercise with additional analysisand discussions at the experiment station level and eventually recommended tile establishment of five national commodity programs (rice cornsorghum beans coffeeand livestock) and one additional major research program (natural resources) In Ecuador the prioritized lists were widely distributed and discussed in the Ministry oi Agriculture the Board of FEIDIA the Commission on Science and Technology arid in AID There was concern in Ecuador with the weights placed on the individual criteria and therefore additional sensitivity analysis was conducted by the Planning Office of the Ministry There seemed to be a recognition in Ecuador that priority setting with a weighted criteria model is an iterative process and that much 4t the models value stems from tile discussion of criteria among the decision makers and from the sensitivityanalysis The high-priority list of comniodities in Ecuador included thL ten highest-priority commodities listed in Table 2b A project paper has been written forAI) which recommends funding For Fli)IAlbr three commodities (coffee soft corn for the Sierra and dairy) Coffee and soft coitn were the number 2 and 3 ranked prioritiesin tile weighted criteria model and dairy was number 9 although it was the top-rankedlivestock commodity

In Urugu ay the results from the first run with tile model were discussed with the research directors Ilie directors made small changes to the weights placed on criteriaand the mode was rerun resulting in the prioritized list in Appendix 2 Personnel in CIAAB are CUrrcwtv undertaking additional sensitivity analysis and with tie assistance of ISNA R are developing a plan for implementation

An Assessment of the Weighted Criteria Models Used in the Dominican RepublicEcuador and Uruguay

The Dominican Republic Ecuador and Uruguay studies were perhaps the first majorattempts to implement weighted criteria models with weights explicitly elicited from decision makers in NARS in less-developed countries A number of strengths andweaknesses in the procedures were identified during the analyses Some of thL weaknesses were corrected fron one study to the next some can be corrected in future studies and others are inherent in the approach

Trhe first strength mentioned earlier is the ability of the procedure to systematicallyincorporate both quantitative and qualitative information related to a set of multiplegoals and criteria in order to prioritize a lorng ist of commodities and research areas in a relatively short period of time Second the procedure proved relatively easy for both research administrators and the loc- -nalysts to understand with the exception of the )ominican Republic where the research administrators were not directly involved7

Third the analysis as applied in Ecuador and ULuguay with the direct involvement of research system administrators at various stages forced those decision makers to

12

consciously identifv and trade off goals and criteria Fourth the use of spreadsheet programs in EcUadfor and Uruguay facilitated sensitivity analysis after the initial set of priorities was deermimd Fifth the procedure provided a relatively objectiveassessment of prioriies because individuals were not allowed to rank commodities orresearch areas directly but had to weight criteria

The first weakness inherent in ex ante research priority setting and thus in theprocedure is that there is a large amount of subjectiveness Although tileapproach isless subjective than unstructured judgment there is subjectiveness in the responses toquestiorns related to some of the criteria and also ini the weights placed by decision makers on the criteria Second it proved diflicult to specify independent criteria with no overlap Third while the procedure required less time than an expected economic surplus analysis by commodity it required more time than asking respondents to directlyrank commodities and research areas In E-cuador the consiultant worked withpersonnel in INIAP and FEI)IA for one week at the start of the analysis to determinegoals and criteria to arrive at atlist of commodities and researh areas to list secondarydata needs to formulate questions to be asked of administrators and scientists to decide who should be asked particular questions and to explain the procedure itself includingthe spreadsheet program The required iidormation was collected over the Followingmonth and the consultant returned and worked with the Planning Director in INIAP for one week to conduct the analysis and write tip the results Revisions to the initial draftof the report we-e then made over the lollowing weeks A similar procedure to the onefollowed in Ecuador was used in l_ruguay althaugli more time was spent discussing the wording of questions to be asked of scientists Furthermore n Uruguay more care was taken to match the criteria with the goas

Certain of the criteria wil always be diflicult to eplain to tileinterviewees Thecriterion on comparative advantage is difficult for some respondents because of a lack of knowledge about neighboring countries If suflicient data time and resources are available this criterion may be rnea4ured more precisely using comparative advantage or domestic resource cost procedures suggested by longmire and Winkelnann Anotherdiflicult criterion to estiniate is the potential foture demand for the product The respondents in Fcuador were asked to consider how the demands for different foodswould change as income and populatiot grow Many of them had a hard timeconceiving that the total or per capita demand for certain Foods could actually decline as income grew In Uiruguay (ibe cmiparative advantage and future dema nd questions were answered by a small group of economists from the ministries of agriculture and trade In the Ecuador study more attention was devoted to assessing the most appropriate people for answering different questions than was the case in the Dominican Republic Ilowever some criticism surfaced about the weights placed on criteria InUruguay decision makers were given the opportunity to change their weights afterviewing the initial results This was very useftl because it demonstrated the implicationsof placing different weights on the various goals and criteria

Computer spreadsheet programs were a major help in Ecuador and Uruguay Additional work needs to be devoted to developing a more mentnu-driven prcgram but the people inthe planning office of INIAP were able to use the SuperCalcR program without too tmucht difliculty despite ialack of previous experience Ihe analyst in Uruguay was veryfamiliar with lotusl which proved most helpfil

It became clear from working inall three countries that it is preferable to work directlywith the final decision makers or their designees Inthe l)ominican Republic the study was conducted by a very competent set of consultants from the local agricultural universi ty but the procedure was not institutionalized to allow for additional sensitivityanalysis or future priority-setting efforts InEcuador and especially in Uruguay the

13

decision makers were directly involved conducted sensitivity analyses and were able to assess why certain commodities or research areas received high priority as a result of the analyses In both Ecuador and Uruguay they have a basic understanding of the strengths and weaknesses of tileprocedure and therelre can explain their subsequentdecisions to groups who might disagree with tilepriorities they establish

The corntrast between the Dominican Republic and tie Uruguay studies illustrates a veryimportant point No matter which technique is used to structure the analysispriority-setting exercises must integrate the decision makers (users) into the processPriority-setting procedures must be institutionalited the results of initial analysisdiscussed thoroughly with decision makers and sensitivity anilsis conlducteU alone the lines suggested by that discussion Tecinological change generates a surplus which can be appropriated by dilkircnt sectors or groups within the economy Who benelitdepends in part on the cotaniodities selected for research and the types of technoloLies generated One of the purposes of institutionalizing a formal prioritN- setting procedureis to help mediate conflicts that may arise (for example between producer groups and consumers) as a result of research-induced improved technologies

Other issues arose during the analyses First the initial grouping and elimination of certain commodities and research areas before lhe irialvis is an important step but one which must be taken with carelul consideration of possible coniplmentarities in research on particular productS or types of rearch Second some of the commodities such as forages are inputs into other commodities such as milk (Corseqtentlv these commodities llay have to he considered itscinps because it is the value-added and not tilegross value of output that is relevant when one coniiodity requires other commodities sinputs and when criteria such as value of production are used This was an issue in I ruguav where livestock are quite important liird little attention was devoted in atv of the studies to defining appropriate criteria Ior the increased inoome stability eoal Future studies if this be an ideltJiiLd LOal shOUld define risk criteria

In sutImary the weighted criteria model as applied in Lcuador and iuguav which had the advantage of bit ildin Oillessons lCarICd illdie l)ominican RCpublic appears to have been a uselul priority-setting tool It ha its ereatest advartage ill situations ill which there are rulItiple goals to consider a relatively major reassesnieiit of prioritiesis contenplated and the list of commodities dd resen rch areas to consider is largeWeighted criteria models described in this paper were applied in small countries with limited resources but with direct involvement (except in the Dominican Republic) ofthC major decision makers These models itav be most useful in the small country situationwhere decision mAkers are more likely to be involved in tile priority-settirt process and are not just using tables of results Perhaps in larger systet7 where decisioii makine is more decentralited tileweighted criteria model can he applied at the regional level before relining research priorities at the national level

Another factor that can be uniortant in large svvtern is tle complerientarity between resea-h aid educat ion [hie larger the svstcnt the clser the linkages tend to be between universities and tileresearchi system Where universities are important tor agriclltural research an aidditional criterion may hc rteedCd to place value on the educationlal benefits of certain types of reseirch

Ihere are several additional iniplications with respect to how weighted criteria models relate to alterniative priori t-setting Inethod parIicularl the expected economic surplusapproach Befoce exploring these relationships kcx aspects of the surplus approacliare described based on aii application in Peru

14

FIJR 1 CONCEPTUAL ELEHNTS IN THE HEIGHTED CRITERIA MODEL FOR PAWING RESEARCH PRIORITIES BY COtlDITY

GOALS fficiency ]

CONCEPTUAL( )CRITERIA Pri7] Probabi i+y Research Distriutjon Iincome

ii J of Succes Resource Eff o L=Acplusmn a rilbility

CRITERIA 1 Value of 1 Scientist 1Rl ofto toNumberPrcduction Estimates 1 AnnualInternational Producers Variability2 Foreign Centers 2 Price in Gro

Exchange 2 Current Effect3 Future Raturn3)Emphasis 3 HomeDemand 3 Private Consumpt ion

4 Comparative Sector Advantage Incentives

Notes

(-n future studies an additional conceptural criteria under the security goal might be health (J4tri ional criteria such as grams of potein or kilo=alories could be included as additional easures of Distributional ikipacts

4ot used in Dominican Republic Ecuador and Uruojuay studies

his assistant In Uruguay the directors of all the regional stations were called into CIIAB headquarters for a aweting to discuss the study and this was followed up byon-site interviews and meetings with scientists and directors at the regional experimentstations as well as with economists in the Ministry of Agriculture These interviews and meetings were conducted by a person assigned to work wvith the Director of Cl LAB on the priority-setting study In the Dominican Republic a survey of scientists extensionworkers and administrators was conducted by the ISNAR consultant from the Instituto Superior de Agricultura who visited the various experiment stations Less care wastaken in the Dominican Repubic than in Ecuador and Uruguay to match the questionspertaining -o individual criteria to particular individuals This probably reduced theusefulness of the responses because those most familiar with the probability of research success For example may not be the same -s those most Familiar with comparativeadvantage

Step 3 Elicit 11eihts on Critria -- Relative weights to place on the different criteria were obtained Fronm national and regional research system administrators In Ecuador34 peopic verc used to determine the weights These wveights were established separatelyIor the commoditv and the research area criteria In 1iuguay fewer people (sevensystem and station directors) were used to determine the weights and more attention was devoted to grouping the criteria pertaining to particular goals A Delphi procedure was used in which the s7even directors were shown the average of the group and provided with an opportunity to adjust their weight

Step 4 )erive Rankinqs hv Commodity and By Research Area -- Once the basic information was collected axd organized into a seies of tables one procedure was followed to arrive at a final ranking of research priorities by commodity A second procedure resulted in lists of priorities by research area In the Dominican Republicmost of the calculations were made by hand in Ecuador and Uruguay microcomputerspreadsheet programs were used to assist with the calculations and to conduct sensitivity analysis

The steps in the procedure for ranking research priorities by commodity and research area are sumnmaried below and are illustrated with a simple example in Appendix 1 Brieily the steps for ranking commodities were

a) Th conniodities were ranked for each criterion for which quantitative data were available

b) Tihese rankings were multiplied by the weights assigned to each criterion and then the weighted criteria were added across to arrive at one sub-ranking for the

Luantitative criteria c) Each commodity was given a high low or none designation for each qualitative

(subjective) criterion The response which implied a need for greater research priority was assigned the number 2 the intermediate response was assigned I and the response which implied lower research priority was assigned 0 1herefore each comniodity received a 2 1 or 0 for each criterion

d) The weights assigned to each qualitative criterion were then multiplied by these numbers and the results were added across all criteria to arrive at a subtotal for each commoditv These subtotals were ranked from highest to lowest to provide a second sub-ranking

e) The sub-rankings for the quantitative and qualitative criteria were then given their corresponding weights and added together to arrive at a final ranking by commodityThe steps for ranking research priorities by research area were the same as steps (c)and (d) above because the criteria were all qualitative

Step 5 Analsis and Interpretationof Resuls -- In the Dominican Republic the resuLts of this weighted criteria analysis were uscd to determine a small set of cormodities and

II

research areas with the highest priority Further assessment was then made of humanphysical administrative and other resources needed to structure research programsfocused on these commodities and resea-ch topics In Ecuador and Uruguay the prioritized list of commodities was split into a high-priority group an intermediate-priority group and a low-priority group Research area priorities were identified for each region of tile country A more detailed analysis was then made of the human physical and other resources needed to support the high and intermediate priorities

In summary five major -rens were involved in operationalizing the weighted criteria models A complete set of rsults for each study included more than 20 tables and are not reproduced here Ilowever the final rankings for commodities and for research areas are shown in Tables I and 2 in Appendix 2 In the Dominican Republic a team of consultants followed up the initial priority-setting exercise with additional analysisand discussions at the experiment station level and eventually recommended tile establishment of five national commodity programs (rice cornsorghum beans coffeeand livestock) and one additional major research program (natural resources) In Ecuador the prioritized lists were widely distributed and discussed in the Ministry oi Agriculture the Board of FEIDIA the Commission on Science and Technology arid in AID There was concern in Ecuador with the weights placed on the individual criteria and therefore additional sensitivity analysis was conducted by the Planning Office of the Ministry There seemed to be a recognition in Ecuador that priority setting with a weighted criteria model is an iterative process and that much 4t the models value stems from tile discussion of criteria among the decision makers and from the sensitivityanalysis The high-priority list of comniodities in Ecuador included thL ten highest-priority commodities listed in Table 2b A project paper has been written forAI) which recommends funding For Fli)IAlbr three commodities (coffee soft corn for the Sierra and dairy) Coffee and soft coitn were the number 2 and 3 ranked prioritiesin tile weighted criteria model and dairy was number 9 although it was the top-rankedlivestock commodity

In Urugu ay the results from the first run with tile model were discussed with the research directors Ilie directors made small changes to the weights placed on criteriaand the mode was rerun resulting in the prioritized list in Appendix 2 Personnel in CIAAB are CUrrcwtv undertaking additional sensitivity analysis and with tie assistance of ISNA R are developing a plan for implementation

An Assessment of the Weighted Criteria Models Used in the Dominican RepublicEcuador and Uruguay

The Dominican Republic Ecuador and Uruguay studies were perhaps the first majorattempts to implement weighted criteria models with weights explicitly elicited from decision makers in NARS in less-developed countries A number of strengths andweaknesses in the procedures were identified during the analyses Some of thL weaknesses were corrected fron one study to the next some can be corrected in future studies and others are inherent in the approach

Trhe first strength mentioned earlier is the ability of the procedure to systematicallyincorporate both quantitative and qualitative information related to a set of multiplegoals and criteria in order to prioritize a lorng ist of commodities and research areas in a relatively short period of time Second the procedure proved relatively easy for both research administrators and the loc- -nalysts to understand with the exception of the )ominican Republic where the research administrators were not directly involved7

Third the analysis as applied in Ecuador and ULuguay with the direct involvement of research system administrators at various stages forced those decision makers to

12

consciously identifv and trade off goals and criteria Fourth the use of spreadsheet programs in EcUadfor and Uruguay facilitated sensitivity analysis after the initial set of priorities was deermimd Fifth the procedure provided a relatively objectiveassessment of prioriies because individuals were not allowed to rank commodities orresearch areas directly but had to weight criteria

The first weakness inherent in ex ante research priority setting and thus in theprocedure is that there is a large amount of subjectiveness Although tileapproach isless subjective than unstructured judgment there is subjectiveness in the responses toquestiorns related to some of the criteria and also ini the weights placed by decision makers on the criteria Second it proved diflicult to specify independent criteria with no overlap Third while the procedure required less time than an expected economic surplus analysis by commodity it required more time than asking respondents to directlyrank commodities and research areas In E-cuador the consiultant worked withpersonnel in INIAP and FEI)IA for one week at the start of the analysis to determinegoals and criteria to arrive at atlist of commodities and researh areas to list secondarydata needs to formulate questions to be asked of administrators and scientists to decide who should be asked particular questions and to explain the procedure itself includingthe spreadsheet program The required iidormation was collected over the Followingmonth and the consultant returned and worked with the Planning Director in INIAP for one week to conduct the analysis and write tip the results Revisions to the initial draftof the report we-e then made over the lollowing weeks A similar procedure to the onefollowed in Ecuador was used in l_ruguay althaugli more time was spent discussing the wording of questions to be asked of scientists Furthermore n Uruguay more care was taken to match the criteria with the goas

Certain of the criteria wil always be diflicult to eplain to tileinterviewees Thecriterion on comparative advantage is difficult for some respondents because of a lack of knowledge about neighboring countries If suflicient data time and resources are available this criterion may be rnea4ured more precisely using comparative advantage or domestic resource cost procedures suggested by longmire and Winkelnann Anotherdiflicult criterion to estiniate is the potential foture demand for the product The respondents in Fcuador were asked to consider how the demands for different foodswould change as income and populatiot grow Many of them had a hard timeconceiving that the total or per capita demand for certain Foods could actually decline as income grew In Uiruguay (ibe cmiparative advantage and future dema nd questions were answered by a small group of economists from the ministries of agriculture and trade In the Ecuador study more attention was devoted to assessing the most appropriate people for answering different questions than was the case in the Dominican Republic Ilowever some criticism surfaced about the weights placed on criteria InUruguay decision makers were given the opportunity to change their weights afterviewing the initial results This was very useftl because it demonstrated the implicationsof placing different weights on the various goals and criteria

Computer spreadsheet programs were a major help in Ecuador and Uruguay Additional work needs to be devoted to developing a more mentnu-driven prcgram but the people inthe planning office of INIAP were able to use the SuperCalcR program without too tmucht difliculty despite ialack of previous experience Ihe analyst in Uruguay was veryfamiliar with lotusl which proved most helpfil

It became clear from working inall three countries that it is preferable to work directlywith the final decision makers or their designees Inthe l)ominican Republic the study was conducted by a very competent set of consultants from the local agricultural universi ty but the procedure was not institutionalized to allow for additional sensitivityanalysis or future priority-setting efforts InEcuador and especially in Uruguay the

13

decision makers were directly involved conducted sensitivity analyses and were able to assess why certain commodities or research areas received high priority as a result of the analyses In both Ecuador and Uruguay they have a basic understanding of the strengths and weaknesses of tileprocedure and therelre can explain their subsequentdecisions to groups who might disagree with tilepriorities they establish

The corntrast between the Dominican Republic and tie Uruguay studies illustrates a veryimportant point No matter which technique is used to structure the analysispriority-setting exercises must integrate the decision makers (users) into the processPriority-setting procedures must be institutionalited the results of initial analysisdiscussed thoroughly with decision makers and sensitivity anilsis conlducteU alone the lines suggested by that discussion Tecinological change generates a surplus which can be appropriated by dilkircnt sectors or groups within the economy Who benelitdepends in part on the cotaniodities selected for research and the types of technoloLies generated One of the purposes of institutionalizing a formal prioritN- setting procedureis to help mediate conflicts that may arise (for example between producer groups and consumers) as a result of research-induced improved technologies

Other issues arose during the analyses First the initial grouping and elimination of certain commodities and research areas before lhe irialvis is an important step but one which must be taken with carelul consideration of possible coniplmentarities in research on particular productS or types of rearch Second some of the commodities such as forages are inputs into other commodities such as milk (Corseqtentlv these commodities llay have to he considered itscinps because it is the value-added and not tilegross value of output that is relevant when one coniiodity requires other commodities sinputs and when criteria such as value of production are used This was an issue in I ruguav where livestock are quite important liird little attention was devoted in atv of the studies to defining appropriate criteria Ior the increased inoome stability eoal Future studies if this be an ideltJiiLd LOal shOUld define risk criteria

In sutImary the weighted criteria model as applied in Lcuador and iuguav which had the advantage of bit ildin Oillessons lCarICd illdie l)ominican RCpublic appears to have been a uselul priority-setting tool It ha its ereatest advartage ill situations ill which there are rulItiple goals to consider a relatively major reassesnieiit of prioritiesis contenplated and the list of commodities dd resen rch areas to consider is largeWeighted criteria models described in this paper were applied in small countries with limited resources but with direct involvement (except in the Dominican Republic) ofthC major decision makers These models itav be most useful in the small country situationwhere decision mAkers are more likely to be involved in tile priority-settirt process and are not just using tables of results Perhaps in larger systet7 where decisioii makine is more decentralited tileweighted criteria model can he applied at the regional level before relining research priorities at the national level

Another factor that can be uniortant in large svvtern is tle complerientarity between resea-h aid educat ion [hie larger the svstcnt the clser the linkages tend to be between universities and tileresearchi system Where universities are important tor agriclltural research an aidditional criterion may hc rteedCd to place value on the educationlal benefits of certain types of reseirch

Ihere are several additional iniplications with respect to how weighted criteria models relate to alterniative priori t-setting Inethod parIicularl the expected economic surplusapproach Befoce exploring these relationships kcx aspects of the surplus approacliare described based on aii application in Peru

14

his assistant In Uruguay the directors of all the regional stations were called into CIIAB headquarters for a aweting to discuss the study and this was followed up byon-site interviews and meetings with scientists and directors at the regional experimentstations as well as with economists in the Ministry of Agriculture These interviews and meetings were conducted by a person assigned to work wvith the Director of Cl LAB on the priority-setting study In the Dominican Republic a survey of scientists extensionworkers and administrators was conducted by the ISNAR consultant from the Instituto Superior de Agricultura who visited the various experiment stations Less care wastaken in the Dominican Repubic than in Ecuador and Uruguay to match the questionspertaining -o individual criteria to particular individuals This probably reduced theusefulness of the responses because those most familiar with the probability of research success For example may not be the same -s those most Familiar with comparativeadvantage

Step 3 Elicit 11eihts on Critria -- Relative weights to place on the different criteria were obtained Fronm national and regional research system administrators In Ecuador34 peopic verc used to determine the weights These wveights were established separatelyIor the commoditv and the research area criteria In 1iuguay fewer people (sevensystem and station directors) were used to determine the weights and more attention was devoted to grouping the criteria pertaining to particular goals A Delphi procedure was used in which the s7even directors were shown the average of the group and provided with an opportunity to adjust their weight

Step 4 )erive Rankinqs hv Commodity and By Research Area -- Once the basic information was collected axd organized into a seies of tables one procedure was followed to arrive at a final ranking of research priorities by commodity A second procedure resulted in lists of priorities by research area In the Dominican Republicmost of the calculations were made by hand in Ecuador and Uruguay microcomputerspreadsheet programs were used to assist with the calculations and to conduct sensitivity analysis

The steps in the procedure for ranking research priorities by commodity and research area are sumnmaried below and are illustrated with a simple example in Appendix 1 Brieily the steps for ranking commodities were

a) Th conniodities were ranked for each criterion for which quantitative data were available

b) Tihese rankings were multiplied by the weights assigned to each criterion and then the weighted criteria were added across to arrive at one sub-ranking for the

Luantitative criteria c) Each commodity was given a high low or none designation for each qualitative

(subjective) criterion The response which implied a need for greater research priority was assigned the number 2 the intermediate response was assigned I and the response which implied lower research priority was assigned 0 1herefore each comniodity received a 2 1 or 0 for each criterion

d) The weights assigned to each qualitative criterion were then multiplied by these numbers and the results were added across all criteria to arrive at a subtotal for each commoditv These subtotals were ranked from highest to lowest to provide a second sub-ranking

e) The sub-rankings for the quantitative and qualitative criteria were then given their corresponding weights and added together to arrive at a final ranking by commodityThe steps for ranking research priorities by research area were the same as steps (c)and (d) above because the criteria were all qualitative

Step 5 Analsis and Interpretationof Resuls -- In the Dominican Republic the resuLts of this weighted criteria analysis were uscd to determine a small set of cormodities and

II

research areas with the highest priority Further assessment was then made of humanphysical administrative and other resources needed to structure research programsfocused on these commodities and resea-ch topics In Ecuador and Uruguay the prioritized list of commodities was split into a high-priority group an intermediate-priority group and a low-priority group Research area priorities were identified for each region of tile country A more detailed analysis was then made of the human physical and other resources needed to support the high and intermediate priorities

In summary five major -rens were involved in operationalizing the weighted criteria models A complete set of rsults for each study included more than 20 tables and are not reproduced here Ilowever the final rankings for commodities and for research areas are shown in Tables I and 2 in Appendix 2 In the Dominican Republic a team of consultants followed up the initial priority-setting exercise with additional analysisand discussions at the experiment station level and eventually recommended tile establishment of five national commodity programs (rice cornsorghum beans coffeeand livestock) and one additional major research program (natural resources) In Ecuador the prioritized lists were widely distributed and discussed in the Ministry oi Agriculture the Board of FEIDIA the Commission on Science and Technology arid in AID There was concern in Ecuador with the weights placed on the individual criteria and therefore additional sensitivity analysis was conducted by the Planning Office of the Ministry There seemed to be a recognition in Ecuador that priority setting with a weighted criteria model is an iterative process and that much 4t the models value stems from tile discussion of criteria among the decision makers and from the sensitivityanalysis The high-priority list of comniodities in Ecuador included thL ten highest-priority commodities listed in Table 2b A project paper has been written forAI) which recommends funding For Fli)IAlbr three commodities (coffee soft corn for the Sierra and dairy) Coffee and soft coitn were the number 2 and 3 ranked prioritiesin tile weighted criteria model and dairy was number 9 although it was the top-rankedlivestock commodity

In Urugu ay the results from the first run with tile model were discussed with the research directors Ilie directors made small changes to the weights placed on criteriaand the mode was rerun resulting in the prioritized list in Appendix 2 Personnel in CIAAB are CUrrcwtv undertaking additional sensitivity analysis and with tie assistance of ISNA R are developing a plan for implementation

An Assessment of the Weighted Criteria Models Used in the Dominican RepublicEcuador and Uruguay

The Dominican Republic Ecuador and Uruguay studies were perhaps the first majorattempts to implement weighted criteria models with weights explicitly elicited from decision makers in NARS in less-developed countries A number of strengths andweaknesses in the procedures were identified during the analyses Some of thL weaknesses were corrected fron one study to the next some can be corrected in future studies and others are inherent in the approach

Trhe first strength mentioned earlier is the ability of the procedure to systematicallyincorporate both quantitative and qualitative information related to a set of multiplegoals and criteria in order to prioritize a lorng ist of commodities and research areas in a relatively short period of time Second the procedure proved relatively easy for both research administrators and the loc- -nalysts to understand with the exception of the )ominican Republic where the research administrators were not directly involved7

Third the analysis as applied in Ecuador and ULuguay with the direct involvement of research system administrators at various stages forced those decision makers to

12

consciously identifv and trade off goals and criteria Fourth the use of spreadsheet programs in EcUadfor and Uruguay facilitated sensitivity analysis after the initial set of priorities was deermimd Fifth the procedure provided a relatively objectiveassessment of prioriies because individuals were not allowed to rank commodities orresearch areas directly but had to weight criteria

The first weakness inherent in ex ante research priority setting and thus in theprocedure is that there is a large amount of subjectiveness Although tileapproach isless subjective than unstructured judgment there is subjectiveness in the responses toquestiorns related to some of the criteria and also ini the weights placed by decision makers on the criteria Second it proved diflicult to specify independent criteria with no overlap Third while the procedure required less time than an expected economic surplus analysis by commodity it required more time than asking respondents to directlyrank commodities and research areas In E-cuador the consiultant worked withpersonnel in INIAP and FEI)IA for one week at the start of the analysis to determinegoals and criteria to arrive at atlist of commodities and researh areas to list secondarydata needs to formulate questions to be asked of administrators and scientists to decide who should be asked particular questions and to explain the procedure itself includingthe spreadsheet program The required iidormation was collected over the Followingmonth and the consultant returned and worked with the Planning Director in INIAP for one week to conduct the analysis and write tip the results Revisions to the initial draftof the report we-e then made over the lollowing weeks A similar procedure to the onefollowed in Ecuador was used in l_ruguay althaugli more time was spent discussing the wording of questions to be asked of scientists Furthermore n Uruguay more care was taken to match the criteria with the goas

Certain of the criteria wil always be diflicult to eplain to tileinterviewees Thecriterion on comparative advantage is difficult for some respondents because of a lack of knowledge about neighboring countries If suflicient data time and resources are available this criterion may be rnea4ured more precisely using comparative advantage or domestic resource cost procedures suggested by longmire and Winkelnann Anotherdiflicult criterion to estiniate is the potential foture demand for the product The respondents in Fcuador were asked to consider how the demands for different foodswould change as income and populatiot grow Many of them had a hard timeconceiving that the total or per capita demand for certain Foods could actually decline as income grew In Uiruguay (ibe cmiparative advantage and future dema nd questions were answered by a small group of economists from the ministries of agriculture and trade In the Ecuador study more attention was devoted to assessing the most appropriate people for answering different questions than was the case in the Dominican Republic Ilowever some criticism surfaced about the weights placed on criteria InUruguay decision makers were given the opportunity to change their weights afterviewing the initial results This was very useftl because it demonstrated the implicationsof placing different weights on the various goals and criteria

Computer spreadsheet programs were a major help in Ecuador and Uruguay Additional work needs to be devoted to developing a more mentnu-driven prcgram but the people inthe planning office of INIAP were able to use the SuperCalcR program without too tmucht difliculty despite ialack of previous experience Ihe analyst in Uruguay was veryfamiliar with lotusl which proved most helpfil

It became clear from working inall three countries that it is preferable to work directlywith the final decision makers or their designees Inthe l)ominican Republic the study was conducted by a very competent set of consultants from the local agricultural universi ty but the procedure was not institutionalized to allow for additional sensitivityanalysis or future priority-setting efforts InEcuador and especially in Uruguay the

13

decision makers were directly involved conducted sensitivity analyses and were able to assess why certain commodities or research areas received high priority as a result of the analyses In both Ecuador and Uruguay they have a basic understanding of the strengths and weaknesses of tileprocedure and therelre can explain their subsequentdecisions to groups who might disagree with tilepriorities they establish

The corntrast between the Dominican Republic and tie Uruguay studies illustrates a veryimportant point No matter which technique is used to structure the analysispriority-setting exercises must integrate the decision makers (users) into the processPriority-setting procedures must be institutionalited the results of initial analysisdiscussed thoroughly with decision makers and sensitivity anilsis conlducteU alone the lines suggested by that discussion Tecinological change generates a surplus which can be appropriated by dilkircnt sectors or groups within the economy Who benelitdepends in part on the cotaniodities selected for research and the types of technoloLies generated One of the purposes of institutionalizing a formal prioritN- setting procedureis to help mediate conflicts that may arise (for example between producer groups and consumers) as a result of research-induced improved technologies

Other issues arose during the analyses First the initial grouping and elimination of certain commodities and research areas before lhe irialvis is an important step but one which must be taken with carelul consideration of possible coniplmentarities in research on particular productS or types of rearch Second some of the commodities such as forages are inputs into other commodities such as milk (Corseqtentlv these commodities llay have to he considered itscinps because it is the value-added and not tilegross value of output that is relevant when one coniiodity requires other commodities sinputs and when criteria such as value of production are used This was an issue in I ruguav where livestock are quite important liird little attention was devoted in atv of the studies to defining appropriate criteria Ior the increased inoome stability eoal Future studies if this be an ideltJiiLd LOal shOUld define risk criteria

In sutImary the weighted criteria model as applied in Lcuador and iuguav which had the advantage of bit ildin Oillessons lCarICd illdie l)ominican RCpublic appears to have been a uselul priority-setting tool It ha its ereatest advartage ill situations ill which there are rulItiple goals to consider a relatively major reassesnieiit of prioritiesis contenplated and the list of commodities dd resen rch areas to consider is largeWeighted criteria models described in this paper were applied in small countries with limited resources but with direct involvement (except in the Dominican Republic) ofthC major decision makers These models itav be most useful in the small country situationwhere decision mAkers are more likely to be involved in tile priority-settirt process and are not just using tables of results Perhaps in larger systet7 where decisioii makine is more decentralited tileweighted criteria model can he applied at the regional level before relining research priorities at the national level

Another factor that can be uniortant in large svvtern is tle complerientarity between resea-h aid educat ion [hie larger the svstcnt the clser the linkages tend to be between universities and tileresearchi system Where universities are important tor agriclltural research an aidditional criterion may hc rteedCd to place value on the educationlal benefits of certain types of reseirch

Ihere are several additional iniplications with respect to how weighted criteria models relate to alterniative priori t-setting Inethod parIicularl the expected economic surplusapproach Befoce exploring these relationships kcx aspects of the surplus approacliare described based on aii application in Peru

14

research areas with the highest priority Further assessment was then made of humanphysical administrative and other resources needed to structure research programsfocused on these commodities and resea-ch topics In Ecuador and Uruguay the prioritized list of commodities was split into a high-priority group an intermediate-priority group and a low-priority group Research area priorities were identified for each region of tile country A more detailed analysis was then made of the human physical and other resources needed to support the high and intermediate priorities

In summary five major -rens were involved in operationalizing the weighted criteria models A complete set of rsults for each study included more than 20 tables and are not reproduced here Ilowever the final rankings for commodities and for research areas are shown in Tables I and 2 in Appendix 2 In the Dominican Republic a team of consultants followed up the initial priority-setting exercise with additional analysisand discussions at the experiment station level and eventually recommended tile establishment of five national commodity programs (rice cornsorghum beans coffeeand livestock) and one additional major research program (natural resources) In Ecuador the prioritized lists were widely distributed and discussed in the Ministry oi Agriculture the Board of FEIDIA the Commission on Science and Technology arid in AID There was concern in Ecuador with the weights placed on the individual criteria and therefore additional sensitivity analysis was conducted by the Planning Office of the Ministry There seemed to be a recognition in Ecuador that priority setting with a weighted criteria model is an iterative process and that much 4t the models value stems from tile discussion of criteria among the decision makers and from the sensitivityanalysis The high-priority list of comniodities in Ecuador included thL ten highest-priority commodities listed in Table 2b A project paper has been written forAI) which recommends funding For Fli)IAlbr three commodities (coffee soft corn for the Sierra and dairy) Coffee and soft coitn were the number 2 and 3 ranked prioritiesin tile weighted criteria model and dairy was number 9 although it was the top-rankedlivestock commodity

In Urugu ay the results from the first run with tile model were discussed with the research directors Ilie directors made small changes to the weights placed on criteriaand the mode was rerun resulting in the prioritized list in Appendix 2 Personnel in CIAAB are CUrrcwtv undertaking additional sensitivity analysis and with tie assistance of ISNA R are developing a plan for implementation

An Assessment of the Weighted Criteria Models Used in the Dominican RepublicEcuador and Uruguay

The Dominican Republic Ecuador and Uruguay studies were perhaps the first majorattempts to implement weighted criteria models with weights explicitly elicited from decision makers in NARS in less-developed countries A number of strengths andweaknesses in the procedures were identified during the analyses Some of thL weaknesses were corrected fron one study to the next some can be corrected in future studies and others are inherent in the approach

Trhe first strength mentioned earlier is the ability of the procedure to systematicallyincorporate both quantitative and qualitative information related to a set of multiplegoals and criteria in order to prioritize a lorng ist of commodities and research areas in a relatively short period of time Second the procedure proved relatively easy for both research administrators and the loc- -nalysts to understand with the exception of the )ominican Republic where the research administrators were not directly involved7

Third the analysis as applied in Ecuador and ULuguay with the direct involvement of research system administrators at various stages forced those decision makers to

12

consciously identifv and trade off goals and criteria Fourth the use of spreadsheet programs in EcUadfor and Uruguay facilitated sensitivity analysis after the initial set of priorities was deermimd Fifth the procedure provided a relatively objectiveassessment of prioriies because individuals were not allowed to rank commodities orresearch areas directly but had to weight criteria

The first weakness inherent in ex ante research priority setting and thus in theprocedure is that there is a large amount of subjectiveness Although tileapproach isless subjective than unstructured judgment there is subjectiveness in the responses toquestiorns related to some of the criteria and also ini the weights placed by decision makers on the criteria Second it proved diflicult to specify independent criteria with no overlap Third while the procedure required less time than an expected economic surplus analysis by commodity it required more time than asking respondents to directlyrank commodities and research areas In E-cuador the consiultant worked withpersonnel in INIAP and FEI)IA for one week at the start of the analysis to determinegoals and criteria to arrive at atlist of commodities and researh areas to list secondarydata needs to formulate questions to be asked of administrators and scientists to decide who should be asked particular questions and to explain the procedure itself includingthe spreadsheet program The required iidormation was collected over the Followingmonth and the consultant returned and worked with the Planning Director in INIAP for one week to conduct the analysis and write tip the results Revisions to the initial draftof the report we-e then made over the lollowing weeks A similar procedure to the onefollowed in Ecuador was used in l_ruguay althaugli more time was spent discussing the wording of questions to be asked of scientists Furthermore n Uruguay more care was taken to match the criteria with the goas

Certain of the criteria wil always be diflicult to eplain to tileinterviewees Thecriterion on comparative advantage is difficult for some respondents because of a lack of knowledge about neighboring countries If suflicient data time and resources are available this criterion may be rnea4ured more precisely using comparative advantage or domestic resource cost procedures suggested by longmire and Winkelnann Anotherdiflicult criterion to estiniate is the potential foture demand for the product The respondents in Fcuador were asked to consider how the demands for different foodswould change as income and populatiot grow Many of them had a hard timeconceiving that the total or per capita demand for certain Foods could actually decline as income grew In Uiruguay (ibe cmiparative advantage and future dema nd questions were answered by a small group of economists from the ministries of agriculture and trade In the Ecuador study more attention was devoted to assessing the most appropriate people for answering different questions than was the case in the Dominican Republic Ilowever some criticism surfaced about the weights placed on criteria InUruguay decision makers were given the opportunity to change their weights afterviewing the initial results This was very useftl because it demonstrated the implicationsof placing different weights on the various goals and criteria

Computer spreadsheet programs were a major help in Ecuador and Uruguay Additional work needs to be devoted to developing a more mentnu-driven prcgram but the people inthe planning office of INIAP were able to use the SuperCalcR program without too tmucht difliculty despite ialack of previous experience Ihe analyst in Uruguay was veryfamiliar with lotusl which proved most helpfil

It became clear from working inall three countries that it is preferable to work directlywith the final decision makers or their designees Inthe l)ominican Republic the study was conducted by a very competent set of consultants from the local agricultural universi ty but the procedure was not institutionalized to allow for additional sensitivityanalysis or future priority-setting efforts InEcuador and especially in Uruguay the

13

decision makers were directly involved conducted sensitivity analyses and were able to assess why certain commodities or research areas received high priority as a result of the analyses In both Ecuador and Uruguay they have a basic understanding of the strengths and weaknesses of tileprocedure and therelre can explain their subsequentdecisions to groups who might disagree with tilepriorities they establish

The corntrast between the Dominican Republic and tie Uruguay studies illustrates a veryimportant point No matter which technique is used to structure the analysispriority-setting exercises must integrate the decision makers (users) into the processPriority-setting procedures must be institutionalited the results of initial analysisdiscussed thoroughly with decision makers and sensitivity anilsis conlducteU alone the lines suggested by that discussion Tecinological change generates a surplus which can be appropriated by dilkircnt sectors or groups within the economy Who benelitdepends in part on the cotaniodities selected for research and the types of technoloLies generated One of the purposes of institutionalizing a formal prioritN- setting procedureis to help mediate conflicts that may arise (for example between producer groups and consumers) as a result of research-induced improved technologies

Other issues arose during the analyses First the initial grouping and elimination of certain commodities and research areas before lhe irialvis is an important step but one which must be taken with carelul consideration of possible coniplmentarities in research on particular productS or types of rearch Second some of the commodities such as forages are inputs into other commodities such as milk (Corseqtentlv these commodities llay have to he considered itscinps because it is the value-added and not tilegross value of output that is relevant when one coniiodity requires other commodities sinputs and when criteria such as value of production are used This was an issue in I ruguav where livestock are quite important liird little attention was devoted in atv of the studies to defining appropriate criteria Ior the increased inoome stability eoal Future studies if this be an ideltJiiLd LOal shOUld define risk criteria

In sutImary the weighted criteria model as applied in Lcuador and iuguav which had the advantage of bit ildin Oillessons lCarICd illdie l)ominican RCpublic appears to have been a uselul priority-setting tool It ha its ereatest advartage ill situations ill which there are rulItiple goals to consider a relatively major reassesnieiit of prioritiesis contenplated and the list of commodities dd resen rch areas to consider is largeWeighted criteria models described in this paper were applied in small countries with limited resources but with direct involvement (except in the Dominican Republic) ofthC major decision makers These models itav be most useful in the small country situationwhere decision mAkers are more likely to be involved in tile priority-settirt process and are not just using tables of results Perhaps in larger systet7 where decisioii makine is more decentralited tileweighted criteria model can he applied at the regional level before relining research priorities at the national level

Another factor that can be uniortant in large svvtern is tle complerientarity between resea-h aid educat ion [hie larger the svstcnt the clser the linkages tend to be between universities and tileresearchi system Where universities are important tor agriclltural research an aidditional criterion may hc rteedCd to place value on the educationlal benefits of certain types of reseirch

Ihere are several additional iniplications with respect to how weighted criteria models relate to alterniative priori t-setting Inethod parIicularl the expected economic surplusapproach Befoce exploring these relationships kcx aspects of the surplus approacliare described based on aii application in Peru

14

consciously identifv and trade off goals and criteria Fourth the use of spreadsheet programs in EcUadfor and Uruguay facilitated sensitivity analysis after the initial set of priorities was deermimd Fifth the procedure provided a relatively objectiveassessment of prioriies because individuals were not allowed to rank commodities orresearch areas directly but had to weight criteria

The first weakness inherent in ex ante research priority setting and thus in theprocedure is that there is a large amount of subjectiveness Although tileapproach isless subjective than unstructured judgment there is subjectiveness in the responses toquestiorns related to some of the criteria and also ini the weights placed by decision makers on the criteria Second it proved diflicult to specify independent criteria with no overlap Third while the procedure required less time than an expected economic surplus analysis by commodity it required more time than asking respondents to directlyrank commodities and research areas In E-cuador the consiultant worked withpersonnel in INIAP and FEI)IA for one week at the start of the analysis to determinegoals and criteria to arrive at atlist of commodities and researh areas to list secondarydata needs to formulate questions to be asked of administrators and scientists to decide who should be asked particular questions and to explain the procedure itself includingthe spreadsheet program The required iidormation was collected over the Followingmonth and the consultant returned and worked with the Planning Director in INIAP for one week to conduct the analysis and write tip the results Revisions to the initial draftof the report we-e then made over the lollowing weeks A similar procedure to the onefollowed in Ecuador was used in l_ruguay althaugli more time was spent discussing the wording of questions to be asked of scientists Furthermore n Uruguay more care was taken to match the criteria with the goas

Certain of the criteria wil always be diflicult to eplain to tileinterviewees Thecriterion on comparative advantage is difficult for some respondents because of a lack of knowledge about neighboring countries If suflicient data time and resources are available this criterion may be rnea4ured more precisely using comparative advantage or domestic resource cost procedures suggested by longmire and Winkelnann Anotherdiflicult criterion to estiniate is the potential foture demand for the product The respondents in Fcuador were asked to consider how the demands for different foodswould change as income and populatiot grow Many of them had a hard timeconceiving that the total or per capita demand for certain Foods could actually decline as income grew In Uiruguay (ibe cmiparative advantage and future dema nd questions were answered by a small group of economists from the ministries of agriculture and trade In the Ecuador study more attention was devoted to assessing the most appropriate people for answering different questions than was the case in the Dominican Republic Ilowever some criticism surfaced about the weights placed on criteria InUruguay decision makers were given the opportunity to change their weights afterviewing the initial results This was very useftl because it demonstrated the implicationsof placing different weights on the various goals and criteria

Computer spreadsheet programs were a major help in Ecuador and Uruguay Additional work needs to be devoted to developing a more mentnu-driven prcgram but the people inthe planning office of INIAP were able to use the SuperCalcR program without too tmucht difliculty despite ialack of previous experience Ihe analyst in Uruguay was veryfamiliar with lotusl which proved most helpfil

It became clear from working inall three countries that it is preferable to work directlywith the final decision makers or their designees Inthe l)ominican Republic the study was conducted by a very competent set of consultants from the local agricultural universi ty but the procedure was not institutionalized to allow for additional sensitivityanalysis or future priority-setting efforts InEcuador and especially in Uruguay the

13

decision makers were directly involved conducted sensitivity analyses and were able to assess why certain commodities or research areas received high priority as a result of the analyses In both Ecuador and Uruguay they have a basic understanding of the strengths and weaknesses of tileprocedure and therelre can explain their subsequentdecisions to groups who might disagree with tilepriorities they establish

The corntrast between the Dominican Republic and tie Uruguay studies illustrates a veryimportant point No matter which technique is used to structure the analysispriority-setting exercises must integrate the decision makers (users) into the processPriority-setting procedures must be institutionalited the results of initial analysisdiscussed thoroughly with decision makers and sensitivity anilsis conlducteU alone the lines suggested by that discussion Tecinological change generates a surplus which can be appropriated by dilkircnt sectors or groups within the economy Who benelitdepends in part on the cotaniodities selected for research and the types of technoloLies generated One of the purposes of institutionalizing a formal prioritN- setting procedureis to help mediate conflicts that may arise (for example between producer groups and consumers) as a result of research-induced improved technologies

Other issues arose during the analyses First the initial grouping and elimination of certain commodities and research areas before lhe irialvis is an important step but one which must be taken with carelul consideration of possible coniplmentarities in research on particular productS or types of rearch Second some of the commodities such as forages are inputs into other commodities such as milk (Corseqtentlv these commodities llay have to he considered itscinps because it is the value-added and not tilegross value of output that is relevant when one coniiodity requires other commodities sinputs and when criteria such as value of production are used This was an issue in I ruguav where livestock are quite important liird little attention was devoted in atv of the studies to defining appropriate criteria Ior the increased inoome stability eoal Future studies if this be an ideltJiiLd LOal shOUld define risk criteria

In sutImary the weighted criteria model as applied in Lcuador and iuguav which had the advantage of bit ildin Oillessons lCarICd illdie l)ominican RCpublic appears to have been a uselul priority-setting tool It ha its ereatest advartage ill situations ill which there are rulItiple goals to consider a relatively major reassesnieiit of prioritiesis contenplated and the list of commodities dd resen rch areas to consider is largeWeighted criteria models described in this paper were applied in small countries with limited resources but with direct involvement (except in the Dominican Republic) ofthC major decision makers These models itav be most useful in the small country situationwhere decision mAkers are more likely to be involved in tile priority-settirt process and are not just using tables of results Perhaps in larger systet7 where decisioii makine is more decentralited tileweighted criteria model can he applied at the regional level before relining research priorities at the national level

Another factor that can be uniortant in large svvtern is tle complerientarity between resea-h aid educat ion [hie larger the svstcnt the clser the linkages tend to be between universities and tileresearchi system Where universities are important tor agriclltural research an aidditional criterion may hc rteedCd to place value on the educationlal benefits of certain types of reseirch

Ihere are several additional iniplications with respect to how weighted criteria models relate to alterniative priori t-setting Inethod parIicularl the expected economic surplusapproach Befoce exploring these relationships kcx aspects of the surplus approacliare described based on aii application in Peru

14

decision makers were directly involved conducted sensitivity analyses and were able to assess why certain commodities or research areas received high priority as a result of the analyses In both Ecuador and Uruguay they have a basic understanding of the strengths and weaknesses of tileprocedure and therelre can explain their subsequentdecisions to groups who might disagree with tilepriorities they establish

The corntrast between the Dominican Republic and tie Uruguay studies illustrates a veryimportant point No matter which technique is used to structure the analysispriority-setting exercises must integrate the decision makers (users) into the processPriority-setting procedures must be institutionalited the results of initial analysisdiscussed thoroughly with decision makers and sensitivity anilsis conlducteU alone the lines suggested by that discussion Tecinological change generates a surplus which can be appropriated by dilkircnt sectors or groups within the economy Who benelitdepends in part on the cotaniodities selected for research and the types of technoloLies generated One of the purposes of institutionalizing a formal prioritN- setting procedureis to help mediate conflicts that may arise (for example between producer groups and consumers) as a result of research-induced improved technologies

Other issues arose during the analyses First the initial grouping and elimination of certain commodities and research areas before lhe irialvis is an important step but one which must be taken with carelul consideration of possible coniplmentarities in research on particular productS or types of rearch Second some of the commodities such as forages are inputs into other commodities such as milk (Corseqtentlv these commodities llay have to he considered itscinps because it is the value-added and not tilegross value of output that is relevant when one coniiodity requires other commodities sinputs and when criteria such as value of production are used This was an issue in I ruguav where livestock are quite important liird little attention was devoted in atv of the studies to defining appropriate criteria Ior the increased inoome stability eoal Future studies if this be an ideltJiiLd LOal shOUld define risk criteria

In sutImary the weighted criteria model as applied in Lcuador and iuguav which had the advantage of bit ildin Oillessons lCarICd illdie l)ominican RCpublic appears to have been a uselul priority-setting tool It ha its ereatest advartage ill situations ill which there are rulItiple goals to consider a relatively major reassesnieiit of prioritiesis contenplated and the list of commodities dd resen rch areas to consider is largeWeighted criteria models described in this paper were applied in small countries with limited resources but with direct involvement (except in the Dominican Republic) ofthC major decision makers These models itav be most useful in the small country situationwhere decision mAkers are more likely to be involved in tile priority-settirt process and are not just using tables of results Perhaps in larger systet7 where decisioii makine is more decentralited tileweighted criteria model can he applied at the regional level before relining research priorities at the national level

Another factor that can be uniortant in large svvtern is tle complerientarity between resea-h aid educat ion [hie larger the svstcnt the clser the linkages tend to be between universities and tileresearchi system Where universities are important tor agriclltural research an aidditional criterion may hc rteedCd to place value on the educationlal benefits of certain types of reseirch

Ihere are several additional iniplications with respect to how weighted criteria models relate to alterniative priori t-setting Inethod parIicularl the expected economic surplusapproach Befoce exploring these relationships kcx aspects of the surplus approacliare described based on aii application in Peru

14

IV THE USE OF EXPECTED ECONOMIC SURPLUS ANALYSIS IN PERU

In 1985 an expecteJ economic iirpls anAlysis ssas conducted of the five majorcormnoditv research and extenion Iamp F) prorams in Peru These R amp F programsfor rice corn small vramspotatoes and beans were begun in 19Sl and the benefits werejust begimling to flow at the time of the analv sis ( onsetlfuentlv the study was basicallyan evaluation of Itnture benefits and previous costs Ihe intent was not to prioritizeamong tilelive commodity programs but to estimate rates of return to the It amp 1ivestment and to calculate tie distribution of benefits between producers andconsumer to provide infociimation to the new eovernment about those programsHIowever the same basic procedure could le used for priority setting In fact f)avisOrant and Ryan ofCIAR and II PRI has e recently developed an expected economicsurplus model to assist IR in establislin its research funding piorities Thefbllowing asoFssment of the Peru studv provides a useful perspective on tie potentialapplication of tis approach iI a devclopil COuIIry context

[he proxdures brieflv aniaried here are described in more detail in Norton G(anozaand lPomarcda Ilie irsi step involved deCloping questionnaires to be used ininterviews sWith rCSeil hes [lld sXtctio workcr lortv-five experiment station)researchers were einerctssd t)obtiii tlicir projections of the most likely yield or costchanlgcs dte to particuLar reaerll proje1 Sprobabuies of success and tine lags fur therelease of new technoloic Iortv evstion workers were interviewed to obtain theirprojection of the itimtu xv eCootaphical lead of new tccltioloi-ies and estimates oftie deprcitmItl ol preMt tcchnoloie R search and extension woikers were askedabout additiontl inputs neCCded t1 sCilthe improsed itiethods possible expansion ofarea Aiutiited aTd or rpl icut of eitnc crops and about the spread of new tchnologIe s ssith 1t1udvItlL ltCxctClstoti

fhe benefits of imertultural ampR1 re quantiticd usitie an expected economic surpluscriterion ( iae il pioducer oid coti umicr surplus rcsultiti trom rightward shifts inthe stlpplstlrvc thit il ocI [rilc or sere projected to occur due to technologicallatc scrc ilculaitcd Separate tials s were coIiucted lor cih comitioditY anddiffertit beneit loriiil--s were ued delMeLdiL ot1tilesituation 1tr eaich commodity withCespect Ito mip)rt or cports titarketable rplus shilts in demand over itiie due to

populiton nd iiL0otc at1d covernmiient pricitg policies Internal rates of return toreseirh were il]klited ald tiledistributioiI of benefits to producers and consumers tvssesed Reserch co t data for the atnalvis came from INI PA records Consumer

111I poucer surplus nelssis rsqlires estimiAtes of tileprice responsiveness (elasticities)osupply at1d fcIingd PlbsiNe1d expenditure elastiitiCs were iscd to approximateitncotie ClItiitIe Atd to cailcIlitC filLe Chisticitv of demtiatnd estimates Ilie incomeel 1sticitcs ere needed to help issess shifits in demand over time Supply elasticityCsttates Acre not available ind altcriiatic assumptions were used illthe analysisCAlculatoti vereperformed uin ut microcomputer spreadsheet progrim (StuprCakc5 )Atnd a suhstamtia a01onlat of s stlVltsi)iiis45 was performed Summarized results from that studv are shon in tblcs 3 and 4 ofAppendix 2

An Assessment of tileExpecieui LEconomic Sirlius Procedure Used in Peru

fhe expeclted eotuotImc sUplus ippr0LIh used ili Peru proved particularly useful forpturiln the cllct ()Alternitz c pricige pliiCsott both tiletotal Ard the distributin

of reseairch heefits i hirt-vei cs were uIcluded inthe spreadsheet program tO allvfor different dcitind trade paiiiy d other wishimptiotv The study was conducted ouer a fhur-mnth period cf time Becaeuse a litlcr-t spreadsheet analysis must beonomplhted or etch conimmdity and because of the detailed information required onVictors such s iticonie and price elasticities it would 1edifficult to apply the procedure

15

used in Peru to a long list of commodities It may make sense in future studies however to narrow down the list of alternatives using a weighted criteria procedure to a small set (10-20) and then use the expected cOnIOnliC surplus approach to prioritize those alternatives using the w ciehts derived trom the first step to keigt ht the eilIcienCe and distribution goal

It is dilicul t to use the expected economic surplus piok edore far rinking reseearch areas because of tile problem of appivitif it tol certain arascI ilc t Ocis-ccllofllics to relatively basic rcsca ch and to systcms or inteidiscipliniary research IlOvever it Ciln be used to CiiIate the ecoioticIL etiiins of Ilaliv other types of resealch sich as plant breeding reedand dica c control etc

The expcted econIoitc irtllus approach doe require a higher level of economic training On tholt of the lcA ianaixst than dOCs the weighted criteria a pproach Il addition the procedure Is currently implementcd appears to be more of a black box for decision lllakers It wits Caicr to explain fie lioic (I the Ac ilclited criteria procedure to administrators than it was the expected econOMnc siu nprplocedure Finally the expected eCOIIOIic spls procckll ctnnot readily incorlmrate tain cliteria such as prsMate-seCtor iticlitIctics to the coltdtl otrsc(rIli

V iM PIICAHlONS F)R I)[V L()PNIINT OlF IMPROVLEI) RIrS [AIRCIIF PRI) RIIY-S IN PRO )CEDIRES

Ie transler of new tecliniolis fepends ott how siitible those teclinoloicis are to local conditios [he sai e i ttiuc or institutional innovatiOnsI sUch as iiproved research prioiit-Cttint I-re dirsc Not i lv irc multiple voals Ind criterii itpOrtliit inl Illos stratecic decimilkin qtatis iII -RS but decisions of different levels of i pOta e ire It i ifl eltm tsilllto tittle and reNsairtcs are ivailable for Illlttkilt the ifcIiil [ tiites rcst rch stews are subl ntiallv rc trlucturCd At other trimeCs I Iorn2ec II thC tlaiili II tc cotultries the basic research lllstitlit ii 11ci i re ivail iblc for research but fii others the1ti t itisttltilns Ire illIu It- c iap f (oncfuiitl there is a need Ior t flexille Ipproalih thli can [ ii I the tune fi itcct iices availale adl the eCOitOtiiuc ittiportlliC I the dcci t it l l iie prioiitv-settilg eercises sLch aIs those fcscribcd kc requirc Iiliunil flow-up aIctivities of several types Receaich

projects ie he ened iciti e 1iiClcsiois iii ide anid pioerillilig completed for other )pti c r ctiitv

Wlciel cl s ni c arcIes i I lir the dec itiakiill proces lie timite I-a le nit is h rt (or thel o c of I llocatlli is relatively ItLncmpl llrlt1ic 0 deision wllt encril e tIcie pr1ltd ro ccIsil lakrs based on) economic pri icinlcs rtIi hem sti ci itakes )toky resea-ch pority- citing issues ltiwt~ic ur R in tid lr re c[lli Ili ati irticle a lfew vears avo Nortonaid itd (lalwcl IPi0 decloped i t i dliics baiscd onl fit applying the value of protduction crllteri aitd tlhcI follovilnlu fll tip vithi ast of eticlOills thicli decision natkcrs lii i-k ticls litirrie inLtil Jcii Pa1 rtiularlv forplaneliilmeetles priflci or i ei t 1ic l lcisions sth cid h ill n lt pr[ uLsefl becaise o1 their reicc llotC r r n1_ i itii is ld tic illhiti C Of ui ertaitV iiltereit ili theInc i pt[tlliiil sulcces2lts It lrrlilf prfqccts

As thl I1l(1tice f the lecii tc t-ses l al ilt)[ otuiill pieces dlesired the Wre ic- criteia ritIlIld lia hc liclpiil ptrticularly vhen the coi lities or reseirch ilreis ecllcrc-d ire oti taian where eerch )ilputs tie dilicult to quatItifI1tie IlnfIIbeCr Of c Ltinn(iodtIc to c)iiisidi is slill the rtsewiich oullputsllre relatively

16

easy to quantify and additional time and resources are available then the expectedeconomic surplus approach mentioned earlier mry he useful With that approach theconomic rates of return to research on diflerent commodities or rescarch areas can be calculated and a more accurate projection made of the distribution of benelits between producers and consumers Benelits also can be rcgionalied within a country and between countries b considering spillover ellects itssuggested by l)avis Oran and Ryan0

It is important to recognie the direct relationhip between the weighted criteria method and the expected economic sirpilus approach lssenrially the expected economic surplus procedure explicitly asNsues that the researii goals are raising the level o national income (elliciencv) and elluitV (distribution) hliereiore it captures most of the criteria mentioned in he weighted criteria discussion and removes the need to weight individual criteria [here remains a rjeed to weight the cliciencv and distributional goals but the sarime procedure used in tie weilite criteria model can be used to elicit weiehts for tlese coals Vithin the set ofdistrihutional goals weights can be applied to dilferent regiois and tO pre uLr consutiers the economic (internal)versus If onlil rate of return is used to ratnk canicdities then all the weight is implicitly placed theon ellicien cy coal

1CLca Lse the expected econiunir surplu s approach subsumes a large array of variables into a manageabe nut ber of indicators it ailows For consistent and reproducibleutegration of both econoinic Mid scientilic viriables into the priority-setting decision process Ihese include

a) Economic Variables

Sq uarntity of specific cornurodities produced and consumed colmodlit prices price responisiseness of supply and demand demand shifters (popilation income) degree md nature of distortions due to price policies bull discount raite cost of research

b) Fechnical Variatiles

probability I-or success of each research program direct (outprt-enhanlcirIg or cost-reducing) effects of research over time (for

example expected yield gains from research) bull expected time rite of adoption and geographical spread ofresearch results (inter-regional spillover ellects of research results

InI summary gcenral guidelines weighted criteria models expected economic surplusmodels and others have their own comparative advantage and can also be used togetherTherefore one logicai solutio to meeting the needs for priority-setting mechanisms in NARS is to develop a flexible nuenri-driven intelactive cimputer program that can assist Ce alyst in selecting and Lnplo iii alternative procedures [he procedure selected kwld depend on the aderqtiac of the data resources and time available and econonic importance of the decision This program could then be used as parz of anill stitutuonaliocd priority-setting and research management process

17

V1 CONCLUSIONS

Potenwial exists for developing a fornralized Ilexible procedure for use by NARS managers in research priority setting Ii pocedU-e build on existing weighted)s can criteria and expected economic surplus models hut additional refinements are neededNARS administrators in Euador and rucuat were Vel receptive to the structured weighted criteria mnodel Research administralors in the l)omini_an Republic who were not involved in the moe development were ipparently ler receptive which poinats out the n1ced to institutiot1Alize the model in the research ilgency during the models initial use in a particular countr-V BceWrs at Liuctmied priority-setting procedures aregenerated as much by the process of decision ma cr onsidering ciiteria and weights on goals as they are by tie final rcsuils

Key elements to be included in werEhred criteria priority-settine procedures include (1)eliciting tIre major development goals to which the research svstem is expected to(outribute (2) delfning a set ofe tcria for commodity ard research area selection (3)sueggestin a set of itcasures to assist in ranking comnmdities or research areas for each critcria (4)specilfig seco larY and primary information nces resulting from the nMeasures (5 providing questi naitires and i ) definng the weighting procedur (or

aggregating across criteria to arrive ait a final set of research priorities If the initialnumbe of ltierniatives is small or is niirrowed down alier applvi g the weighted criteria model tire por ntiil exists for employing art expected economic surplus analysisproedure Itl that case consumer and produer surplus wOUld provide the measures of several of the crit-ria hut weights would still need to b( applied to tIre efliciencv and distributriil oal lotential also exists fsr plating weights on those goals and also pfl~(lg weights on idditional gois nd criterl i such as income sta bility is)ni-duplication

dillrent procedCIr rCqureC dilerent miomit

of priviate-scetor reseirth tire erittierfA benIefits ol research or environmental ustaitl bi litv

It iirpartt to remcilcr thait)Ip orit-ie nrocedures are subjective Also a time (an1itst and decision makers)

tesources Lirid iiriiONrrIIu ni iiilt lircirts itt the procedures currently employed inNARS irust imore titan piy ioi tcirrslves through improved agricultural productivityitsa result of miprov d iillotitrils of research money Recent methodologicalinproveients in priorit -ettirie picedures and enhanced mrpiter capabilitiesincrease the cliices of nrak ig soq-iiJectie changes in priority-settinI procedures in NA RS

i8

Foot notes

1) See Ruttan for a discussion of several of these studies 2) Rules of thumb aresometimes used such as the congruence approach where Funds

are allocated across research programs in proportion to the value of agriculturalproduction in each commodity See 5jalmon or a discussion and apptcation of this approach

3) Changes often result as ivll frora the introduction of forcign-isistad research projects

4) l takes time Ir agricultural research to yield new results Ior cx miple Pardeyreports art average research gestation lag for the 1S state agricultUrA cI xperhnentstations durig the early 1970s of around three to four years

5) Additionl discussionl of these methods cau be 1bund in Shuoswav Rtltian- hiuhand Tollini Parton ct al Anderson and Parton Scobie and Norton and I)

6) See lia vlicek and Norton for a sommirv of several of these procedures (ai tio) isneeded when ipplyg quantitaivC procedures to research project selection becausescientists are often the best judge of tue appropriat projects within a given research area

7) Tile Dominicani Republic was in the process ofrestructuring its agricultural researchsystem at the tieni of the priority-setting study and the newly created aency(Il)FA ) was VJtihIO(t a (drector

S) The ned for discussion and sensitivity analysis k evident in the list of research priorities by commodity shown in fable la ofppendix 2 iat list includes somesurprises compared withI a ranking based on valut of productioni

9) [he expected economic surplus approach as described by Davis ranl and Rvanis particularly oseful for international agenicie because of its emphasis on spilloverstcroSs cCnitrics

10) A l and ISNRaP ire currently refining the Dasis Ormn and Ryan expectedC ooin c su pluS procedure to incorporate regional spillovers within countries II ( ~leart the Ceononiic-teCiical dichotomy preseoted here is principally forexpository purp( sec lechni-al ariables such as the expected rate of adoptionand spillovr clects are responsivc to both technical and economic forcesNeveitheless the dichotom is osetul whei1 thinking about the principal sources ofdata Manv of the ecnnoiic Ntriables can be obtained from secondary sources

while sone of the teclii-ical aintbles relire direct inrut from scientirts and program leaders In the ibsine of hiard data the sensitivity of the results can betested against a reasonable rane of parameter estimates

19

References

Alston JMG V Edwards and J W Freebairn Market Distortions and Benefitsfrom Research Department of Agriculture and Rural Affairs in Victoria and Lairobe University March 19 S7 Minimeo

Anderson J R and K A Parton Technologies for Guiding the Allocation ofResources A1mong Rural Research State of the ArtProjects Iraethus 1(1983)180-20l

Araji A R J Sim and R 1 Gardner Returns to Agricultural Research andExtcnsion lPro era m An lx Anrte pproach r gr lcon 60(1978) 96-4-8

Binswanger 11P and I6 Ryan Flficiency and lIqtuitv IssueS in Ex-Ante lljcaionof Research Resources Idlia 1 o L32(1977) 217-31

Centro de hivestieacio 1 Agricola Alherto lioerger (CIAI) Identificacion ySeleccion )e Prioridades de luvestig cion eropccnaria en elUruguay Reportprepared lor CIAB by (9 lcrrcira G Norton and C Valverde May 1987

Consultative Group on International giculmral Research (CGIAR) TechicalAdvisory Committee 1) C Rview f (C IAR Priorities ad Ftm SrtiiesTAC Secretariat Food aiod Agriculture Oigonization of the United Nations ugust [985

Dais JS PA ()ram tiid J( Ryan Assessing ofregate Agricultural research Internatonal AustralianPrioitics n Perspective Centre for International

ericnltriral Pcarcli (ACIl() and the liternttional Food Policy Research Iustitute (11 [R I)D1711 Alrfgust I9Mo

Edwards G V W ehairo 1laspuill a Cnlrs Gains Frt ResearchlhJeopv i10 licutin t(Rural kcsea -11 llraia A Report to the I onuonweIti Cou cil and iExtension Australianfor Ruri] Rescarci Canberra Goerntnult l vhing crvice 1981

Edwards (j W aid JV icebairn The Gains From Research into Tradeable Uoiataoditics Amir1 er Lcon 66(1 4) 41-419

Espinosa P G Norton and llD Gross identificacion Y Seleccion De Prioridades De lnvestigacion Agropecnaria en el Ecuador iwntituto Nacional de InvestigacionAgropecuaria (INIP ))raft ()ctober 19S6

Fishel XVI The Minnesota Agricultura IResca rechReource Allocation System and Experiment Resource 11lwain in lgricukral esearch ed XVI Fishel Minneapolis Minnwesota (niversitY of Minnesota Pres 19)71

Ilavlicek 1G aild i V Norton Review of Research Project Ivaluation andSelection Methods A Report to ARDl1 Stai Paper SP-13-SI l)epartment of gricultural icoonoics Virginia Polytechnic initute and State University November 1981

Instituto Superior de gricultura (ISA) Contenido lecnico de la Investigacion delInstituto l)omnicnno de Investigaciones gropecuarias Report prepared for

29

ISNAR in the Dominicin Republic by W Moscoso A Coutu I) Bandy and G W Norton June 1986

Longmire 1and 1) Vinkelmann Research Resource Allocation and ComparativeAdvantace CIMI YT paner presented at the XIX (ongre s of InternationalAgricultural Economics Association MIalaga Spain August 1985

Mahlstede J P Iong-Rangqe Planning at tie Iowa Agricnltural and IIonie EconomicsExperimental Station R surce Allocatioin I tritdrtur l Researcih cdIishel Minneapolis Minnesota rnversitv of Minnesota Press 1971

WL

Norton C and IS l)avis Val ittin cturns to Agicultural Research ARe ew for Olr lc(t 6 1)( -609Q o3t 1S

Norton GW aid V(G (Gioalhe lienclit s of A cricultural Research and Extensionin Peru paper proired for INI PA and b USI D Lima Peru June 1985

Norton (X and V( (Ganoa (4uideliTeCs fr Allocations of ResourcesAgricultural Researchland Extension

to in fcru paper prepared Ior INIPA and for

USAID Lima Peru 1chrue rv 1980

Nortoi GV (4 (ianoa and C Pmiarel IPotential Benefits For giculturalResearch and Extension it Peru i1hgr(ti( 1987) 247-257E-ct

lcu dLILti Paper presented to tle Ntion if S11npositim ott I valuatiag Agricultural Research Productivitl

Pardev P(G [Ie riculturUl Know i I [oIIIILction An Impirical Look

tlantai(e rgi Iinn 7lt V I9

Parton K IRAndcreon irid I N 1htm lantion Mtods1fr Iustralialltod Rsr1 XL iCult Ra IlI lCononlics Bulletin No 29 Armidale IiiscrsitV of New laid I1

Paulsen A ind I) R Kildor Tluation and Planning of Research in the ExperimentStation tr Jl 1olo 51t(l9oS 11 1902

flinstr up-- et seti P aid I) 1 rainklin A Systems Approach to Aeriultural ResearchRcsmrce llcatlon IIII I)c elopirig (Ci)lit es Resource Allocation andrd tit to Xi Namland bltletooatol 6-Aarch ed TM Arndt )Gl)Driiplc avd V XV Rnttani Minneapolis Minnesota University of innesota lres 1)77

Rannalho de Castro P and ( IE Schuh An Empirical Test of An Econometric Mode] for Istablishing Research Priorities A Brazil Case Study Resource llocauoaind Iroductivitv in National aiid [iternationalResearch ed IM ArndtI)G I)alrymple and VX Ruttan Minneapoli Minnesota Lniversity of imncota Press 1977

Russell 1)( Resource Allocation in Aguricultural Research Using Socio-EconomicE valuationoand MathematicidModels Canadian ir Econ 23(1977) 29-52

Ruttan VW 1gricuiiralRwarch Plt Bfaltimore Johns Ilopkins University Press 1982

Salmon D( Congruence of Agricultural Re tearch in Indonesia 1974-78 EconomicI)evclopment Center Bulletin 83-1 University of Minnesota St Paul 1983

21

Scobie GM Investment in Agricultural Research Some Economic Principles C immNYT Working Paper Mexico 1984

Shumway CR and 1I McCracken Use of Scoring Models in Ivaluation Research Programs liner Agr Econ 57(1975) 714-18

Von Oppen M an2 1G Ryan Rescarch RCsource Allocation Determining Regiona Priorities fod PoliWv 10 (19S5) 253-2o4

Williamson C lhe Joint l)epartmcnt of Agriculture and State lExperiment Stations Study of Research Needs Resource Allocation in lgricuturalResearch ed WL Fishel Minneapolis Minnesota University of Minnesota Press 1971

22

Appendix I

Simplified Hypothetical Example for Selecting Commodity Priorities

Step a Fill in a table for each quantitative criterion and rank commodities in each table in order ofimportance

Product Foreign Number Value Exchange of Farms Calories Protein

I Coffee 1 Colee 1Plantain 1Rice I Rice 2 Rice 2 Maize 2 Rice 2 Plantain 2 Maize3 Plantan 3 Rice 3 Cassava 3 Cassava 3 Plantain 1Cassava 4 Cassava 4 Coffee 4 Maize 4Cassava 5Maize 5 Plantain 5 Maize 5 C 5 Coffee

Step b Multiply the weights which have been assigned to each criterion (for example30 10 10 5 5) by the number in the ranking fbr each quantitative critcrionand add across to arrive at a ranking for that group of criteria

RankCoffee (I X3)+(l X l4(4 X l)+(5 X 05)--(5 X 05) = 13 2 Rice (2 X 3)+(3 X 1)e(2 X I)+( I X 5)+( I X 05) = 12 1Plantain (3 X 3)+ (5N)-I X 1)+(2 XA5)+(3 X 05) = 175 3Cassava (4 X 3)+(4 X X)(31 )+(3 X 05)+(4 X 05) = 225 4Maize (5X 3)+(2 X 1)+(5 X l)+(4 X 05)+-(2 X 05) = 25 5

Step c Fill in a table f r each qualitative criterion placing each commodity in a highmedium or low category for each criterion In each table give a 2 to those commodities for which the criterion implies the greatest need for moreresearch and a 0 to those commodities for which the criterion implies the least need for research

Yield Success Relation to Current Private Gap Probability Int Ctrs Emphasis Incentive

Coffee Rice Plantain Cassava Maize

2 1 I 0 0

1 2 1 0 2

0 2 0 2 2

I 2 I

0 I

0 1 2 2 1

Comparative Price Home Future Advantage Effect Consumption Demand

Coffee 2 2 0 1 Rice I I I I Plantain Cassava Maize

1 0 1

0 0 2

2 2 1

0 0 2

23

Step d Multiply the weights assigned to each of the qualitative criteria (for example 4 4 4 6 4 8 2 2 6) by the number assigned each commodity (2 1 or 0) for each citerion and add across to arrive at a ranking for the group of qualitative criteria

Coffee (2 X0-)+(U X04)+(O X04) +(1 X 03)+(OX04)+ Rice (1 X04)+(2 X04)+(2 X01)+(2 X 06)1(I X04)+ Plantain (0 X04)+(I X04) (0 X04)+(I X 06)+(2 X04)-+ Cassava (0 X04) +(0 X04) +(2 X04) + (0 X 06)1(2 X04) -Maize (0 X04) 1-(2 X04) 1 (2 X04) + (1 X 06) I (HX 04) +

Rank Coffee (2 X08) + (2 X02) 4 (0 X02) 1-( X 06) =44 3 Rice (0X08)+(0 X02)+( X02)+(H X 06)=54 1 Plantain (1 X08) +(0 X02) f (2 X02) + (0 X06) =34 4 Cassava (0 X08) (0 X02) 1 (2 X02) 1-(0 X 06) =20 5 Maize tI X08)+(2 X02) +(IX02)1 (2 X06)=52 2

Step e Now there are two rankings (one For the group of quantitative criteria and one for the goup of qualitative criteria) Weight these two rankings by the total weights assigned to their respective sets of criteria to arrive at a final ranking

Coffee

Rice

Plantain

Cassava

Maize

Final Rank (2 X6) 1-(3 X 1) 24 2 (I X61+ ( X 4)= I I

(3X6) f( X 4)= 3A 3 (4 X6) 1(5 X 4) =44 5 (5 X6) +(2 X A)=38 4

Step f Divide the final rarking into 3 groups those commodities with highest priority those with ower priority and those with the least priority This is a very subjective step and depends on the resources available for research but has proven to be a useful step

Group I Group 2 Group 3

Rice Plantain Cassava Coflee Maize

24

Steps for Implenienting Procedure for Research Area Selection

Step a Each criterion for research area election is qualitative Fill in a table for eachcriteriou assigning a 2 to those research areas for which the criterion impliesthe greatest need for more research a I to those for which the criterionimplies some need for research and a 0 to those for which the criterion implieslittle or no necd for research These tables should be constructed separatelyfor each region in the country for certain criteria

InternationalResource Problem Center Private CurrentAbundance Impoiltance Relations lIcentive Programs Plant Breeding 2 2 2 0 2Cultural Practice (Lrops) 1 1 1 1 2Plant Protection 1 2 1 0 2Soil Fertility 2 1 1 1Soil Conservation 2 2 1 1 1Water Use Efficiency 1 2 0 1 0Mechanization 0 1 0 0

I I I 1 1 0Socio-econornics

Technology Transfer 2 1 I 1 1Seed Production I I I IPosi-harvest Technology I 1 0 1 1Agro-forestry 2 1 1 2 1Animal Breeding 2 1 c

0 0 1Animal lealth 1 2 0 0 0Animal Nutrition 2 2 1 0 2Aquaculture 1 0 0 1 0

Step h Assign weight (out of a total of 100) to each criterion (for example 35 35I I 1) multiply by the number in Step 1 and sum across to get a singleranking

Plant Breeding (2 ) 35) + (2 X 35) + (2X 1)+ (OX 1)+ (2X 1)= Rank

18 ICultural Practices (crops) (IX 35) + (IX 35) + (IX 1)4- (1 X 1)+ (2X 1)= 11 10Plant Protectioll (IX 35) + (2X 35) + (IX 1)+ (0X1)+ (2X I)= 135 6Soil Fertility (2X 35) + (IX 35) + (1X 1)+ (I X 1)+ (2X l)= 145Soil Conservation (2 X 35) (2 X 35) 4

+ + (1 X 1) + (I X 1) + (I X 1) = 17 2Water Uselfliciency (I X 35) + (2 X 35) + (OX 1) + (IX 1) -- (0X 1) = 115 8Mechanization (0 X 35) + (IX 35) + (0X ) + (0 X 1) + (0 x 1) = 35 16Socio-ecornomics (1 X 35) + (I X 35) + (IX 1)+ (1 X i + (I X 1) = 10 12Tech Transfer (2 X 35) + (IX 35) + (IX i) + (I X ) + (1 X 1) = 135 6Seed Production (1 X 35) (1X 35) + + + (1 X 1)= 12 - (1X ) (1X 10Post-harvest (I X 35) + (IX 35) + (0 X 1) + (IX 1) + (I X 1) = 9 14Agro-forestry (2 X 35) + (1 X 35) + (I X 1) 4- (2X 1) + (I X 1) = 145 4Animal Breeding (2 X 35) (1 X 35) ++ (0X 1) + (0X 1) + (I X 1) = 115 8Animal Health (IX 35) + (2 X 35) + (0X A)+ (OX ) + (0X 1) 105 11Animal Nutrition (2 X 35) + (2 X 35) + (I X I) + (0X1)+ =(2 X) 17 2Aquaculture (I X 35) (0 X 35)+ + (0 X 1)+ (I X 1) + (0 X 1) = 45 15

25

Appendix 2 -- Results

Table Ia Ranking of Research Priorities by Commodity from the Weighted-Criteria Model Applied in the Dominican Republic

Commodity Rank Comrnodi) Rank

47Arroz (cascara) I AguacateCafe (cerezo) 2 Lechuga 48Leche 3 Limon Dulce 49Frijol 4 Zapote 50 Came de Res 5 Toronja 51Maiz (grano) 6 Melon 52Coco de Agua 7 Name 53Tornate 8 Nispero 54Cacao (grano) 9 Molondron 55 Huevos 10 Jagua 56 Naranja Dulce 11 Bercnjena 57Auyama 12 Repo[o 58Carbon Vegetal 13 Lechoza 59 Guandul 14 Ru10 60Batata 15 Pcpino 61Lena Familiar 16 anahoria 62 Cana Azucar 17 Cabuya (Sisal) 63[laba 18 Remolacha 64Platano 19 Arveja 65Came de Ayes 20 Jengibre 66 Naanja Agna 21 Ajonjoli 67Came de Cerdo 22 1liguereta 68 Algodon (iama) 23 Tayota 69Fruto de Palma 24 Mapuey 70Yuca 25 Bija 71Mani (cascara) 26 Cajuil 72 Tabaco (rama) 27 Rabano 73

74Came OvinaCaprin 28 Tamaruido Ajo 29 Guineo 30 YaUtia 31 Limon Agrio 32 Oregano 33 Cera de Abejas 34 Papa 35 Aji 36 Garbanzo 37 Cebolla 38 Miel de Abejas 39 Ceb Uin 40 Guanabana 41 Lena Industrial 42 Mango 43 Pesca 44 Otros Frijoles 45 Pina 46

26

Table lb Ranking of Research Priorities by Commodity from the Weighted-Criteria Mode

Applied in Ecuador

Commodity

Arroz Cafe Maiz Sierra Cebada platanoPapa Baiano Cacao Ganado de Leche laiz Costs

Trigo Fruitas lioja Caduca Frejol Ganado de Carrie Lcguminosas and Mfen Yuca Fruitas Citricos ILenteja Lcgurainosas Costa Cercalcs dinos Porcinos Fruitales Tropicales

Cereales Menores Cultivos wtd Nlenores Main Ovinos Fniitaes Subtropical fIort de Clima Ffio IlorlTropicalAyes Caprinos Chontaduro Soya Palna Africana Sorgo Algodon

Especies Menores Oleaginosas Menores Cana Ifort Subtropical Tabaco [ores Jojoba e

Rank

I 2 3 4 5 6 7 8 9 10

11 12 13 14 15 16 17 18 19 20 21 22

23 24 25 26 27 28 29 30 31 32 33 34 35 36

37 38 39 40 41 42 43 44

Priority Level

High

Medium

Low

None

27

Table Ic Ranking of Research Priorities by Commodity from the Weighted-Criteria Model

Applied in Uruguay (Preliminary Results) Commodity

Beef Cattle Wheat Forages

Dairy Citrus Potatoes Maize SheepRice Other Fruits Grapes Sunlloers Soybeans Vegetables

Sorghum Barley Tomatoes Swine Peanuts Linseed Grain legumes Frutilla

Rank

I 2 3

4 5 6 7 8 9

10 II 12 13 14

15 16 17 18 19 20 21 22

Priority Level

tHligh

Medium

Low

28

Table 2a Ranking Assigned to Research Aeas by the Weighted-Critera Model in the Dominican Republic

Research Area

Natural Resources (sods water forests)

Plant Breeding Cultural Practices Crop Systems

Amral Nutrition and Mxnagement

Iechnology Transfer

Animal genetics and reproduction

Socioeconomics

Plant Protection

Post lwuvest lechnology

Mechmization

Ranking

I

2

3

4

5

6

7

8

9

29

Table 2b Ranking Assigned to Research Areas by the Weighted-Criteria Model in Ecilador

Research Aggregate Ranking by Region Area Ranking I 2 3 4 5 6 789 9 10 11

Cultural Practices (crops) 7 I 8 6 8 9 t10 8 8 2 6

Agro-Forestry 2 2 2 2 1 3 3 4 3 2 7 7

Animal Nutrition 3 3 3 7 11 5 5 2 4 10 2 3

Soil Conservation 4 4 4 3 3 6 12 6 5 4 9 9

Socio-economics 10 5 6 8 4 7 6 15 11 5 3 10

eclmology Transfer 4 6 7 4 5 8 7 7 6 6 4 4

5d duction 4 7 5 5 6 10 8 3 7 7 5 5

Plant Breeding 1 8 1 1 7 1 1 1 1 1 1 1

Soil Fertility 7 9 9 9 9 2 2 11 2 9 6 2

Post-tlarvest technology 13 10 10 10 10 15 10 12 13 14 13 13

Plant Protection 9 11 12 12 2 4 4 5 9 3 8 8

Water Use 11 12 14 14 1 11 13 8 14 12 10 11

Animal lealth 16 13 16 16 11 16 15 16 16 16 16 16

Livestock Breeding amp Management 14 14 13 13 12 13 11 14 0 11 15 15

Aquaculture I1 15 15 15 14 14 14 9 12 13 11 12

Mechanization 15 16 11 11 16 12 16 13 15 15 14 14

Region 1= Region 2 = Region 3 =

Tropical and dry Subtropical Occidentad de Pichincha Fstacion Experimental Santo Domingo

Region 4 = Region 5 =

Subtropico Seco-Loja lrovincia del Ovo

Region 6 = Region 7 =

Elstacion Experimental PortoviejoEstacion Experimental Boliche

Region 8 = E E NAPO PayaminoRegion 9 = E ETropical PichilingueRegion 10 = EESanta Catalina Region 11= EEChuquipata

30

Table 2c Ranking Assigned to Research Areas by the Weighilted-Criteria Model in Uruguay

Research Area

Plant Protection Agroclimatology Plant Breeding Livestock Management Livestock Nutrition Production Systems Pasture Management Soil Management Animal Ilealth Plant Nutrition amp PhysiologyCultural Practices Sociocconomics Animal Reproduction valuation of Products

Animal Improvement IPost-I larest Tech

Region I El Nort Region 2 = la Fstanzuela Region 3 = Salto Rcoon 4 = [ as Brujas Region 5 = Fst

Aggregate Ranking

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

Ranking by Region1 2 3 4 5 7 7 1 10 1

11 9 2 4 5 12 1 S 1 6 2 5 11 13 7 9 4 13 11 8 8 3 10 7 12 4 6 15 14 2

13 2 4 8 3 3 11 16 16 4

14 13 5 9 9 5 8 3 6 11

10 14 7 3 13 1 10 12 12 10

15 16 9 5 16 6 12 14 15 14

16 Is 6 2 15

31

Table 3 Summary of Internal Rates of Return to INIPA Rescarch and Jxterjsio a )

RICELb) CORN WII EAT POTATOES BEANS AGGREGATP)

Research Investment

from 1981 to 1986

and Extension from

1981 to 1990

Free Trade

Pivotal Shift 17 10 18 17

in Stpply Curve

Parallel Shift 35 23 28 33

in Supply Curve

No Trade

Pivotal Shift 18 22 14

in Supply Curve

Prallel Shift 37 42 24

in Supply Curve

Research Investment

from 191 to 1992

and Extension from

LIU_Q 199

FreeTrade

Pivotal Shift 30 20 28 25

in Supply Curve

Parallel Shift 44 31 36 38

in Supply Curve

No Trade

Pivotal Shift 22 14

in Supply Curve

Parall Shift 42 24

IIIsupply Curve

(asrumes no expansion of cultivated area

(1vihen ropansion in cultivated area of 1deg1per year was assumed these rates more than doubled For example the return to research and extension on rice for the 1981 to 1986 research investment and 1981 to 19W) extension investment

changed from 17 to 48

(ceither Free Trade nor No Tra

32

Tble 4 Percent Distribution of Total Net Economic Surplus to INIPA Research and Extension for

No Trade ScenarioJO

Pivotal Supply Shift Parallel Supply Shift

Consumer Producer Consumer Producer

Gain Gain Gain Gain

Rice n = - 76 83 17

n = -39 105 -5 52 48

n = -27 115 -15

Potatoes n = -64 72 28

n = -34 88 12 44 56

n = -24 95 5

Beans n = - 61 74 26

n = -31 (1 9 46 54

n = -21 99 1

KlUbenefits accrue to producers for the trade scenario for rice corn and wheat when the price elasticity

of demand is perfectly elastic

33

Foot notes

1) See Ruttan for a discussion of several of these studies 2) Rules of thumb aresometimes used such as the congruence approach where Funds

are allocated across research programs in proportion to the value of agriculturalproduction in each commodity See 5jalmon or a discussion and apptcation of this approach

3) Changes often result as ivll frora the introduction of forcign-isistad research projects

4) l takes time Ir agricultural research to yield new results Ior cx miple Pardeyreports art average research gestation lag for the 1S state agricultUrA cI xperhnentstations durig the early 1970s of around three to four years

5) Additionl discussionl of these methods cau be 1bund in Shuoswav Rtltian- hiuhand Tollini Parton ct al Anderson and Parton Scobie and Norton and I)

6) See lia vlicek and Norton for a sommirv of several of these procedures (ai tio) isneeded when ipplyg quantitaivC procedures to research project selection becausescientists are often the best judge of tue appropriat projects within a given research area

7) Tile Dominicani Republic was in the process ofrestructuring its agricultural researchsystem at the tieni of the priority-setting study and the newly created aency(Il)FA ) was VJtihIO(t a (drector

S) The ned for discussion and sensitivity analysis k evident in the list of research priorities by commodity shown in fable la ofppendix 2 iat list includes somesurprises compared withI a ranking based on valut of productioni

9) [he expected economic surplus approach as described by Davis ranl and Rvanis particularly oseful for international agenicie because of its emphasis on spilloverstcroSs cCnitrics

10) A l and ISNRaP ire currently refining the Dasis Ormn and Ryan expectedC ooin c su pluS procedure to incorporate regional spillovers within countries II ( ~leart the Ceononiic-teCiical dichotomy preseoted here is principally forexpository purp( sec lechni-al ariables such as the expected rate of adoptionand spillovr clects are responsivc to both technical and economic forcesNeveitheless the dichotom is osetul whei1 thinking about the principal sources ofdata Manv of the ecnnoiic Ntriables can be obtained from secondary sources

while sone of the teclii-ical aintbles relire direct inrut from scientirts and program leaders In the ibsine of hiard data the sensitivity of the results can betested against a reasonable rane of parameter estimates

19

References

Alston JMG V Edwards and J W Freebairn Market Distortions and Benefitsfrom Research Department of Agriculture and Rural Affairs in Victoria and Lairobe University March 19 S7 Minimeo

Anderson J R and K A Parton Technologies for Guiding the Allocation ofResources A1mong Rural Research State of the ArtProjects Iraethus 1(1983)180-20l

Araji A R J Sim and R 1 Gardner Returns to Agricultural Research andExtcnsion lPro era m An lx Anrte pproach r gr lcon 60(1978) 96-4-8

Binswanger 11P and I6 Ryan Flficiency and lIqtuitv IssueS in Ex-Ante lljcaionof Research Resources Idlia 1 o L32(1977) 217-31

Centro de hivestieacio 1 Agricola Alherto lioerger (CIAI) Identificacion ySeleccion )e Prioridades de luvestig cion eropccnaria en elUruguay Reportprepared lor CIAB by (9 lcrrcira G Norton and C Valverde May 1987

Consultative Group on International giculmral Research (CGIAR) TechicalAdvisory Committee 1) C Rview f (C IAR Priorities ad Ftm SrtiiesTAC Secretariat Food aiod Agriculture Oigonization of the United Nations ugust [985

Dais JS PA ()ram tiid J( Ryan Assessing ofregate Agricultural research Internatonal AustralianPrioitics n Perspective Centre for International

ericnltriral Pcarcli (ACIl() and the liternttional Food Policy Research Iustitute (11 [R I)D1711 Alrfgust I9Mo

Edwards G V W ehairo 1laspuill a Cnlrs Gains Frt ResearchlhJeopv i10 licutin t(Rural kcsea -11 llraia A Report to the I onuonweIti Cou cil and iExtension Australianfor Ruri] Rescarci Canberra Goerntnult l vhing crvice 1981

Edwards (j W aid JV icebairn The Gains From Research into Tradeable Uoiataoditics Amir1 er Lcon 66(1 4) 41-419

Espinosa P G Norton and llD Gross identificacion Y Seleccion De Prioridades De lnvestigacion Agropecnaria en el Ecuador iwntituto Nacional de InvestigacionAgropecuaria (INIP ))raft ()ctober 19S6

Fishel XVI The Minnesota Agricultura IResca rechReource Allocation System and Experiment Resource 11lwain in lgricukral esearch ed XVI Fishel Minneapolis Minnwesota (niversitY of Minnesota Pres 19)71

Ilavlicek 1G aild i V Norton Review of Research Project Ivaluation andSelection Methods A Report to ARDl1 Stai Paper SP-13-SI l)epartment of gricultural icoonoics Virginia Polytechnic initute and State University November 1981

Instituto Superior de gricultura (ISA) Contenido lecnico de la Investigacion delInstituto l)omnicnno de Investigaciones gropecuarias Report prepared for

29

ISNAR in the Dominicin Republic by W Moscoso A Coutu I) Bandy and G W Norton June 1986

Longmire 1and 1) Vinkelmann Research Resource Allocation and ComparativeAdvantace CIMI YT paner presented at the XIX (ongre s of InternationalAgricultural Economics Association MIalaga Spain August 1985

Mahlstede J P Iong-Rangqe Planning at tie Iowa Agricnltural and IIonie EconomicsExperimental Station R surce Allocatioin I tritdrtur l Researcih cdIishel Minneapolis Minnesota rnversitv of Minnesota Press 1971

WL

Norton C and IS l)avis Val ittin cturns to Agicultural Research ARe ew for Olr lc(t 6 1)( -609Q o3t 1S

Norton GW aid V(G (Gioalhe lienclit s of A cricultural Research and Extensionin Peru paper proired for INI PA and b USI D Lima Peru June 1985

Norton (X and V( (Ganoa (4uideliTeCs fr Allocations of ResourcesAgricultural Researchland Extension

to in fcru paper prepared Ior INIPA and for

USAID Lima Peru 1chrue rv 1980

Nortoi GV (4 (ianoa and C Pmiarel IPotential Benefits For giculturalResearch and Extension it Peru i1hgr(ti( 1987) 247-257E-ct

lcu dLILti Paper presented to tle Ntion if S11npositim ott I valuatiag Agricultural Research Productivitl

Pardev P(G [Ie riculturUl Know i I [oIIIILction An Impirical Look

tlantai(e rgi Iinn 7lt V I9

Parton K IRAndcreon irid I N 1htm lantion Mtods1fr Iustralialltod Rsr1 XL iCult Ra IlI lCononlics Bulletin No 29 Armidale IiiscrsitV of New laid I1

Paulsen A ind I) R Kildor Tluation and Planning of Research in the ExperimentStation tr Jl 1olo 51t(l9oS 11 1902

flinstr up-- et seti P aid I) 1 rainklin A Systems Approach to Aeriultural ResearchRcsmrce llcatlon IIII I)c elopirig (Ci)lit es Resource Allocation andrd tit to Xi Namland bltletooatol 6-Aarch ed TM Arndt )Gl)Driiplc avd V XV Rnttani Minneapolis Minnesota University of innesota lres 1)77

Rannalho de Castro P and ( IE Schuh An Empirical Test of An Econometric Mode] for Istablishing Research Priorities A Brazil Case Study Resource llocauoaind Iroductivitv in National aiid [iternationalResearch ed IM ArndtI)G I)alrymple and VX Ruttan Minneapoli Minnesota Lniversity of imncota Press 1977

Russell 1)( Resource Allocation in Aguricultural Research Using Socio-EconomicE valuationoand MathematicidModels Canadian ir Econ 23(1977) 29-52

Ruttan VW 1gricuiiralRwarch Plt Bfaltimore Johns Ilopkins University Press 1982

Salmon D( Congruence of Agricultural Re tearch in Indonesia 1974-78 EconomicI)evclopment Center Bulletin 83-1 University of Minnesota St Paul 1983

21

Scobie GM Investment in Agricultural Research Some Economic Principles C immNYT Working Paper Mexico 1984

Shumway CR and 1I McCracken Use of Scoring Models in Ivaluation Research Programs liner Agr Econ 57(1975) 714-18

Von Oppen M an2 1G Ryan Rescarch RCsource Allocation Determining Regiona Priorities fod PoliWv 10 (19S5) 253-2o4

Williamson C lhe Joint l)epartmcnt of Agriculture and State lExperiment Stations Study of Research Needs Resource Allocation in lgricuturalResearch ed WL Fishel Minneapolis Minnesota University of Minnesota Press 1971

22

Appendix I

Simplified Hypothetical Example for Selecting Commodity Priorities

Step a Fill in a table for each quantitative criterion and rank commodities in each table in order ofimportance

Product Foreign Number Value Exchange of Farms Calories Protein

I Coffee 1 Colee 1Plantain 1Rice I Rice 2 Rice 2 Maize 2 Rice 2 Plantain 2 Maize3 Plantan 3 Rice 3 Cassava 3 Cassava 3 Plantain 1Cassava 4 Cassava 4 Coffee 4 Maize 4Cassava 5Maize 5 Plantain 5 Maize 5 C 5 Coffee

Step b Multiply the weights which have been assigned to each criterion (for example30 10 10 5 5) by the number in the ranking fbr each quantitative critcrionand add across to arrive at a ranking for that group of criteria

RankCoffee (I X3)+(l X l4(4 X l)+(5 X 05)--(5 X 05) = 13 2 Rice (2 X 3)+(3 X 1)e(2 X I)+( I X 5)+( I X 05) = 12 1Plantain (3 X 3)+ (5N)-I X 1)+(2 XA5)+(3 X 05) = 175 3Cassava (4 X 3)+(4 X X)(31 )+(3 X 05)+(4 X 05) = 225 4Maize (5X 3)+(2 X 1)+(5 X l)+(4 X 05)+-(2 X 05) = 25 5

Step c Fill in a table f r each qualitative criterion placing each commodity in a highmedium or low category for each criterion In each table give a 2 to those commodities for which the criterion implies the greatest need for moreresearch and a 0 to those commodities for which the criterion implies the least need for research

Yield Success Relation to Current Private Gap Probability Int Ctrs Emphasis Incentive

Coffee Rice Plantain Cassava Maize

2 1 I 0 0

1 2 1 0 2

0 2 0 2 2

I 2 I

0 I

0 1 2 2 1

Comparative Price Home Future Advantage Effect Consumption Demand

Coffee 2 2 0 1 Rice I I I I Plantain Cassava Maize

1 0 1

0 0 2

2 2 1

0 0 2

23

Step d Multiply the weights assigned to each of the qualitative criteria (for example 4 4 4 6 4 8 2 2 6) by the number assigned each commodity (2 1 or 0) for each citerion and add across to arrive at a ranking for the group of qualitative criteria

Coffee (2 X0-)+(U X04)+(O X04) +(1 X 03)+(OX04)+ Rice (1 X04)+(2 X04)+(2 X01)+(2 X 06)1(I X04)+ Plantain (0 X04)+(I X04) (0 X04)+(I X 06)+(2 X04)-+ Cassava (0 X04) +(0 X04) +(2 X04) + (0 X 06)1(2 X04) -Maize (0 X04) 1-(2 X04) 1 (2 X04) + (1 X 06) I (HX 04) +

Rank Coffee (2 X08) + (2 X02) 4 (0 X02) 1-( X 06) =44 3 Rice (0X08)+(0 X02)+( X02)+(H X 06)=54 1 Plantain (1 X08) +(0 X02) f (2 X02) + (0 X06) =34 4 Cassava (0 X08) (0 X02) 1 (2 X02) 1-(0 X 06) =20 5 Maize tI X08)+(2 X02) +(IX02)1 (2 X06)=52 2

Step e Now there are two rankings (one For the group of quantitative criteria and one for the goup of qualitative criteria) Weight these two rankings by the total weights assigned to their respective sets of criteria to arrive at a final ranking

Coffee

Rice

Plantain

Cassava

Maize

Final Rank (2 X6) 1-(3 X 1) 24 2 (I X61+ ( X 4)= I I

(3X6) f( X 4)= 3A 3 (4 X6) 1(5 X 4) =44 5 (5 X6) +(2 X A)=38 4

Step f Divide the final rarking into 3 groups those commodities with highest priority those with ower priority and those with the least priority This is a very subjective step and depends on the resources available for research but has proven to be a useful step

Group I Group 2 Group 3

Rice Plantain Cassava Coflee Maize

24

Steps for Implenienting Procedure for Research Area Selection

Step a Each criterion for research area election is qualitative Fill in a table for eachcriteriou assigning a 2 to those research areas for which the criterion impliesthe greatest need for more research a I to those for which the criterionimplies some need for research and a 0 to those for which the criterion implieslittle or no necd for research These tables should be constructed separatelyfor each region in the country for certain criteria

InternationalResource Problem Center Private CurrentAbundance Impoiltance Relations lIcentive Programs Plant Breeding 2 2 2 0 2Cultural Practice (Lrops) 1 1 1 1 2Plant Protection 1 2 1 0 2Soil Fertility 2 1 1 1Soil Conservation 2 2 1 1 1Water Use Efficiency 1 2 0 1 0Mechanization 0 1 0 0

I I I 1 1 0Socio-econornics

Technology Transfer 2 1 I 1 1Seed Production I I I IPosi-harvest Technology I 1 0 1 1Agro-forestry 2 1 1 2 1Animal Breeding 2 1 c

0 0 1Animal lealth 1 2 0 0 0Animal Nutrition 2 2 1 0 2Aquaculture 1 0 0 1 0

Step h Assign weight (out of a total of 100) to each criterion (for example 35 35I I 1) multiply by the number in Step 1 and sum across to get a singleranking

Plant Breeding (2 ) 35) + (2 X 35) + (2X 1)+ (OX 1)+ (2X 1)= Rank

18 ICultural Practices (crops) (IX 35) + (IX 35) + (IX 1)4- (1 X 1)+ (2X 1)= 11 10Plant Protectioll (IX 35) + (2X 35) + (IX 1)+ (0X1)+ (2X I)= 135 6Soil Fertility (2X 35) + (IX 35) + (1X 1)+ (I X 1)+ (2X l)= 145Soil Conservation (2 X 35) (2 X 35) 4

+ + (1 X 1) + (I X 1) + (I X 1) = 17 2Water Uselfliciency (I X 35) + (2 X 35) + (OX 1) + (IX 1) -- (0X 1) = 115 8Mechanization (0 X 35) + (IX 35) + (0X ) + (0 X 1) + (0 x 1) = 35 16Socio-ecornomics (1 X 35) + (I X 35) + (IX 1)+ (1 X i + (I X 1) = 10 12Tech Transfer (2 X 35) + (IX 35) + (IX i) + (I X ) + (1 X 1) = 135 6Seed Production (1 X 35) (1X 35) + + + (1 X 1)= 12 - (1X ) (1X 10Post-harvest (I X 35) + (IX 35) + (0 X 1) + (IX 1) + (I X 1) = 9 14Agro-forestry (2 X 35) + (1 X 35) + (I X 1) 4- (2X 1) + (I X 1) = 145 4Animal Breeding (2 X 35) (1 X 35) ++ (0X 1) + (0X 1) + (I X 1) = 115 8Animal Health (IX 35) + (2 X 35) + (0X A)+ (OX ) + (0X 1) 105 11Animal Nutrition (2 X 35) + (2 X 35) + (I X I) + (0X1)+ =(2 X) 17 2Aquaculture (I X 35) (0 X 35)+ + (0 X 1)+ (I X 1) + (0 X 1) = 45 15

25

Appendix 2 -- Results

Table Ia Ranking of Research Priorities by Commodity from the Weighted-Criteria Model Applied in the Dominican Republic

Commodity Rank Comrnodi) Rank

47Arroz (cascara) I AguacateCafe (cerezo) 2 Lechuga 48Leche 3 Limon Dulce 49Frijol 4 Zapote 50 Came de Res 5 Toronja 51Maiz (grano) 6 Melon 52Coco de Agua 7 Name 53Tornate 8 Nispero 54Cacao (grano) 9 Molondron 55 Huevos 10 Jagua 56 Naranja Dulce 11 Bercnjena 57Auyama 12 Repo[o 58Carbon Vegetal 13 Lechoza 59 Guandul 14 Ru10 60Batata 15 Pcpino 61Lena Familiar 16 anahoria 62 Cana Azucar 17 Cabuya (Sisal) 63[laba 18 Remolacha 64Platano 19 Arveja 65Came de Ayes 20 Jengibre 66 Naanja Agna 21 Ajonjoli 67Came de Cerdo 22 1liguereta 68 Algodon (iama) 23 Tayota 69Fruto de Palma 24 Mapuey 70Yuca 25 Bija 71Mani (cascara) 26 Cajuil 72 Tabaco (rama) 27 Rabano 73

74Came OvinaCaprin 28 Tamaruido Ajo 29 Guineo 30 YaUtia 31 Limon Agrio 32 Oregano 33 Cera de Abejas 34 Papa 35 Aji 36 Garbanzo 37 Cebolla 38 Miel de Abejas 39 Ceb Uin 40 Guanabana 41 Lena Industrial 42 Mango 43 Pesca 44 Otros Frijoles 45 Pina 46

26

Table lb Ranking of Research Priorities by Commodity from the Weighted-Criteria Mode

Applied in Ecuador

Commodity

Arroz Cafe Maiz Sierra Cebada platanoPapa Baiano Cacao Ganado de Leche laiz Costs

Trigo Fruitas lioja Caduca Frejol Ganado de Carrie Lcguminosas and Mfen Yuca Fruitas Citricos ILenteja Lcgurainosas Costa Cercalcs dinos Porcinos Fruitales Tropicales

Cereales Menores Cultivos wtd Nlenores Main Ovinos Fniitaes Subtropical fIort de Clima Ffio IlorlTropicalAyes Caprinos Chontaduro Soya Palna Africana Sorgo Algodon

Especies Menores Oleaginosas Menores Cana Ifort Subtropical Tabaco [ores Jojoba e

Rank

I 2 3 4 5 6 7 8 9 10

11 12 13 14 15 16 17 18 19 20 21 22

23 24 25 26 27 28 29 30 31 32 33 34 35 36

37 38 39 40 41 42 43 44

Priority Level

High

Medium

Low

None

27

Table Ic Ranking of Research Priorities by Commodity from the Weighted-Criteria Model

Applied in Uruguay (Preliminary Results) Commodity

Beef Cattle Wheat Forages

Dairy Citrus Potatoes Maize SheepRice Other Fruits Grapes Sunlloers Soybeans Vegetables

Sorghum Barley Tomatoes Swine Peanuts Linseed Grain legumes Frutilla

Rank

I 2 3

4 5 6 7 8 9

10 II 12 13 14

15 16 17 18 19 20 21 22

Priority Level

tHligh

Medium

Low

28

Table 2a Ranking Assigned to Research Aeas by the Weighted-Critera Model in the Dominican Republic

Research Area

Natural Resources (sods water forests)

Plant Breeding Cultural Practices Crop Systems

Amral Nutrition and Mxnagement

Iechnology Transfer

Animal genetics and reproduction

Socioeconomics

Plant Protection

Post lwuvest lechnology

Mechmization

Ranking

I

2

3

4

5

6

7

8

9

29

Table 2b Ranking Assigned to Research Areas by the Weighted-Criteria Model in Ecilador

Research Aggregate Ranking by Region Area Ranking I 2 3 4 5 6 789 9 10 11

Cultural Practices (crops) 7 I 8 6 8 9 t10 8 8 2 6

Agro-Forestry 2 2 2 2 1 3 3 4 3 2 7 7

Animal Nutrition 3 3 3 7 11 5 5 2 4 10 2 3

Soil Conservation 4 4 4 3 3 6 12 6 5 4 9 9

Socio-economics 10 5 6 8 4 7 6 15 11 5 3 10

eclmology Transfer 4 6 7 4 5 8 7 7 6 6 4 4

5d duction 4 7 5 5 6 10 8 3 7 7 5 5

Plant Breeding 1 8 1 1 7 1 1 1 1 1 1 1

Soil Fertility 7 9 9 9 9 2 2 11 2 9 6 2

Post-tlarvest technology 13 10 10 10 10 15 10 12 13 14 13 13

Plant Protection 9 11 12 12 2 4 4 5 9 3 8 8

Water Use 11 12 14 14 1 11 13 8 14 12 10 11

Animal lealth 16 13 16 16 11 16 15 16 16 16 16 16

Livestock Breeding amp Management 14 14 13 13 12 13 11 14 0 11 15 15

Aquaculture I1 15 15 15 14 14 14 9 12 13 11 12

Mechanization 15 16 11 11 16 12 16 13 15 15 14 14

Region 1= Region 2 = Region 3 =

Tropical and dry Subtropical Occidentad de Pichincha Fstacion Experimental Santo Domingo

Region 4 = Region 5 =

Subtropico Seco-Loja lrovincia del Ovo

Region 6 = Region 7 =

Elstacion Experimental PortoviejoEstacion Experimental Boliche

Region 8 = E E NAPO PayaminoRegion 9 = E ETropical PichilingueRegion 10 = EESanta Catalina Region 11= EEChuquipata

30

Table 2c Ranking Assigned to Research Areas by the Weighilted-Criteria Model in Uruguay

Research Area

Plant Protection Agroclimatology Plant Breeding Livestock Management Livestock Nutrition Production Systems Pasture Management Soil Management Animal Ilealth Plant Nutrition amp PhysiologyCultural Practices Sociocconomics Animal Reproduction valuation of Products

Animal Improvement IPost-I larest Tech

Region I El Nort Region 2 = la Fstanzuela Region 3 = Salto Rcoon 4 = [ as Brujas Region 5 = Fst

Aggregate Ranking

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

Ranking by Region1 2 3 4 5 7 7 1 10 1

11 9 2 4 5 12 1 S 1 6 2 5 11 13 7 9 4 13 11 8 8 3 10 7 12 4 6 15 14 2

13 2 4 8 3 3 11 16 16 4

14 13 5 9 9 5 8 3 6 11

10 14 7 3 13 1 10 12 12 10

15 16 9 5 16 6 12 14 15 14

16 Is 6 2 15

31

Table 3 Summary of Internal Rates of Return to INIPA Rescarch and Jxterjsio a )

RICELb) CORN WII EAT POTATOES BEANS AGGREGATP)

Research Investment

from 1981 to 1986

and Extension from

1981 to 1990

Free Trade

Pivotal Shift 17 10 18 17

in Stpply Curve

Parallel Shift 35 23 28 33

in Supply Curve

No Trade

Pivotal Shift 18 22 14

in Supply Curve

Prallel Shift 37 42 24

in Supply Curve

Research Investment

from 191 to 1992

and Extension from

LIU_Q 199

FreeTrade

Pivotal Shift 30 20 28 25

in Supply Curve

Parallel Shift 44 31 36 38

in Supply Curve

No Trade

Pivotal Shift 22 14

in Supply Curve

Parall Shift 42 24

IIIsupply Curve

(asrumes no expansion of cultivated area

(1vihen ropansion in cultivated area of 1deg1per year was assumed these rates more than doubled For example the return to research and extension on rice for the 1981 to 1986 research investment and 1981 to 19W) extension investment

changed from 17 to 48

(ceither Free Trade nor No Tra

32

Tble 4 Percent Distribution of Total Net Economic Surplus to INIPA Research and Extension for

No Trade ScenarioJO

Pivotal Supply Shift Parallel Supply Shift

Consumer Producer Consumer Producer

Gain Gain Gain Gain

Rice n = - 76 83 17

n = -39 105 -5 52 48

n = -27 115 -15

Potatoes n = -64 72 28

n = -34 88 12 44 56

n = -24 95 5

Beans n = - 61 74 26

n = -31 (1 9 46 54

n = -21 99 1

KlUbenefits accrue to producers for the trade scenario for rice corn and wheat when the price elasticity

of demand is perfectly elastic

33

References

Alston JMG V Edwards and J W Freebairn Market Distortions and Benefitsfrom Research Department of Agriculture and Rural Affairs in Victoria and Lairobe University March 19 S7 Minimeo

Anderson J R and K A Parton Technologies for Guiding the Allocation ofResources A1mong Rural Research State of the ArtProjects Iraethus 1(1983)180-20l

Araji A R J Sim and R 1 Gardner Returns to Agricultural Research andExtcnsion lPro era m An lx Anrte pproach r gr lcon 60(1978) 96-4-8

Binswanger 11P and I6 Ryan Flficiency and lIqtuitv IssueS in Ex-Ante lljcaionof Research Resources Idlia 1 o L32(1977) 217-31

Centro de hivestieacio 1 Agricola Alherto lioerger (CIAI) Identificacion ySeleccion )e Prioridades de luvestig cion eropccnaria en elUruguay Reportprepared lor CIAB by (9 lcrrcira G Norton and C Valverde May 1987

Consultative Group on International giculmral Research (CGIAR) TechicalAdvisory Committee 1) C Rview f (C IAR Priorities ad Ftm SrtiiesTAC Secretariat Food aiod Agriculture Oigonization of the United Nations ugust [985

Dais JS PA ()ram tiid J( Ryan Assessing ofregate Agricultural research Internatonal AustralianPrioitics n Perspective Centre for International

ericnltriral Pcarcli (ACIl() and the liternttional Food Policy Research Iustitute (11 [R I)D1711 Alrfgust I9Mo

Edwards G V W ehairo 1laspuill a Cnlrs Gains Frt ResearchlhJeopv i10 licutin t(Rural kcsea -11 llraia A Report to the I onuonweIti Cou cil and iExtension Australianfor Ruri] Rescarci Canberra Goerntnult l vhing crvice 1981

Edwards (j W aid JV icebairn The Gains From Research into Tradeable Uoiataoditics Amir1 er Lcon 66(1 4) 41-419

Espinosa P G Norton and llD Gross identificacion Y Seleccion De Prioridades De lnvestigacion Agropecnaria en el Ecuador iwntituto Nacional de InvestigacionAgropecuaria (INIP ))raft ()ctober 19S6

Fishel XVI The Minnesota Agricultura IResca rechReource Allocation System and Experiment Resource 11lwain in lgricukral esearch ed XVI Fishel Minneapolis Minnwesota (niversitY of Minnesota Pres 19)71

Ilavlicek 1G aild i V Norton Review of Research Project Ivaluation andSelection Methods A Report to ARDl1 Stai Paper SP-13-SI l)epartment of gricultural icoonoics Virginia Polytechnic initute and State University November 1981

Instituto Superior de gricultura (ISA) Contenido lecnico de la Investigacion delInstituto l)omnicnno de Investigaciones gropecuarias Report prepared for

29

ISNAR in the Dominicin Republic by W Moscoso A Coutu I) Bandy and G W Norton June 1986

Longmire 1and 1) Vinkelmann Research Resource Allocation and ComparativeAdvantace CIMI YT paner presented at the XIX (ongre s of InternationalAgricultural Economics Association MIalaga Spain August 1985

Mahlstede J P Iong-Rangqe Planning at tie Iowa Agricnltural and IIonie EconomicsExperimental Station R surce Allocatioin I tritdrtur l Researcih cdIishel Minneapolis Minnesota rnversitv of Minnesota Press 1971

WL

Norton C and IS l)avis Val ittin cturns to Agicultural Research ARe ew for Olr lc(t 6 1)( -609Q o3t 1S

Norton GW aid V(G (Gioalhe lienclit s of A cricultural Research and Extensionin Peru paper proired for INI PA and b USI D Lima Peru June 1985

Norton (X and V( (Ganoa (4uideliTeCs fr Allocations of ResourcesAgricultural Researchland Extension

to in fcru paper prepared Ior INIPA and for

USAID Lima Peru 1chrue rv 1980

Nortoi GV (4 (ianoa and C Pmiarel IPotential Benefits For giculturalResearch and Extension it Peru i1hgr(ti( 1987) 247-257E-ct

lcu dLILti Paper presented to tle Ntion if S11npositim ott I valuatiag Agricultural Research Productivitl

Pardev P(G [Ie riculturUl Know i I [oIIIILction An Impirical Look

tlantai(e rgi Iinn 7lt V I9

Parton K IRAndcreon irid I N 1htm lantion Mtods1fr Iustralialltod Rsr1 XL iCult Ra IlI lCononlics Bulletin No 29 Armidale IiiscrsitV of New laid I1

Paulsen A ind I) R Kildor Tluation and Planning of Research in the ExperimentStation tr Jl 1olo 51t(l9oS 11 1902

flinstr up-- et seti P aid I) 1 rainklin A Systems Approach to Aeriultural ResearchRcsmrce llcatlon IIII I)c elopirig (Ci)lit es Resource Allocation andrd tit to Xi Namland bltletooatol 6-Aarch ed TM Arndt )Gl)Driiplc avd V XV Rnttani Minneapolis Minnesota University of innesota lres 1)77

Rannalho de Castro P and ( IE Schuh An Empirical Test of An Econometric Mode] for Istablishing Research Priorities A Brazil Case Study Resource llocauoaind Iroductivitv in National aiid [iternationalResearch ed IM ArndtI)G I)alrymple and VX Ruttan Minneapoli Minnesota Lniversity of imncota Press 1977

Russell 1)( Resource Allocation in Aguricultural Research Using Socio-EconomicE valuationoand MathematicidModels Canadian ir Econ 23(1977) 29-52

Ruttan VW 1gricuiiralRwarch Plt Bfaltimore Johns Ilopkins University Press 1982

Salmon D( Congruence of Agricultural Re tearch in Indonesia 1974-78 EconomicI)evclopment Center Bulletin 83-1 University of Minnesota St Paul 1983

21

Scobie GM Investment in Agricultural Research Some Economic Principles C immNYT Working Paper Mexico 1984

Shumway CR and 1I McCracken Use of Scoring Models in Ivaluation Research Programs liner Agr Econ 57(1975) 714-18

Von Oppen M an2 1G Ryan Rescarch RCsource Allocation Determining Regiona Priorities fod PoliWv 10 (19S5) 253-2o4

Williamson C lhe Joint l)epartmcnt of Agriculture and State lExperiment Stations Study of Research Needs Resource Allocation in lgricuturalResearch ed WL Fishel Minneapolis Minnesota University of Minnesota Press 1971

22

Appendix I

Simplified Hypothetical Example for Selecting Commodity Priorities

Step a Fill in a table for each quantitative criterion and rank commodities in each table in order ofimportance

Product Foreign Number Value Exchange of Farms Calories Protein

I Coffee 1 Colee 1Plantain 1Rice I Rice 2 Rice 2 Maize 2 Rice 2 Plantain 2 Maize3 Plantan 3 Rice 3 Cassava 3 Cassava 3 Plantain 1Cassava 4 Cassava 4 Coffee 4 Maize 4Cassava 5Maize 5 Plantain 5 Maize 5 C 5 Coffee

Step b Multiply the weights which have been assigned to each criterion (for example30 10 10 5 5) by the number in the ranking fbr each quantitative critcrionand add across to arrive at a ranking for that group of criteria

RankCoffee (I X3)+(l X l4(4 X l)+(5 X 05)--(5 X 05) = 13 2 Rice (2 X 3)+(3 X 1)e(2 X I)+( I X 5)+( I X 05) = 12 1Plantain (3 X 3)+ (5N)-I X 1)+(2 XA5)+(3 X 05) = 175 3Cassava (4 X 3)+(4 X X)(31 )+(3 X 05)+(4 X 05) = 225 4Maize (5X 3)+(2 X 1)+(5 X l)+(4 X 05)+-(2 X 05) = 25 5

Step c Fill in a table f r each qualitative criterion placing each commodity in a highmedium or low category for each criterion In each table give a 2 to those commodities for which the criterion implies the greatest need for moreresearch and a 0 to those commodities for which the criterion implies the least need for research

Yield Success Relation to Current Private Gap Probability Int Ctrs Emphasis Incentive

Coffee Rice Plantain Cassava Maize

2 1 I 0 0

1 2 1 0 2

0 2 0 2 2

I 2 I

0 I

0 1 2 2 1

Comparative Price Home Future Advantage Effect Consumption Demand

Coffee 2 2 0 1 Rice I I I I Plantain Cassava Maize

1 0 1

0 0 2

2 2 1

0 0 2

23

Step d Multiply the weights assigned to each of the qualitative criteria (for example 4 4 4 6 4 8 2 2 6) by the number assigned each commodity (2 1 or 0) for each citerion and add across to arrive at a ranking for the group of qualitative criteria

Coffee (2 X0-)+(U X04)+(O X04) +(1 X 03)+(OX04)+ Rice (1 X04)+(2 X04)+(2 X01)+(2 X 06)1(I X04)+ Plantain (0 X04)+(I X04) (0 X04)+(I X 06)+(2 X04)-+ Cassava (0 X04) +(0 X04) +(2 X04) + (0 X 06)1(2 X04) -Maize (0 X04) 1-(2 X04) 1 (2 X04) + (1 X 06) I (HX 04) +

Rank Coffee (2 X08) + (2 X02) 4 (0 X02) 1-( X 06) =44 3 Rice (0X08)+(0 X02)+( X02)+(H X 06)=54 1 Plantain (1 X08) +(0 X02) f (2 X02) + (0 X06) =34 4 Cassava (0 X08) (0 X02) 1 (2 X02) 1-(0 X 06) =20 5 Maize tI X08)+(2 X02) +(IX02)1 (2 X06)=52 2

Step e Now there are two rankings (one For the group of quantitative criteria and one for the goup of qualitative criteria) Weight these two rankings by the total weights assigned to their respective sets of criteria to arrive at a final ranking

Coffee

Rice

Plantain

Cassava

Maize

Final Rank (2 X6) 1-(3 X 1) 24 2 (I X61+ ( X 4)= I I

(3X6) f( X 4)= 3A 3 (4 X6) 1(5 X 4) =44 5 (5 X6) +(2 X A)=38 4

Step f Divide the final rarking into 3 groups those commodities with highest priority those with ower priority and those with the least priority This is a very subjective step and depends on the resources available for research but has proven to be a useful step

Group I Group 2 Group 3

Rice Plantain Cassava Coflee Maize

24

Steps for Implenienting Procedure for Research Area Selection

Step a Each criterion for research area election is qualitative Fill in a table for eachcriteriou assigning a 2 to those research areas for which the criterion impliesthe greatest need for more research a I to those for which the criterionimplies some need for research and a 0 to those for which the criterion implieslittle or no necd for research These tables should be constructed separatelyfor each region in the country for certain criteria

InternationalResource Problem Center Private CurrentAbundance Impoiltance Relations lIcentive Programs Plant Breeding 2 2 2 0 2Cultural Practice (Lrops) 1 1 1 1 2Plant Protection 1 2 1 0 2Soil Fertility 2 1 1 1Soil Conservation 2 2 1 1 1Water Use Efficiency 1 2 0 1 0Mechanization 0 1 0 0

I I I 1 1 0Socio-econornics

Technology Transfer 2 1 I 1 1Seed Production I I I IPosi-harvest Technology I 1 0 1 1Agro-forestry 2 1 1 2 1Animal Breeding 2 1 c

0 0 1Animal lealth 1 2 0 0 0Animal Nutrition 2 2 1 0 2Aquaculture 1 0 0 1 0

Step h Assign weight (out of a total of 100) to each criterion (for example 35 35I I 1) multiply by the number in Step 1 and sum across to get a singleranking

Plant Breeding (2 ) 35) + (2 X 35) + (2X 1)+ (OX 1)+ (2X 1)= Rank

18 ICultural Practices (crops) (IX 35) + (IX 35) + (IX 1)4- (1 X 1)+ (2X 1)= 11 10Plant Protectioll (IX 35) + (2X 35) + (IX 1)+ (0X1)+ (2X I)= 135 6Soil Fertility (2X 35) + (IX 35) + (1X 1)+ (I X 1)+ (2X l)= 145Soil Conservation (2 X 35) (2 X 35) 4

+ + (1 X 1) + (I X 1) + (I X 1) = 17 2Water Uselfliciency (I X 35) + (2 X 35) + (OX 1) + (IX 1) -- (0X 1) = 115 8Mechanization (0 X 35) + (IX 35) + (0X ) + (0 X 1) + (0 x 1) = 35 16Socio-ecornomics (1 X 35) + (I X 35) + (IX 1)+ (1 X i + (I X 1) = 10 12Tech Transfer (2 X 35) + (IX 35) + (IX i) + (I X ) + (1 X 1) = 135 6Seed Production (1 X 35) (1X 35) + + + (1 X 1)= 12 - (1X ) (1X 10Post-harvest (I X 35) + (IX 35) + (0 X 1) + (IX 1) + (I X 1) = 9 14Agro-forestry (2 X 35) + (1 X 35) + (I X 1) 4- (2X 1) + (I X 1) = 145 4Animal Breeding (2 X 35) (1 X 35) ++ (0X 1) + (0X 1) + (I X 1) = 115 8Animal Health (IX 35) + (2 X 35) + (0X A)+ (OX ) + (0X 1) 105 11Animal Nutrition (2 X 35) + (2 X 35) + (I X I) + (0X1)+ =(2 X) 17 2Aquaculture (I X 35) (0 X 35)+ + (0 X 1)+ (I X 1) + (0 X 1) = 45 15

25

Appendix 2 -- Results

Table Ia Ranking of Research Priorities by Commodity from the Weighted-Criteria Model Applied in the Dominican Republic

Commodity Rank Comrnodi) Rank

47Arroz (cascara) I AguacateCafe (cerezo) 2 Lechuga 48Leche 3 Limon Dulce 49Frijol 4 Zapote 50 Came de Res 5 Toronja 51Maiz (grano) 6 Melon 52Coco de Agua 7 Name 53Tornate 8 Nispero 54Cacao (grano) 9 Molondron 55 Huevos 10 Jagua 56 Naranja Dulce 11 Bercnjena 57Auyama 12 Repo[o 58Carbon Vegetal 13 Lechoza 59 Guandul 14 Ru10 60Batata 15 Pcpino 61Lena Familiar 16 anahoria 62 Cana Azucar 17 Cabuya (Sisal) 63[laba 18 Remolacha 64Platano 19 Arveja 65Came de Ayes 20 Jengibre 66 Naanja Agna 21 Ajonjoli 67Came de Cerdo 22 1liguereta 68 Algodon (iama) 23 Tayota 69Fruto de Palma 24 Mapuey 70Yuca 25 Bija 71Mani (cascara) 26 Cajuil 72 Tabaco (rama) 27 Rabano 73

74Came OvinaCaprin 28 Tamaruido Ajo 29 Guineo 30 YaUtia 31 Limon Agrio 32 Oregano 33 Cera de Abejas 34 Papa 35 Aji 36 Garbanzo 37 Cebolla 38 Miel de Abejas 39 Ceb Uin 40 Guanabana 41 Lena Industrial 42 Mango 43 Pesca 44 Otros Frijoles 45 Pina 46

26

Table lb Ranking of Research Priorities by Commodity from the Weighted-Criteria Mode

Applied in Ecuador

Commodity

Arroz Cafe Maiz Sierra Cebada platanoPapa Baiano Cacao Ganado de Leche laiz Costs

Trigo Fruitas lioja Caduca Frejol Ganado de Carrie Lcguminosas and Mfen Yuca Fruitas Citricos ILenteja Lcgurainosas Costa Cercalcs dinos Porcinos Fruitales Tropicales

Cereales Menores Cultivos wtd Nlenores Main Ovinos Fniitaes Subtropical fIort de Clima Ffio IlorlTropicalAyes Caprinos Chontaduro Soya Palna Africana Sorgo Algodon

Especies Menores Oleaginosas Menores Cana Ifort Subtropical Tabaco [ores Jojoba e

Rank

I 2 3 4 5 6 7 8 9 10

11 12 13 14 15 16 17 18 19 20 21 22

23 24 25 26 27 28 29 30 31 32 33 34 35 36

37 38 39 40 41 42 43 44

Priority Level

High

Medium

Low

None

27

Table Ic Ranking of Research Priorities by Commodity from the Weighted-Criteria Model

Applied in Uruguay (Preliminary Results) Commodity

Beef Cattle Wheat Forages

Dairy Citrus Potatoes Maize SheepRice Other Fruits Grapes Sunlloers Soybeans Vegetables

Sorghum Barley Tomatoes Swine Peanuts Linseed Grain legumes Frutilla

Rank

I 2 3

4 5 6 7 8 9

10 II 12 13 14

15 16 17 18 19 20 21 22

Priority Level

tHligh

Medium

Low

28

Table 2a Ranking Assigned to Research Aeas by the Weighted-Critera Model in the Dominican Republic

Research Area

Natural Resources (sods water forests)

Plant Breeding Cultural Practices Crop Systems

Amral Nutrition and Mxnagement

Iechnology Transfer

Animal genetics and reproduction

Socioeconomics

Plant Protection

Post lwuvest lechnology

Mechmization

Ranking

I

2

3

4

5

6

7

8

9

29

Table 2b Ranking Assigned to Research Areas by the Weighted-Criteria Model in Ecilador

Research Aggregate Ranking by Region Area Ranking I 2 3 4 5 6 789 9 10 11

Cultural Practices (crops) 7 I 8 6 8 9 t10 8 8 2 6

Agro-Forestry 2 2 2 2 1 3 3 4 3 2 7 7

Animal Nutrition 3 3 3 7 11 5 5 2 4 10 2 3

Soil Conservation 4 4 4 3 3 6 12 6 5 4 9 9

Socio-economics 10 5 6 8 4 7 6 15 11 5 3 10

eclmology Transfer 4 6 7 4 5 8 7 7 6 6 4 4

5d duction 4 7 5 5 6 10 8 3 7 7 5 5

Plant Breeding 1 8 1 1 7 1 1 1 1 1 1 1

Soil Fertility 7 9 9 9 9 2 2 11 2 9 6 2

Post-tlarvest technology 13 10 10 10 10 15 10 12 13 14 13 13

Plant Protection 9 11 12 12 2 4 4 5 9 3 8 8

Water Use 11 12 14 14 1 11 13 8 14 12 10 11

Animal lealth 16 13 16 16 11 16 15 16 16 16 16 16

Livestock Breeding amp Management 14 14 13 13 12 13 11 14 0 11 15 15

Aquaculture I1 15 15 15 14 14 14 9 12 13 11 12

Mechanization 15 16 11 11 16 12 16 13 15 15 14 14

Region 1= Region 2 = Region 3 =

Tropical and dry Subtropical Occidentad de Pichincha Fstacion Experimental Santo Domingo

Region 4 = Region 5 =

Subtropico Seco-Loja lrovincia del Ovo

Region 6 = Region 7 =

Elstacion Experimental PortoviejoEstacion Experimental Boliche

Region 8 = E E NAPO PayaminoRegion 9 = E ETropical PichilingueRegion 10 = EESanta Catalina Region 11= EEChuquipata

30

Table 2c Ranking Assigned to Research Areas by the Weighilted-Criteria Model in Uruguay

Research Area

Plant Protection Agroclimatology Plant Breeding Livestock Management Livestock Nutrition Production Systems Pasture Management Soil Management Animal Ilealth Plant Nutrition amp PhysiologyCultural Practices Sociocconomics Animal Reproduction valuation of Products

Animal Improvement IPost-I larest Tech

Region I El Nort Region 2 = la Fstanzuela Region 3 = Salto Rcoon 4 = [ as Brujas Region 5 = Fst

Aggregate Ranking

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

Ranking by Region1 2 3 4 5 7 7 1 10 1

11 9 2 4 5 12 1 S 1 6 2 5 11 13 7 9 4 13 11 8 8 3 10 7 12 4 6 15 14 2

13 2 4 8 3 3 11 16 16 4

14 13 5 9 9 5 8 3 6 11

10 14 7 3 13 1 10 12 12 10

15 16 9 5 16 6 12 14 15 14

16 Is 6 2 15

31

Table 3 Summary of Internal Rates of Return to INIPA Rescarch and Jxterjsio a )

RICELb) CORN WII EAT POTATOES BEANS AGGREGATP)

Research Investment

from 1981 to 1986

and Extension from

1981 to 1990

Free Trade

Pivotal Shift 17 10 18 17

in Stpply Curve

Parallel Shift 35 23 28 33

in Supply Curve

No Trade

Pivotal Shift 18 22 14

in Supply Curve

Prallel Shift 37 42 24

in Supply Curve

Research Investment

from 191 to 1992

and Extension from

LIU_Q 199

FreeTrade

Pivotal Shift 30 20 28 25

in Supply Curve

Parallel Shift 44 31 36 38

in Supply Curve

No Trade

Pivotal Shift 22 14

in Supply Curve

Parall Shift 42 24

IIIsupply Curve

(asrumes no expansion of cultivated area

(1vihen ropansion in cultivated area of 1deg1per year was assumed these rates more than doubled For example the return to research and extension on rice for the 1981 to 1986 research investment and 1981 to 19W) extension investment

changed from 17 to 48

(ceither Free Trade nor No Tra

32

Tble 4 Percent Distribution of Total Net Economic Surplus to INIPA Research and Extension for

No Trade ScenarioJO

Pivotal Supply Shift Parallel Supply Shift

Consumer Producer Consumer Producer

Gain Gain Gain Gain

Rice n = - 76 83 17

n = -39 105 -5 52 48

n = -27 115 -15

Potatoes n = -64 72 28

n = -34 88 12 44 56

n = -24 95 5

Beans n = - 61 74 26

n = -31 (1 9 46 54

n = -21 99 1

KlUbenefits accrue to producers for the trade scenario for rice corn and wheat when the price elasticity

of demand is perfectly elastic

33

ISNAR in the Dominicin Republic by W Moscoso A Coutu I) Bandy and G W Norton June 1986

Longmire 1and 1) Vinkelmann Research Resource Allocation and ComparativeAdvantace CIMI YT paner presented at the XIX (ongre s of InternationalAgricultural Economics Association MIalaga Spain August 1985

Mahlstede J P Iong-Rangqe Planning at tie Iowa Agricnltural and IIonie EconomicsExperimental Station R surce Allocatioin I tritdrtur l Researcih cdIishel Minneapolis Minnesota rnversitv of Minnesota Press 1971

WL

Norton C and IS l)avis Val ittin cturns to Agicultural Research ARe ew for Olr lc(t 6 1)( -609Q o3t 1S

Norton GW aid V(G (Gioalhe lienclit s of A cricultural Research and Extensionin Peru paper proired for INI PA and b USI D Lima Peru June 1985

Norton (X and V( (Ganoa (4uideliTeCs fr Allocations of ResourcesAgricultural Researchland Extension

to in fcru paper prepared Ior INIPA and for

USAID Lima Peru 1chrue rv 1980

Nortoi GV (4 (ianoa and C Pmiarel IPotential Benefits For giculturalResearch and Extension it Peru i1hgr(ti( 1987) 247-257E-ct

lcu dLILti Paper presented to tle Ntion if S11npositim ott I valuatiag Agricultural Research Productivitl

Pardev P(G [Ie riculturUl Know i I [oIIIILction An Impirical Look

tlantai(e rgi Iinn 7lt V I9

Parton K IRAndcreon irid I N 1htm lantion Mtods1fr Iustralialltod Rsr1 XL iCult Ra IlI lCononlics Bulletin No 29 Armidale IiiscrsitV of New laid I1

Paulsen A ind I) R Kildor Tluation and Planning of Research in the ExperimentStation tr Jl 1olo 51t(l9oS 11 1902

flinstr up-- et seti P aid I) 1 rainklin A Systems Approach to Aeriultural ResearchRcsmrce llcatlon IIII I)c elopirig (Ci)lit es Resource Allocation andrd tit to Xi Namland bltletooatol 6-Aarch ed TM Arndt )Gl)Driiplc avd V XV Rnttani Minneapolis Minnesota University of innesota lres 1)77

Rannalho de Castro P and ( IE Schuh An Empirical Test of An Econometric Mode] for Istablishing Research Priorities A Brazil Case Study Resource llocauoaind Iroductivitv in National aiid [iternationalResearch ed IM ArndtI)G I)alrymple and VX Ruttan Minneapoli Minnesota Lniversity of imncota Press 1977

Russell 1)( Resource Allocation in Aguricultural Research Using Socio-EconomicE valuationoand MathematicidModels Canadian ir Econ 23(1977) 29-52

Ruttan VW 1gricuiiralRwarch Plt Bfaltimore Johns Ilopkins University Press 1982

Salmon D( Congruence of Agricultural Re tearch in Indonesia 1974-78 EconomicI)evclopment Center Bulletin 83-1 University of Minnesota St Paul 1983

21

Scobie GM Investment in Agricultural Research Some Economic Principles C immNYT Working Paper Mexico 1984

Shumway CR and 1I McCracken Use of Scoring Models in Ivaluation Research Programs liner Agr Econ 57(1975) 714-18

Von Oppen M an2 1G Ryan Rescarch RCsource Allocation Determining Regiona Priorities fod PoliWv 10 (19S5) 253-2o4

Williamson C lhe Joint l)epartmcnt of Agriculture and State lExperiment Stations Study of Research Needs Resource Allocation in lgricuturalResearch ed WL Fishel Minneapolis Minnesota University of Minnesota Press 1971

22

Appendix I

Simplified Hypothetical Example for Selecting Commodity Priorities

Step a Fill in a table for each quantitative criterion and rank commodities in each table in order ofimportance

Product Foreign Number Value Exchange of Farms Calories Protein

I Coffee 1 Colee 1Plantain 1Rice I Rice 2 Rice 2 Maize 2 Rice 2 Plantain 2 Maize3 Plantan 3 Rice 3 Cassava 3 Cassava 3 Plantain 1Cassava 4 Cassava 4 Coffee 4 Maize 4Cassava 5Maize 5 Plantain 5 Maize 5 C 5 Coffee

Step b Multiply the weights which have been assigned to each criterion (for example30 10 10 5 5) by the number in the ranking fbr each quantitative critcrionand add across to arrive at a ranking for that group of criteria

RankCoffee (I X3)+(l X l4(4 X l)+(5 X 05)--(5 X 05) = 13 2 Rice (2 X 3)+(3 X 1)e(2 X I)+( I X 5)+( I X 05) = 12 1Plantain (3 X 3)+ (5N)-I X 1)+(2 XA5)+(3 X 05) = 175 3Cassava (4 X 3)+(4 X X)(31 )+(3 X 05)+(4 X 05) = 225 4Maize (5X 3)+(2 X 1)+(5 X l)+(4 X 05)+-(2 X 05) = 25 5

Step c Fill in a table f r each qualitative criterion placing each commodity in a highmedium or low category for each criterion In each table give a 2 to those commodities for which the criterion implies the greatest need for moreresearch and a 0 to those commodities for which the criterion implies the least need for research

Yield Success Relation to Current Private Gap Probability Int Ctrs Emphasis Incentive

Coffee Rice Plantain Cassava Maize

2 1 I 0 0

1 2 1 0 2

0 2 0 2 2

I 2 I

0 I

0 1 2 2 1

Comparative Price Home Future Advantage Effect Consumption Demand

Coffee 2 2 0 1 Rice I I I I Plantain Cassava Maize

1 0 1

0 0 2

2 2 1

0 0 2

23

Step d Multiply the weights assigned to each of the qualitative criteria (for example 4 4 4 6 4 8 2 2 6) by the number assigned each commodity (2 1 or 0) for each citerion and add across to arrive at a ranking for the group of qualitative criteria

Coffee (2 X0-)+(U X04)+(O X04) +(1 X 03)+(OX04)+ Rice (1 X04)+(2 X04)+(2 X01)+(2 X 06)1(I X04)+ Plantain (0 X04)+(I X04) (0 X04)+(I X 06)+(2 X04)-+ Cassava (0 X04) +(0 X04) +(2 X04) + (0 X 06)1(2 X04) -Maize (0 X04) 1-(2 X04) 1 (2 X04) + (1 X 06) I (HX 04) +

Rank Coffee (2 X08) + (2 X02) 4 (0 X02) 1-( X 06) =44 3 Rice (0X08)+(0 X02)+( X02)+(H X 06)=54 1 Plantain (1 X08) +(0 X02) f (2 X02) + (0 X06) =34 4 Cassava (0 X08) (0 X02) 1 (2 X02) 1-(0 X 06) =20 5 Maize tI X08)+(2 X02) +(IX02)1 (2 X06)=52 2

Step e Now there are two rankings (one For the group of quantitative criteria and one for the goup of qualitative criteria) Weight these two rankings by the total weights assigned to their respective sets of criteria to arrive at a final ranking

Coffee

Rice

Plantain

Cassava

Maize

Final Rank (2 X6) 1-(3 X 1) 24 2 (I X61+ ( X 4)= I I

(3X6) f( X 4)= 3A 3 (4 X6) 1(5 X 4) =44 5 (5 X6) +(2 X A)=38 4

Step f Divide the final rarking into 3 groups those commodities with highest priority those with ower priority and those with the least priority This is a very subjective step and depends on the resources available for research but has proven to be a useful step

Group I Group 2 Group 3

Rice Plantain Cassava Coflee Maize

24

Steps for Implenienting Procedure for Research Area Selection

Step a Each criterion for research area election is qualitative Fill in a table for eachcriteriou assigning a 2 to those research areas for which the criterion impliesthe greatest need for more research a I to those for which the criterionimplies some need for research and a 0 to those for which the criterion implieslittle or no necd for research These tables should be constructed separatelyfor each region in the country for certain criteria

InternationalResource Problem Center Private CurrentAbundance Impoiltance Relations lIcentive Programs Plant Breeding 2 2 2 0 2Cultural Practice (Lrops) 1 1 1 1 2Plant Protection 1 2 1 0 2Soil Fertility 2 1 1 1Soil Conservation 2 2 1 1 1Water Use Efficiency 1 2 0 1 0Mechanization 0 1 0 0

I I I 1 1 0Socio-econornics

Technology Transfer 2 1 I 1 1Seed Production I I I IPosi-harvest Technology I 1 0 1 1Agro-forestry 2 1 1 2 1Animal Breeding 2 1 c

0 0 1Animal lealth 1 2 0 0 0Animal Nutrition 2 2 1 0 2Aquaculture 1 0 0 1 0

Step h Assign weight (out of a total of 100) to each criterion (for example 35 35I I 1) multiply by the number in Step 1 and sum across to get a singleranking

Plant Breeding (2 ) 35) + (2 X 35) + (2X 1)+ (OX 1)+ (2X 1)= Rank

18 ICultural Practices (crops) (IX 35) + (IX 35) + (IX 1)4- (1 X 1)+ (2X 1)= 11 10Plant Protectioll (IX 35) + (2X 35) + (IX 1)+ (0X1)+ (2X I)= 135 6Soil Fertility (2X 35) + (IX 35) + (1X 1)+ (I X 1)+ (2X l)= 145Soil Conservation (2 X 35) (2 X 35) 4

+ + (1 X 1) + (I X 1) + (I X 1) = 17 2Water Uselfliciency (I X 35) + (2 X 35) + (OX 1) + (IX 1) -- (0X 1) = 115 8Mechanization (0 X 35) + (IX 35) + (0X ) + (0 X 1) + (0 x 1) = 35 16Socio-ecornomics (1 X 35) + (I X 35) + (IX 1)+ (1 X i + (I X 1) = 10 12Tech Transfer (2 X 35) + (IX 35) + (IX i) + (I X ) + (1 X 1) = 135 6Seed Production (1 X 35) (1X 35) + + + (1 X 1)= 12 - (1X ) (1X 10Post-harvest (I X 35) + (IX 35) + (0 X 1) + (IX 1) + (I X 1) = 9 14Agro-forestry (2 X 35) + (1 X 35) + (I X 1) 4- (2X 1) + (I X 1) = 145 4Animal Breeding (2 X 35) (1 X 35) ++ (0X 1) + (0X 1) + (I X 1) = 115 8Animal Health (IX 35) + (2 X 35) + (0X A)+ (OX ) + (0X 1) 105 11Animal Nutrition (2 X 35) + (2 X 35) + (I X I) + (0X1)+ =(2 X) 17 2Aquaculture (I X 35) (0 X 35)+ + (0 X 1)+ (I X 1) + (0 X 1) = 45 15

25

Appendix 2 -- Results

Table Ia Ranking of Research Priorities by Commodity from the Weighted-Criteria Model Applied in the Dominican Republic

Commodity Rank Comrnodi) Rank

47Arroz (cascara) I AguacateCafe (cerezo) 2 Lechuga 48Leche 3 Limon Dulce 49Frijol 4 Zapote 50 Came de Res 5 Toronja 51Maiz (grano) 6 Melon 52Coco de Agua 7 Name 53Tornate 8 Nispero 54Cacao (grano) 9 Molondron 55 Huevos 10 Jagua 56 Naranja Dulce 11 Bercnjena 57Auyama 12 Repo[o 58Carbon Vegetal 13 Lechoza 59 Guandul 14 Ru10 60Batata 15 Pcpino 61Lena Familiar 16 anahoria 62 Cana Azucar 17 Cabuya (Sisal) 63[laba 18 Remolacha 64Platano 19 Arveja 65Came de Ayes 20 Jengibre 66 Naanja Agna 21 Ajonjoli 67Came de Cerdo 22 1liguereta 68 Algodon (iama) 23 Tayota 69Fruto de Palma 24 Mapuey 70Yuca 25 Bija 71Mani (cascara) 26 Cajuil 72 Tabaco (rama) 27 Rabano 73

74Came OvinaCaprin 28 Tamaruido Ajo 29 Guineo 30 YaUtia 31 Limon Agrio 32 Oregano 33 Cera de Abejas 34 Papa 35 Aji 36 Garbanzo 37 Cebolla 38 Miel de Abejas 39 Ceb Uin 40 Guanabana 41 Lena Industrial 42 Mango 43 Pesca 44 Otros Frijoles 45 Pina 46

26

Table lb Ranking of Research Priorities by Commodity from the Weighted-Criteria Mode

Applied in Ecuador

Commodity

Arroz Cafe Maiz Sierra Cebada platanoPapa Baiano Cacao Ganado de Leche laiz Costs

Trigo Fruitas lioja Caduca Frejol Ganado de Carrie Lcguminosas and Mfen Yuca Fruitas Citricos ILenteja Lcgurainosas Costa Cercalcs dinos Porcinos Fruitales Tropicales

Cereales Menores Cultivos wtd Nlenores Main Ovinos Fniitaes Subtropical fIort de Clima Ffio IlorlTropicalAyes Caprinos Chontaduro Soya Palna Africana Sorgo Algodon

Especies Menores Oleaginosas Menores Cana Ifort Subtropical Tabaco [ores Jojoba e

Rank

I 2 3 4 5 6 7 8 9 10

11 12 13 14 15 16 17 18 19 20 21 22

23 24 25 26 27 28 29 30 31 32 33 34 35 36

37 38 39 40 41 42 43 44

Priority Level

High

Medium

Low

None

27

Table Ic Ranking of Research Priorities by Commodity from the Weighted-Criteria Model

Applied in Uruguay (Preliminary Results) Commodity

Beef Cattle Wheat Forages

Dairy Citrus Potatoes Maize SheepRice Other Fruits Grapes Sunlloers Soybeans Vegetables

Sorghum Barley Tomatoes Swine Peanuts Linseed Grain legumes Frutilla

Rank

I 2 3

4 5 6 7 8 9

10 II 12 13 14

15 16 17 18 19 20 21 22

Priority Level

tHligh

Medium

Low

28

Table 2a Ranking Assigned to Research Aeas by the Weighted-Critera Model in the Dominican Republic

Research Area

Natural Resources (sods water forests)

Plant Breeding Cultural Practices Crop Systems

Amral Nutrition and Mxnagement

Iechnology Transfer

Animal genetics and reproduction

Socioeconomics

Plant Protection

Post lwuvest lechnology

Mechmization

Ranking

I

2

3

4

5

6

7

8

9

29

Table 2b Ranking Assigned to Research Areas by the Weighted-Criteria Model in Ecilador

Research Aggregate Ranking by Region Area Ranking I 2 3 4 5 6 789 9 10 11

Cultural Practices (crops) 7 I 8 6 8 9 t10 8 8 2 6

Agro-Forestry 2 2 2 2 1 3 3 4 3 2 7 7

Animal Nutrition 3 3 3 7 11 5 5 2 4 10 2 3

Soil Conservation 4 4 4 3 3 6 12 6 5 4 9 9

Socio-economics 10 5 6 8 4 7 6 15 11 5 3 10

eclmology Transfer 4 6 7 4 5 8 7 7 6 6 4 4

5d duction 4 7 5 5 6 10 8 3 7 7 5 5

Plant Breeding 1 8 1 1 7 1 1 1 1 1 1 1

Soil Fertility 7 9 9 9 9 2 2 11 2 9 6 2

Post-tlarvest technology 13 10 10 10 10 15 10 12 13 14 13 13

Plant Protection 9 11 12 12 2 4 4 5 9 3 8 8

Water Use 11 12 14 14 1 11 13 8 14 12 10 11

Animal lealth 16 13 16 16 11 16 15 16 16 16 16 16

Livestock Breeding amp Management 14 14 13 13 12 13 11 14 0 11 15 15

Aquaculture I1 15 15 15 14 14 14 9 12 13 11 12

Mechanization 15 16 11 11 16 12 16 13 15 15 14 14

Region 1= Region 2 = Region 3 =

Tropical and dry Subtropical Occidentad de Pichincha Fstacion Experimental Santo Domingo

Region 4 = Region 5 =

Subtropico Seco-Loja lrovincia del Ovo

Region 6 = Region 7 =

Elstacion Experimental PortoviejoEstacion Experimental Boliche

Region 8 = E E NAPO PayaminoRegion 9 = E ETropical PichilingueRegion 10 = EESanta Catalina Region 11= EEChuquipata

30

Table 2c Ranking Assigned to Research Areas by the Weighilted-Criteria Model in Uruguay

Research Area

Plant Protection Agroclimatology Plant Breeding Livestock Management Livestock Nutrition Production Systems Pasture Management Soil Management Animal Ilealth Plant Nutrition amp PhysiologyCultural Practices Sociocconomics Animal Reproduction valuation of Products

Animal Improvement IPost-I larest Tech

Region I El Nort Region 2 = la Fstanzuela Region 3 = Salto Rcoon 4 = [ as Brujas Region 5 = Fst

Aggregate Ranking

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

Ranking by Region1 2 3 4 5 7 7 1 10 1

11 9 2 4 5 12 1 S 1 6 2 5 11 13 7 9 4 13 11 8 8 3 10 7 12 4 6 15 14 2

13 2 4 8 3 3 11 16 16 4

14 13 5 9 9 5 8 3 6 11

10 14 7 3 13 1 10 12 12 10

15 16 9 5 16 6 12 14 15 14

16 Is 6 2 15

31

Table 3 Summary of Internal Rates of Return to INIPA Rescarch and Jxterjsio a )

RICELb) CORN WII EAT POTATOES BEANS AGGREGATP)

Research Investment

from 1981 to 1986

and Extension from

1981 to 1990

Free Trade

Pivotal Shift 17 10 18 17

in Stpply Curve

Parallel Shift 35 23 28 33

in Supply Curve

No Trade

Pivotal Shift 18 22 14

in Supply Curve

Prallel Shift 37 42 24

in Supply Curve

Research Investment

from 191 to 1992

and Extension from

LIU_Q 199

FreeTrade

Pivotal Shift 30 20 28 25

in Supply Curve

Parallel Shift 44 31 36 38

in Supply Curve

No Trade

Pivotal Shift 22 14

in Supply Curve

Parall Shift 42 24

IIIsupply Curve

(asrumes no expansion of cultivated area

(1vihen ropansion in cultivated area of 1deg1per year was assumed these rates more than doubled For example the return to research and extension on rice for the 1981 to 1986 research investment and 1981 to 19W) extension investment

changed from 17 to 48

(ceither Free Trade nor No Tra

32

Tble 4 Percent Distribution of Total Net Economic Surplus to INIPA Research and Extension for

No Trade ScenarioJO

Pivotal Supply Shift Parallel Supply Shift

Consumer Producer Consumer Producer

Gain Gain Gain Gain

Rice n = - 76 83 17

n = -39 105 -5 52 48

n = -27 115 -15

Potatoes n = -64 72 28

n = -34 88 12 44 56

n = -24 95 5

Beans n = - 61 74 26

n = -31 (1 9 46 54

n = -21 99 1

KlUbenefits accrue to producers for the trade scenario for rice corn and wheat when the price elasticity

of demand is perfectly elastic

33

Scobie GM Investment in Agricultural Research Some Economic Principles C immNYT Working Paper Mexico 1984

Shumway CR and 1I McCracken Use of Scoring Models in Ivaluation Research Programs liner Agr Econ 57(1975) 714-18

Von Oppen M an2 1G Ryan Rescarch RCsource Allocation Determining Regiona Priorities fod PoliWv 10 (19S5) 253-2o4

Williamson C lhe Joint l)epartmcnt of Agriculture and State lExperiment Stations Study of Research Needs Resource Allocation in lgricuturalResearch ed WL Fishel Minneapolis Minnesota University of Minnesota Press 1971

22

Appendix I

Simplified Hypothetical Example for Selecting Commodity Priorities

Step a Fill in a table for each quantitative criterion and rank commodities in each table in order ofimportance

Product Foreign Number Value Exchange of Farms Calories Protein

I Coffee 1 Colee 1Plantain 1Rice I Rice 2 Rice 2 Maize 2 Rice 2 Plantain 2 Maize3 Plantan 3 Rice 3 Cassava 3 Cassava 3 Plantain 1Cassava 4 Cassava 4 Coffee 4 Maize 4Cassava 5Maize 5 Plantain 5 Maize 5 C 5 Coffee

Step b Multiply the weights which have been assigned to each criterion (for example30 10 10 5 5) by the number in the ranking fbr each quantitative critcrionand add across to arrive at a ranking for that group of criteria

RankCoffee (I X3)+(l X l4(4 X l)+(5 X 05)--(5 X 05) = 13 2 Rice (2 X 3)+(3 X 1)e(2 X I)+( I X 5)+( I X 05) = 12 1Plantain (3 X 3)+ (5N)-I X 1)+(2 XA5)+(3 X 05) = 175 3Cassava (4 X 3)+(4 X X)(31 )+(3 X 05)+(4 X 05) = 225 4Maize (5X 3)+(2 X 1)+(5 X l)+(4 X 05)+-(2 X 05) = 25 5

Step c Fill in a table f r each qualitative criterion placing each commodity in a highmedium or low category for each criterion In each table give a 2 to those commodities for which the criterion implies the greatest need for moreresearch and a 0 to those commodities for which the criterion implies the least need for research

Yield Success Relation to Current Private Gap Probability Int Ctrs Emphasis Incentive

Coffee Rice Plantain Cassava Maize

2 1 I 0 0

1 2 1 0 2

0 2 0 2 2

I 2 I

0 I

0 1 2 2 1

Comparative Price Home Future Advantage Effect Consumption Demand

Coffee 2 2 0 1 Rice I I I I Plantain Cassava Maize

1 0 1

0 0 2

2 2 1

0 0 2

23

Step d Multiply the weights assigned to each of the qualitative criteria (for example 4 4 4 6 4 8 2 2 6) by the number assigned each commodity (2 1 or 0) for each citerion and add across to arrive at a ranking for the group of qualitative criteria

Coffee (2 X0-)+(U X04)+(O X04) +(1 X 03)+(OX04)+ Rice (1 X04)+(2 X04)+(2 X01)+(2 X 06)1(I X04)+ Plantain (0 X04)+(I X04) (0 X04)+(I X 06)+(2 X04)-+ Cassava (0 X04) +(0 X04) +(2 X04) + (0 X 06)1(2 X04) -Maize (0 X04) 1-(2 X04) 1 (2 X04) + (1 X 06) I (HX 04) +

Rank Coffee (2 X08) + (2 X02) 4 (0 X02) 1-( X 06) =44 3 Rice (0X08)+(0 X02)+( X02)+(H X 06)=54 1 Plantain (1 X08) +(0 X02) f (2 X02) + (0 X06) =34 4 Cassava (0 X08) (0 X02) 1 (2 X02) 1-(0 X 06) =20 5 Maize tI X08)+(2 X02) +(IX02)1 (2 X06)=52 2

Step e Now there are two rankings (one For the group of quantitative criteria and one for the goup of qualitative criteria) Weight these two rankings by the total weights assigned to their respective sets of criteria to arrive at a final ranking

Coffee

Rice

Plantain

Cassava

Maize

Final Rank (2 X6) 1-(3 X 1) 24 2 (I X61+ ( X 4)= I I

(3X6) f( X 4)= 3A 3 (4 X6) 1(5 X 4) =44 5 (5 X6) +(2 X A)=38 4

Step f Divide the final rarking into 3 groups those commodities with highest priority those with ower priority and those with the least priority This is a very subjective step and depends on the resources available for research but has proven to be a useful step

Group I Group 2 Group 3

Rice Plantain Cassava Coflee Maize

24

Steps for Implenienting Procedure for Research Area Selection

Step a Each criterion for research area election is qualitative Fill in a table for eachcriteriou assigning a 2 to those research areas for which the criterion impliesthe greatest need for more research a I to those for which the criterionimplies some need for research and a 0 to those for which the criterion implieslittle or no necd for research These tables should be constructed separatelyfor each region in the country for certain criteria

InternationalResource Problem Center Private CurrentAbundance Impoiltance Relations lIcentive Programs Plant Breeding 2 2 2 0 2Cultural Practice (Lrops) 1 1 1 1 2Plant Protection 1 2 1 0 2Soil Fertility 2 1 1 1Soil Conservation 2 2 1 1 1Water Use Efficiency 1 2 0 1 0Mechanization 0 1 0 0

I I I 1 1 0Socio-econornics

Technology Transfer 2 1 I 1 1Seed Production I I I IPosi-harvest Technology I 1 0 1 1Agro-forestry 2 1 1 2 1Animal Breeding 2 1 c

0 0 1Animal lealth 1 2 0 0 0Animal Nutrition 2 2 1 0 2Aquaculture 1 0 0 1 0

Step h Assign weight (out of a total of 100) to each criterion (for example 35 35I I 1) multiply by the number in Step 1 and sum across to get a singleranking

Plant Breeding (2 ) 35) + (2 X 35) + (2X 1)+ (OX 1)+ (2X 1)= Rank

18 ICultural Practices (crops) (IX 35) + (IX 35) + (IX 1)4- (1 X 1)+ (2X 1)= 11 10Plant Protectioll (IX 35) + (2X 35) + (IX 1)+ (0X1)+ (2X I)= 135 6Soil Fertility (2X 35) + (IX 35) + (1X 1)+ (I X 1)+ (2X l)= 145Soil Conservation (2 X 35) (2 X 35) 4

+ + (1 X 1) + (I X 1) + (I X 1) = 17 2Water Uselfliciency (I X 35) + (2 X 35) + (OX 1) + (IX 1) -- (0X 1) = 115 8Mechanization (0 X 35) + (IX 35) + (0X ) + (0 X 1) + (0 x 1) = 35 16Socio-ecornomics (1 X 35) + (I X 35) + (IX 1)+ (1 X i + (I X 1) = 10 12Tech Transfer (2 X 35) + (IX 35) + (IX i) + (I X ) + (1 X 1) = 135 6Seed Production (1 X 35) (1X 35) + + + (1 X 1)= 12 - (1X ) (1X 10Post-harvest (I X 35) + (IX 35) + (0 X 1) + (IX 1) + (I X 1) = 9 14Agro-forestry (2 X 35) + (1 X 35) + (I X 1) 4- (2X 1) + (I X 1) = 145 4Animal Breeding (2 X 35) (1 X 35) ++ (0X 1) + (0X 1) + (I X 1) = 115 8Animal Health (IX 35) + (2 X 35) + (0X A)+ (OX ) + (0X 1) 105 11Animal Nutrition (2 X 35) + (2 X 35) + (I X I) + (0X1)+ =(2 X) 17 2Aquaculture (I X 35) (0 X 35)+ + (0 X 1)+ (I X 1) + (0 X 1) = 45 15

25

Appendix 2 -- Results

Table Ia Ranking of Research Priorities by Commodity from the Weighted-Criteria Model Applied in the Dominican Republic

Commodity Rank Comrnodi) Rank

47Arroz (cascara) I AguacateCafe (cerezo) 2 Lechuga 48Leche 3 Limon Dulce 49Frijol 4 Zapote 50 Came de Res 5 Toronja 51Maiz (grano) 6 Melon 52Coco de Agua 7 Name 53Tornate 8 Nispero 54Cacao (grano) 9 Molondron 55 Huevos 10 Jagua 56 Naranja Dulce 11 Bercnjena 57Auyama 12 Repo[o 58Carbon Vegetal 13 Lechoza 59 Guandul 14 Ru10 60Batata 15 Pcpino 61Lena Familiar 16 anahoria 62 Cana Azucar 17 Cabuya (Sisal) 63[laba 18 Remolacha 64Platano 19 Arveja 65Came de Ayes 20 Jengibre 66 Naanja Agna 21 Ajonjoli 67Came de Cerdo 22 1liguereta 68 Algodon (iama) 23 Tayota 69Fruto de Palma 24 Mapuey 70Yuca 25 Bija 71Mani (cascara) 26 Cajuil 72 Tabaco (rama) 27 Rabano 73

74Came OvinaCaprin 28 Tamaruido Ajo 29 Guineo 30 YaUtia 31 Limon Agrio 32 Oregano 33 Cera de Abejas 34 Papa 35 Aji 36 Garbanzo 37 Cebolla 38 Miel de Abejas 39 Ceb Uin 40 Guanabana 41 Lena Industrial 42 Mango 43 Pesca 44 Otros Frijoles 45 Pina 46

26

Table lb Ranking of Research Priorities by Commodity from the Weighted-Criteria Mode

Applied in Ecuador

Commodity

Arroz Cafe Maiz Sierra Cebada platanoPapa Baiano Cacao Ganado de Leche laiz Costs

Trigo Fruitas lioja Caduca Frejol Ganado de Carrie Lcguminosas and Mfen Yuca Fruitas Citricos ILenteja Lcgurainosas Costa Cercalcs dinos Porcinos Fruitales Tropicales

Cereales Menores Cultivos wtd Nlenores Main Ovinos Fniitaes Subtropical fIort de Clima Ffio IlorlTropicalAyes Caprinos Chontaduro Soya Palna Africana Sorgo Algodon

Especies Menores Oleaginosas Menores Cana Ifort Subtropical Tabaco [ores Jojoba e

Rank

I 2 3 4 5 6 7 8 9 10

11 12 13 14 15 16 17 18 19 20 21 22

23 24 25 26 27 28 29 30 31 32 33 34 35 36

37 38 39 40 41 42 43 44

Priority Level

High

Medium

Low

None

27

Table Ic Ranking of Research Priorities by Commodity from the Weighted-Criteria Model

Applied in Uruguay (Preliminary Results) Commodity

Beef Cattle Wheat Forages

Dairy Citrus Potatoes Maize SheepRice Other Fruits Grapes Sunlloers Soybeans Vegetables

Sorghum Barley Tomatoes Swine Peanuts Linseed Grain legumes Frutilla

Rank

I 2 3

4 5 6 7 8 9

10 II 12 13 14

15 16 17 18 19 20 21 22

Priority Level

tHligh

Medium

Low

28

Table 2a Ranking Assigned to Research Aeas by the Weighted-Critera Model in the Dominican Republic

Research Area

Natural Resources (sods water forests)

Plant Breeding Cultural Practices Crop Systems

Amral Nutrition and Mxnagement

Iechnology Transfer

Animal genetics and reproduction

Socioeconomics

Plant Protection

Post lwuvest lechnology

Mechmization

Ranking

I

2

3

4

5

6

7

8

9

29

Table 2b Ranking Assigned to Research Areas by the Weighted-Criteria Model in Ecilador

Research Aggregate Ranking by Region Area Ranking I 2 3 4 5 6 789 9 10 11

Cultural Practices (crops) 7 I 8 6 8 9 t10 8 8 2 6

Agro-Forestry 2 2 2 2 1 3 3 4 3 2 7 7

Animal Nutrition 3 3 3 7 11 5 5 2 4 10 2 3

Soil Conservation 4 4 4 3 3 6 12 6 5 4 9 9

Socio-economics 10 5 6 8 4 7 6 15 11 5 3 10

eclmology Transfer 4 6 7 4 5 8 7 7 6 6 4 4

5d duction 4 7 5 5 6 10 8 3 7 7 5 5

Plant Breeding 1 8 1 1 7 1 1 1 1 1 1 1

Soil Fertility 7 9 9 9 9 2 2 11 2 9 6 2

Post-tlarvest technology 13 10 10 10 10 15 10 12 13 14 13 13

Plant Protection 9 11 12 12 2 4 4 5 9 3 8 8

Water Use 11 12 14 14 1 11 13 8 14 12 10 11

Animal lealth 16 13 16 16 11 16 15 16 16 16 16 16

Livestock Breeding amp Management 14 14 13 13 12 13 11 14 0 11 15 15

Aquaculture I1 15 15 15 14 14 14 9 12 13 11 12

Mechanization 15 16 11 11 16 12 16 13 15 15 14 14

Region 1= Region 2 = Region 3 =

Tropical and dry Subtropical Occidentad de Pichincha Fstacion Experimental Santo Domingo

Region 4 = Region 5 =

Subtropico Seco-Loja lrovincia del Ovo

Region 6 = Region 7 =

Elstacion Experimental PortoviejoEstacion Experimental Boliche

Region 8 = E E NAPO PayaminoRegion 9 = E ETropical PichilingueRegion 10 = EESanta Catalina Region 11= EEChuquipata

30

Table 2c Ranking Assigned to Research Areas by the Weighilted-Criteria Model in Uruguay

Research Area

Plant Protection Agroclimatology Plant Breeding Livestock Management Livestock Nutrition Production Systems Pasture Management Soil Management Animal Ilealth Plant Nutrition amp PhysiologyCultural Practices Sociocconomics Animal Reproduction valuation of Products

Animal Improvement IPost-I larest Tech

Region I El Nort Region 2 = la Fstanzuela Region 3 = Salto Rcoon 4 = [ as Brujas Region 5 = Fst

Aggregate Ranking

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

Ranking by Region1 2 3 4 5 7 7 1 10 1

11 9 2 4 5 12 1 S 1 6 2 5 11 13 7 9 4 13 11 8 8 3 10 7 12 4 6 15 14 2

13 2 4 8 3 3 11 16 16 4

14 13 5 9 9 5 8 3 6 11

10 14 7 3 13 1 10 12 12 10

15 16 9 5 16 6 12 14 15 14

16 Is 6 2 15

31

Table 3 Summary of Internal Rates of Return to INIPA Rescarch and Jxterjsio a )

RICELb) CORN WII EAT POTATOES BEANS AGGREGATP)

Research Investment

from 1981 to 1986

and Extension from

1981 to 1990

Free Trade

Pivotal Shift 17 10 18 17

in Stpply Curve

Parallel Shift 35 23 28 33

in Supply Curve

No Trade

Pivotal Shift 18 22 14

in Supply Curve

Prallel Shift 37 42 24

in Supply Curve

Research Investment

from 191 to 1992

and Extension from

LIU_Q 199

FreeTrade

Pivotal Shift 30 20 28 25

in Supply Curve

Parallel Shift 44 31 36 38

in Supply Curve

No Trade

Pivotal Shift 22 14

in Supply Curve

Parall Shift 42 24

IIIsupply Curve

(asrumes no expansion of cultivated area

(1vihen ropansion in cultivated area of 1deg1per year was assumed these rates more than doubled For example the return to research and extension on rice for the 1981 to 1986 research investment and 1981 to 19W) extension investment

changed from 17 to 48

(ceither Free Trade nor No Tra

32

Tble 4 Percent Distribution of Total Net Economic Surplus to INIPA Research and Extension for

No Trade ScenarioJO

Pivotal Supply Shift Parallel Supply Shift

Consumer Producer Consumer Producer

Gain Gain Gain Gain

Rice n = - 76 83 17

n = -39 105 -5 52 48

n = -27 115 -15

Potatoes n = -64 72 28

n = -34 88 12 44 56

n = -24 95 5

Beans n = - 61 74 26

n = -31 (1 9 46 54

n = -21 99 1

KlUbenefits accrue to producers for the trade scenario for rice corn and wheat when the price elasticity

of demand is perfectly elastic

33

Appendix I

Simplified Hypothetical Example for Selecting Commodity Priorities

Step a Fill in a table for each quantitative criterion and rank commodities in each table in order ofimportance

Product Foreign Number Value Exchange of Farms Calories Protein

I Coffee 1 Colee 1Plantain 1Rice I Rice 2 Rice 2 Maize 2 Rice 2 Plantain 2 Maize3 Plantan 3 Rice 3 Cassava 3 Cassava 3 Plantain 1Cassava 4 Cassava 4 Coffee 4 Maize 4Cassava 5Maize 5 Plantain 5 Maize 5 C 5 Coffee

Step b Multiply the weights which have been assigned to each criterion (for example30 10 10 5 5) by the number in the ranking fbr each quantitative critcrionand add across to arrive at a ranking for that group of criteria

RankCoffee (I X3)+(l X l4(4 X l)+(5 X 05)--(5 X 05) = 13 2 Rice (2 X 3)+(3 X 1)e(2 X I)+( I X 5)+( I X 05) = 12 1Plantain (3 X 3)+ (5N)-I X 1)+(2 XA5)+(3 X 05) = 175 3Cassava (4 X 3)+(4 X X)(31 )+(3 X 05)+(4 X 05) = 225 4Maize (5X 3)+(2 X 1)+(5 X l)+(4 X 05)+-(2 X 05) = 25 5

Step c Fill in a table f r each qualitative criterion placing each commodity in a highmedium or low category for each criterion In each table give a 2 to those commodities for which the criterion implies the greatest need for moreresearch and a 0 to those commodities for which the criterion implies the least need for research

Yield Success Relation to Current Private Gap Probability Int Ctrs Emphasis Incentive

Coffee Rice Plantain Cassava Maize

2 1 I 0 0

1 2 1 0 2

0 2 0 2 2

I 2 I

0 I

0 1 2 2 1

Comparative Price Home Future Advantage Effect Consumption Demand

Coffee 2 2 0 1 Rice I I I I Plantain Cassava Maize

1 0 1

0 0 2

2 2 1

0 0 2

23

Step d Multiply the weights assigned to each of the qualitative criteria (for example 4 4 4 6 4 8 2 2 6) by the number assigned each commodity (2 1 or 0) for each citerion and add across to arrive at a ranking for the group of qualitative criteria

Coffee (2 X0-)+(U X04)+(O X04) +(1 X 03)+(OX04)+ Rice (1 X04)+(2 X04)+(2 X01)+(2 X 06)1(I X04)+ Plantain (0 X04)+(I X04) (0 X04)+(I X 06)+(2 X04)-+ Cassava (0 X04) +(0 X04) +(2 X04) + (0 X 06)1(2 X04) -Maize (0 X04) 1-(2 X04) 1 (2 X04) + (1 X 06) I (HX 04) +

Rank Coffee (2 X08) + (2 X02) 4 (0 X02) 1-( X 06) =44 3 Rice (0X08)+(0 X02)+( X02)+(H X 06)=54 1 Plantain (1 X08) +(0 X02) f (2 X02) + (0 X06) =34 4 Cassava (0 X08) (0 X02) 1 (2 X02) 1-(0 X 06) =20 5 Maize tI X08)+(2 X02) +(IX02)1 (2 X06)=52 2

Step e Now there are two rankings (one For the group of quantitative criteria and one for the goup of qualitative criteria) Weight these two rankings by the total weights assigned to their respective sets of criteria to arrive at a final ranking

Coffee

Rice

Plantain

Cassava

Maize

Final Rank (2 X6) 1-(3 X 1) 24 2 (I X61+ ( X 4)= I I

(3X6) f( X 4)= 3A 3 (4 X6) 1(5 X 4) =44 5 (5 X6) +(2 X A)=38 4

Step f Divide the final rarking into 3 groups those commodities with highest priority those with ower priority and those with the least priority This is a very subjective step and depends on the resources available for research but has proven to be a useful step

Group I Group 2 Group 3

Rice Plantain Cassava Coflee Maize

24

Steps for Implenienting Procedure for Research Area Selection

Step a Each criterion for research area election is qualitative Fill in a table for eachcriteriou assigning a 2 to those research areas for which the criterion impliesthe greatest need for more research a I to those for which the criterionimplies some need for research and a 0 to those for which the criterion implieslittle or no necd for research These tables should be constructed separatelyfor each region in the country for certain criteria

InternationalResource Problem Center Private CurrentAbundance Impoiltance Relations lIcentive Programs Plant Breeding 2 2 2 0 2Cultural Practice (Lrops) 1 1 1 1 2Plant Protection 1 2 1 0 2Soil Fertility 2 1 1 1Soil Conservation 2 2 1 1 1Water Use Efficiency 1 2 0 1 0Mechanization 0 1 0 0

I I I 1 1 0Socio-econornics

Technology Transfer 2 1 I 1 1Seed Production I I I IPosi-harvest Technology I 1 0 1 1Agro-forestry 2 1 1 2 1Animal Breeding 2 1 c

0 0 1Animal lealth 1 2 0 0 0Animal Nutrition 2 2 1 0 2Aquaculture 1 0 0 1 0

Step h Assign weight (out of a total of 100) to each criterion (for example 35 35I I 1) multiply by the number in Step 1 and sum across to get a singleranking

Plant Breeding (2 ) 35) + (2 X 35) + (2X 1)+ (OX 1)+ (2X 1)= Rank

18 ICultural Practices (crops) (IX 35) + (IX 35) + (IX 1)4- (1 X 1)+ (2X 1)= 11 10Plant Protectioll (IX 35) + (2X 35) + (IX 1)+ (0X1)+ (2X I)= 135 6Soil Fertility (2X 35) + (IX 35) + (1X 1)+ (I X 1)+ (2X l)= 145Soil Conservation (2 X 35) (2 X 35) 4

+ + (1 X 1) + (I X 1) + (I X 1) = 17 2Water Uselfliciency (I X 35) + (2 X 35) + (OX 1) + (IX 1) -- (0X 1) = 115 8Mechanization (0 X 35) + (IX 35) + (0X ) + (0 X 1) + (0 x 1) = 35 16Socio-ecornomics (1 X 35) + (I X 35) + (IX 1)+ (1 X i + (I X 1) = 10 12Tech Transfer (2 X 35) + (IX 35) + (IX i) + (I X ) + (1 X 1) = 135 6Seed Production (1 X 35) (1X 35) + + + (1 X 1)= 12 - (1X ) (1X 10Post-harvest (I X 35) + (IX 35) + (0 X 1) + (IX 1) + (I X 1) = 9 14Agro-forestry (2 X 35) + (1 X 35) + (I X 1) 4- (2X 1) + (I X 1) = 145 4Animal Breeding (2 X 35) (1 X 35) ++ (0X 1) + (0X 1) + (I X 1) = 115 8Animal Health (IX 35) + (2 X 35) + (0X A)+ (OX ) + (0X 1) 105 11Animal Nutrition (2 X 35) + (2 X 35) + (I X I) + (0X1)+ =(2 X) 17 2Aquaculture (I X 35) (0 X 35)+ + (0 X 1)+ (I X 1) + (0 X 1) = 45 15

25

Appendix 2 -- Results

Table Ia Ranking of Research Priorities by Commodity from the Weighted-Criteria Model Applied in the Dominican Republic

Commodity Rank Comrnodi) Rank

47Arroz (cascara) I AguacateCafe (cerezo) 2 Lechuga 48Leche 3 Limon Dulce 49Frijol 4 Zapote 50 Came de Res 5 Toronja 51Maiz (grano) 6 Melon 52Coco de Agua 7 Name 53Tornate 8 Nispero 54Cacao (grano) 9 Molondron 55 Huevos 10 Jagua 56 Naranja Dulce 11 Bercnjena 57Auyama 12 Repo[o 58Carbon Vegetal 13 Lechoza 59 Guandul 14 Ru10 60Batata 15 Pcpino 61Lena Familiar 16 anahoria 62 Cana Azucar 17 Cabuya (Sisal) 63[laba 18 Remolacha 64Platano 19 Arveja 65Came de Ayes 20 Jengibre 66 Naanja Agna 21 Ajonjoli 67Came de Cerdo 22 1liguereta 68 Algodon (iama) 23 Tayota 69Fruto de Palma 24 Mapuey 70Yuca 25 Bija 71Mani (cascara) 26 Cajuil 72 Tabaco (rama) 27 Rabano 73

74Came OvinaCaprin 28 Tamaruido Ajo 29 Guineo 30 YaUtia 31 Limon Agrio 32 Oregano 33 Cera de Abejas 34 Papa 35 Aji 36 Garbanzo 37 Cebolla 38 Miel de Abejas 39 Ceb Uin 40 Guanabana 41 Lena Industrial 42 Mango 43 Pesca 44 Otros Frijoles 45 Pina 46

26

Table lb Ranking of Research Priorities by Commodity from the Weighted-Criteria Mode

Applied in Ecuador

Commodity

Arroz Cafe Maiz Sierra Cebada platanoPapa Baiano Cacao Ganado de Leche laiz Costs

Trigo Fruitas lioja Caduca Frejol Ganado de Carrie Lcguminosas and Mfen Yuca Fruitas Citricos ILenteja Lcgurainosas Costa Cercalcs dinos Porcinos Fruitales Tropicales

Cereales Menores Cultivos wtd Nlenores Main Ovinos Fniitaes Subtropical fIort de Clima Ffio IlorlTropicalAyes Caprinos Chontaduro Soya Palna Africana Sorgo Algodon

Especies Menores Oleaginosas Menores Cana Ifort Subtropical Tabaco [ores Jojoba e

Rank

I 2 3 4 5 6 7 8 9 10

11 12 13 14 15 16 17 18 19 20 21 22

23 24 25 26 27 28 29 30 31 32 33 34 35 36

37 38 39 40 41 42 43 44

Priority Level

High

Medium

Low

None

27

Table Ic Ranking of Research Priorities by Commodity from the Weighted-Criteria Model

Applied in Uruguay (Preliminary Results) Commodity

Beef Cattle Wheat Forages

Dairy Citrus Potatoes Maize SheepRice Other Fruits Grapes Sunlloers Soybeans Vegetables

Sorghum Barley Tomatoes Swine Peanuts Linseed Grain legumes Frutilla

Rank

I 2 3

4 5 6 7 8 9

10 II 12 13 14

15 16 17 18 19 20 21 22

Priority Level

tHligh

Medium

Low

28

Table 2a Ranking Assigned to Research Aeas by the Weighted-Critera Model in the Dominican Republic

Research Area

Natural Resources (sods water forests)

Plant Breeding Cultural Practices Crop Systems

Amral Nutrition and Mxnagement

Iechnology Transfer

Animal genetics and reproduction

Socioeconomics

Plant Protection

Post lwuvest lechnology

Mechmization

Ranking

I

2

3

4

5

6

7

8

9

29

Table 2b Ranking Assigned to Research Areas by the Weighted-Criteria Model in Ecilador

Research Aggregate Ranking by Region Area Ranking I 2 3 4 5 6 789 9 10 11

Cultural Practices (crops) 7 I 8 6 8 9 t10 8 8 2 6

Agro-Forestry 2 2 2 2 1 3 3 4 3 2 7 7

Animal Nutrition 3 3 3 7 11 5 5 2 4 10 2 3

Soil Conservation 4 4 4 3 3 6 12 6 5 4 9 9

Socio-economics 10 5 6 8 4 7 6 15 11 5 3 10

eclmology Transfer 4 6 7 4 5 8 7 7 6 6 4 4

5d duction 4 7 5 5 6 10 8 3 7 7 5 5

Plant Breeding 1 8 1 1 7 1 1 1 1 1 1 1

Soil Fertility 7 9 9 9 9 2 2 11 2 9 6 2

Post-tlarvest technology 13 10 10 10 10 15 10 12 13 14 13 13

Plant Protection 9 11 12 12 2 4 4 5 9 3 8 8

Water Use 11 12 14 14 1 11 13 8 14 12 10 11

Animal lealth 16 13 16 16 11 16 15 16 16 16 16 16

Livestock Breeding amp Management 14 14 13 13 12 13 11 14 0 11 15 15

Aquaculture I1 15 15 15 14 14 14 9 12 13 11 12

Mechanization 15 16 11 11 16 12 16 13 15 15 14 14

Region 1= Region 2 = Region 3 =

Tropical and dry Subtropical Occidentad de Pichincha Fstacion Experimental Santo Domingo

Region 4 = Region 5 =

Subtropico Seco-Loja lrovincia del Ovo

Region 6 = Region 7 =

Elstacion Experimental PortoviejoEstacion Experimental Boliche

Region 8 = E E NAPO PayaminoRegion 9 = E ETropical PichilingueRegion 10 = EESanta Catalina Region 11= EEChuquipata

30

Table 2c Ranking Assigned to Research Areas by the Weighilted-Criteria Model in Uruguay

Research Area

Plant Protection Agroclimatology Plant Breeding Livestock Management Livestock Nutrition Production Systems Pasture Management Soil Management Animal Ilealth Plant Nutrition amp PhysiologyCultural Practices Sociocconomics Animal Reproduction valuation of Products

Animal Improvement IPost-I larest Tech

Region I El Nort Region 2 = la Fstanzuela Region 3 = Salto Rcoon 4 = [ as Brujas Region 5 = Fst

Aggregate Ranking

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

Ranking by Region1 2 3 4 5 7 7 1 10 1

11 9 2 4 5 12 1 S 1 6 2 5 11 13 7 9 4 13 11 8 8 3 10 7 12 4 6 15 14 2

13 2 4 8 3 3 11 16 16 4

14 13 5 9 9 5 8 3 6 11

10 14 7 3 13 1 10 12 12 10

15 16 9 5 16 6 12 14 15 14

16 Is 6 2 15

31

Table 3 Summary of Internal Rates of Return to INIPA Rescarch and Jxterjsio a )

RICELb) CORN WII EAT POTATOES BEANS AGGREGATP)

Research Investment

from 1981 to 1986

and Extension from

1981 to 1990

Free Trade

Pivotal Shift 17 10 18 17

in Stpply Curve

Parallel Shift 35 23 28 33

in Supply Curve

No Trade

Pivotal Shift 18 22 14

in Supply Curve

Prallel Shift 37 42 24

in Supply Curve

Research Investment

from 191 to 1992

and Extension from

LIU_Q 199

FreeTrade

Pivotal Shift 30 20 28 25

in Supply Curve

Parallel Shift 44 31 36 38

in Supply Curve

No Trade

Pivotal Shift 22 14

in Supply Curve

Parall Shift 42 24

IIIsupply Curve

(asrumes no expansion of cultivated area

(1vihen ropansion in cultivated area of 1deg1per year was assumed these rates more than doubled For example the return to research and extension on rice for the 1981 to 1986 research investment and 1981 to 19W) extension investment

changed from 17 to 48

(ceither Free Trade nor No Tra

32

Tble 4 Percent Distribution of Total Net Economic Surplus to INIPA Research and Extension for

No Trade ScenarioJO

Pivotal Supply Shift Parallel Supply Shift

Consumer Producer Consumer Producer

Gain Gain Gain Gain

Rice n = - 76 83 17

n = -39 105 -5 52 48

n = -27 115 -15

Potatoes n = -64 72 28

n = -34 88 12 44 56

n = -24 95 5

Beans n = - 61 74 26

n = -31 (1 9 46 54

n = -21 99 1

KlUbenefits accrue to producers for the trade scenario for rice corn and wheat when the price elasticity

of demand is perfectly elastic

33

Step d Multiply the weights assigned to each of the qualitative criteria (for example 4 4 4 6 4 8 2 2 6) by the number assigned each commodity (2 1 or 0) for each citerion and add across to arrive at a ranking for the group of qualitative criteria

Coffee (2 X0-)+(U X04)+(O X04) +(1 X 03)+(OX04)+ Rice (1 X04)+(2 X04)+(2 X01)+(2 X 06)1(I X04)+ Plantain (0 X04)+(I X04) (0 X04)+(I X 06)+(2 X04)-+ Cassava (0 X04) +(0 X04) +(2 X04) + (0 X 06)1(2 X04) -Maize (0 X04) 1-(2 X04) 1 (2 X04) + (1 X 06) I (HX 04) +

Rank Coffee (2 X08) + (2 X02) 4 (0 X02) 1-( X 06) =44 3 Rice (0X08)+(0 X02)+( X02)+(H X 06)=54 1 Plantain (1 X08) +(0 X02) f (2 X02) + (0 X06) =34 4 Cassava (0 X08) (0 X02) 1 (2 X02) 1-(0 X 06) =20 5 Maize tI X08)+(2 X02) +(IX02)1 (2 X06)=52 2

Step e Now there are two rankings (one For the group of quantitative criteria and one for the goup of qualitative criteria) Weight these two rankings by the total weights assigned to their respective sets of criteria to arrive at a final ranking

Coffee

Rice

Plantain

Cassava

Maize

Final Rank (2 X6) 1-(3 X 1) 24 2 (I X61+ ( X 4)= I I

(3X6) f( X 4)= 3A 3 (4 X6) 1(5 X 4) =44 5 (5 X6) +(2 X A)=38 4

Step f Divide the final rarking into 3 groups those commodities with highest priority those with ower priority and those with the least priority This is a very subjective step and depends on the resources available for research but has proven to be a useful step

Group I Group 2 Group 3

Rice Plantain Cassava Coflee Maize

24

Steps for Implenienting Procedure for Research Area Selection

Step a Each criterion for research area election is qualitative Fill in a table for eachcriteriou assigning a 2 to those research areas for which the criterion impliesthe greatest need for more research a I to those for which the criterionimplies some need for research and a 0 to those for which the criterion implieslittle or no necd for research These tables should be constructed separatelyfor each region in the country for certain criteria

InternationalResource Problem Center Private CurrentAbundance Impoiltance Relations lIcentive Programs Plant Breeding 2 2 2 0 2Cultural Practice (Lrops) 1 1 1 1 2Plant Protection 1 2 1 0 2Soil Fertility 2 1 1 1Soil Conservation 2 2 1 1 1Water Use Efficiency 1 2 0 1 0Mechanization 0 1 0 0

I I I 1 1 0Socio-econornics

Technology Transfer 2 1 I 1 1Seed Production I I I IPosi-harvest Technology I 1 0 1 1Agro-forestry 2 1 1 2 1Animal Breeding 2 1 c

0 0 1Animal lealth 1 2 0 0 0Animal Nutrition 2 2 1 0 2Aquaculture 1 0 0 1 0

Step h Assign weight (out of a total of 100) to each criterion (for example 35 35I I 1) multiply by the number in Step 1 and sum across to get a singleranking

Plant Breeding (2 ) 35) + (2 X 35) + (2X 1)+ (OX 1)+ (2X 1)= Rank

18 ICultural Practices (crops) (IX 35) + (IX 35) + (IX 1)4- (1 X 1)+ (2X 1)= 11 10Plant Protectioll (IX 35) + (2X 35) + (IX 1)+ (0X1)+ (2X I)= 135 6Soil Fertility (2X 35) + (IX 35) + (1X 1)+ (I X 1)+ (2X l)= 145Soil Conservation (2 X 35) (2 X 35) 4

+ + (1 X 1) + (I X 1) + (I X 1) = 17 2Water Uselfliciency (I X 35) + (2 X 35) + (OX 1) + (IX 1) -- (0X 1) = 115 8Mechanization (0 X 35) + (IX 35) + (0X ) + (0 X 1) + (0 x 1) = 35 16Socio-ecornomics (1 X 35) + (I X 35) + (IX 1)+ (1 X i + (I X 1) = 10 12Tech Transfer (2 X 35) + (IX 35) + (IX i) + (I X ) + (1 X 1) = 135 6Seed Production (1 X 35) (1X 35) + + + (1 X 1)= 12 - (1X ) (1X 10Post-harvest (I X 35) + (IX 35) + (0 X 1) + (IX 1) + (I X 1) = 9 14Agro-forestry (2 X 35) + (1 X 35) + (I X 1) 4- (2X 1) + (I X 1) = 145 4Animal Breeding (2 X 35) (1 X 35) ++ (0X 1) + (0X 1) + (I X 1) = 115 8Animal Health (IX 35) + (2 X 35) + (0X A)+ (OX ) + (0X 1) 105 11Animal Nutrition (2 X 35) + (2 X 35) + (I X I) + (0X1)+ =(2 X) 17 2Aquaculture (I X 35) (0 X 35)+ + (0 X 1)+ (I X 1) + (0 X 1) = 45 15

25

Appendix 2 -- Results

Table Ia Ranking of Research Priorities by Commodity from the Weighted-Criteria Model Applied in the Dominican Republic

Commodity Rank Comrnodi) Rank

47Arroz (cascara) I AguacateCafe (cerezo) 2 Lechuga 48Leche 3 Limon Dulce 49Frijol 4 Zapote 50 Came de Res 5 Toronja 51Maiz (grano) 6 Melon 52Coco de Agua 7 Name 53Tornate 8 Nispero 54Cacao (grano) 9 Molondron 55 Huevos 10 Jagua 56 Naranja Dulce 11 Bercnjena 57Auyama 12 Repo[o 58Carbon Vegetal 13 Lechoza 59 Guandul 14 Ru10 60Batata 15 Pcpino 61Lena Familiar 16 anahoria 62 Cana Azucar 17 Cabuya (Sisal) 63[laba 18 Remolacha 64Platano 19 Arveja 65Came de Ayes 20 Jengibre 66 Naanja Agna 21 Ajonjoli 67Came de Cerdo 22 1liguereta 68 Algodon (iama) 23 Tayota 69Fruto de Palma 24 Mapuey 70Yuca 25 Bija 71Mani (cascara) 26 Cajuil 72 Tabaco (rama) 27 Rabano 73

74Came OvinaCaprin 28 Tamaruido Ajo 29 Guineo 30 YaUtia 31 Limon Agrio 32 Oregano 33 Cera de Abejas 34 Papa 35 Aji 36 Garbanzo 37 Cebolla 38 Miel de Abejas 39 Ceb Uin 40 Guanabana 41 Lena Industrial 42 Mango 43 Pesca 44 Otros Frijoles 45 Pina 46

26

Table lb Ranking of Research Priorities by Commodity from the Weighted-Criteria Mode

Applied in Ecuador

Commodity

Arroz Cafe Maiz Sierra Cebada platanoPapa Baiano Cacao Ganado de Leche laiz Costs

Trigo Fruitas lioja Caduca Frejol Ganado de Carrie Lcguminosas and Mfen Yuca Fruitas Citricos ILenteja Lcgurainosas Costa Cercalcs dinos Porcinos Fruitales Tropicales

Cereales Menores Cultivos wtd Nlenores Main Ovinos Fniitaes Subtropical fIort de Clima Ffio IlorlTropicalAyes Caprinos Chontaduro Soya Palna Africana Sorgo Algodon

Especies Menores Oleaginosas Menores Cana Ifort Subtropical Tabaco [ores Jojoba e

Rank

I 2 3 4 5 6 7 8 9 10

11 12 13 14 15 16 17 18 19 20 21 22

23 24 25 26 27 28 29 30 31 32 33 34 35 36

37 38 39 40 41 42 43 44

Priority Level

High

Medium

Low

None

27

Table Ic Ranking of Research Priorities by Commodity from the Weighted-Criteria Model

Applied in Uruguay (Preliminary Results) Commodity

Beef Cattle Wheat Forages

Dairy Citrus Potatoes Maize SheepRice Other Fruits Grapes Sunlloers Soybeans Vegetables

Sorghum Barley Tomatoes Swine Peanuts Linseed Grain legumes Frutilla

Rank

I 2 3

4 5 6 7 8 9

10 II 12 13 14

15 16 17 18 19 20 21 22

Priority Level

tHligh

Medium

Low

28

Table 2a Ranking Assigned to Research Aeas by the Weighted-Critera Model in the Dominican Republic

Research Area

Natural Resources (sods water forests)

Plant Breeding Cultural Practices Crop Systems

Amral Nutrition and Mxnagement

Iechnology Transfer

Animal genetics and reproduction

Socioeconomics

Plant Protection

Post lwuvest lechnology

Mechmization

Ranking

I

2

3

4

5

6

7

8

9

29

Table 2b Ranking Assigned to Research Areas by the Weighted-Criteria Model in Ecilador

Research Aggregate Ranking by Region Area Ranking I 2 3 4 5 6 789 9 10 11

Cultural Practices (crops) 7 I 8 6 8 9 t10 8 8 2 6

Agro-Forestry 2 2 2 2 1 3 3 4 3 2 7 7

Animal Nutrition 3 3 3 7 11 5 5 2 4 10 2 3

Soil Conservation 4 4 4 3 3 6 12 6 5 4 9 9

Socio-economics 10 5 6 8 4 7 6 15 11 5 3 10

eclmology Transfer 4 6 7 4 5 8 7 7 6 6 4 4

5d duction 4 7 5 5 6 10 8 3 7 7 5 5

Plant Breeding 1 8 1 1 7 1 1 1 1 1 1 1

Soil Fertility 7 9 9 9 9 2 2 11 2 9 6 2

Post-tlarvest technology 13 10 10 10 10 15 10 12 13 14 13 13

Plant Protection 9 11 12 12 2 4 4 5 9 3 8 8

Water Use 11 12 14 14 1 11 13 8 14 12 10 11

Animal lealth 16 13 16 16 11 16 15 16 16 16 16 16

Livestock Breeding amp Management 14 14 13 13 12 13 11 14 0 11 15 15

Aquaculture I1 15 15 15 14 14 14 9 12 13 11 12

Mechanization 15 16 11 11 16 12 16 13 15 15 14 14

Region 1= Region 2 = Region 3 =

Tropical and dry Subtropical Occidentad de Pichincha Fstacion Experimental Santo Domingo

Region 4 = Region 5 =

Subtropico Seco-Loja lrovincia del Ovo

Region 6 = Region 7 =

Elstacion Experimental PortoviejoEstacion Experimental Boliche

Region 8 = E E NAPO PayaminoRegion 9 = E ETropical PichilingueRegion 10 = EESanta Catalina Region 11= EEChuquipata

30

Table 2c Ranking Assigned to Research Areas by the Weighilted-Criteria Model in Uruguay

Research Area

Plant Protection Agroclimatology Plant Breeding Livestock Management Livestock Nutrition Production Systems Pasture Management Soil Management Animal Ilealth Plant Nutrition amp PhysiologyCultural Practices Sociocconomics Animal Reproduction valuation of Products

Animal Improvement IPost-I larest Tech

Region I El Nort Region 2 = la Fstanzuela Region 3 = Salto Rcoon 4 = [ as Brujas Region 5 = Fst

Aggregate Ranking

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

Ranking by Region1 2 3 4 5 7 7 1 10 1

11 9 2 4 5 12 1 S 1 6 2 5 11 13 7 9 4 13 11 8 8 3 10 7 12 4 6 15 14 2

13 2 4 8 3 3 11 16 16 4

14 13 5 9 9 5 8 3 6 11

10 14 7 3 13 1 10 12 12 10

15 16 9 5 16 6 12 14 15 14

16 Is 6 2 15

31

Table 3 Summary of Internal Rates of Return to INIPA Rescarch and Jxterjsio a )

RICELb) CORN WII EAT POTATOES BEANS AGGREGATP)

Research Investment

from 1981 to 1986

and Extension from

1981 to 1990

Free Trade

Pivotal Shift 17 10 18 17

in Stpply Curve

Parallel Shift 35 23 28 33

in Supply Curve

No Trade

Pivotal Shift 18 22 14

in Supply Curve

Prallel Shift 37 42 24

in Supply Curve

Research Investment

from 191 to 1992

and Extension from

LIU_Q 199

FreeTrade

Pivotal Shift 30 20 28 25

in Supply Curve

Parallel Shift 44 31 36 38

in Supply Curve

No Trade

Pivotal Shift 22 14

in Supply Curve

Parall Shift 42 24

IIIsupply Curve

(asrumes no expansion of cultivated area

(1vihen ropansion in cultivated area of 1deg1per year was assumed these rates more than doubled For example the return to research and extension on rice for the 1981 to 1986 research investment and 1981 to 19W) extension investment

changed from 17 to 48

(ceither Free Trade nor No Tra

32

Tble 4 Percent Distribution of Total Net Economic Surplus to INIPA Research and Extension for

No Trade ScenarioJO

Pivotal Supply Shift Parallel Supply Shift

Consumer Producer Consumer Producer

Gain Gain Gain Gain

Rice n = - 76 83 17

n = -39 105 -5 52 48

n = -27 115 -15

Potatoes n = -64 72 28

n = -34 88 12 44 56

n = -24 95 5

Beans n = - 61 74 26

n = -31 (1 9 46 54

n = -21 99 1

KlUbenefits accrue to producers for the trade scenario for rice corn and wheat when the price elasticity

of demand is perfectly elastic

33

Steps for Implenienting Procedure for Research Area Selection

Step a Each criterion for research area election is qualitative Fill in a table for eachcriteriou assigning a 2 to those research areas for which the criterion impliesthe greatest need for more research a I to those for which the criterionimplies some need for research and a 0 to those for which the criterion implieslittle or no necd for research These tables should be constructed separatelyfor each region in the country for certain criteria

InternationalResource Problem Center Private CurrentAbundance Impoiltance Relations lIcentive Programs Plant Breeding 2 2 2 0 2Cultural Practice (Lrops) 1 1 1 1 2Plant Protection 1 2 1 0 2Soil Fertility 2 1 1 1Soil Conservation 2 2 1 1 1Water Use Efficiency 1 2 0 1 0Mechanization 0 1 0 0

I I I 1 1 0Socio-econornics

Technology Transfer 2 1 I 1 1Seed Production I I I IPosi-harvest Technology I 1 0 1 1Agro-forestry 2 1 1 2 1Animal Breeding 2 1 c

0 0 1Animal lealth 1 2 0 0 0Animal Nutrition 2 2 1 0 2Aquaculture 1 0 0 1 0

Step h Assign weight (out of a total of 100) to each criterion (for example 35 35I I 1) multiply by the number in Step 1 and sum across to get a singleranking

Plant Breeding (2 ) 35) + (2 X 35) + (2X 1)+ (OX 1)+ (2X 1)= Rank

18 ICultural Practices (crops) (IX 35) + (IX 35) + (IX 1)4- (1 X 1)+ (2X 1)= 11 10Plant Protectioll (IX 35) + (2X 35) + (IX 1)+ (0X1)+ (2X I)= 135 6Soil Fertility (2X 35) + (IX 35) + (1X 1)+ (I X 1)+ (2X l)= 145Soil Conservation (2 X 35) (2 X 35) 4

+ + (1 X 1) + (I X 1) + (I X 1) = 17 2Water Uselfliciency (I X 35) + (2 X 35) + (OX 1) + (IX 1) -- (0X 1) = 115 8Mechanization (0 X 35) + (IX 35) + (0X ) + (0 X 1) + (0 x 1) = 35 16Socio-ecornomics (1 X 35) + (I X 35) + (IX 1)+ (1 X i + (I X 1) = 10 12Tech Transfer (2 X 35) + (IX 35) + (IX i) + (I X ) + (1 X 1) = 135 6Seed Production (1 X 35) (1X 35) + + + (1 X 1)= 12 - (1X ) (1X 10Post-harvest (I X 35) + (IX 35) + (0 X 1) + (IX 1) + (I X 1) = 9 14Agro-forestry (2 X 35) + (1 X 35) + (I X 1) 4- (2X 1) + (I X 1) = 145 4Animal Breeding (2 X 35) (1 X 35) ++ (0X 1) + (0X 1) + (I X 1) = 115 8Animal Health (IX 35) + (2 X 35) + (0X A)+ (OX ) + (0X 1) 105 11Animal Nutrition (2 X 35) + (2 X 35) + (I X I) + (0X1)+ =(2 X) 17 2Aquaculture (I X 35) (0 X 35)+ + (0 X 1)+ (I X 1) + (0 X 1) = 45 15

25

Appendix 2 -- Results

Table Ia Ranking of Research Priorities by Commodity from the Weighted-Criteria Model Applied in the Dominican Republic

Commodity Rank Comrnodi) Rank

47Arroz (cascara) I AguacateCafe (cerezo) 2 Lechuga 48Leche 3 Limon Dulce 49Frijol 4 Zapote 50 Came de Res 5 Toronja 51Maiz (grano) 6 Melon 52Coco de Agua 7 Name 53Tornate 8 Nispero 54Cacao (grano) 9 Molondron 55 Huevos 10 Jagua 56 Naranja Dulce 11 Bercnjena 57Auyama 12 Repo[o 58Carbon Vegetal 13 Lechoza 59 Guandul 14 Ru10 60Batata 15 Pcpino 61Lena Familiar 16 anahoria 62 Cana Azucar 17 Cabuya (Sisal) 63[laba 18 Remolacha 64Platano 19 Arveja 65Came de Ayes 20 Jengibre 66 Naanja Agna 21 Ajonjoli 67Came de Cerdo 22 1liguereta 68 Algodon (iama) 23 Tayota 69Fruto de Palma 24 Mapuey 70Yuca 25 Bija 71Mani (cascara) 26 Cajuil 72 Tabaco (rama) 27 Rabano 73

74Came OvinaCaprin 28 Tamaruido Ajo 29 Guineo 30 YaUtia 31 Limon Agrio 32 Oregano 33 Cera de Abejas 34 Papa 35 Aji 36 Garbanzo 37 Cebolla 38 Miel de Abejas 39 Ceb Uin 40 Guanabana 41 Lena Industrial 42 Mango 43 Pesca 44 Otros Frijoles 45 Pina 46

26

Table lb Ranking of Research Priorities by Commodity from the Weighted-Criteria Mode

Applied in Ecuador

Commodity

Arroz Cafe Maiz Sierra Cebada platanoPapa Baiano Cacao Ganado de Leche laiz Costs

Trigo Fruitas lioja Caduca Frejol Ganado de Carrie Lcguminosas and Mfen Yuca Fruitas Citricos ILenteja Lcgurainosas Costa Cercalcs dinos Porcinos Fruitales Tropicales

Cereales Menores Cultivos wtd Nlenores Main Ovinos Fniitaes Subtropical fIort de Clima Ffio IlorlTropicalAyes Caprinos Chontaduro Soya Palna Africana Sorgo Algodon

Especies Menores Oleaginosas Menores Cana Ifort Subtropical Tabaco [ores Jojoba e

Rank

I 2 3 4 5 6 7 8 9 10

11 12 13 14 15 16 17 18 19 20 21 22

23 24 25 26 27 28 29 30 31 32 33 34 35 36

37 38 39 40 41 42 43 44

Priority Level

High

Medium

Low

None

27

Table Ic Ranking of Research Priorities by Commodity from the Weighted-Criteria Model

Applied in Uruguay (Preliminary Results) Commodity

Beef Cattle Wheat Forages

Dairy Citrus Potatoes Maize SheepRice Other Fruits Grapes Sunlloers Soybeans Vegetables

Sorghum Barley Tomatoes Swine Peanuts Linseed Grain legumes Frutilla

Rank

I 2 3

4 5 6 7 8 9

10 II 12 13 14

15 16 17 18 19 20 21 22

Priority Level

tHligh

Medium

Low

28

Table 2a Ranking Assigned to Research Aeas by the Weighted-Critera Model in the Dominican Republic

Research Area

Natural Resources (sods water forests)

Plant Breeding Cultural Practices Crop Systems

Amral Nutrition and Mxnagement

Iechnology Transfer

Animal genetics and reproduction

Socioeconomics

Plant Protection

Post lwuvest lechnology

Mechmization

Ranking

I

2

3

4

5

6

7

8

9

29

Table 2b Ranking Assigned to Research Areas by the Weighted-Criteria Model in Ecilador

Research Aggregate Ranking by Region Area Ranking I 2 3 4 5 6 789 9 10 11

Cultural Practices (crops) 7 I 8 6 8 9 t10 8 8 2 6

Agro-Forestry 2 2 2 2 1 3 3 4 3 2 7 7

Animal Nutrition 3 3 3 7 11 5 5 2 4 10 2 3

Soil Conservation 4 4 4 3 3 6 12 6 5 4 9 9

Socio-economics 10 5 6 8 4 7 6 15 11 5 3 10

eclmology Transfer 4 6 7 4 5 8 7 7 6 6 4 4

5d duction 4 7 5 5 6 10 8 3 7 7 5 5

Plant Breeding 1 8 1 1 7 1 1 1 1 1 1 1

Soil Fertility 7 9 9 9 9 2 2 11 2 9 6 2

Post-tlarvest technology 13 10 10 10 10 15 10 12 13 14 13 13

Plant Protection 9 11 12 12 2 4 4 5 9 3 8 8

Water Use 11 12 14 14 1 11 13 8 14 12 10 11

Animal lealth 16 13 16 16 11 16 15 16 16 16 16 16

Livestock Breeding amp Management 14 14 13 13 12 13 11 14 0 11 15 15

Aquaculture I1 15 15 15 14 14 14 9 12 13 11 12

Mechanization 15 16 11 11 16 12 16 13 15 15 14 14

Region 1= Region 2 = Region 3 =

Tropical and dry Subtropical Occidentad de Pichincha Fstacion Experimental Santo Domingo

Region 4 = Region 5 =

Subtropico Seco-Loja lrovincia del Ovo

Region 6 = Region 7 =

Elstacion Experimental PortoviejoEstacion Experimental Boliche

Region 8 = E E NAPO PayaminoRegion 9 = E ETropical PichilingueRegion 10 = EESanta Catalina Region 11= EEChuquipata

30

Table 2c Ranking Assigned to Research Areas by the Weighilted-Criteria Model in Uruguay

Research Area

Plant Protection Agroclimatology Plant Breeding Livestock Management Livestock Nutrition Production Systems Pasture Management Soil Management Animal Ilealth Plant Nutrition amp PhysiologyCultural Practices Sociocconomics Animal Reproduction valuation of Products

Animal Improvement IPost-I larest Tech

Region I El Nort Region 2 = la Fstanzuela Region 3 = Salto Rcoon 4 = [ as Brujas Region 5 = Fst

Aggregate Ranking

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

Ranking by Region1 2 3 4 5 7 7 1 10 1

11 9 2 4 5 12 1 S 1 6 2 5 11 13 7 9 4 13 11 8 8 3 10 7 12 4 6 15 14 2

13 2 4 8 3 3 11 16 16 4

14 13 5 9 9 5 8 3 6 11

10 14 7 3 13 1 10 12 12 10

15 16 9 5 16 6 12 14 15 14

16 Is 6 2 15

31

Table 3 Summary of Internal Rates of Return to INIPA Rescarch and Jxterjsio a )

RICELb) CORN WII EAT POTATOES BEANS AGGREGATP)

Research Investment

from 1981 to 1986

and Extension from

1981 to 1990

Free Trade

Pivotal Shift 17 10 18 17

in Stpply Curve

Parallel Shift 35 23 28 33

in Supply Curve

No Trade

Pivotal Shift 18 22 14

in Supply Curve

Prallel Shift 37 42 24

in Supply Curve

Research Investment

from 191 to 1992

and Extension from

LIU_Q 199

FreeTrade

Pivotal Shift 30 20 28 25

in Supply Curve

Parallel Shift 44 31 36 38

in Supply Curve

No Trade

Pivotal Shift 22 14

in Supply Curve

Parall Shift 42 24

IIIsupply Curve

(asrumes no expansion of cultivated area

(1vihen ropansion in cultivated area of 1deg1per year was assumed these rates more than doubled For example the return to research and extension on rice for the 1981 to 1986 research investment and 1981 to 19W) extension investment

changed from 17 to 48

(ceither Free Trade nor No Tra

32

Tble 4 Percent Distribution of Total Net Economic Surplus to INIPA Research and Extension for

No Trade ScenarioJO

Pivotal Supply Shift Parallel Supply Shift

Consumer Producer Consumer Producer

Gain Gain Gain Gain

Rice n = - 76 83 17

n = -39 105 -5 52 48

n = -27 115 -15

Potatoes n = -64 72 28

n = -34 88 12 44 56

n = -24 95 5

Beans n = - 61 74 26

n = -31 (1 9 46 54

n = -21 99 1

KlUbenefits accrue to producers for the trade scenario for rice corn and wheat when the price elasticity

of demand is perfectly elastic

33

Appendix 2 -- Results

Table Ia Ranking of Research Priorities by Commodity from the Weighted-Criteria Model Applied in the Dominican Republic

Commodity Rank Comrnodi) Rank

47Arroz (cascara) I AguacateCafe (cerezo) 2 Lechuga 48Leche 3 Limon Dulce 49Frijol 4 Zapote 50 Came de Res 5 Toronja 51Maiz (grano) 6 Melon 52Coco de Agua 7 Name 53Tornate 8 Nispero 54Cacao (grano) 9 Molondron 55 Huevos 10 Jagua 56 Naranja Dulce 11 Bercnjena 57Auyama 12 Repo[o 58Carbon Vegetal 13 Lechoza 59 Guandul 14 Ru10 60Batata 15 Pcpino 61Lena Familiar 16 anahoria 62 Cana Azucar 17 Cabuya (Sisal) 63[laba 18 Remolacha 64Platano 19 Arveja 65Came de Ayes 20 Jengibre 66 Naanja Agna 21 Ajonjoli 67Came de Cerdo 22 1liguereta 68 Algodon (iama) 23 Tayota 69Fruto de Palma 24 Mapuey 70Yuca 25 Bija 71Mani (cascara) 26 Cajuil 72 Tabaco (rama) 27 Rabano 73

74Came OvinaCaprin 28 Tamaruido Ajo 29 Guineo 30 YaUtia 31 Limon Agrio 32 Oregano 33 Cera de Abejas 34 Papa 35 Aji 36 Garbanzo 37 Cebolla 38 Miel de Abejas 39 Ceb Uin 40 Guanabana 41 Lena Industrial 42 Mango 43 Pesca 44 Otros Frijoles 45 Pina 46

26

Table lb Ranking of Research Priorities by Commodity from the Weighted-Criteria Mode

Applied in Ecuador

Commodity

Arroz Cafe Maiz Sierra Cebada platanoPapa Baiano Cacao Ganado de Leche laiz Costs

Trigo Fruitas lioja Caduca Frejol Ganado de Carrie Lcguminosas and Mfen Yuca Fruitas Citricos ILenteja Lcgurainosas Costa Cercalcs dinos Porcinos Fruitales Tropicales

Cereales Menores Cultivos wtd Nlenores Main Ovinos Fniitaes Subtropical fIort de Clima Ffio IlorlTropicalAyes Caprinos Chontaduro Soya Palna Africana Sorgo Algodon

Especies Menores Oleaginosas Menores Cana Ifort Subtropical Tabaco [ores Jojoba e

Rank

I 2 3 4 5 6 7 8 9 10

11 12 13 14 15 16 17 18 19 20 21 22

23 24 25 26 27 28 29 30 31 32 33 34 35 36

37 38 39 40 41 42 43 44

Priority Level

High

Medium

Low

None

27

Table Ic Ranking of Research Priorities by Commodity from the Weighted-Criteria Model

Applied in Uruguay (Preliminary Results) Commodity

Beef Cattle Wheat Forages

Dairy Citrus Potatoes Maize SheepRice Other Fruits Grapes Sunlloers Soybeans Vegetables

Sorghum Barley Tomatoes Swine Peanuts Linseed Grain legumes Frutilla

Rank

I 2 3

4 5 6 7 8 9

10 II 12 13 14

15 16 17 18 19 20 21 22

Priority Level

tHligh

Medium

Low

28

Table 2a Ranking Assigned to Research Aeas by the Weighted-Critera Model in the Dominican Republic

Research Area

Natural Resources (sods water forests)

Plant Breeding Cultural Practices Crop Systems

Amral Nutrition and Mxnagement

Iechnology Transfer

Animal genetics and reproduction

Socioeconomics

Plant Protection

Post lwuvest lechnology

Mechmization

Ranking

I

2

3

4

5

6

7

8

9

29

Table 2b Ranking Assigned to Research Areas by the Weighted-Criteria Model in Ecilador

Research Aggregate Ranking by Region Area Ranking I 2 3 4 5 6 789 9 10 11

Cultural Practices (crops) 7 I 8 6 8 9 t10 8 8 2 6

Agro-Forestry 2 2 2 2 1 3 3 4 3 2 7 7

Animal Nutrition 3 3 3 7 11 5 5 2 4 10 2 3

Soil Conservation 4 4 4 3 3 6 12 6 5 4 9 9

Socio-economics 10 5 6 8 4 7 6 15 11 5 3 10

eclmology Transfer 4 6 7 4 5 8 7 7 6 6 4 4

5d duction 4 7 5 5 6 10 8 3 7 7 5 5

Plant Breeding 1 8 1 1 7 1 1 1 1 1 1 1

Soil Fertility 7 9 9 9 9 2 2 11 2 9 6 2

Post-tlarvest technology 13 10 10 10 10 15 10 12 13 14 13 13

Plant Protection 9 11 12 12 2 4 4 5 9 3 8 8

Water Use 11 12 14 14 1 11 13 8 14 12 10 11

Animal lealth 16 13 16 16 11 16 15 16 16 16 16 16

Livestock Breeding amp Management 14 14 13 13 12 13 11 14 0 11 15 15

Aquaculture I1 15 15 15 14 14 14 9 12 13 11 12

Mechanization 15 16 11 11 16 12 16 13 15 15 14 14

Region 1= Region 2 = Region 3 =

Tropical and dry Subtropical Occidentad de Pichincha Fstacion Experimental Santo Domingo

Region 4 = Region 5 =

Subtropico Seco-Loja lrovincia del Ovo

Region 6 = Region 7 =

Elstacion Experimental PortoviejoEstacion Experimental Boliche

Region 8 = E E NAPO PayaminoRegion 9 = E ETropical PichilingueRegion 10 = EESanta Catalina Region 11= EEChuquipata

30

Table 2c Ranking Assigned to Research Areas by the Weighilted-Criteria Model in Uruguay

Research Area

Plant Protection Agroclimatology Plant Breeding Livestock Management Livestock Nutrition Production Systems Pasture Management Soil Management Animal Ilealth Plant Nutrition amp PhysiologyCultural Practices Sociocconomics Animal Reproduction valuation of Products

Animal Improvement IPost-I larest Tech

Region I El Nort Region 2 = la Fstanzuela Region 3 = Salto Rcoon 4 = [ as Brujas Region 5 = Fst

Aggregate Ranking

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

Ranking by Region1 2 3 4 5 7 7 1 10 1

11 9 2 4 5 12 1 S 1 6 2 5 11 13 7 9 4 13 11 8 8 3 10 7 12 4 6 15 14 2

13 2 4 8 3 3 11 16 16 4

14 13 5 9 9 5 8 3 6 11

10 14 7 3 13 1 10 12 12 10

15 16 9 5 16 6 12 14 15 14

16 Is 6 2 15

31

Table 3 Summary of Internal Rates of Return to INIPA Rescarch and Jxterjsio a )

RICELb) CORN WII EAT POTATOES BEANS AGGREGATP)

Research Investment

from 1981 to 1986

and Extension from

1981 to 1990

Free Trade

Pivotal Shift 17 10 18 17

in Stpply Curve

Parallel Shift 35 23 28 33

in Supply Curve

No Trade

Pivotal Shift 18 22 14

in Supply Curve

Prallel Shift 37 42 24

in Supply Curve

Research Investment

from 191 to 1992

and Extension from

LIU_Q 199

FreeTrade

Pivotal Shift 30 20 28 25

in Supply Curve

Parallel Shift 44 31 36 38

in Supply Curve

No Trade

Pivotal Shift 22 14

in Supply Curve

Parall Shift 42 24

IIIsupply Curve

(asrumes no expansion of cultivated area

(1vihen ropansion in cultivated area of 1deg1per year was assumed these rates more than doubled For example the return to research and extension on rice for the 1981 to 1986 research investment and 1981 to 19W) extension investment

changed from 17 to 48

(ceither Free Trade nor No Tra

32

Tble 4 Percent Distribution of Total Net Economic Surplus to INIPA Research and Extension for

No Trade ScenarioJO

Pivotal Supply Shift Parallel Supply Shift

Consumer Producer Consumer Producer

Gain Gain Gain Gain

Rice n = - 76 83 17

n = -39 105 -5 52 48

n = -27 115 -15

Potatoes n = -64 72 28

n = -34 88 12 44 56

n = -24 95 5

Beans n = - 61 74 26

n = -31 (1 9 46 54

n = -21 99 1

KlUbenefits accrue to producers for the trade scenario for rice corn and wheat when the price elasticity

of demand is perfectly elastic

33

Table lb Ranking of Research Priorities by Commodity from the Weighted-Criteria Mode

Applied in Ecuador

Commodity

Arroz Cafe Maiz Sierra Cebada platanoPapa Baiano Cacao Ganado de Leche laiz Costs

Trigo Fruitas lioja Caduca Frejol Ganado de Carrie Lcguminosas and Mfen Yuca Fruitas Citricos ILenteja Lcgurainosas Costa Cercalcs dinos Porcinos Fruitales Tropicales

Cereales Menores Cultivos wtd Nlenores Main Ovinos Fniitaes Subtropical fIort de Clima Ffio IlorlTropicalAyes Caprinos Chontaduro Soya Palna Africana Sorgo Algodon

Especies Menores Oleaginosas Menores Cana Ifort Subtropical Tabaco [ores Jojoba e

Rank

I 2 3 4 5 6 7 8 9 10

11 12 13 14 15 16 17 18 19 20 21 22

23 24 25 26 27 28 29 30 31 32 33 34 35 36

37 38 39 40 41 42 43 44

Priority Level

High

Medium

Low

None

27

Table Ic Ranking of Research Priorities by Commodity from the Weighted-Criteria Model

Applied in Uruguay (Preliminary Results) Commodity

Beef Cattle Wheat Forages

Dairy Citrus Potatoes Maize SheepRice Other Fruits Grapes Sunlloers Soybeans Vegetables

Sorghum Barley Tomatoes Swine Peanuts Linseed Grain legumes Frutilla

Rank

I 2 3

4 5 6 7 8 9

10 II 12 13 14

15 16 17 18 19 20 21 22

Priority Level

tHligh

Medium

Low

28

Table 2a Ranking Assigned to Research Aeas by the Weighted-Critera Model in the Dominican Republic

Research Area

Natural Resources (sods water forests)

Plant Breeding Cultural Practices Crop Systems

Amral Nutrition and Mxnagement

Iechnology Transfer

Animal genetics and reproduction

Socioeconomics

Plant Protection

Post lwuvest lechnology

Mechmization

Ranking

I

2

3

4

5

6

7

8

9

29

Table 2b Ranking Assigned to Research Areas by the Weighted-Criteria Model in Ecilador

Research Aggregate Ranking by Region Area Ranking I 2 3 4 5 6 789 9 10 11

Cultural Practices (crops) 7 I 8 6 8 9 t10 8 8 2 6

Agro-Forestry 2 2 2 2 1 3 3 4 3 2 7 7

Animal Nutrition 3 3 3 7 11 5 5 2 4 10 2 3

Soil Conservation 4 4 4 3 3 6 12 6 5 4 9 9

Socio-economics 10 5 6 8 4 7 6 15 11 5 3 10

eclmology Transfer 4 6 7 4 5 8 7 7 6 6 4 4

5d duction 4 7 5 5 6 10 8 3 7 7 5 5

Plant Breeding 1 8 1 1 7 1 1 1 1 1 1 1

Soil Fertility 7 9 9 9 9 2 2 11 2 9 6 2

Post-tlarvest technology 13 10 10 10 10 15 10 12 13 14 13 13

Plant Protection 9 11 12 12 2 4 4 5 9 3 8 8

Water Use 11 12 14 14 1 11 13 8 14 12 10 11

Animal lealth 16 13 16 16 11 16 15 16 16 16 16 16

Livestock Breeding amp Management 14 14 13 13 12 13 11 14 0 11 15 15

Aquaculture I1 15 15 15 14 14 14 9 12 13 11 12

Mechanization 15 16 11 11 16 12 16 13 15 15 14 14

Region 1= Region 2 = Region 3 =

Tropical and dry Subtropical Occidentad de Pichincha Fstacion Experimental Santo Domingo

Region 4 = Region 5 =

Subtropico Seco-Loja lrovincia del Ovo

Region 6 = Region 7 =

Elstacion Experimental PortoviejoEstacion Experimental Boliche

Region 8 = E E NAPO PayaminoRegion 9 = E ETropical PichilingueRegion 10 = EESanta Catalina Region 11= EEChuquipata

30

Table 2c Ranking Assigned to Research Areas by the Weighilted-Criteria Model in Uruguay

Research Area

Plant Protection Agroclimatology Plant Breeding Livestock Management Livestock Nutrition Production Systems Pasture Management Soil Management Animal Ilealth Plant Nutrition amp PhysiologyCultural Practices Sociocconomics Animal Reproduction valuation of Products

Animal Improvement IPost-I larest Tech

Region I El Nort Region 2 = la Fstanzuela Region 3 = Salto Rcoon 4 = [ as Brujas Region 5 = Fst

Aggregate Ranking

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

Ranking by Region1 2 3 4 5 7 7 1 10 1

11 9 2 4 5 12 1 S 1 6 2 5 11 13 7 9 4 13 11 8 8 3 10 7 12 4 6 15 14 2

13 2 4 8 3 3 11 16 16 4

14 13 5 9 9 5 8 3 6 11

10 14 7 3 13 1 10 12 12 10

15 16 9 5 16 6 12 14 15 14

16 Is 6 2 15

31

Table 3 Summary of Internal Rates of Return to INIPA Rescarch and Jxterjsio a )

RICELb) CORN WII EAT POTATOES BEANS AGGREGATP)

Research Investment

from 1981 to 1986

and Extension from

1981 to 1990

Free Trade

Pivotal Shift 17 10 18 17

in Stpply Curve

Parallel Shift 35 23 28 33

in Supply Curve

No Trade

Pivotal Shift 18 22 14

in Supply Curve

Prallel Shift 37 42 24

in Supply Curve

Research Investment

from 191 to 1992

and Extension from

LIU_Q 199

FreeTrade

Pivotal Shift 30 20 28 25

in Supply Curve

Parallel Shift 44 31 36 38

in Supply Curve

No Trade

Pivotal Shift 22 14

in Supply Curve

Parall Shift 42 24

IIIsupply Curve

(asrumes no expansion of cultivated area

(1vihen ropansion in cultivated area of 1deg1per year was assumed these rates more than doubled For example the return to research and extension on rice for the 1981 to 1986 research investment and 1981 to 19W) extension investment

changed from 17 to 48

(ceither Free Trade nor No Tra

32

Tble 4 Percent Distribution of Total Net Economic Surplus to INIPA Research and Extension for

No Trade ScenarioJO

Pivotal Supply Shift Parallel Supply Shift

Consumer Producer Consumer Producer

Gain Gain Gain Gain

Rice n = - 76 83 17

n = -39 105 -5 52 48

n = -27 115 -15

Potatoes n = -64 72 28

n = -34 88 12 44 56

n = -24 95 5

Beans n = - 61 74 26

n = -31 (1 9 46 54

n = -21 99 1

KlUbenefits accrue to producers for the trade scenario for rice corn and wheat when the price elasticity

of demand is perfectly elastic

33

Table Ic Ranking of Research Priorities by Commodity from the Weighted-Criteria Model

Applied in Uruguay (Preliminary Results) Commodity

Beef Cattle Wheat Forages

Dairy Citrus Potatoes Maize SheepRice Other Fruits Grapes Sunlloers Soybeans Vegetables

Sorghum Barley Tomatoes Swine Peanuts Linseed Grain legumes Frutilla

Rank

I 2 3

4 5 6 7 8 9

10 II 12 13 14

15 16 17 18 19 20 21 22

Priority Level

tHligh

Medium

Low

28

Table 2a Ranking Assigned to Research Aeas by the Weighted-Critera Model in the Dominican Republic

Research Area

Natural Resources (sods water forests)

Plant Breeding Cultural Practices Crop Systems

Amral Nutrition and Mxnagement

Iechnology Transfer

Animal genetics and reproduction

Socioeconomics

Plant Protection

Post lwuvest lechnology

Mechmization

Ranking

I

2

3

4

5

6

7

8

9

29

Table 2b Ranking Assigned to Research Areas by the Weighted-Criteria Model in Ecilador

Research Aggregate Ranking by Region Area Ranking I 2 3 4 5 6 789 9 10 11

Cultural Practices (crops) 7 I 8 6 8 9 t10 8 8 2 6

Agro-Forestry 2 2 2 2 1 3 3 4 3 2 7 7

Animal Nutrition 3 3 3 7 11 5 5 2 4 10 2 3

Soil Conservation 4 4 4 3 3 6 12 6 5 4 9 9

Socio-economics 10 5 6 8 4 7 6 15 11 5 3 10

eclmology Transfer 4 6 7 4 5 8 7 7 6 6 4 4

5d duction 4 7 5 5 6 10 8 3 7 7 5 5

Plant Breeding 1 8 1 1 7 1 1 1 1 1 1 1

Soil Fertility 7 9 9 9 9 2 2 11 2 9 6 2

Post-tlarvest technology 13 10 10 10 10 15 10 12 13 14 13 13

Plant Protection 9 11 12 12 2 4 4 5 9 3 8 8

Water Use 11 12 14 14 1 11 13 8 14 12 10 11

Animal lealth 16 13 16 16 11 16 15 16 16 16 16 16

Livestock Breeding amp Management 14 14 13 13 12 13 11 14 0 11 15 15

Aquaculture I1 15 15 15 14 14 14 9 12 13 11 12

Mechanization 15 16 11 11 16 12 16 13 15 15 14 14

Region 1= Region 2 = Region 3 =

Tropical and dry Subtropical Occidentad de Pichincha Fstacion Experimental Santo Domingo

Region 4 = Region 5 =

Subtropico Seco-Loja lrovincia del Ovo

Region 6 = Region 7 =

Elstacion Experimental PortoviejoEstacion Experimental Boliche

Region 8 = E E NAPO PayaminoRegion 9 = E ETropical PichilingueRegion 10 = EESanta Catalina Region 11= EEChuquipata

30

Table 2c Ranking Assigned to Research Areas by the Weighilted-Criteria Model in Uruguay

Research Area

Plant Protection Agroclimatology Plant Breeding Livestock Management Livestock Nutrition Production Systems Pasture Management Soil Management Animal Ilealth Plant Nutrition amp PhysiologyCultural Practices Sociocconomics Animal Reproduction valuation of Products

Animal Improvement IPost-I larest Tech

Region I El Nort Region 2 = la Fstanzuela Region 3 = Salto Rcoon 4 = [ as Brujas Region 5 = Fst

Aggregate Ranking

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

Ranking by Region1 2 3 4 5 7 7 1 10 1

11 9 2 4 5 12 1 S 1 6 2 5 11 13 7 9 4 13 11 8 8 3 10 7 12 4 6 15 14 2

13 2 4 8 3 3 11 16 16 4

14 13 5 9 9 5 8 3 6 11

10 14 7 3 13 1 10 12 12 10

15 16 9 5 16 6 12 14 15 14

16 Is 6 2 15

31

Table 3 Summary of Internal Rates of Return to INIPA Rescarch and Jxterjsio a )

RICELb) CORN WII EAT POTATOES BEANS AGGREGATP)

Research Investment

from 1981 to 1986

and Extension from

1981 to 1990

Free Trade

Pivotal Shift 17 10 18 17

in Stpply Curve

Parallel Shift 35 23 28 33

in Supply Curve

No Trade

Pivotal Shift 18 22 14

in Supply Curve

Prallel Shift 37 42 24

in Supply Curve

Research Investment

from 191 to 1992

and Extension from

LIU_Q 199

FreeTrade

Pivotal Shift 30 20 28 25

in Supply Curve

Parallel Shift 44 31 36 38

in Supply Curve

No Trade

Pivotal Shift 22 14

in Supply Curve

Parall Shift 42 24

IIIsupply Curve

(asrumes no expansion of cultivated area

(1vihen ropansion in cultivated area of 1deg1per year was assumed these rates more than doubled For example the return to research and extension on rice for the 1981 to 1986 research investment and 1981 to 19W) extension investment

changed from 17 to 48

(ceither Free Trade nor No Tra

32

Tble 4 Percent Distribution of Total Net Economic Surplus to INIPA Research and Extension for

No Trade ScenarioJO

Pivotal Supply Shift Parallel Supply Shift

Consumer Producer Consumer Producer

Gain Gain Gain Gain

Rice n = - 76 83 17

n = -39 105 -5 52 48

n = -27 115 -15

Potatoes n = -64 72 28

n = -34 88 12 44 56

n = -24 95 5

Beans n = - 61 74 26

n = -31 (1 9 46 54

n = -21 99 1

KlUbenefits accrue to producers for the trade scenario for rice corn and wheat when the price elasticity

of demand is perfectly elastic

33

Table 2a Ranking Assigned to Research Aeas by the Weighted-Critera Model in the Dominican Republic

Research Area

Natural Resources (sods water forests)

Plant Breeding Cultural Practices Crop Systems

Amral Nutrition and Mxnagement

Iechnology Transfer

Animal genetics and reproduction

Socioeconomics

Plant Protection

Post lwuvest lechnology

Mechmization

Ranking

I

2

3

4

5

6

7

8

9

29

Table 2b Ranking Assigned to Research Areas by the Weighted-Criteria Model in Ecilador

Research Aggregate Ranking by Region Area Ranking I 2 3 4 5 6 789 9 10 11

Cultural Practices (crops) 7 I 8 6 8 9 t10 8 8 2 6

Agro-Forestry 2 2 2 2 1 3 3 4 3 2 7 7

Animal Nutrition 3 3 3 7 11 5 5 2 4 10 2 3

Soil Conservation 4 4 4 3 3 6 12 6 5 4 9 9

Socio-economics 10 5 6 8 4 7 6 15 11 5 3 10

eclmology Transfer 4 6 7 4 5 8 7 7 6 6 4 4

5d duction 4 7 5 5 6 10 8 3 7 7 5 5

Plant Breeding 1 8 1 1 7 1 1 1 1 1 1 1

Soil Fertility 7 9 9 9 9 2 2 11 2 9 6 2

Post-tlarvest technology 13 10 10 10 10 15 10 12 13 14 13 13

Plant Protection 9 11 12 12 2 4 4 5 9 3 8 8

Water Use 11 12 14 14 1 11 13 8 14 12 10 11

Animal lealth 16 13 16 16 11 16 15 16 16 16 16 16

Livestock Breeding amp Management 14 14 13 13 12 13 11 14 0 11 15 15

Aquaculture I1 15 15 15 14 14 14 9 12 13 11 12

Mechanization 15 16 11 11 16 12 16 13 15 15 14 14

Region 1= Region 2 = Region 3 =

Tropical and dry Subtropical Occidentad de Pichincha Fstacion Experimental Santo Domingo

Region 4 = Region 5 =

Subtropico Seco-Loja lrovincia del Ovo

Region 6 = Region 7 =

Elstacion Experimental PortoviejoEstacion Experimental Boliche

Region 8 = E E NAPO PayaminoRegion 9 = E ETropical PichilingueRegion 10 = EESanta Catalina Region 11= EEChuquipata

30

Table 2c Ranking Assigned to Research Areas by the Weighilted-Criteria Model in Uruguay

Research Area

Plant Protection Agroclimatology Plant Breeding Livestock Management Livestock Nutrition Production Systems Pasture Management Soil Management Animal Ilealth Plant Nutrition amp PhysiologyCultural Practices Sociocconomics Animal Reproduction valuation of Products

Animal Improvement IPost-I larest Tech

Region I El Nort Region 2 = la Fstanzuela Region 3 = Salto Rcoon 4 = [ as Brujas Region 5 = Fst

Aggregate Ranking

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

Ranking by Region1 2 3 4 5 7 7 1 10 1

11 9 2 4 5 12 1 S 1 6 2 5 11 13 7 9 4 13 11 8 8 3 10 7 12 4 6 15 14 2

13 2 4 8 3 3 11 16 16 4

14 13 5 9 9 5 8 3 6 11

10 14 7 3 13 1 10 12 12 10

15 16 9 5 16 6 12 14 15 14

16 Is 6 2 15

31

Table 3 Summary of Internal Rates of Return to INIPA Rescarch and Jxterjsio a )

RICELb) CORN WII EAT POTATOES BEANS AGGREGATP)

Research Investment

from 1981 to 1986

and Extension from

1981 to 1990

Free Trade

Pivotal Shift 17 10 18 17

in Stpply Curve

Parallel Shift 35 23 28 33

in Supply Curve

No Trade

Pivotal Shift 18 22 14

in Supply Curve

Prallel Shift 37 42 24

in Supply Curve

Research Investment

from 191 to 1992

and Extension from

LIU_Q 199

FreeTrade

Pivotal Shift 30 20 28 25

in Supply Curve

Parallel Shift 44 31 36 38

in Supply Curve

No Trade

Pivotal Shift 22 14

in Supply Curve

Parall Shift 42 24

IIIsupply Curve

(asrumes no expansion of cultivated area

(1vihen ropansion in cultivated area of 1deg1per year was assumed these rates more than doubled For example the return to research and extension on rice for the 1981 to 1986 research investment and 1981 to 19W) extension investment

changed from 17 to 48

(ceither Free Trade nor No Tra

32

Tble 4 Percent Distribution of Total Net Economic Surplus to INIPA Research and Extension for

No Trade ScenarioJO

Pivotal Supply Shift Parallel Supply Shift

Consumer Producer Consumer Producer

Gain Gain Gain Gain

Rice n = - 76 83 17

n = -39 105 -5 52 48

n = -27 115 -15

Potatoes n = -64 72 28

n = -34 88 12 44 56

n = -24 95 5

Beans n = - 61 74 26

n = -31 (1 9 46 54

n = -21 99 1

KlUbenefits accrue to producers for the trade scenario for rice corn and wheat when the price elasticity

of demand is perfectly elastic

33

Table 2b Ranking Assigned to Research Areas by the Weighted-Criteria Model in Ecilador

Research Aggregate Ranking by Region Area Ranking I 2 3 4 5 6 789 9 10 11

Cultural Practices (crops) 7 I 8 6 8 9 t10 8 8 2 6

Agro-Forestry 2 2 2 2 1 3 3 4 3 2 7 7

Animal Nutrition 3 3 3 7 11 5 5 2 4 10 2 3

Soil Conservation 4 4 4 3 3 6 12 6 5 4 9 9

Socio-economics 10 5 6 8 4 7 6 15 11 5 3 10

eclmology Transfer 4 6 7 4 5 8 7 7 6 6 4 4

5d duction 4 7 5 5 6 10 8 3 7 7 5 5

Plant Breeding 1 8 1 1 7 1 1 1 1 1 1 1

Soil Fertility 7 9 9 9 9 2 2 11 2 9 6 2

Post-tlarvest technology 13 10 10 10 10 15 10 12 13 14 13 13

Plant Protection 9 11 12 12 2 4 4 5 9 3 8 8

Water Use 11 12 14 14 1 11 13 8 14 12 10 11

Animal lealth 16 13 16 16 11 16 15 16 16 16 16 16

Livestock Breeding amp Management 14 14 13 13 12 13 11 14 0 11 15 15

Aquaculture I1 15 15 15 14 14 14 9 12 13 11 12

Mechanization 15 16 11 11 16 12 16 13 15 15 14 14

Region 1= Region 2 = Region 3 =

Tropical and dry Subtropical Occidentad de Pichincha Fstacion Experimental Santo Domingo

Region 4 = Region 5 =

Subtropico Seco-Loja lrovincia del Ovo

Region 6 = Region 7 =

Elstacion Experimental PortoviejoEstacion Experimental Boliche

Region 8 = E E NAPO PayaminoRegion 9 = E ETropical PichilingueRegion 10 = EESanta Catalina Region 11= EEChuquipata

30

Table 2c Ranking Assigned to Research Areas by the Weighilted-Criteria Model in Uruguay

Research Area

Plant Protection Agroclimatology Plant Breeding Livestock Management Livestock Nutrition Production Systems Pasture Management Soil Management Animal Ilealth Plant Nutrition amp PhysiologyCultural Practices Sociocconomics Animal Reproduction valuation of Products

Animal Improvement IPost-I larest Tech

Region I El Nort Region 2 = la Fstanzuela Region 3 = Salto Rcoon 4 = [ as Brujas Region 5 = Fst

Aggregate Ranking

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

Ranking by Region1 2 3 4 5 7 7 1 10 1

11 9 2 4 5 12 1 S 1 6 2 5 11 13 7 9 4 13 11 8 8 3 10 7 12 4 6 15 14 2

13 2 4 8 3 3 11 16 16 4

14 13 5 9 9 5 8 3 6 11

10 14 7 3 13 1 10 12 12 10

15 16 9 5 16 6 12 14 15 14

16 Is 6 2 15

31

Table 3 Summary of Internal Rates of Return to INIPA Rescarch and Jxterjsio a )

RICELb) CORN WII EAT POTATOES BEANS AGGREGATP)

Research Investment

from 1981 to 1986

and Extension from

1981 to 1990

Free Trade

Pivotal Shift 17 10 18 17

in Stpply Curve

Parallel Shift 35 23 28 33

in Supply Curve

No Trade

Pivotal Shift 18 22 14

in Supply Curve

Prallel Shift 37 42 24

in Supply Curve

Research Investment

from 191 to 1992

and Extension from

LIU_Q 199

FreeTrade

Pivotal Shift 30 20 28 25

in Supply Curve

Parallel Shift 44 31 36 38

in Supply Curve

No Trade

Pivotal Shift 22 14

in Supply Curve

Parall Shift 42 24

IIIsupply Curve

(asrumes no expansion of cultivated area

(1vihen ropansion in cultivated area of 1deg1per year was assumed these rates more than doubled For example the return to research and extension on rice for the 1981 to 1986 research investment and 1981 to 19W) extension investment

changed from 17 to 48

(ceither Free Trade nor No Tra

32

Tble 4 Percent Distribution of Total Net Economic Surplus to INIPA Research and Extension for

No Trade ScenarioJO

Pivotal Supply Shift Parallel Supply Shift

Consumer Producer Consumer Producer

Gain Gain Gain Gain

Rice n = - 76 83 17

n = -39 105 -5 52 48

n = -27 115 -15

Potatoes n = -64 72 28

n = -34 88 12 44 56

n = -24 95 5

Beans n = - 61 74 26

n = -31 (1 9 46 54

n = -21 99 1

KlUbenefits accrue to producers for the trade scenario for rice corn and wheat when the price elasticity

of demand is perfectly elastic

33

Table 2c Ranking Assigned to Research Areas by the Weighilted-Criteria Model in Uruguay

Research Area

Plant Protection Agroclimatology Plant Breeding Livestock Management Livestock Nutrition Production Systems Pasture Management Soil Management Animal Ilealth Plant Nutrition amp PhysiologyCultural Practices Sociocconomics Animal Reproduction valuation of Products

Animal Improvement IPost-I larest Tech

Region I El Nort Region 2 = la Fstanzuela Region 3 = Salto Rcoon 4 = [ as Brujas Region 5 = Fst

Aggregate Ranking

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

Ranking by Region1 2 3 4 5 7 7 1 10 1

11 9 2 4 5 12 1 S 1 6 2 5 11 13 7 9 4 13 11 8 8 3 10 7 12 4 6 15 14 2

13 2 4 8 3 3 11 16 16 4

14 13 5 9 9 5 8 3 6 11

10 14 7 3 13 1 10 12 12 10

15 16 9 5 16 6 12 14 15 14

16 Is 6 2 15

31

Table 3 Summary of Internal Rates of Return to INIPA Rescarch and Jxterjsio a )

RICELb) CORN WII EAT POTATOES BEANS AGGREGATP)

Research Investment

from 1981 to 1986

and Extension from

1981 to 1990

Free Trade

Pivotal Shift 17 10 18 17

in Stpply Curve

Parallel Shift 35 23 28 33

in Supply Curve

No Trade

Pivotal Shift 18 22 14

in Supply Curve

Prallel Shift 37 42 24

in Supply Curve

Research Investment

from 191 to 1992

and Extension from

LIU_Q 199

FreeTrade

Pivotal Shift 30 20 28 25

in Supply Curve

Parallel Shift 44 31 36 38

in Supply Curve

No Trade

Pivotal Shift 22 14

in Supply Curve

Parall Shift 42 24

IIIsupply Curve

(asrumes no expansion of cultivated area

(1vihen ropansion in cultivated area of 1deg1per year was assumed these rates more than doubled For example the return to research and extension on rice for the 1981 to 1986 research investment and 1981 to 19W) extension investment

changed from 17 to 48

(ceither Free Trade nor No Tra

32

Tble 4 Percent Distribution of Total Net Economic Surplus to INIPA Research and Extension for

No Trade ScenarioJO

Pivotal Supply Shift Parallel Supply Shift

Consumer Producer Consumer Producer

Gain Gain Gain Gain

Rice n = - 76 83 17

n = -39 105 -5 52 48

n = -27 115 -15

Potatoes n = -64 72 28

n = -34 88 12 44 56

n = -24 95 5

Beans n = - 61 74 26

n = -31 (1 9 46 54

n = -21 99 1

KlUbenefits accrue to producers for the trade scenario for rice corn and wheat when the price elasticity

of demand is perfectly elastic

33

Table 3 Summary of Internal Rates of Return to INIPA Rescarch and Jxterjsio a )

RICELb) CORN WII EAT POTATOES BEANS AGGREGATP)

Research Investment

from 1981 to 1986

and Extension from

1981 to 1990

Free Trade

Pivotal Shift 17 10 18 17

in Stpply Curve

Parallel Shift 35 23 28 33

in Supply Curve

No Trade

Pivotal Shift 18 22 14

in Supply Curve

Prallel Shift 37 42 24

in Supply Curve

Research Investment

from 191 to 1992

and Extension from

LIU_Q 199

FreeTrade

Pivotal Shift 30 20 28 25

in Supply Curve

Parallel Shift 44 31 36 38

in Supply Curve

No Trade

Pivotal Shift 22 14

in Supply Curve

Parall Shift 42 24

IIIsupply Curve

(asrumes no expansion of cultivated area

(1vihen ropansion in cultivated area of 1deg1per year was assumed these rates more than doubled For example the return to research and extension on rice for the 1981 to 1986 research investment and 1981 to 19W) extension investment

changed from 17 to 48

(ceither Free Trade nor No Tra

32

Tble 4 Percent Distribution of Total Net Economic Surplus to INIPA Research and Extension for

No Trade ScenarioJO

Pivotal Supply Shift Parallel Supply Shift

Consumer Producer Consumer Producer

Gain Gain Gain Gain

Rice n = - 76 83 17

n = -39 105 -5 52 48

n = -27 115 -15

Potatoes n = -64 72 28

n = -34 88 12 44 56

n = -24 95 5

Beans n = - 61 74 26

n = -31 (1 9 46 54

n = -21 99 1

KlUbenefits accrue to producers for the trade scenario for rice corn and wheat when the price elasticity

of demand is perfectly elastic

33

Tble 4 Percent Distribution of Total Net Economic Surplus to INIPA Research and Extension for

No Trade ScenarioJO

Pivotal Supply Shift Parallel Supply Shift

Consumer Producer Consumer Producer

Gain Gain Gain Gain

Rice n = - 76 83 17

n = -39 105 -5 52 48

n = -27 115 -15

Potatoes n = -64 72 28

n = -34 88 12 44 56

n = -24 95 5

Beans n = - 61 74 26

n = -31 (1 9 46 54

n = -21 99 1

KlUbenefits accrue to producers for the trade scenario for rice corn and wheat when the price elasticity

of demand is perfectly elastic

33