IEEE TRANSACTIONS ON ENGINEERING MANAGEMENT, VOL. EM-21, NO. 4, NOVEMBER 1974 ]

A Review of Some Recent Developments in Portfolio Modelling in Applied Research and Development

A. E. GEAR

Abstract-At the present time, very few organizations ap­pear to be using the more complex, portfolio based, resource allocation models proposed in the literature. It is the objective of this paper to discuss some extensions and improvements to a basic linear programming approach to de­cision making in R&D, pointing out their weaknesses whether of a theoretical or practical nature.

INTRODUCTION There is a growing body of management science literature

which deals with the subject of applying mathematically based decision models to the planning of research and development activities. Theoretical reviews have been given by Baker [1], Cetron [2] and Gear [3]. A survey into the evaluation and control procedures used by 112 companies from the "Times 300" was carried out by Allen [4] in 1968. Allen found that only sixteen companies did not use either net present value, payback period, or average rate of return on capital invested when assessing the commercial viability of projects. He men­tions that fourteen of these estimated "probability of com­mercial success" so it maybe assumed that some other form of cash-flow analysis was employed.

In his survey, Allen found that about one-third of the organizations used mathematical models and/or weighted check-lists or project ranking indices as a part of their project assessment procedure. Only about one-quarter of the organi­zations did not use networks, bar charts or graphs to specify time-scales for some part of their R&D programme, with a definite tendency for the more rapidly changing and complex industries to employ these techniques the most.

At the present time very few organizations appear to be using the more complex, portfolio based, resource allocation models proposed in the theoretical literature (reviewed in references [1], [2], [3]). These models have considerable appeal from a theoretical viewpoint. It is the objective of this paper to discuss some extensions and improvements to a basic linear programming approach to decision-making in R&D, pointing out their weaknesses whether of a theoretical or practical nature.

PORTFOLIO PLANNING The planning problem facing the R&D Director of an orga­

nization is to semi-continuously change or modify previous decisions concerned with the allocation of resources to

Manuscript received March 1974; revised June 1974. An earlier version of this paper was presented to the Southern Operational Re­search Group of the Operational Research Society, December 1972.

The author is with The Open University, Milton Keynes, England.

projects, and the acquisition of further resources, in order to maintain compatibility with the organizational objectives as a whole. The portfolio selection problem is created by the probability that a number of project proposals together with existing projects, will make demands on resources in excess of current and forecast capabilities. The object of this discussion is to present and analyze some approaches to the problem.

For each project, whether potential or current, there will be some time interval during which it can be undertaken. Each project will require a set of resources over time which is unlikely to be precisely known at any stage of its life. Similar­ly each project will eventually produce a time series of benefits to the organization which are usually only estimated when decisions regarding selection and continuation have to be made.

Faced with this exceedingly complex problem, a number of authors, for example Bell [5] and [6], Beattie [7] and Waiters [8] have developed models based on linear and/or integer programming frameworks which have only small variations from each other. For present purposes it is sufficient to outline the structure of the model of Bell [6] in the following section.

Bell's Model This model requires a linear and/or integer programming

algorithm for its analysis, and structurally it represents an extension of capital budgeting models (eg., see Weingartner [9] ) to accommodate the special features of an R&D situa­tion. The model has been used by the North Eastern Region of the Central Electricity Generating Board as an aid to portfolio selection and recruitment policy (Read [10] ).

Main features of the model are that it allows planning of projects over a number of future time periods; caters for a variety of resource constraints, particularly\manpower types; considers alternative versions of projects; allows for the flex­ibility of certain manpower types to work in areas other than that in which they are principally categorized, as necessity demands; and includes decision variables representing recruit­ment options.

Mathematically, each project is allowed alternative versions which correspond to different rates of progress (fast, medium, slow, etc), to alternative start delays and to alternative tech­nical approaches. The ;'th version of project i is allocated a variable, x,y, in such a way that solution of the model produces a value for Χη as follows:

Xif = 1 ; indicates selection Xij = 0; indicates rejection.

120 IEEE TRANSACTIONS ON ENGINEERING MANAGEMENT, NOVEMBER 1974

Apart from the aspects of the model concerned with re­cruitment and flexibility of manpower, the formulation is

n mi maximize Z = Ç Σ οηΧ$

subject to; m/ Σ xq< 1 ; i=l, ·■·#!., 7=1

n m/ Σ Σ aijkpXij < Akp ; fc=l,. · . N.f 1 = 1 / = 1 Ρ=1, · · ·Λ, (1)

x,y = 0or l ; i=l , · · ·« , and/=1, · · · ,/w/where bij is the value (or utility) of version / of project /. mi is the number of alternative versions of project i. n is the number of projects, both on-going and new at

the time of running the model. üißp is the amount of resource type k planned (or re­

quired) for version; of project / in time period p. Afcp is the overall availability of resource type k in period

p. N is the number of resource categories considered

(money, facilities, manpower types, etc.). P is the number of planning periods into which the

overall planning period is divided from the present up to a defined "planning horizon."

The model can easily be extended to include the recruit­ment and degree of transfer of each manpower type as a series of additional decision variables whose values are found by the model during the optimization process involved in maximizing Z, taking account of the cost of recruitment. These extensions are described by Lockett [11] and Gear [12].

Allen [13] has described a way of allowing for treating ey­as an uncertain quantity, but having a subjectively based probability distribution. This approach is essentially based on repeated sampling of the ft,y values and repeated linear pro­gramming runs. Difficulties arise, however, in interpreting re­sults, and thus in advising management of the portfolio to select.

If the model is solved using a linear programming algorithm (0 < Xij < 1), some of the solution values of xzy can be fractional. The problem of interpreting fractional solutions is discussed by Bell [14]. In practice, most of the solution variables naturally appear as 0 or 1 so that the problem is not usually serious.

Discussion of Bell's Model An advantage of Bell's model is that it has been tested as to

feasibility in several locations (see references [11] and [14]). Another is that the approach caters for a multiplicity of resource types and can break up future time into a number of planning periods. More complex models based on dynamic programming, such as that described by Hess [15], do not appear capable of extension to multiple resource constraints without incurring prohibitive computational problems. How­ever, the model of Hess does allow the muiti-stage nature of many projects to be incorporated in the analysis. Bell's model, and nesfly all others, make the implicit assumption that once

selected a project is continued to completion, ignoring the sequence of decisions which may be involved during the evolu­tion of a project. These decisions may be during the R&D phase of the project and/or during its subsequent exploitation.

Another defect of Bell's model is that it assumes that the time series of resource requirements for each project version are exactly known in advance, and that a single value to represent the benefit of undertaking each version can be derived. In practice both resource requirements and benefits are likely to be based on uncertain estimates about the future course of events, making Bell's deterministic framework diffi­cult to reconcile with reality.

Practically all the models assume that only one objective is involved in the resource allocation problem. In practice most situations involve more than one objective, and so benefits (or disbenefits) in more than one area affect the worthiness of each project. For example, in road research the benefits of projects may be concerned with reducing accidents and journey times. In industry, returns in the short and/or longer term are likely to be important, as well as other factors such as the working capital involved at the implementation stage(s).

The following sections discuss ways of overcoming the defects of Bell's model which have been outlined. There is no intention to imply that these are the only shortcomings of the model, though they are certainly important ones. It is shown that the sequential nature and uncertain aspects of projects can be considered by using a decision tree format to plan projects, and several approaches to modelling multiple objec­tive situations are presented.

DECISION TREE DIAGRAMMING The representational framework of decision trees allows the

multi-stage nature, and uncertainties connected with each project, to be displayed diagrammatically. Lockett [16] has suggested this approach and called the resulting diagram a "project tree." A project tree, taken from reference [16] is shown in Fig. 1, which extends over three time periods and involves two resource types. The project has a decision as to the level of effort to apply in period 1, followed by an uncertain intermediate technical outcome. If the higher level

KEY

Decision Node

!

ΐ .

Chance Node

<x: Estimated resource consumption

probability

probability

Amount of ! Amount of Typel Type 2

Fig. 1. Project tree for project /.

GEAR: PORTFOLIO MODELLING IN APPLIED R&D 121

of effort is selected in period 1, and the outcome with prob­ability 0.7 results, then a further decision stage is reached regarding the level of effort for the following two time periods.

The project tree format allows each opportunity, with its own pattern of chance and choice milestones, to be planned on a common, pre-determined time scale divided into discrete intervals. It is assumed that the intervals are fine enough to assume that milestones occur only at their beginnings. The method allows a great variety of project types to be repre­sented. For example, at one extreme is the basic research project, from which spin-off development projects are ex­pected from time to time with a small probability but high potential value. At the other extreme a development project with just a starting decision can be displayed. A set of six projects is shown in Fig. 2 from reference [16], which demonstrates the flexibility of the approach to represent projects with alternative features and strategies. The method can represent uncertainties in the duration, resource inputs, technical and commercial outcome at each stage, and project objective(s). Project trees obtained from a number of field studies are described by Gillespie [17].

Project 1

Project 2

Project 3

Project 4

Project 5

Project 6

L-CO

Overall resource

availability

fc^St 1 ' 0.1

HUT I I o.

nnzoiixzn Fig. 2. Example problem of 6 project trees.

Analysis If a portfolio of opportunities is modelled using the project

tree approach, and resources may be scarce, the resource allocation problem can be stated as: to what sub-set of oppor­tunities should resources be allocated in the first time interval in order to be on an optimal path in terms of the overall objective(s) of the laboratory?

Assuming that the objective is to maximize the overall expected value of the sum of terminal values of the projects which are completed, Gear [18] has shown that the problem can be analyzed as a stochastic linear programming problem. Furthermore, there is no inherent difficulty to adding uncer­tainties associated with resource availabilities or penalties asso­ciated with over-runs to the problem.

Using the six project situation of Fig. 2 as an example, the formulation of the problem as an extended linear pro­gramming tableau is shown in Table I. Rows 1 to 6 ensure that each project is not selected in more than one starting version; row 7 ensures that the first period availability of the single resource type involved (10 units) is not exceeded; rows 8 to 19 ensure that the resource availability of 9 and 8 units is not exceeded in time periods two and three for all possible futures; rows 20 to 29 ensure that the sequencing of projects 4 and 6 is maintained under all possible futures; row 30 is the objective function. A full explanation of the tableau is given in reference [18].

The tableau can be solved using a linear or integer pro­gramming algorithm. Ine integer and linear solutions of the

TABLE I Linear Programming Tableau under Uncertainty

Row Not.

1 2 3 4 5 6 7 8 9

10 11 12 13 14

1 i « 16 17 18 19 20 21

I — 23 24 25

) 26 27 28 29

-^Variables

N a m e \ ^ ^ PROJ 1 PROJ 2 PROJ 3 PROJ 4 PROJ 5 PROJ 6 PERIOD 1 PE 2 FUT 1 P E 2 F U T 2 P E 2 F U T 3 PE 2 FUT 4 PE 2 FUT 1 P E 3 F U T 2 PE 3 FUT 3 " 3 FUT 4 P E 3 F U T 5 P E 3 F U T 6 PE 3 FUT 7 P E 3 F U T 8 PROJ 4 FUT 1 PR0J4 FUT 2 PROJ£ FUT î PROJ6FUT2 PROJ6 FUT 3 PROJ6FUT4 PROJ6FUT5 PROJ6FUT6 PROJ6 FUT 7 PROJ6 FUT 8 OBJECTIVE

,-1

2 3 3 3 3 3 3 3 3 3 3 3 3

5

j-

1

0 3 3 3 3 3 3 3 3 3 3 3 3

8

a

1

3 3 3 3 3 0 0 0 0 0 0 0 0

10

£

1

4

€ 1

2 2 ? 2 2 0 0 0 0 0 0 0 0

6

f 1

0 2 ? 2 2 2 2 2 2 ? 2 2 2

5

£

1

3 1 1

1 J_

4.5

£

1

1 3 fi 3 6 3 3 6 6 3 3 6 6

_.

9

-

1 1

K[~

...__,

.,._ ■ \

■1 • 1 -1

1-1 [•T ■1

0

.

10

ΗΞ

2 2

j

—

2.45

£̂

2 2

±..

2.45

i?

0 0

1

0

|

d o 1

1

1°

£

~y

ί

[ 0.08

X

il

—

0.54

: : iZ

j _ _

0.06

£

£ 2._

u. 1 _

0.54

£ £

| ! — t - -

'-Ί::: τ\-

1* ! 1

__!_" i.~_

1

0.14| 1.26

i 1 ] $

i I 1 i '

...... . . . i . . . r . J

i l i . ! 1 J « i 1 ! 1

r—: -i J . ^"4_f - T- -

j - - i 4 '

2 — . 1 2 _

. ._ r i j ! . . . j _ . .

j

1

__u 0.U|V26

u 4 _.

! ' ■

1—| - i | 1

0.135

1 ! 1

! t j ■

1.215 0135

1 £ S

; j

• ) ... - 4

;;:. : ) _«._+ --·-

.„ 4—.

— ...

1

1.215

1

0.315

L—„ ' 4

£ ί

..... — f -

! -~i — i _ j 4 i

~"

J

2S3S

μ θ ! 1

I ! t'

| 1

1 0.315J2.835

r o

e

« * £

* * «f

i-

i-* . 1 .

- i _

I

:

«

< X I n i

oc 551 1 1

] 1_J 1 1 1

10 9 9_J 9 | 9 !

" 8 " ] O 8 ! 8 8 8 8

" 8 0 0 0 0 0 0 0 Q 0 0

122 IEEE TRANSACTIONS ON ENGINEERING MANAGEMENT, NOVEMBER 1974

TABLE II Integer and Linear Programming Solutions to the

Tableau of Table I

Variable

χΠ,

x 2 1 ,

x22,

x 3 1 ,

x32,

* 3 3 1

x41 ,

x 5 1 |

x 6 1 ,

x621

x 4 1 2 1 2

x 4 , 2 2 2

x 4 2 2 1 2

x42 2 2 2 I

X6 I3 I3

x61 3 2 3

x61 3 3 3

x61 3 4 3

"61 3 5 3

x61 3 7 3

x61 3 8 3

x 6 2 3 1 3

x62 3 2 3

κ62 3 3 3

x62 3 4 3

x62 3 5 3

x62363

x62 3 7 3

x62 3 8 3

I Portfolio

| Value

Integer Solution J

1

1

1

1

1.0

1.0

1.0

1.0

1.0

1.0

1.0

1.0

1.0

1.0

32~4

Linear Solut ion j

Ö44

1.0

1.0

1.0

0.11

1.0

0.5

1.0

0.5

1.0

i 1 0

1.0

1.0

1.0

1.0

1.0

1.0

319

First

stage

decision

variables

Future

stage

decision

variables

6- project problem are shown in Table II. The solutions give the optimal set of first period decisions in the light of the flexibility to modify subsequent actions in the light of chance outcomes. In practice, once the first period solution is decided, the model would be up-dated and re-run at the commencement of peripd 2.

The above approach to analyzing a portfolio of project trees under conditions of uncertainty and resource scarcity can easily lead to exceedingly large problems involving many thousands of rows and columns. An alternative approach to the analysis of large problems, based on the use of heuristics and simulation, has been suggested by Lockett [16] and tested on data from a field study, Lockett [19]. This approach will usually generate a number of "good" solutions, leaving the decision-taker to make a final selection. More recently, Lockett [20] has developed a promising approach to analyzing large problems of this type based on the use of heuristics to find upper and lower bounding solutions. It is too early to say whether this method will prove desirable with data from real situations rather than with the fictitious data employed to develop the method.

MULTIPLE OBJECTIVES

Introduction In many real-life problems in which management science is

seeking to aid decision-takers to reach optimal decisions from among large numbers of feasible options, the problem of

trading off one benefit or dis-benefit against another often arises. A given action (or decision) leads to a set of conse­quences (outcomes) in a number of heterogeneous areas.

The general problem is one of searching for an optimal solution in a multiple objective situation. But 'optimal' no longer has a clear-cut analytical meaning, for mathematically one can only optimize with respect to one objective function at a time. Some examples of this type of problem are as follows:

Production-maximize profits, minimize defectives, etc. Transportation-maximize delivery speed, minimize de­

livery costs, etc. Resource Allocation—minimize total cost, minimize slack

resources, maximize short term benefits, maximize long term benefits, minimize risks, etc.

Road Research—maximize lives saved, maximize injuries prevented, minimize road building and maintenance costs, etc.

Telecommunications—minimize cost of service, maximize level of service

Health Services-minimize cost of service, maximize level of service.

The above examples demonstrate the common problem of conflicting objectives. The R&D decision-taker is confronted with this type of situation in both the private and public sectors. In what follows, a number of approaches to the problem are introduced. Essentially all the methods are seek­ing a "best" compromise rather than an optimal solution. The approaches discussed may be categorized into:

1. Objective ordering 2. Indifference surface determination 3. Cost/Benefit analysis 4. Utility maximization 5. Best compromise.

Approaches (i) Objective Ordering: This method involves the following steps. 1. List objectives in order of decreasing importance/

relevance. 2. Optimize with respect to the first objective on the list

only. 3. Study the solution to this problem to decide if the other

objectives on the list are met at a "satisfactory" level. 4. If the solution for step 3 is not satisfactory, continue

the process by optimizing with respect to the second objective in the list, and so on.

Hopefully this process will generate a satisfactory solution, or suggest the addition of constraint rows to improve the solution to a satisfactory level.

(ii) Indifference Surfaces: This method involves treating all the objectives other than

the principal one as constraints. By changing the degree to which the constraints operate in a systematic way a family of feasible solutions is generated. Final selection from this set is then left to the decision-taker.

(Hi) Cost j Benefit Analysis: All the costs and benefits associated with the objectives are

GEAR: PORTFOLIO MODELLING IN APPLIED R&D 123

converted into monetary terms using unit costs. This involves assigning costs (or monetary benefits) to such heterogeneous variables as time, lives, amenity, environment, etc. Having done this the problem is reduced to maximizing a single monetary function which can be handled mathematically. But the assignment of unit monetary values to the factors involved inevitably introduces many arguable assumptions. For example, how does one discount the value of a life lost in, say, 1980 in order to compare with the value of a life lost in the present?

(iv) Maximize Expected Utility: This approach is a straightforward extension of utility

theory in which it is stated that the decision-takers' objective is to maximize the function E[U(R)], where E represents take an expectation of U(R), and U is the utility to the decision-taker of portfolio return R. When the return is multi­dimensional, the function generalizes to:

maximum £[£/(R)], where

R = ( / ? ! . . . , / ? / . . . , * „ ) . (2)

That is, utility U is a function of n attributes or benefit areas. The problem becomes extremely difficult to handle practically due to the difficulty of determining U(R), which involves subjective questioning to discover indifference levels between simple alternative options presented to the decision-taker. If n is greater than 3 this becomes a weighty undertaking. The resultant function, because it covers a large volume of R-space, is normally non-linear. So even if the function is isolated, specially designed methods of finding the optimal solution usually have to be employed. If additive utilities are assumed, i.e.,

U(R) = £/,(/?!) . . . Uf(Rf) . . . + Un(Rn) (3)

the problem simplifies, but can of course still be non-linear. The approach handles uncertainty of return. For a given

decision, there is an exhaustive set of chance outcomes Rj, R2, etc., with probabilities of occurrence P\,P2, etc. where Σ/>=1.

Then the objective is:

Max Σ U(R)P. (4) all outcomes

(v) Best Compromise: This normative approach has been suggested independently

by Freeman [21] and by Benayoun [22]. The method as­sumes that a "best compromise" solution exists which leads to a benefit in each area usually less than the maximum attain­able, yet optimal in terms of an overall tradeoff. Because this approach appears powerful, the mathematical formulation is now outlined. An example application is described in reference [21].

Formulation of Best Compromise Approach Let Xi (/=1, · · ♦ ,iV)be the variable representing project /, and

for each-project define: eft—the resource requirement of project i under resource

category k. (fc=l, · · · , K). bjj—Ûie benefit of undertaking project i as measured

against criterion / (in arbitrary units). / is the number of benefit and/or dis-benefit areas, (pi, ··,«/)·

In addition define: Wj —the linear weight attached to objective /. Bj —the portfolio objective value for criterion /. Q —the availability of resource type k.

With these definitions it is usual to adopt the following formulation:

P N 1 Maximize! Σ w,· Σ x{bii I ·

L'=1 i=i J N

Subject to Σ XjCik < C#, fc=l, · · · , K f = l

o <X(< 1, /=1, · - · ,7V(and possibly integral). (5)

This is a special case of maximizing a utility function in which a linear weighted sum is assumed. Thus we have fixed values for the weights uy. It would be an improvement to drop this restriction on the uy and allow each wj to be a function, fj, of (B\, · · ·,7?y).However, in the best compromise approach it is not necessary to find the functions, fj. Instead it is assumed that the weights are locally linear near B*, where

B* = (**/, ·..,/?*/)> (6)

and B*j is the optimum value of equations (5) but with objective:

N Maximize Σ χφη. (7)

f = l

In general B* will not correspond to a feasible solution with given constraints. But for any B corresponding to a feasible solution we can calculate:

Bj*-Bi B1*

Clearly ABj, which lies in the range 0 <Bj <1 , can be interpreted as a normalized measure of the suboptimality (regret) in benefit area/. Our objective is now taken to be that of choosing the JC/ such that the ABj are as close to zero as possible. This is still a multiple objective problem.

If Wj is now interpreted as a weight representing the relative amount of normalized sub-optimality in benefit area /, near B*, a "best compromise" problem can be formulated as:

Minimize R subject to N Σ x£ik<Ck9 fc=l,...,tf.,

0 < Xf < 1, i= 1, · · · , N. (and possibly integral),

ABjWj^R, 7 = 1 , · · , / . (9)

This formulation corresponds to choosing a portfolio of projects with limited resources which minimizes the maximum allowable regret.

Alternatively, the portfolio of projects which minimizes the

124 IEEE TRANSACTIONS ON ENGINEERING MANAGEMENT, NOVEMBER 1974

weighted sum of the regrets can be found if the objective function

Maximize Σ ABjWj, (10) 7 = 1

is adopted, with the constraints as for equations (5). In practice, the weights in the region of B* may be obtained by subjective questioning with the following relationship in mind:

ABj wj = constant, (11)

where AZ?*represents a "small" reduction in Bj at B*. It is important to note that the best compromise ap­

proaches, by assuming that the relationship [11] holds over the region of solution space of interest, lead to linear pro­gramming formulations, ((9) and (10)), of the multiple objec­tive problem.

To close this section, there is a risk that the management scientist will allow the manager to impose "sloppy" thinking as an easy way out by introducing multiple objectives unneces­sarily. A careful check must be made to ensure that the factors cannot be put onto a common (eg., monetary) footing. On the other hand, there can be no final agreement, and on moral grounds perhaps one should not seek one, for equivalence of, say, lives saved and road building costs.

MODEL-BUILDING AND ANALYSIS In the previous sections, a method of representing the

sequential and uncertain aspects of many R&D projects has been described, and some methods of handling multiple objective problems have been outlined. However, in many practical situations both project trees and multiple objectives will be involved. Now, it has been shown that the problems, either alone or in combination, can be formulatetLâs a linear programme. But if more than a few projects are involved the resultant tableau can become very large, perhaps running into millions of rows and columns, far beyond the capacity of existing methods of solution.

Several courses of action are open to the model-builder faced with this situation. Firstly, he can deliberately reduce the size of the problem by making simplifying assumptions and solving a simpler problem. Or, he can abandon the idea of seeking an "optimal solution," and instead explore the possi­bility of developing heuristic rules and other approaches which lead to sufficiently good solutions to the original problem. In the former case, a methodology is required to test the effects of simplifying the problem, and in the latter case practical procedures are required. Some suggestions along these lines are presented in the following two sections.

A Model-Building Methodology In order to make decisions in any situation some form of

model of that situation is required. The model, which may be purely a subjective idealization of the real world in the decision-taker's mind, is an attempt to abstract those factors which he feels have the most bearing on the decision in question. If the situation is complex, the decision-taker may build, or have built, some form of mathematical model with which to explore the options involved. The model may be very simple, requiring little data and analysis, or, at the other extreme, a very detailed and complex set of relationships

Fig. 3. Flow diagram of activities to compare models A and B.

representing a close approach to "reality." Between these extremes lie a great number of alternative models which could be used as a basis for decision-taking. The question is, in an uncertain world, which model is a sufficient representation of the actual situation in order to allow the decision-taker to reach a "good" decision? This model selection problem faces all model-builders in management science, and is usually ap­proached on an intuitive basis. A formalized approach to the problem is outlined in the following paragraphs, based on a paper by Gillespie [23], in which some examples of field study applications are described.

The elements of a mathematical model are the objective function, the constraints, and the decision-variables, and so models may differ in one or more of these aspects. Given a block of data relating to a particular situational decision problem, two models A and £, will in general lead to different solutions, £(,4) and S(B). If followed, due to uncertainties, the actual outcome of following the decisions indicated by either solution can only be expressed as a probability distribution of the objective function in each case. Gillespie [23] describes a process for deriving the probability distributions of the solu­tion of two models in a form which is suitable for comparison (Fig. 3), and suggests some quantitative tests of significance. The tests are based on

1. the area of overlap of the two probability distributions 2. the probability of a random sample from one dis­

tribution exceeding the value of a random sample from the other

3. the difference between the probabilities of each distri­bution exceeding a defined value of the objective func­tion.

An Inter-Active Procedure Faced with the complex problem of deciding and con­

tinually modifying a complete R&D programme, the decision-takers have to use their judgment and experience to allocate (and acquire additional) resources. In a situation with many uncertainties and a multiplicity of objectives, perhaps the best assistance that a model can provide is simply to give a forward looking summary of the consequences of taking a given set of decisions on the set of projects involved. This could be in graphical form, summarizing many factors on a time scale, particularly the resource requirements, probability of over­runs, and benefits of a given set of decisions.

If all the projects are represented in the form of project

GEAR: PORTFOLIO MODELLING IN APPLIED R&D 125

I miiiiiiniH i * i ι · i f I i I i I i i i i I

1971/72 1972/73 1973/74 1974/75 1975/76 1976/77 1977/78 1978/79 1979/80 Year

Fig. 4. Overall royalty receipts from the current set of sponsored projects. The mean and sidebands at plus or minus one standard error are shown.

trees as outlined in Lockett [16], then this approach becomes computationally feasible. Ideally, the decision-taker(s) would interact directly with the computer using a terminal in the office, exploring a number of alternative sets of decisions until a "good compromise" is reached, taking account of the value of benefits in different objective areas as well as resource requirements. Part of the output for a set of 12 projects taken from a field study is shown in Fig. 4.

SUMMARY AND CONCLUSIONS The great flexibility of decision tree diagramming and linear

programming formulations to handle many of the complex aspects of decision-modelling in R&D are apparently demon­strated from a theoretical viewpoint. But, in practice, approx­imations may be necessary in order to make the resulting problem soluble using existing analytical procedures. Also the data demands become onerous as the model becomes larger and more complex. There is an outstanding need to develop methodologies for assessing and comparing alternative de­cision-models of a situation.

There is also an urgent need to test various models in field studies. The adoption of a model by an organization, while continuing with traditional approaches to decision-taking, will help to pin-point problems of data collection and estimation as well as highlighting limitations of the model itself. With time and patience this consistency of data input and analysis over time appears highly likely to yield improvements.

ACKNOWLEDGMENT I wish to thank the following for many helpful discussions

and co-operation over a number of years: Dr. D. Allen, J. Allen, D. C. Bell, C. J. Beattie, Dr. P. Freeman, J. S. Gillespie, D. Hunter, P. Johnson, A. G. Lockett, R. Moore, K. P. Norris, A. W. Pearson, S. Rainbow, Dr. A. W. Read, S. Teal. Needless to say, this list is far from complete.

REFERENCES [1] Baker, N. R. and Pound, W. H. "R&D Project Selection: Where

we stand." IEEE Trans. Eng. Manag., voLEM-11, Dec. 1964, pp. 124-134.

[2] Cetron, M. J., Martino, J. and Roepeke, L. "The selection of R&D program content—Survey of quantitative methods." IEEE Trans. Eng. Manag., vol. EM-14, March 1967, pp. 4-13.

[3] Gear, A. E., Lockert, A. G. and Pearson, A. W. "An analysis of some portfolio selection models for research and development." IEEE Trans. Eng. Manag. voL EM-18, May 1971.

[4] Allen, J. M. "A survey into the R&D evaluation and control procedures currently used in industry." /. of Industrial Eco­nomics, vol. 18, No. 2,1970. pp. 161-181.

[5] BeU, D. C, Chilcott, J. F., Read, A. W. and Salway, R. A. "Applications of a research project selection method in the north eastern region scientific services department." United Kingdom C.E.G.B. R&D Department Report No. RD/H/R2.

[6] Bell, D. C. and Read, A. W. "The application of a research project selection method." R&D Management, vol. 1, No. 1,1970.

[7] Beattie, C. J. "Allocating resources to research in practice." In the proceedings of the NATO Conference entitled Applications of Mathematical Programming Techniques", June 1968. Pub. by English Universities Press.

[8] Watters, L. D. "Research and development project selection: Interdependence and multi-period probabilistic budget con­straints." PhD dissertation. Arizona State University. Tempe, 1967.

[9] Weingartner, H. M. "Mathematical Programming and the Analysis of Capital Budgeting Problems." Englewood Cliffs, N.J. Prentice-Hall, 1963.

[ 10] Read, A. W. Private communication. [11] Lockett, A. G. and Gear, A. E. "Programme selection in research

and development." Manag. Sci. vol. 18, No. 10. June 1972 [12] Gear, A. E., Gillespie, J. S., Lockett, A. G. and Pearson, A. W.

"Manpower Modelling: A Study in Research and Development." Conference of the Inst. of Manag. Sci. London, July 1970. Published by English Universities Press.

[13] Allen, D. and Johnson, T. "Realism in LP modelling for project selection." R&D Management, vol. 1, No. 2, 1971.

[14] Bell, D. C. Freeman, P., Gear, A. E., Lockett, A. G., Rainbow, S. and Read, A. W. "Resource allocation modelling." Presented at a Conference entitled "Practical aids to research management," organized by the Operational Research Society, London, Feb­ruary 1970.

[15] Hess, S. W. "A dynamic programming approach to R&D budget­ing and project selection." IRE Trans, on Eng. Manag., vol. EM-9, 1962 pp. 170-179.

[16] Lockett, A. G. and Gear, A. E. "An approach to dynamic modelling in R&D." Presented at the 38th Meeting of the Operations Research Society of America. Detroit, October, 1970.

[17] Gillespie, J. S. and Gear, A. E. "Decision networks in applied research and development." Presented at the INTERNET Con­ference, Stockholm, May 1972.

[18] Gear, A. E. and Lockett, A. G. "A dynamic model of some multi-stage aspects of research and development portfolios." IEEE Trans, on Eng. Manag, vol. EM-20, No. 1, 1973.

[19] Lockett, A. G. and Gear, A. E. "Representation and analysis of multi-stage problems in R&D." Manag. Sci vol. 19, No. 8 pp. 947-960,1973.

[20] Lockett, A. G. and Muhlemann, A. P. "Solution bounds on a method of project selection using discrete stochastic integer programming with recourse." Presented at the Joint ORS A/TIMS Conference, Boston, Mass., April 1974.

[21] Freeman, P. "Development of linear programming models for portfolio selection in research and development." PhD disserta­tion. Manchester University, 1972.

[22] Benauoun, R., de Montgolfier, J., Tergny, J. and Laritchev, O. "Linear programming with multiple objectives: Step method (STEM)." Mathematical Programming, vol. 1,1971. pp. 366-375.

[23] Gillespie, J. S. and Gear, A. E. "An analytical methodology for comparing the suitability of management science models." IEEE Trans, on Eng. Man., vol. EM-20, No. 4 ,1973.