lida2007 hayes

Upload: anurag-sharma

Post on 06-Apr-2018

229 views

Category:

Documents


0 download

TRANSCRIPT

  • 8/2/2019 Lida2007 Hayes

    1/128

    Data Envelopment Analysis

    Robert M. Hayes

  • 8/2/2019 Lida2007 Hayes

    2/128

    Overview

    Introduction

    Data Envelopment Analysis

    DEA Models

    Extensions to includea priori Valuations

    Strengths and Weaknesses of DEA

    Implementation of DEA

    The Example of Libraries

    Annals of Operations Research 66

    Annals of Operations Research 73

  • 8/2/2019 Lida2007 Hayes

    3/128

    Introduction

    Utility Functions

    Cost/Effectiveness

    Interpretation for Libraries

  • 8/2/2019 Lida2007 Hayes

    4/128

    Utility Functions

    A fundamental requirement in applying operations research

    models is the identification of a "utility function" whichcombines all variables relevant to a decision problem into asingle variable which is to be optimized. Underlying theconcept of a utility function is the view that it should representthe decision-maker's perceptions of the relative importance ofthe variables involved rather than being regarded as uniformacross all decision-makers or externally imposed.

    The problem, of course, is that the resulting utility functionsmay bear no relationship to each other and it is therefore

    difficult to make comparisons from one decision context toanother. Indeed, not only may it not be possible to comparetwo different decision-makers but it may not be possible tocompare the utility functions of a single decision-maker fromone context to another.

  • 8/2/2019 Lida2007 Hayes

    5/128

    Cost/Effectiveness

    A traditional way to combine variables in a utility

    function is to use a cost/effectiveness ratio, called an

    "efficiency" measure. It measures utility by the "cost

    per unit produced". On the surface, that would appearto make comparison between two contexts possible by

    comparing the two cost/effectiveness ratios. The

    problem, though, is that two different decision-makers

    may place different emphases on the two variables.

  • 8/2/2019 Lida2007 Hayes

    6/128

    Cost/Effectiveness

    It also must be recognized that effectiveness will usuallyentail consideration of a number of products and servicesand costs a number of sources of costs. Cost/effectivenessmeasurement requires combining the sources of cost intoa single measure of cost and the products and services

    into a single measure of effectiveness. Again, the problem of different emphases of importance

    must be recognized. This is especially the case for theseveral measures of effectiveness. But it may also be thecase with the several measure of costs, since some costs

    may be regarded as more important than others eventhough they may all be measured in dollars. When somecosts cannot be measured in dollars, the problem iscompounded.

  • 8/2/2019 Lida2007 Hayes

    7/128

    Cost/Effectiveness

    More generally, instead of costs and effectiveness, the

    variables may be identified as "input" and "output".

    The efficiency ratio is then no long characterized as

    cost/effectiveness but as "output/input", but the issues

    identified above are the same.

  • 8/2/2019 Lida2007 Hayes

    8/128

    Interpretation for Libraries

    This issue can be illustrated by evaluating libraryperformance. Effectiveness here is the extent to whichlibrary services meet the expectations or goals set bythe organization served. It is likely to be measured by

    several services which are the outputs of libraryoperationsmaking a collection available for use,circulation or other uses of materials, answering ofinformation questions, instructing and consulting.

    Inputs are represented by acquisitions, staff, and space,to which evident costs can be assigned, but they are alsorepresented by measures of the populations served.

  • 8/2/2019 Lida2007 Hayes

    9/128

    Interpretation for Libraries

    Efficiency measures the librarys ability to transformits inputs (resources and demands) into production ofoutputs (services). The objective in doing so is tooptimize the balance between the level of outputs and

    the level of inputs. The success of the library, like thatof other organizations, depends on its ability to behaveboth effectively and efficiently.

    The issue at hand, then is how to combine the several

    measures of input and output into a single measure ofefficiency. The method we will examine is that called"data envelopment analysis".

  • 8/2/2019 Lida2007 Hayes

    10/128

    Data Envelopment Analysis

    Data Envelopment Analysis (DEA) measures the relativeefficiencies of organizations with multiple inputs andmultiple outputs. The organizations are called thedecision-making units, or DMUs.

    DEA assigns weights to the inputs and outputs of a DMUthat give it the best possible efficiency. It thus arrives at aweighting of the relative importance of the input andoutput variables that reflects the emphasis that appears tohave been placed on them for that particular DMU.

    At the same time, though, DEA then gives all the otherDMUs the same weights and compares the resultingefficiencies with that for the DMU of focus.

  • 8/2/2019 Lida2007 Hayes

    11/128

    Data Envelopment Analysis

    If the focus DMU looks at least as good as any otherDMU, it receives a maximum efficiency score. But if

    some other DMU looks better than the focus DMU, the

    weights having been calculated to be most favorable to

    the focus DMU, then it will receive an efficiency score

    less than maximum.

  • 8/2/2019 Lida2007 Hayes

    12/128

    Graphical Illustration To illustrate, consider seven DMUs which each have

    one input and one output: L1 = (2,2), L2 = (3,5), L3 =(6,7), L4 = (9,8), L5 = (5,3), L6 = (4,1), L7 = (10,7).

    0

    1

    2

    3

    4

    5

    6

    7

    8

    9

    0 1 2 3 4 5 6 7 8 9 10 11

    Input

    Output

    L1

    L2

    L3

    L4

    L5

    L6

    L7

  • 8/2/2019 Lida2007 Hayes

    13/128

    Graphical Illustration

    DEA identifies the units in the comparison set which lieat the top and to the left, as represented by L1, L2, L3,and L4. These units are called the efficient units, andthe line connecting them is called the "envelopmentsurface" because it envelops all the cases.

    DMUs L5 through L7 are not on the envelopmentsurface and thus are evaluated as inefficient by theDEA analysis. There are two ways to explain theirweakness. One is to say that, for example, L5 could

    perhaps produce as much output as it does, but withless input (comparing with L1 and L2); the other is tosay it could produce more output with the same input(comparing with L2 and L3).

  • 8/2/2019 Lida2007 Hayes

    14/128

    Graphical Illustration

    Thus, there are two possible definitions of efficiencydepending on the purpose of the evaluation. One mightbe interested in possible reduction of inputs (in DEAthis is called the input orientation) or augmentation ofoutputs (the output orientation) in achieving technicalefficiency. Depending on the purpose of the evaluation,the analysis provides different sets of peer groups towhich to compare.

    However, there are times when reduction of inputs oraugmentation of outputs is not sufficient. In our

    example, even when L6 reduces its input from 4 unitsto 2, there is still a gap between it and its peer L1 in theamount of one unit of output. In DEA, this is called the"slack" which means excess input or missing outputthat exists even after the proportional change in the

    input or the outputs.

  • 8/2/2019 Lida2007 Hayes

    15/128

    Graphical Illustration

    This example will be used to illustrate the several forms

    that the DEA model can take.

    In each case, the results presented are based on the

    implementation of DEA that will be discussed later in

    this presentation. It is an Excel spreadsheet using the

    add-in Solver capability.

    The spreadsheet is identical for all of the forms, but the

    application of Solver differs in the target optimized and

    in the values to be varied, so for each form the target

    and the values to be varied will be identified.

  • 8/2/2019 Lida2007 Hayes

    16/128

    DEA Models

    The Basic EDA Concept

    Variations of DEA Formulation

    Formulation: Primal or Dual Orientation: Input or Output

    Returns to Scale: Fixed or Variable

  • 8/2/2019 Lida2007 Hayes

    17/128

    The Basic EDA Concept

    Assume that each DMU has values for a set of inputsand a set of outputs.

    Choose non-negative weights to be applied to the inputsand outputs for a focus DMU so as to maximize theratio of weighted outputs divided by weighted inputs

    But do so subject to the condition that, if the sameweights are applied to each of the DMUs (including thefocus DMU), the corresponding ratios are not greaterthan 1

    Do that for each DMU.

    The resulting value of the ratio for each DMU is itsEDA efficiency. It is 1 if the DMU is efficient and lessthan 1 if it is not.

  • 8/2/2019 Lida2007 Hayes

    18/128

    Formulation

    Let (Yk,Xk) = (Yki,Xkj), k = 1 to n, i = 1 to s, j = 1 to m

    Maximize Yk/Xk for each value of k from 1 to n,subject in each case to Yj/nXj = 0

    The solution is the set of maximum values for Yk/Xkand the associated values for and

  • 8/2/2019 Lida2007 Hayes

    19/128

    Basic Linear Programming Model

    For solution, this optimization problem is transformedinto a linear programming problem, schematicallydisplayed as follows:

    In a moment, we will interpret this display as it istranslated into alternative formulations of theoptimization target and conditional inequalities.

    Min Yj -Xj

  • 8/2/2019 Lida2007 Hayes

    20/128

    Variations of DEA Formulation

    But first, it is necessary to identify several sources ofvariation in the basic DEA formulation, leading to avariety of different models for implementation:

    We will now examine and illustrate each of thosesources of variation.

    (1) Formulation Primal Form Dual Form

    (2) Orientation Input Minimization Output Maximization(3) Returns to Scale Fixed Returns Variable Returns

    (4) Discretionary? Discretionary Variables Non-discretionary Variables

    (5) Models Additive Multiplicative

  • 8/2/2019 Lida2007 Hayes

    21/128

    (1) Formulation: Primal or Dual

    The first source of variation is interpretation of thedisplay for the linear programming model.

    One interpretation, called the Primal, treats the rows of

    the display as representing the model.

    The other interpretation, called the Dual, treats thecolumns as representing the model.

    Lets examine each of those in turn.

  • 8/2/2019 Lida2007 Hayes

    22/128

    Primal Formulation

    The rows of this display are interpreted as follows:

    (M) Maximize W = Yk

    Xk

    subject to

    (1) YjXj

  • 8/2/2019 Lida2007 Hayes

    23/128

    The Dual Formulation

    The Columns of this display are interpreted as follows:

    (m) Minimize W = -a - b subject to

    (1) Yja >= Yk (2) Xj - b >= -Xk

    (m) Yj -Xj 0a -I -I

    b -I -I

    >= >=

    Yk - Xk

  • 8/2/2019 Lida2007 Hayes

    24/128

    The Choice of Formulation

    Since the results from the two formulation are equal,though expressed differently, the choice between them

    is based on computational efficiency or, perhaps, ease

    of interpretation.

    The Dual form is more efficient in computation if the

    number of DMUs is large compared to the number of

    input and output variables. Note that the Primal form

    entails n conditions (n being the number of DMUs)

    which, in the Dual form, are replaced by just m + s

    conditions (m being the number of input variables ands, the number of output variables)

  • 8/2/2019 Lida2007 Hayes

    25/128

    Illustration

    To illustrate, consider the example previously presented. Thetarget to be minimized in the Dual form is W =ab. The

    values to be varied are (, a, b), or (,. The following table shows the solution for both forms:

    X Y W a b L1 2 2 - 1.33 1.33 = 0.67 1 1.67L2 3 5 0.00 0.00 = 00 1 1.67L3 6 7 - 3.00 3.00 = 00 1 1.67L4 9 8 - 7.00 7.00 = 00 1 1.67L5 5 3 - 5.33 5.33 = 1 1.67L6 4 1 - 5.67 5.67 = 1 1.67L7 10 7 - 9.67 9.67 = 1 1.67

  • 8/2/2019 Lida2007 Hayes

    26/128

    Illustration

    Graphically, the results are as follows:

    The maximum value for W, over all cases, is at L2, whereW = 0 and the ratio of Y/X is a maximum. The slack foreach other case is the vertical distance to the line which

    goes from the origin (0,0) through L2 (3,5).

    0

    5

    10

    15

    20

    25

    0 2 4 6 8 10 12

  • 8/2/2019 Lida2007 Hayes

    27/128

    (2) Orientation: Input or Output

    The second source of variation, orientation, providesthe means for focusing on minimizing input or on

    maximizing output.

    These represent two quite different objectives in

    making assessments of efficiency. Is the objective to be

    minimally expensive (e.g., to save money) or is it to be

    maximally effective?

  • 8/2/2019 Lida2007 Hayes

    28/128

    Orientation to Input

    The linear programming display for the inputorientation is as follows:

    It adds one additional condition, Xk

  • 8/2/2019 Lida2007 Hayes

    29/128

    Orientation to Input

    The resulting Dual formulation is as follows: (m) Minimize W = c-1 subject to

    (1) Yja >= Yk (2)Xjb + (c1)Xk >= -Xk or Xk + b = >=

    Max Yk - Xk

  • 8/2/2019 Lida2007 Hayes

    30/128

    Orientation to Input

    Continuing with the same example, the following table

    shows the solutions in both formulations. The target isW = c1. Values to be varied are now (, a, b, c) or( and .

    Note that L2 still dominates the solution, but the graph

    is now quite different,

    X Y W=c-1 a b L1 2 2 - 0.40 = 0.40 0.30 0.50L2 3 5 0.00 = 00 0.20 0.33L3 6 7 - 0.30 = 0 0.10 0.17L4 9 8 - 0.46 = 0 0.07 0.11L5 5 3 - 0.64 = 00 0.12 0.20L6 4 1 - 0.85 = 00 0.15 0.25L7 10 7 - 0.58 = 0 0.06 0.10

  • 8/2/2019 Lida2007 Hayes

    31/128

    Orientation to Input

    0

    2

    4

    6

    8

    10

    12

    0 2 4 6 8 10 12

  • 8/2/2019 Lida2007 Hayes

    32/128

    Orientation to Output

    The linear programming display for the outputorientation is as follows:

    It adds one additional condition, Yk

  • 8/2/2019 Lida2007 Hayes

    33/128

    Orientation to Output

    The resulting Dual formulation is as follows:

    (m) Minimize W = 1c subject to (1) Yja >= cYk (2)Xjb >=Xk or Xk + b = >=

    Max Yk - Xk

  • 8/2/2019 Lida2007 Hayes

    34/128

    Orientation to Output

    Continuing with the same example, the following table

    shows the solutions in both formulations. The target isW = 1c. Values to be varied are still (, a, b, c) or( and .

    Note that L2 still dominates the solution, but the graph

    is now quite different,

    X Y W=1-c a b L1 2 2 - 0.67 = 0.67 0.50 0.83L2 3 5 0.00 = 00 0.20 0.33L3 6 7 - 0.43 = 00 0.14 0.24L4 9 8 - 0.87 = 00 0.13 0.21L5 5 3 - 1.78 = 0.33 0.56L6 4 1 - 5.67 = 1.00 1.67L7 10 7 - 1.38 = 0.14 0.24

  • 8/2/2019 Lida2007 Hayes

    35/128

    Orientation to Output

    Note that the graphical display is identical to that forthe general form, though the interpretation is

    somewhat different (replacing efficiencies by slacks).

    0

    5

    10

    15

    20

    25

    0 2 4 6 8 10 12

  • 8/2/2019 Lida2007 Hayes

    36/128

    (3) Returns to Scale: Fixed or Variable

    The third basis for variation among DEA models isreturns to scale.

    The examples presented to this point have each involvedconstant returns to scale. That is, the ratio Y/X canbe replaced by the inequality YX

  • 8/2/2019 Lida2007 Hayes

    37/128

    Variable Returns to Scale, Basic Model

    The linear programming display for the basic DEAmodel is as follows:

    It adds the variable u to the display.

    u Min Yj -Xj I

  • 8/2/2019 Lida2007 Hayes

    38/128

    Variable Returns: Orientation to Input

    The linear programming display for the variablesreturns to scale, input orientation is as follows:

    It adds one additional condition, Xk

  • 8/2/2019 Lida2007 Hayes

    39/128

    Orientation to Input The resulting Dual formulation is as follows:

    (m) Minimize W = c-1 subject to (1) Yja >= Yk (2)Xjb + (c1)Xk >= -Xk or Xk + b = 1

    The new, third condition makes things interesting.

    u (m) Yj -Xj I 0a -I 0

    b -I 0

    c - 1 Xk I>= >=

    Max Yk - Xk I

  • 8/2/2019 Lida2007 Hayes

    40/128

    Orientation to Input

    Continuing with the same example, the

    following table shows the solutions in bothformulations. The target is W = c1. Values to

    be varied are now (, a, b, c) or (, , u.X Y W=c-1 a b u

    L1 2 2 0.00 0.00 =1.00 0.00 0.00 0.00L2 3 5 0.00 0.00 = 1.00 0.00 0.00 0.00L3 6 7 0.00 0.00 = 1.00 0.00 0.00 0.00L4 9 8 0.00 0.00 = 1.00 0.00 0.00 0.00L5 5 3 - 4.00 2.00 = 1.00 0.00 0.00 0.00L6 4 1 - 5.00 4.00 = 1.00 0.00 0.00 0.00L7 10 7 - 4.00 0.00 = 1.00 0.00 0.00 0.00

  • 8/2/2019 Lida2007 Hayes

    41/128

    Orientation to Input

    0

    2

    4

    6

    8

    10

    12

    0 2 4 6 8 10 12

  • 8/2/2019 Lida2007 Hayes

    42/128

    Orientation to Output

    The linear programming display for the outputorientation is as follows:

    It adds one additional condition, Yk

  • 8/2/2019 Lida2007 Hayes

    43/128

    Orientation to Output

    The resulting Dual formulation is as follows:

    (m) Minimize W = 1c subject to

    (1) Yja >= cYk (2)Xjb >=Xk or Xk + b = >=

    Max Yk - Xk

  • 8/2/2019 Lida2007 Hayes

    44/128

    Orientation to Output

    Continuing with the same example, the following tableshows the solutions in both formulations. The target isW = 1c. Values to be varied are still (, a, b, c) or( and .

    Note that L2 still dominates the solution, but the graph

    is now quite different,

    X Y W=1-c a b L1 2 2 - 0.67 = 0.67 0.50 0.83L2 3 5 0.00 = 00 0.20 0.33L3 6 7 - 0.43 = 00 0.14 0.24L4 9 8 - 0.87 = 00 0.13 0.21L5 5 3 - 1.78 = 0.33 0.56L6 4 1 - 5.67 = 1.00 1.67L7 10 7 - 1.38 = 0.14 0.24

  • 8/2/2019 Lida2007 Hayes

    45/128

    Orientation to Output

    Note that the graphical display is identical to that forthe general form, though the interpretation is

    somewhat different (replacing efficiencies by slacks).

    0

    5

    10

    15

    20

    25

    0 2 4 6 8 10 12

  • 8/2/2019 Lida2007 Hayes

    46/128

    Extensions to includea priori Valuations

    To this point, DEA has been essentially a mathematical process inwhich the data for input and output are taken as given, withoutfurther interpretation with respect to the reality of operations.

    But reality needs to be recognized, so there are several extensions

    that can be made to the basic DEA model, applicable to any of thevariations.

    They fall into seven categories:

    (1) Discretionary and Non-discretionary Variables

    (2) Categorical Variables

    (3)A priori restrictions on Weights (4) Relationships between Weights on Variables

    (5)A priori assessments of Efficient Units

    (6) Substitutability of Variables

    (7) Discrimination among Efficient Units

  • 8/2/2019 Lida2007 Hayes

    47/128

    Discretionary & Non-discretionary

    In identifying input and output variables, one wants toinclude all that are relevant to the operation. Forexample, the level of output is determined not only bywhat the unit itself does but by the size of the market towhich the output is delivered.

    The result, though, is that some relevant variables, suchas the size of the market, are not under the control ofmanagement. Such variables, called non-discretionary,are in contrast to those that are under managementcontrol, called discretionary.

    In assessing efficiency, all variables are considered, butin determining the criterion function to be maximized orminimized, only the discretionary variables are included.

  • 8/2/2019 Lida2007 Hayes

    48/128

    Categorical Variables

    In the DEA model as so far presented, the variables aretreated as essentially quantitative, but sometimes one

    would like to identify non-quantitative variables, such

    as ordinal or nominal variables.

    For example, one might like to compare institutions ofthe same type, such as public or private universities.

    This is accomplished by introducing categorical

    variables containing numbers for order or identifiers

    for names.

  • 8/2/2019 Lida2007 Hayes

    49/128

    A priori Restrictions on Weights

    In the model as presented, the weights are limited onlyby the requirements that they be non-negative.

    However, there may be reason to require that weightsbe further limited.

    For example, it may be felt that a given variablemustbe included in the assessment so its weight must have atleast a minimal value greater than zero. This mightrepresent an output that is essential in assessment.

    As another example, a variable may be such a large

    weight would over-emphasize itsa priori importance sothat there should be an upper limit on the weight. Thismight represent an output variable that is counter-productive.

  • 8/2/2019 Lida2007 Hayes

    50/128

    Relationships between Weights

    Sometimes,a priori knowledge may imply that there is

    a necessary relationship among variables. For example,

    an output variable may absolutely require some level of

    an input variable. Sucha priori knowledge may be expressed as a ratio

    between the weights assigned to the related variables.

  • 8/2/2019 Lida2007 Hayes

    51/128

    A priori assessments of Efficient Units

    Some DMUs may be regarded, based ona priori

    knowledge, as eminently efficient or notoriously

    inefficient. While one might argue about the validity of

    such a priori judgments, frequently they must berecognized.

    To do so, additional conditions may be imposed upon

    the choice of weights. For example, the condition

    Yj/nXj

  • 8/2/2019 Lida2007 Hayes

    52/128

    Substitutability of Variables

    A still unresolved issue is the means for representing

    substitutability of variables. For example, two input

    variables may represent two different type of labor

    which may be, to some extent, substitutable for eachother.

    How is such substitutability to be incorporated?

    Lets explore this issue a bit further since, by doing so,

    we can illuminate some additional perspectives on thebasic DEA model.

  • 8/2/2019 Lida2007 Hayes

    53/128

    Substitutability of Variables

    For simplicity in description, consider two input variables

    and a single output variable that has the same value forall DMUs. The graphic representation of the envelopmentsurface can now best be presented not in terms of therelationship between output and input, as previouslyshown, but between the variables of input.

    The two variables are Professional Staff and Non-Professional Staff. The assumption is that they arecompletely substitutable and that physicians differ only intheir styles of providing service, represented by the mix

    of the two means for doing so. The efficient DMUs are located on the red envelopment

    surface, which shows the minimums in use of variables.

  • 8/2/2019 Lida2007 Hayes

    54/128

    Substitutability of Variables

    0

    1

    2

    3

    4

    5

    6

    7

    8

    9

    10

    -1 0 1 2 3 4 5

    Professional Staff

    Non-ProfessionalStaff

    Style 1

    Style 4

    Style 2 Style 3

  • 8/2/2019 Lida2007 Hayes

    55/128

    Discrimination among Efficient Units

  • 8/2/2019 Lida2007 Hayes

    56/128

    Strengths & Weaknesses of DEA

    Strengths

    DEA can handle multiple inputs and multiple outputs

    DEA doesn't require relating inputs to outputs.

    Comparisons are directly against peers

    Inputs and outputs can have very different units

    Weaknesses

    Measurement error can cause significant problems

    DEA does not measure"absolute" efficiency

    Statistical tests are not applicable

    Large problems can be computationally intensive

  • 8/2/2019 Lida2007 Hayes

    57/128

    Implementation of DEA

    Structure

    Spreadsheet implementation

    Choice of Model

    Spreadsheet Structure

    Spreadsheet Calculations

    Solver Elements in Spreadsheet

    Visual Basic Program

    Access to the Implementation X

    The data included in the spreadsheet is for ARLlibraries in 1996.

    http://f/External%20Disk%20Backup/Fundamental%20Backup/My%20Writings/Class%20CD/PowerPoint%20Files/Economics%20PPT/DEA-ARL-Complete.xlshttp://dea-arl-complete.xls/http://dea-arl-complete.xls/http://f/External%20Disk%20Backup/Fundamental%20Backup/My%20Writings/Class%20CD/PowerPoint%20Files/Economics%20PPT/DEA-ARL-Complete.xls
  • 8/2/2019 Lida2007 Hayes

    58/128

    Choice of Model

    The spreadsheet includes means to identify the choice

    of model by means of three parameters: Form: Dual represented by 0 and Primal by 1

    Orientation: Addition by 0, Input by 1, Output by 2

    Convexity: No by 0, Yes by 1

    Given the specification, solution of the resulting modelis initiated by pressing Ctrl-q.

    The solution is effected by a Visual Basic program thatdetermines the model from the parameters and thenlaunches the Excel Add-In called Solver.

    The program then produces the output on Sheet 3 thatshows the results.

  • 8/2/2019 Lida2007 Hayes

    59/128

    Spreadsheet Structure

    The DEA Spreadsheet for application to ARL libraries

    consists of three main parts:

    (1) The source data, stored in cells B16:R117

    (2) The spreadsheet calculations, stored in cells D5:R15

    (3) The Solver related calculations, stored in cells

    B1:B15, A7:A117, T12:T117

    The source data consists of the 10 input and 5 output

    variables for each of the ARL institutions plus, in rowB16:R16, a set of normalizing factors, one for each of

    the variables.

  • 8/2/2019 Lida2007 Hayes

    60/128

    Spreadsheet Calculations

    The Spreadsheet calculations in D5:R14 can be

    illustrated by D5:D14 and N5:N14:

    C D5 Discretionary? 16 Weights 0.0000017

    8

    9 Comp =SUMPRODUCT(Mult,D17:D113)*D1610Slacks 15.207341022937811Mod Comp =D9+D10

    12=INDEX(C17:C126,MATCH($B$12,$B$17:$B$126,0),1) =INDEX(Data,MATCH($B$12,$B$17:$B$126,0),COLUMN()-3)*D1613 =D12*$B$1314 =IF($B$2=1,D13,D12)

  • 8/2/2019 Lida2007 Hayes

    61/128

    Spreadsheet Calculations

    The Spreadsheet calculations in D5:R14 can be

    illustrated by D5:D14 and N5:N14:

    C N5 Discretionary? 16 Weights 9.99999999999265E-077

    8

    9 Comp =SUMPRODUCT(Mult,N17:N113)*N1610Slacks 5.5626973172299511Mod Comp =N9-N10

    12=INDEX(C17:C126,MATCH($B$12,$B$17:$B$126,0),1) =INDEX(Data,MATCH($B$12,$B$17:$B$126,0),COLUMN()-3)*N1613 =N12*$B$1314 =IF($B$2=2,N13,N12)

  • 8/2/2019 Lida2007 Hayes

    62/128

    Solver Elements in Spreadsheet

    # B1 B2 B3 Target Vary Conditions

    0 0 0 0 B7 Min $D$10:$R$10,$A$17:$A$113 $D$11:$R$11= $D$14:$R$14 $A$17:$A$113>= 01 0 0 1 B7 Min $D$10:$R$10,$A$17:$A$113 $D$11:$R$11= $D$14:$R$14 $A$17:$A$113>= 0 $B$127= 1

    2 0 1 0 B8 Min $D$10:$R$10,$A$17:$A$113,$B$13 $D$11:$R$11= $D$14:$R$14 $A$17:$A$113>= 0

    3 0 1 1 B8 Min $D$10:$R$10,$A$17:$A$113,$B$13 $D$11:$R$11= $D$14:$R$14 $A$17:$A$113>= 0 $B$127= 1

    4 0 2 0 B9 Min $D$10:$R$10,$A$17:$A$113,$B$13 $D$11:$R$11= $D$14:$R$14 $A$17:$A$113>= 0

    5 0 2 1 B9 Min $D$10:$R$10,$A$17:$A$113,$B$13 $D$11:$R$11= $D$14:$R$14 $A$17:$A$113>= 0 $B$127= 1

    6 1 0 0 B6 Max $D$6:$R$6 $T$17:$T$113

  • 8/2/2019 Lida2007 Hayes

    63/128

    Visual Basic ProgramApplication.Range("B3").SelectConvex = Selection.Value

    A = 6 * Form + 2 * Orient + ConvexSolverReset

    'Set Target, MaxMinVal, ChangeIf A = 0 Or A = 1 Then 'Dual, Addition

    SolverOk SetCell:="B7", MaxMinVal:=2, ValueOf:="0", _ByChange:= "$D$10:$R$10,$A$17:$A$113"

    End IfIf A = 2 Or A = 3 Then 'Dual, Input

    SolverOk SetCell:="B8", MaxMinVal:=2, ValueOf:="0", _ByChange:= "$D$10:$R$10,$A$17:$A$113,$B$13"

    End IfIf A = 4 Or A = 5 Then 'Dual, Output

    SolverOk SetCell:="B9", MaxMinVal:=2, ValueOf:="0", _ByChange:= "$D$10:$R$10,$A$17:$A$113,$B$13"

    End If

    If A = 6 Or A = 8 Or A = 10 Then 'Primal, Not Convex (Constant Returns to Scale)SolverOk SetCell:="B6", MaxMinVal:=1, ValueOf:="0", _

    ByChange:= "$D$6:$R$6"End IfIf A = 7 Or A = 9 Or A = 11 Then 'Primal, Convex (Variable Returns to Scale

    SolverOk SetCell:="B6", MaxMinVal:=1, ValueOf:="0", _ByChange:= "$D$6:$R$6,$S$6"

    End If

  • 8/2/2019 Lida2007 Hayes

    64/128

    Vi l B i P

  • 8/2/2019 Lida2007 Hayes

    65/128

    Visual Basic Program

    For m = 1 To 97Application.StatusBar = "Calculating Efficiency for unit " & Str(m)

    ' Paste unit 0's number to model worksheetSheets("Sheet1").SelectApplication.Goto Reference:="unit"Selection.Value = m

    ' Run Solver (with the dialog box turned off)SolverSolve (True)

    ' Paste unit's number and name to All Results sheetSheets("Sheet1").Select

    Application.Range("C12").SelectSelection.Copy

    Sheets("Sheet3").SelectRange("A2").Offset(m, 0).SelectSelection.PasteSpecial Paste:=xlValues

    Vi l B i P

  • 8/2/2019 Lida2007 Hayes

    66/128

    Visual Basic Program

    Sheets("Sheet1").Select

    Application.Goto Reference:="Target1"Selection.CopySheets("Sheet3").SelectRange("A2").Offset(m, 1).SelectSelection.Value = mRange("A2").Offset(m, 2).SelectSelection.PasteSpecial Paste:=xlValues

    Sheets("Sheet1").SelectApplication.Goto Reference:="Results"Selection.CopySheets("Sheet3").SelectRange("A2").Offset(m, 1).SelectSelection.Value = mRange("A2").Offset(m, 3).SelectSelection.PasteSpecial Paste:=xlValues

    Next mApplication.Goto Reference:="Start"

    End Sub

  • 8/2/2019 Lida2007 Hayes

    67/128

    The Example of Libraries

    Selection of Data

    Input Variables (10):

    Collection Characteristics (Discretionary)

    Staff Characteristics (Discretionary) University Characteristics (Non-discretionary)

    Output Variables (5):

    Scaling of Data

    Constraints on Weights

    Results

    Effects of the several Variables

  • 8/2/2019 Lida2007 Hayes

    68/128

    Selection of Data

  • 8/2/2019 Lida2007 Hayes

    69/128

    The Variables

  • 8/2/2019 Lida2007 Hayes

    70/128

    Scaling of the Variables

  • 8/2/2019 Lida2007 Hayes

    71/128

    Constraints on Weights

  • 8/2/2019 Lida2007 Hayes

    72/128

    Results

    Efficiency Distribution

  • 8/2/2019 Lida2007 Hayes

    73/128

    Efficiency Distribution

    The following chart display the efficiency distribution

    for the 97 U.S. ARL libraries.

    The input and output components for each institution

    have been multiplied by the size of the collection.

    Note the cluster of inefficient institutions below the

    3,000,000 volumes of holdings.

    There appear to be three groups of institutions:

    The efficient ones, lying on the red line

    The seven that are more then 4 million and mildly inefficient

    Those that are less than 4 million and range in efficiency

  • 8/2/2019 Lida2007 Hayes

    74/128

    1.00

    3.00

    5.00

    7.00

    9.00

    11.00

    13.00

    1.00 3.00 5.00 7.00 9.00 11.00 13.00

    Collection*Input

    Collec

    tion*Output

    S f P j ti

  • 8/2/2019 Lida2007 Hayes

    75/128

    Sum of Projections

    The following chart show the distribution of the sum of

    the projections as a function of the Intensity.

  • 8/2/2019 Lida2007 Hayes

    76/128

    0,00

    1,00

    2,00

    3,00

    4,00

    5,00

    6,00

    7,00

    8,00

    9,00

    0,00 0,20 0,40 0,60 0,80 1,00 1,20

    Intensity

    Sumof

    Projections

  • 8/2/2019 Lida2007 Hayes

    77/128

    Distribution of Weights

    The following chart shows the magnitudes of the

    weights on each of the Input and Output components

  • 8/2/2019 Lida2007 Hayes

    78/128

    -0,05

    0,00

    0,05

    0,10

    0,15

    0,20

    0,25

    0 2 4 6 8 10 12 14 16

    A l f O ti R h 66

  • 8/2/2019 Lida2007 Hayes

    79/128

    Annals of Operations Research 66

    (1996)

    Preface

    P t I DEA d l th d d

  • 8/2/2019 Lida2007 Hayes

    80/128

    Part I: DEA models, methods and

    interrelations

    Chapter 1. Introduction: Extensions and new

    developments in DEA

    W W Cooper, R.G. Thompson and R.M. Thrall

    Chapter 2. A generalized data envelopment analysis

    model: A unification and extension of existing methods

    for efficiency analysis of decision making units

    G.Yu, Q. Wet and P Binckett

    E t i i DEA

  • 8/2/2019 Lida2007 Hayes

    81/128

    Extensions in DEA

    Covers (1) new measures of efficiency, (2) new models,

    and (3) new implementations.

    The TDT measure of relative efficiency takes thecriterion measure (weighted output/weighted input)relative to the maximum for that measure

    The Pareto-Koopman measure applies the Paretocriterion (no variables can be improved without worseningothers)

    The BCC model (variable returns to scale) is presented.

    Congestion arises when excess inputs interfere withoutputs. It thus represents relationships among variables.

    G li d DEA d l

  • 8/2/2019 Lida2007 Hayes

    82/128

    Generalized DEA model

    Essentially, this paper does what I have been trying to do in

    implementation of DEA.

    It does so by identifying the primal and dual (P and D), thetwo returns to scale (fixed and variable), and three binaryparameters (d1, d2, d3) in the equations

    d1 = d1eT + d1d2(-1)d3n+1 (for the dual)d1d2(-1)d30(for the primal)Values of (d1, d2, d3) include:(0,-,-) the CCR model

    (1,0,-) the BCC model(1,1,0) the FG model

    (1,1,1) the ST model

    The relationships among the several models are discussed.

    Part II: Desirable properties of

  • 8/2/2019 Lida2007 Hayes

    83/128

    Part II: Desirable properties of

    models, measures and solutions (1)

    Chapter 3. Translation invariance in data envelopment

    analysis: A generalization

    J.T Pastor

    Chapter 4. The lack of invariance of optimal dual

    solutions under translation

    R.M. Thrall

    Chapter 5. Duality, classification and slacks in DEA

    R.M. Thrall

  • 8/2/2019 Lida2007 Hayes

    84/128

    Lack of invariance

  • 8/2/2019 Lida2007 Hayes

    85/128

    Lack of invariance

    This paper supplements the prior one. It shows that in

    neither the BCC model nor the additive model are the

    optimal solutions for the dual (i.e., multipler)

    formulation invariant under translation.

    Duality classification and slack

  • 8/2/2019 Lida2007 Hayes

    86/128

    Duality, classification and slack

    This paper considers the role of slacks especially in the

    context of radial measures of efficiency. The effect ofalternative optima is to make slacks difficult to dealwith; the theory presented resolves the difficulties.

    The CCT model presented eliminates the need for non-Archimedean models and permits dealing with zero

    values for the variables.

    The concept of an admissible virtual multiplier isintroduced and the maximizing virtual multiplier w* isthe basis for categorizing efficient DMUs into 3 groups:

    Extreme Efficient: all variables are included in w* Efficient: the variables in w* are all positive

    Weak efficient: w* has at least one zero variable

    Similarly for non-efficient DMUs

    Part II: Desirable properties of

  • 8/2/2019 Lida2007 Hayes

    87/128

    Part II: Desirable properties of

    models, measures and solutions (2)

    Chapter 6. On the construction of strong

    complementarity slackness solutions for DEA linear

    programming problems using a primal-dual interior-

    point method M.D. Gonzdlez-Ltma, R.A. Tapta and R.M. Thralt

    Chapter 7. DEA multiplier analytic center sensitivity

    with an illustrative application to independent oil

    companies R.C. Thompson, PS. Ditarmapala, f Diaz, M.D.

    Gonzdlez-Lima and R.M. Thrall

    Complementarity Slackness Solutions

  • 8/2/2019 Lida2007 Hayes

    88/128

    Complementarity Slackness Solutions

    This paper proposes use of primary-dual interior-point

    methods for solution of the DEA linear programmingproblem (an iterative process that generates interior pointthat converge to the solution).

    The primary form minimizes Cx; the dual formmaximizes By.

    The condition for solution is that Cx = By, called thecomplementarity slackness condition.

    These methods attempt to solve the primary and duallinear programs simultaneously.

    Solutions are classified as radially efficient or inefficientusing the CCT model.

    Multiplier Sensitivity

  • 8/2/2019 Lida2007 Hayes

    89/128

    Multiplier Sensitivity

    The stability of the set E of extreme efficient DMUs is

    examined to determine the sensitivity to changes in the

    data,

    Part III: Frontier shifts and efficiency

  • 8/2/2019 Lida2007 Hayes

    90/128

    Part III: Frontier shifts and efficiency

    evaluations

    Chapter 8. Estimating production frontier shifts: An

    application of DEA to technology assessment

    R.D. Banker and R.C. Morey

    Chapter 9. Moving frontier analysis: An application ofdata envelopment analysis for competitive analysis of a

    high-technology manufacturing plant

    K.K. Sinha

    Chapter 10. Profitability and productivity changes: Anapplication to Swedish pharmacies

    R. Aithin, R. Fare and S. Grosskopf 219

    Production Frontier Shifts

  • 8/2/2019 Lida2007 Hayes

    91/128

    Production Frontier Shifts

    This paper divides the set of DMUs into two categories(representing the use or non-use of a technology). For a

    DMU without the technology, comparison is made only

    with others without the technology; for those with the

    technology, comparison is made with all DMUs.

    The result is a basis for assessment of the impact of the

    technology.

    Moving Frontier Analysis

  • 8/2/2019 Lida2007 Hayes

    92/128

    Moving Frontier Analysis

    This paper proposes a method for assessing when somedata may not be available. It uses aggregate data on

    best practices. It depends upon time series data

    P fit bilit & d ti it h

  • 8/2/2019 Lida2007 Hayes

    93/128

    Profitability & productivity changes

    It is not evident how this relates to DEA.

    Part IV: Statistical and stochastic

  • 8/2/2019 Lida2007 Hayes

    94/128

    Part IV: Statistical and stochastic

    characterizations

    Chapter 11. Simulation studies of efficiency, returns toscale and misspecification with nonlinear functions inDEA

    RD. Banker; H. Chang and WW Cooper Chapter 12. New uses of DEA and statistical

    regressions for efficiency evaluation and estimation -with an illustrative application to public secondaryschools in Texas

    VL Arnold, LR. Bardhan, WW Cooper and S.C.Kumbhakar

    Chapter 13. Satisficing DEA models under chanceconstraints

    W W Cooper Z Huang and S.X. Li

  • 8/2/2019 Lida2007 Hayes

    95/128

    Simulation studies

    Well, so be it.

  • 8/2/2019 Lida2007 Hayes

    96/128

    Satisficing DEA models

  • 8/2/2019 Lida2007 Hayes

    97/128

    Satisficing DEA models

    Introduces stochastic variables (characterized byprobability distributions) and the concept of

    stochastic efficiency.

    It distinguishes between a rule (which has a

    probability of 1) and a policy (which has a

    probability between 0.5 and 1).

  • 8/2/2019 Lida2007 Hayes

    98/128

    Part V: Some new applications

    Chapter 14. Evaluating the efficiency of vehicle

    manufacturing with different products

    G. Zeng

    Chapter 15. DEA/AR analysis of the 1988-1989performance of the Nanjing Textiles Corporation

    J.Zhu 311

  • 8/2/2019 Lida2007 Hayes

    99/128

  • 8/2/2019 Lida2007 Hayes

    100/128

    DEA/AR analysis

    Another application in China.

    Annals of Operations Research 73

  • 8/2/2019 Lida2007 Hayes

    101/128

    Annals of Operations Research 73

    (1997)

    Contents

    Preface

    Foreword

  • 8/2/2019 Lida2007 Hayes

    102/128

    Part VI: Extending Frontiers

    Extending the frontiers of Data Envelopment Analysis

    A.Y Lewin and LM. Seijord

    Weights restrictions and value judgements in Data

    Envelopment Analysis: Evolution, development andfuture directions

    R.Allen, A. Athanassopoulos, R.O. Dyson and F.

    Thanassoulis

  • 8/2/2019 Lida2007 Hayes

    103/128

    Extending the frontiers

    See earlier in this presentation

  • 8/2/2019 Lida2007 Hayes

    104/128

    Weights restrictions & value judgments

    See earlier in this presentation.

  • 8/2/2019 Lida2007 Hayes

    105/128

    Part VII: Applications

    DEA and primary care physician report cards:Deriving preferred practice cones from managed careservice concepts and operating strategies

    IA. Chilingerian and H.D. Sherman

    An analysis of staffing efficiency in U.S.manufacturing: 1983 and 1989

    PT Ward, J.E. Storbeck, S.L. Mangum and REByrnes

    Applications of DEA to measure the efficiency ofsoftware production at two large Canadian banks

    J.C. Paradi, D.N. Reese and D. Rosen

    Primary care physician

  • 8/2/2019 Lida2007 Hayes

    106/128

    Primary care physician

    This papers identifies styles of management based on

    ratios of input variables aimed at input cost minimizing.

    The example used is comparison of hospital days versus

    office visits

    Staffing efficiency

  • 8/2/2019 Lida2007 Hayes

    107/128

    g y

    Again, styles of management are identified, this timebased on ratios of types of staffing (e.g., professional vs.

    non-professional). Industries are divided into types (batch

    vs. line processing industries) and best practices for

    each type are identified by DEA.

    software production

  • 8/2/2019 Lida2007 Hayes

    108/128

    software production

    Input to software production is taken as cost; outputs assize (measure by function points), quality (measured

    by defects or rework hours), and time to market.

    The DEA is compared to performance ratio analyses,

    such as Cost/Function, Defects/Function, Days/Function. Then, constraints on the weights are introduced. One set

    of constraints consisted of bounds on ratios of weights. A

    second set of constraints consisted of tradeoffs between

    variables, again represented by bounds on ratios.

  • 8/2/2019 Lida2007 Hayes

    109/128

    Part VII: Applications

    Restricted best practice selection in DEA: An overview

    with a case study evaluating the socio-economic

    performance of nations

    B.Golany and S. Thore A new measure of baseball batters using DEA

    T.R. Anderson and G.P Sharp

    Efficiency of families managing home health care

    CE. Smith, S. VM. Kiembeck, K. Fernengel and L.S.

    Mayer

    Economic performance of nations

  • 8/2/2019 Lida2007 Hayes

    110/128

    p

    To apply DEA to evaluation of economic performanceof nations, it is necessary to recognize some constraints:

    International requirements (treaties, bilateral agreements)

    Externalities (e.g., mandated quotas)

    Issues of equity These constraints are then incorporated into DEA

    Baseball batters

  • 8/2/2019 Lida2007 Hayes

    111/128

    Baseball batters

    Traditional methods for evaluating batters include

    fixed and variable weight statistics (homers, batting

    average, slugging average, RBI, etc.). The point in this

    article is that use of DEA allows one to determine the

    effect of changes over time. Another effect of interest is noise. To correct for

    noise, the DEA model derates the data for each

    player by a factor based on the players standard

    deviation for each variable

    Efficiency of families

  • 8/2/2019 Lida2007 Hayes

    112/128

    Family home health care is assessed using a steppedprocedure in DEA.

    The stepped procedure involves a series of steps in

    which variables are successively introduced:

    Inputs OutputsStep 1 Direct Costs Medical Expense Family Income

    Step 2 Indirect Costs Training Patient/Caregiver

    Step 1 Step 1

    Step 3 Caring Costs Hours /day Caregiver burden

    Moths/caregiving Caregiver esteemMedication

    Step 2

  • 8/2/2019 Lida2007 Hayes

    113/128

    Part VII: Applications

    A DEA-based analysis of productivity change and

    intertemporal managerial performance

    E.Grifell-Tatje and C.A.K. LoveII

    Use of Data Envelopment Analysis in assessingInformation Technology impact on firm performance

    C.H. Wang, R.D. Gopal and S. Zionts

    Productivity & managerial performance

  • 8/2/2019 Lida2007 Hayes

    114/128

    Examines the productivity of an organization overtime.

    Information Technology impact

  • 8/2/2019 Lida2007 Hayes

    115/128

    gy p

    Examines the impact of information technology onperformance of firms. It divides operations into two

    stages: (1) Accumulation of resources and (2) Use of

    resources. (These are illustrated in banking by (1) the

    collection of funds from depositors and (2) use of those

    funds for generating income).

    It examines separately the effect of information

    technology (represented by ATM machines) on the two

    stages.

  • 8/2/2019 Lida2007 Hayes

    116/128

    Part VIII: Theoretical Extensions

    Comparative advantage and disadvantage in DEA

    A.I. Alt and CS. LeTine

    Model misspecification in Data Envelopment Analysis

    P Smith Dominant Competitive Factors for evaluating program

    efficiency in grouped data

    J.J.Rousseau and J.H. Semple

    DEA-based yardstick competition. The optimality ofbest practice regulation

    P Bogetoft

    Comparative advantage & disadvantage

  • 8/2/2019 Lida2007 Hayes

    117/128

    p g g

    This paper introduces a cost function into DEAanalysis as the means for calculating a comparativeadvantage or disadvantage as the difference betweenthe costs of input and the income from output.

    It interprets the weights in each DMUs optimum as

    prices for the respective inputs and outputs. The resultis virtual cost, revenue, and profit. The profit (orloss) is then compared with the maximum profitobtained by a best practice unit and that of theevaluated unit.

    For an efficient unit, the comparison is between thevirtual profit of the valuated unit and the maximumprofit across all other units.

    Comparative disadvantage

  • 8/2/2019 Lida2007 Hayes

    118/128

    p g

    The DEA model for determining comparativedisadvantage is:

    Max RC + w subject to Min h - w - uY1 + R = -1 - w Y1 + Y + T0r = 0 vX1C = 1 hX1X + T1r1 = 0 uYvX = Iw

  • 8/2/2019 Lida2007 Hayes

    119/128

    The DEA model for determining comparativeadvantage is applied to the set removing the target

    unit:

    MaxR + C + w subject to Min h - w - uY1 + R =1 - w Y1 + Y1 + T0r = 0 vX1 + C = 1 hX1X1 + T1r1 = 0 uY1vX1 = Iw

  • 8/2/2019 Lida2007 Hayes

    120/128

    This paper examines the effects of various types of mis-

    specifications of the DEA model. They include:

    Omission of a necessary input

    Inclusion of an extraneous variable Erroneous assumption about returns to scale

    Dominant Competitive Factors

  • 8/2/2019 Lida2007 Hayes

    121/128

    p

    This paper treats DEA as a tool in game theory. Oneplayer has control over the weights applied to the

    variables, the other over the weights applied to the

    DMUs. Each tries to optimize against the other.

    The solution is of the pair of prime-dual problems:

    Player 1 Maximizes vy0ux0subject to vyjuxj =0

    Player 2 Minimizes a subject to Y + ay0 >= y0, Xax0 = 0, a unrestricted

    Best practice regulation

  • 8/2/2019 Lida2007 Hayes

    122/128

    The use of DEA in regulatory practice is discussed. Theunderlying game is represented by a series of steps:

    Costs and demands for service are observed oridentified

    Schemes are proposed by the regulator

    The schemes are rejected or accepted by the DMUs

    Costs are selected by the DMUs

    Data on performance are observed

    Compensations are paid

    The aim of the regulator is to minimize the expectedcosts of making the DMUs accept, fulfil, and minimizecosts.

    The use of DEA is to determine the best practce norms.

  • 8/2/2019 Lida2007 Hayes

    123/128

    Part VIII: Theoretical Extensions

    A Data Envelopment Analysis approach to

    Discriminant Analysis

    D.L. Retzlaff-Roberts

    Derivation of the Maximum Efficiency Ratio modefrom the maximum decisional efficiency principle

    M.D. Trouft

  • 8/2/2019 Lida2007 Hayes

    124/128

    Discriminant Analysis

    Discriminant analysis is a means for determining group

    classification for a set of similar units or observations.

    It determines a set of factor weights which best

    separate the groups, given units for which membershipis already known.

    This paper proposes the use of DEA as a means for

    doing DA

    Maximum Efficiency Ratio

  • 8/2/2019 Lida2007 Hayes

    125/128

    Maximum efficiency ratio (MER) is intended to

    prioritize the DEA efficient DMUs by defining common

    weights. This paper supposes the existence of a ratio

    form criterion common to all the DMUs but not

    necessarily frontier oriented.

    Maxu,v (Minj (Suryrj/Svixij), subject to Suryrj/Svixij = 0

    Part IX: Computational

  • 8/2/2019 Lida2007 Hayes

    126/128

    Implementation

    A Parallel and hierarchical decomposition approaches

    for solving large-scale Data Envelopment Analysis

    models

    R.S. Barr and M.L. Durchholz

  • 8/2/2019 Lida2007 Hayes

    127/128

    Part X: Abraham Charnes

    Abraham Charnes remembered

    Abraham Charnes, 1917-1992

    A bibliography for Data Envelopment Analysis (1978-

    1996) LM. Setford

  • 8/2/2019 Lida2007 Hayes

    128/128

    The End