lp, excel, and merit – oh my! (w/apologies to frank baum)
DESCRIPTION
CIT Research/Teaching Seminar Series (Oct 4, 2007). LP, Excel, and Merit – Oh My! (w/apologies to Frank Baum). John Seydel. No, It’s Not About Getting Back to Kansas!. Here’s the Problem. Developing merit evaluations of multiple faculty members Some are good all around - PowerPoint PPT PresentationTRANSCRIPT
LP, Excel, and Merit – Oh My! (w/apologies to Frank Baum)
CIT Research/Teaching Seminar Series (Oct 4, 2007)
John Seydel
No, It’s Not About Getting Back to Kansas!
Here’s the ProblemDeveloping merit evaluations of multiple faculty members
Some are good all around Each is good at something Which “somethings” should be considered more/less
important? How much more/less important?
Why not borrow from Economics: concept of Pareto efficiency?
Identify the efficient set of faculty members Avoid answering the “importance” question
We can use LP (linear programming) with some help from Excel to address thisHence: “LP, Excel, and Merit”
What’s LP?Consider a production planning problem Liva’s Lumber (refer to handout)
3 products 3 constraints 1 objective (maximize weekly profit)
Summary table:
Modelling: LP model and Excel model
Product: CDX AC Form
How much: ??? ??? ???
Profit: $ 5.00 $ 7.00 $ 6.00 Available
Cutting: 2 3 10 54,000
Gluing: 4 7 4 24,000
Finishing; 2 3 7 18,000
Now, the Merit Problem
Typical merit criteria: Teaching Research Service
Consider the teaching criterion Our CoB evaluations have 35 dimensions
associated with the teaching criterion What do we do with all those
There are too many to weight So we just average them; i.e., we treat them as if
they’re all equally important! “Follows syllabus” is important as “explains clearly”
Let’s consider a smaller example (Table 1)
Aggregation of Results
Humans want a single performance measuresTypical schemes Simple average (see Table 2) Focus on “overall effectiveness” question
(e.g., #8) Also, weighted average
Weights determined by whom (committee, administrator, statute, . . . )?
Illustrated by MBO So, what’s wrong with a simple average?
Obscures individual strengths and weaknesses Artificially values minor differences
DEA to the Rescue (?)
We want to evaluate the outcomes of behaviors (decisions) on the basis of Multiple criteria to be considered for the
outcomes No generally acceptable set of weights exists
(and no one is willing to determine such)
This is where DEA (data envelopment analysis) can be usefulConsider each instructor to be a DMU (decision-making unit)Apply the concept of economic efficiency . . .
Efficient Set ConceptSet of entities (DMUs) where no entity performs as well or better on all criteria Graphically: convex hull Consider concept from finance: efficient
portfolio Risk Return
Any entity’s weighted multicriteria score will be the same as the others’ scores, if they all get to choose their own weights These entities are called efficient decision making unitsConsider a simple example (subset from Table 2) . . .
Bicriterion Performance Comparision
Criterion Simple Avg Weighted Avg
Instrtuctor Impartial Prepared Value Rank Value Rank
OBA 4.77 3.78 4.28 1 4.08 2
GJB 3.02 2.83 2.93 6 2.89 5
IAB 2.01 2.20 2.11 7 2.14 7
OVB 3.39 4.24 3.82 3 3.99 3
BFH 3.74 2.35 3.05 5 2.77 6
DEI 2.68 3.58 3.13 4 3.31 4
OLK 4.58 3.96 4.27 2 4.15 1
Weight: 0.30 0.70
Graphically Identifying the Efficient Frontier
0.00
1.00
2.00
3.00
4.00
5.00
6.00
0.00 1.00 2.00 3.00 4.00 5.00
Preparedness
Imp
arti
alit
yy
OBA
OLK
OVBBFH
IAB
GJBDEI
Some Basic Definitions
Efficiency = Output / InputMaximum possible efficiency is defined as 100% (i.e., 1.00)Output for an instructor is her/his weighted average evaluation scoreInput for all instructors is theoretically the same (100% of time available)This leads to a model (recall the LP model for Liva’s Lumber) . . .
Efficiency ModelChoose a set of criterion weights for a given instructor so as to
Maximize: Instructor’s Output/Input Subject to:
Each other instructor’s Output/Input <= 1 Weight values are positive
Which is the same as Maximize: Instructor’s Weighted Average Score Subject to:
Each other instructor’s Weighted Average Score <= 1 Weight values are positive
Since each instructor’s input is defined to be 1.00Note, however, that the “weighted average” is now scaled to the 0.00 – 1.00 interval
An Example DEA Output Model for Evaluating Faculty Teaching
Let w1 and w2 be the weights to assign to impartiality and preparedness, respectivelyThen, for instructor GJB (for example, the objective is to
Maximize: 3.02w1 + 2.83w2 (GJB score)
ST: 4.77w1 + 3.78w2 ≤ 1.00 (OBA) 3.02w1 + 2.83w2 ≤ 1.00 (GGB) 2.01w1 + 2.20w2 ≤ 1.00 (IAB) . . . 4.58w1 + 3.96w2 ≤ 1.00 (IAB) w1, w2 > 0.00We can use Excel to model and solve this, but we need to reformulate and solve for every instructorThat’s where macro programming comes in . . .
Now, Let’s Apply This to the Data
Consider the model for QVAThen note the summary tableThings of interest Size of efficient set Rank reversals Comparison with simple average
approach (Figure 1)
Where To From Here?
Constraining the weightsRanking the “efficient” instructorsExpand across other criteria in the merit evaluationsOther DEA applications (decsion support) Comparing ecommerce platforms Vendor selection Other . . . ?
Go looking for more “Lions and tigers and bears (oh my)!”
Appendix
The LP Model for Liva’s Lumber
We can model this mathematically: Let x1 = number of sheets of CDX to produce weekly x2 = number of sheets of form plywood to produce weekly x3 = number of sheets of AC to produce weekly
The objective is to
Maximize: 5x1 + 7x2 + 6x3 (Weekly profit)
ST: 2x1 + 3x2 + 10x3 ≤ 54000 (Cutting) 4x1 + 7x2 + 4x3 ≤ 24000 (Gluing) 2x1 + 3x2 + 7x3 ≤ 36000 (Finishing) Solving is “simply” a matter of determining the best combination of x1, x2, and x3
Enter Excel
Create a spreadsheet table like the summary tableAdd a few formulae
Total profit Total amount of each resource consumed
Solve by trial and error . . . ?Better: use the Solver tool
Find the optimal solution quickly Tinker with parameters and re-solve
Even better: use Solver with a macro button Record macro Call subroutine when editing onClick event for button
Table 1:Example Evaluation Items
Table 2:Example Departmental Summary
DEA Model for Instructor QVL
Results Across Instructors
Figure 1: DEA vs. Simple Averaging