ex-ante evaluation /prioritization multicriteriaapproaches ... · ex-ante evaluation...
TRANSCRIPT
Ex-anteEvaluation/ PrioritizationMulticriteria Approaches
- andthecontext-
SergioSalles-Filho&AdrianaBin04.07.2017
Outline
1. FundamentalconceptstoRDIprioritization
2. Howtoselectthemostsuitableapproachesunderdifferentconditionsofboundeduncertainty/partialknowledge?
3. Fourcases
4. Agenda
1.FundamentalconceptstoRDIprioritization
RDISpecificities
Thingsthatneverhappenedbefore
Thingsthatdependonknowledgetobedeveloped
Thingswedonotknowwhethertheywillwork
Fundamentalconcepts
UncertaintyRisk
Ambiguity
Rationality
Expectation
Intuition
Twocomplementarydefinitions
• “…Itisaworldofchangeinwhichwelive,andaworldofuncertainty.• (…)• Theessenceofthesituationisactionaccordingtoopinion,ofgreaterorlessfoundationandvalue,neitherentireignorancenorcompleteandperfectinformation,butpartialknowledge.“
Risk,Uncertainty,andProfitFrankKnight,1921
Twocomplementarydefinitions
• “Inapredestinateworld,decisionwouldbeillusory• Inaworldofperfectforesight,empty• Inaworldwithoutnaturalorder,powerless• (…)• Sincedecisioninthissenseexcludesbothperfectforesightandanarchyinnatureitmustbedefinedaschoiceinfaceofboundeduncertainty”
DecisionOrderandTimeinHumanAffairsGeorgeShackle(1969)
PartialKnowledge
BoundedUncertainty
Ex-anteEvaluation
1+2: Thestartingpoint
Howprioritizeunderpartialignoranceandboundeduncertainty?
Data&Opinion
Data&Opinion’
Opinion
Opinion’
Data
Data’
Opinion&Data
Opinion&Data’
BasedontheStaceyDiagram(RalphStaceyprof.Hertfordshire)
ProfessorRobertGeyerofLancasterUniversity
StaceyDiagram
2.Howtoselectthemostsuitableapproachesunderdifferentconditionsofboundeduncertainty/partialknowledge?Ageneralproposition
Foresig
htto
ols
DataandTextmining
SurveysandDelphi
Scenarios
Projections
Panels
Roadmapping
Horizonscanning
… MeansofP
rioritiza
tion Mathematicalprogramming
Multicriteria MethodsMCDA
Descriptive/multivariateStatistics
Simplescoring
Potentialsurprise
BayesianStatistics
RealOptions
Geneticalgorithms
Artificialintelligence
...
ForesightToolsandMeansofPrioritization
Severalclassificationsofmethods
MedidasdeBenefício
métodoscomparativos
métodosdepontuação
modeloseconômicos
técnicasdedecisãoemgrupo
ProgramaçãoMatemática
TeoriadosJogos
ModelosdeSimulação
ModelosHeurísticos
EmulaçãoCognitiva
Pontuação
ModelosEconômicos
MétodosInterativos
AnálisedeDecisão
ProgramaçãoMatemática
InteligênciaArtificial
OtimizaçãodePortfolio
AvaliaçãopelosPares
MedidasdeBenefício
ProgramaçãoMatemática
EmulaçãoCognitiva
ModelosHeurísticosedeSimulação
OpçõesReais
Modelosadhoc
MétodosEconômicos
ProgramaçãoMatemática
AnáliseDecisória
MétodosInterativos
Pontuação
ModelosEstratégicos
Verbano eNosella (2010)Iamratanakul etal.(2008)Henriksen eTraynor (1999)
Heidenberger eStummer (1999)
MathematicalProgrammingOptimization
EconomicModelsUtitlity,B/C,RealOptions
MachineLearningModeling AI
ADHOCModelsExperts,Citzen
MCDA- MulticriteriaDecisionAnalysis
MCDA
OperationsResearch
Statistics
MachineLearning
Psychology(BehavioralEconomics)
SocialChoiceTheory
WhatisMCDA?
SlidekindlyprovidedbyProf.LeonardoTomazeli FCA/UNICAMP
MCDA:aninterdisciplinaryfield
SlidekindlyprovidedbyProf.LeonardoTomazeli FCA/UNICAMP
AGGREGATION• MultiAtribute Utility(MAUT)• MultiAtribute ValueTheory(MAVT)
OUTRANKING• Electre• Promethee
MCDA:maincategories
Analytichierarchyprocess(AHP)
Analyticnetworkprocess(ANP)
ELECTRE(Outranking)
Goalprogramming
Innerproductofvectors
UTA,UTAII,UTADIS
NonstructuralFuzzyDecisionSupportSyst.
PAPRIKA
PROMETHEE(Outranking)
Superiorityandinferiorityrankingmet.
TOPSIS
Valueengineering
FuzzyVIKORmethod
Weightedproductmodel
Weightedsummodel
Multi-attributevaluetheory(MAVT)
Multi-attributeutilitytheory(MAUT)
SlidekindlyprovidedbyProf.Anibal Azevedo FCA/UNICAMP
MCDA:maincategories
Goal
Attributionofarelativevalueorofacomparisonstructure
Evaluationofthealternativestotheselectedcriteria
Decisionaiding
DM
Agent1 Agent2 ... Agentk
Alternatives
Criteria
Informationprocessing
MCDAmethod
Problemmodeling
Decisionmakingauthority
Data
SlideofProf.LeonardoTomazeli FCA/UNICAMP– slightlymodified
TheprocessofMCDA
1. Data
1. Agentsinvolved
1. Goals
2. Criteria
3. Alternatives
Subjective
SlideofProf.LeonardoTomazeli FCA/UNICAMP– slightlymodified
ObjectiveandSubjectiveElements
WhyoutrankingMCDAinRDIex-anteevaluation?
• Theyaremoresuitabletodealwithboundeduncertainty• Theyarenotbasedinone-off(best)choice• Theybuildakindoffuzzyrankingcomparingallcriteriaagainsteachother• Theydealequallywithobjectiveandsubjectiveinformation (DATAANDOPINION)• Theyaresuitabletocombinewithanyotherprioritizingmethod
MCDAuses
2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017ModelosdeSimulação 1 1
AprendizadodeMáquina 2 1 1 2 1
MétodosEconômicos 1 1 1 3 1 1
ModelosAd-Hoc 2 1 1 2 2 1 2 1 2 1
ProgramaçãoMatemática 1 1 2 1 2 2 3 2 4 4 2 1 2 2
MétodosMulticritério 1 1 1 2 2 2 1 6 5 8 6 6 4 3 6 5 2
0
2
4
6
8
10
12
14
WoK:2000– 2017à 109papers(R&Dprojectselection+differentmethods)
MathematicalProgramming
Economicmodels
Machinelearning
Adhocmodels
MCDA
Simulation
Massaguer,P.(2017,forthcoming)
MCDAcombinationsShowValues>=1and<=28
AHPAnalyticNetwork
Delphi DEADynamic
programmingLinearprogram
Integerprogramming
Fuzzysets
Goalprogramming
NPVGametheory
Decisiontree
Expertsystems
MonteCarlo
Peerreview
RateofReturn
ScoringDiscountedcash
AHP 28 1 1 2 1AnalyticNetwork 1 24 4 4 1Delphi 1 4 11 1DEA 10 1Dynamicprogramming 8 1Linearprogramming 8Integerprogramming 2 1 7Fuzzysets 5Goalprogramming 4 1 4NPV 1 4 1 2Gametheory 3Decisiontree 1 1 2Expertsystems 2MonteCarlo 1 2Peerreview 2RateofReturn 2 2Scoring 2Discountedcashflow 1
Massaguer,P.(2017,forthcoming)
PrioritysettinganddecisionmakinginRDIusingtheStaceyDiagram (Slightlymodified…)
Agreem
ent
Assertivenessofinformation+ -
-Data&Opinion
Data&Opinion’
Opinion
Opinion’
Data
Data’
Opinion&Data
Opinion&Data’
Math.Programming
EconomicModels
Adhoc models
MCDAMachinelearnning
Machinelearnning
PrioritysettinganddecisionmakinginRDIusingtheStaceyDiagram (Slightlymodified…)
Adhoc models
3.Fourcases
Methodstoemphasis
convergence
Methodsforboth
CalculationMethodstoemphasisinformation
Agreem
ent
Assertivenessofinformation+ -
-Aproposalbasedonthediagram
4 cases
Materialsforpackaging
perceptionofwell-beingin10yearstime
projectstoaccomplishwithan
enforcement
technologiesina
“promising”areaof
knowledge
Agreem
ent
+
-
Assertivenessofinformation+ -
Case1:selectingwell-beingconceptsforcosmeticsindustry
Theobjectiveofthestudy:• Identifyrelevantcomponentstocharacterizeandtomeasureindividualwell-being
TheChallenge:• highlysubjective,non-structured,andvariableinformationtobegatheredandselectedaccordingtoseveral
criteria• Horizonof10years• Lowlevelofagreementaboutwhatpromoteswell-being• Lowlevelofassertivenessofinformation
TheApproach:• Firstlist:literaturereview+Interviews• Internalvalidation• Backgrounddocument• Multi-ExpertPanel:collectivescores• Simpleranking(importancexcontext)
Case1:selectingwell-beingconceptsforcosmeticsindustry
Findings:• Mostadequateconceptstobeexploitedindifferentsituations:
3rdBest:joy–optimism- discomfort(reduction)
2ndBest:emotionalstability- anxiety/distress(reduction)- love
Bestranked:happiness- self-esteem- physicalhealth
OpinionBased
• Literaturereview• Imaginationanddiversity• Heterogeneousopinions• Discussions
Case%1%
future%of%thermoplas0c%
resins%in%packaging%
percep0on%of%well7being%for%
new%cosme0cs%company%
projects%in%order%to%
accomplish%with%an%
enforcement%
technologies%in%a%
“promising”%area%of%
knowledge%
Agreem
ent%
Asser0veness%of%informa0on%+% 7%
7%
Case2:Technologiesforapromisingareaintheelectricitysector:Grids
Theobjectiveofthestudy:• IdentifytechnologiestocomposefutureR&Dprojectportfoliowithfocusongrid/smartgrid
TheChallenge:
• Identifytrends,strongandweaksignals,andopportunitiestodevelop“Grid”technologies• HighlevelofagreementabouttheimportanceofGrid
• Lowlevelofassertivenessofinformation
TheApproach:• Firstlist:Dataandtextmining
• Internalvalidation
• Backgrounddocument
• ExpertPanel• ELECTRE(severalscenarios)
• Classificationof“robust”technologies
• Internalvalidation
••Smart meters (AMI)••Meterdatamanagementsystems(MDMS)integratedwithinformationsystemsoperating(AMI)••Artificialintelligence systems(AdaptiveProtection,Control Technologiesand DynamicReconfiguration)••Virtualandaugmentedrealityplatformsforsimulation(TrainingMethods)
Robustselection
Searchforexpertopinionsandnewdata
• Asmuchdataaspossible• Searchforweaksignals• Expertopinion• Popperianfalseability(KarlPopper)
Case%2%
future%of%thermoplas0c%
resins%in%packaging%
percep0on%of%well7being%for%
new%cosme0cs%company%
projects%in%order%to%
accomplish%with%an%
enforcement%
technologies%in%a%
“promising”%area%of%
knowledge%Ag
reem
ent%
Asser0veness%of%informa0on%+% 7%
7%
Case3:Selectingprojectstoaccomplishwithregulatoryframework
Theobjectiveofthestudy:• SelectthemostsuitableprojectproposalstoaccomplishwiththeBrazilianregulatoryframeworkinthe
electricalsectorwhichobligesfirmstoinvest1%ortheirrevenuesinR&Dperyear(underthethreatofbeingfinnedbytheregulatoryAgency)
TheChallenge:• Selectingtheproposalsthatfulfilltheregulatoryrequirementsandreducetheregulatoryrisk• Highlevelofagreementaboutthealternatives• Highlevelofassertivenessofinformation
TheApproach:• Firstlist:Proposalsreceivedbythecompany• Optimization+multicriteria method(knapsack+Promethee)+MonteCarlo• ClassificationoftheBestsolutionbasedonoptimizationandoutranking
5 10 15 20 25 300
10
20
30
40
50
60
Total number of projects to be selected
Mean
cons
umpti
on (%
)
KNAPSACK-OUTRANKINGPROMETHEE IIPROMETHEE V
MaximizationApproaches
• Optimization(knapsack)problem• +Multicriteria• +MonteCarlo• Etc.
Case%3%
future%of%thermoplas0c%
resins%in%packaging%
percep0on%of%well7being%for%
new%cosme0cs%company%
projects%in%order%to%
accomplish%with%an%
enforcement%
technologies%in%a%
“promising”%area%of%
knowledge%Ag
reem
ent%
Asser0veness%of%informa0on%+% 7%
7%
Case4:Applicationsofthermoplasticsinpackaging
Theobjectiveofthestudy:• Identifynewapplicationsofdifferenttypesofresinstowardspackaging
TheChallenge:• Identifynewpossibilitiesofusingthermoplasticresinstodevelopnew(ortoreplaceexisting)packaging
materials• Lowlevelofagreementaboutthefuture(environmentalandconsumptiontrends)• Highlevelofassertivenessofinformation(knownmaterialproperties)
TheApproach:• Firstlist:marketandtechnicalavailabledata• Backgrounddocument• ExpertPanel• Multiplecorrespondenceanalysis• Selectionofnewdevelopmentsandpotentialreplacements
Buildingconvergence
• Monitoring(dataandtextmining)• Trendsanddriversconstantlyadjusted• Searchforconvergence
Case%4%
future%of%thermoplas0c%
resins%in%packaging%
percep0on%of%well7being%for%
new%cosme0cs%company%
projects%in%order%to%
accomplish%with%an%
enforcement%
technologies%in%a%
“promising”%area%of%
knowledge%
Agreem
ent%
Asser0veness%of%informa0on%+% 7%
7%
Agreem
ent
Assertivenessofinformation+ -
-Packaging
“convergence”Well-beingKnowledge+convergence
RegulatoryconstraintsOptimization
Gridknowledge
4cases
4.Agenda
Agendaonmeansofcalculation
• Workmoreonpossibilitiesthaninprobabilities• Shackleanapproach• Horizonscanningandmonitoring
• Workonmethodsofconstantrecalculation• Bayesianapproaches
• Workonmethodsrelatedtoevolution• Geneticalgorithmsapproaches
• WorkonArtificialIntelligenceapproaches• BigDataDrivenapproach– Watsonandbeyond
[email protected]@ige.unicamp.br