integrating decision support systems: expert, group, and collective intelligence
DESCRIPTION
ICAI Conference pptTRANSCRIPT
Integrating Decision Support Systems: Expert, Group, and
Collective IntelligenceSteve Diasio* & Núria Agell
ESADE Business School- BarcelonaGREC Research Group
*This research has been partially supported by the AURA research project (TIN2005-08873-C02), funded by the Spanish Ministry of Science and Information Technology and the Commission for Universities and Research of the Ministry of Innovation, Universities, and Enterprises of the Government of Catalonia.
IC’ AI 09 Las Vegas, 2009
Road Map
• Introduction and Motivation• Framework for Integration• Terms and Concepts• Leveraging Expertise in
Decision Support Technology– Expert Systems (ESs)– Group Decision Support
Systems (GDSSs)– Collective Intelligence
Tools (CI Tools)• Enhancing Decision-Making
and CI Tools• Conclusions and Future
Work
Empty trading pit @ CBOT
Introduction and Motivation
• Organizations today face a changing environment; – external conditions change rapidly (Ilinitch et al, 1996).– organizational structures flat and dispersed (Malone, 2006).– traditional roles of experts have been “squeezed” or of decreased
importance (Mauboussin, 2008).• Today’s new environment places a premium on collaboration
creating renewed interest in decision support technology to survive and remain competitive (Hamel & Breen, 2008).
• Information technology is playing an increasing role in facilitating a firm’s success and is woven thread in the fabric of the organization (Zammuto et al, 2007).
• The paper aims to understand how integration of expert systems (ESs), group decision support systems (GDSSs), and collective intelligence tools (CI tools) can enhance decision-making.
Framework for Integration
• Abundance of decision support tools at their disposal.• Tools have been independently built (Turban & Watkins,
1986) for individual problems but be flexible to adapt to the changing conditions and needs.
• Individually shown advantages of using such systems, however have not extended or offered in theory or practice an integrated system that supports organizational needs in expertise for decision-making.
Expert Systems GDSSs CI Tools
Integrated System
Proposed integrated support system
• Abundance of decision support tools at their disposal.• Tools have been independently built (Turban & Watkins, 1986) for
individual problems but be flexible to adapt to the changing conditions and needs.
Expert System GDSSs CI Tools
Integrated System
Framework for Integration
Proposed integrated support system
• Individually shown advantages of using such systems, however have not extended or offered in theory or practice an integrated system that supports organizational needs in expertise for decision-making.
Terms and Concepts
What is Expertise?• Multi-dimensional (Sternberg,
1997) with expert knowledge as the essential part (Tynjala, 1999)
• Short supply and difficult to represent
• Highly specialized or domain specific (Chi, Glaser, & Farr, 1988)
• Skills honed through practice (Jackson, 1999)
• Perform consistently more accurate in relation to others (Hartely, 1985)
Practical Knowledge Self-r
egula
tive
Know
ledgeFo
rmal
Kno
wled
ge
Expert Knowledge Dimensions
Expertise in Law
Formal Knowledge Practical Knowledge
Self-regulative Knowledge
•Reflective skill•Evaluation of action•Monitor argument and presentation to jury
•Factual knowledge•Learning of explicit information•In school or cases
•Intuition•Experience in legal setting•Tacit and difficult to express
Lawyer Expertise
Expertise by Means of Technology
• Expertise not limited to humans
• Technology built to capture knowledge or represent expertise (Barton, 1987; Liou & Nunamaker, 1990; Smith, 1994)
• Level of expertise can be augmented by increasing the amount of participants in the decision-making process
Expertise in Design
Level of Expertise in Systems Design
Number of People
Expert Systems
GDSSs
Collective Intelligence Tools
Objective: To represent expertise to its users for decision-making when a human expert can not be found or is in short supply.
Playing a critical role for organizations and are a source for competitive advantage (Gill, 1995).
Contributing to decision-making through their representation of knowledge and reasoning of human experts (Weiss & Kulikowski, 1984).
By mimicking and replicating the cognitive process of a human expert, novice users can be supported to perform as well as experts (Cascante et al, 2002).
ES are a technology that facilitates learning through the transfer of tacit and explicit knowledge (Yoon et al., 1995; Gregor & Benasat, 1999).
Leveraging Expertise Expert Systems
Attributes:
Objective: To capture the knowledge and contribution from the individual users to facilitate solutions to problems.
Occupies the center point for the aggregation of information and expertise from each participant.
Support the changing organizational structure, project basedteams, dispersed workforce, and greater emphasis on collaboration.
Aided groups to deal with to the changing dynamics characterized by greater knowledge, complexity, and turbulence (Huber, 1982; 1984).
Shown to reduce time, costs (Gallup, 1985), foster collaboration, communication, deliberation, and negotiations (Kull, 1982).
Leveraging Expertise Group Decision Support Systems
Attributes:
What is Collective Intelligence?
• The collective judgment of group can predict or forecast better than experts or groups of experts (Surowiecki, 2004)
• Diverging from traditional thought- high levels of expertise are the best source for decision-making
• Including many people in decision-making by harnessing lower levels of expertise for peak solutions (Page, 2007)
Objective: To facilitate the summative body of knowledge,information, and resources of its users.
Democratize decision-making by including many people in and outside the organization into the information gathering and decision-making process.
Prediction markets, incubates information scattered around the organization or network that allows non-experts to produce expert like results.
Challenges traditional roles of experts, may change answer givers to inquiry mediators in effort to harness the knowledge of the masses in decision-making.
Offer an additional tool in decision-making.
Leveraging Expertise Collective Intelligence Tools
Attributes:
Enhancing Decision-Making and CI Tools
• Past attempts have made steps (Aiken et al. 1991; Turban & Watkins, 1986).
• Opportunities for system integration to solve a wider spectrum of problems.
• AI techniques to CI Tools– Transforming from
passive to active agents– Intelligent components
to increase participation – Managing interaction
and collaboration between users
Ill- Structured
Problem Structure
Many
ES
Group Size SupportedFew
Well- Structured
GDSS
CI Tools
DSS
Decision Support Technologies *
*Figure adapted from Aiken et al. 1991
Differences Between ES, GDSSs, CI Tools
Attributes ES GDSS CI TOOLS
Objective Replicate or mimic human expertsFacilitate solutions for a group of people
To sum the knowledge and information of many people
Who makes the recommendation (decision)?
The system or heavily weighted if human is involved
The group and/ or systems through ranking The System/ Tool
Major orientation (characteristic)Transfer of expertise (human-machine-human) Build group consensus
Transfer of hard to find information or qualitative to quantitative data
Nature of support Individual or group Group Individual or group
Problem area characteristic Narrow domain Semi/ Unstructured, broad Limited variability
Type of problem treated Repetitive Unique/ not often / importantForecasting/ dispersed collaborators/ Probabilistic
Reasoning capability Yes (deduction) NoYes (depending on the tool (induction)
Assumptions Closed-world Limited to users boundaries Changing
Expertise Level or In-depth knowledge of problem Specific/ Expert Level Dependent on task or problem
All levels including learning capacity with use
Figure 4 Differences between ES, GDSS, CI ToolsAdapted from Aiken et al, 1991]
Shown Indicated
ExploredHighlighted
Conclusions
an evolutionary perspective of expertise supported by decision support technologies.
how organizational use of expertise is changing which reflects the new roles of experts and non-experts in decision-making
how organizational expertise in short supply can be augmented
issues of design for integration with existing decision support technology
Thank You!
Steve Diasio & Núria Agell{stephen.diasio; nuria.agell} @esade.edu
ESADE Business School- BarcelonaGREC Research Group
References[1] M. Aiken, O. Liu Sheng and D. Vogel, “Integrating expert systems with group decision support systems”,
ACM Transactions on Information Systems (TOIS), Jan. 1991. (9)1, 75-95.[2] L. Bañón and K. Schmidt, “CSCW: Four characters in search of a context”, Proceedings of the First
European Conference on Computer-Supported Cooperative Work - EC-CSCW Gatwick, London, Sept., 1989, 358-372.
[3] A. Barton, “Experiences in Expert Systems”, The Journal of the Operational Research Society, The Practice of OR in Central Government Oct., 38(10), 1987, 965-974.
[4] J. Berg and T. Rietz, “Prediction Markets as Decision Support Systems”, Information Systems Frontiers, 5(1), 2003, 79-93.
[5] J. Berg, F. Nelson and T Rietz, “Accuracy and forecast standard error of prediction markets”, Working Paper, The University of Iowa, 2001.
[6] E. Bonabeau, “Decisions 2.0: The Power of Collective Intelligence”, MIT Sloan Management Review. Winter 50(2), 2009.
[7] B.G. Buchan, D. Barstow, R. Bechtel, J. Bennett, W. Clancey, C. Kulikowski, T. Mitchell and D.A. Waterman, “Constructing and Expert System”, Building Expert Systems, Hayes-Roth, F., Waterman, D.A. and Lenat, D.B. (eds), 1983. pp 127-167, Addison-Wesley, Reading, MA.
[8] L. Cascante, M. Plaisent, P. Bernard and L. Maguiraga, “The impact of expert decision support systems on the performance of new employee”, Information Resource Management Journal; Oct-Dec 15(4), 2002, 67-78.
[9] M. T. H. Chi, R. Glaser, and M. J. Farr, eds., The Nature of Expertise, 1988. Hillsdale, NJ: Erlbaum.[10] B. Cowgil, J. Wolfers, and E. Zitzewitz, “Using Prediction Markets to Track Information Flows: Evidence
from Google”, 2008, Working Paper.[11] K. Crowston and T. W. Malone, “Intelligent software agents”, Byte Magazine, Dec., 1988, 267-272.[12] G. DeSantics and R. Gallup, “A Foundation for the Study of Group Decision Support Systems”.
Management Science, 33(5), 1987, 589-609.[13] S. Diasio, “Collaborative Fusion: Expert, Group, and Collective Intelligence in Decision-making
Support”, unpublished paper, 2008.[14] R. Dye, “The Promise of Prediction Markets: A Roundtable”. McKinsey Quarterly (2) 2008.
References[15] E.A Feigenbaum, “The Art of Artificial Intelligence: Themes and Case Studies of Knowledge Engineering’,
International Joint Conference on Artificial Intelligence, 1977, 1014-1029.[16] B. Gallup, “The impact of task difficulty on the use of a group decision support system”. Ph.D. dissertation,
Dept. of Information and Decision Sciences, Univ. of Minnesota, 1985.[17] M. J. Gannon, Organizational Behavior: A Managerial and Organizational Perspective, Boston: Little, Brown,
1979.[18] T. Gill, “Early Expert Systems: Where are they now?”, MIS Quarterly. March, 1995.[19] R. Hanson, “Decision Markets”, IEEE Intelligent Systems, 14, 1999, 16-19.[20] R. Hartley, “Expert System Methodology: A Conceptual Analysis”, International Journal of Systems Research
and Information Science, 1, 1985, 11-22.[21] T. Ho, and K. Chen, (2007). “New Product Blockbusters: The Magic and Science of Prediction Markets”,
California Management Review. Fall, 50(1), Berkeley, California, 2007.[22] J. Howe, Crowdsourcing: Why the Power of the Crowd Is Driving the Future of Business, New York, Crown
Business, 2008.[23] G. P. Huber, “Group decision support systems as aids in the use of structured group management
techniques”, In DDS-82 Conference Proceedings, 1982, 96-108.[24] G. P. Huber, “Issues in the Design of Group Decision Support Systems”, MIS Quarterly, Sept. 8(3), 1984,
195- 204.[25] P. Jackson, Introduction to Expert Systems, (3rd) ed, Pearson Addison Wesley, 1999.[26] D. Kull, “Group decisions: Can a computer help?”, Computer Decisions, vol. 15(5), 1982, 64-70.[27] Y. Liou, and J. Nunamaker, “Using a Group Decision Support System Environment for Knowledge Acquisition:
A Field Study”, Proceedings of the ACM SIGBDP conference on Trends and directions in expert systems. Orlando, Florida, United States, 1990, 212 – 236.
[28] D. S. Murphy, and S. A. Yetmar, “Auditor evidence evaluation: Expert systems as credible sources”, Behavior and Information Technology, 15, 1996, 14-23.
[29] S. Page, The Difference: How the Power of Diversity Creates Better Groups, Firms, Schools, and Societies, Princeton University Press, 2007.
[30] E. H. Shortliffe, and B. Buchanan, “Rule-Based Expert Systems: The MYCIN Experiments of the Stanford Heuristic Programming Project”, 1984.
References[31] E. H. Shortliffe, “Computer-based medical consultation MYCIN”, American Elsevier, 1976.[32] H. Simon, Administrative Behavior: A Study of Decision- Making Processes in Administrative
Organizations, 4th ed. The Free Press, 1997.[33] B. Smith, Collective Intelligence in Computer-Based Collaboration, Hillsdale, NJ: Lawrence Erlbaum,
1994.[34] R. J. Sternberg, “Cognitive conceptions of expertise”, In P.J. Feltovich. K. M. Ford and R. R. Hoffman.
Expertise in context. Human and Machine, Menlo Park, CA: AAAI Press/ The MIT Press, 1997. 149-162.[35] J. Surowiecki, The Wisdom of Crowds: Why the Many Are Smarter Than the Few and How Collective
Wisdom Shapes Business, Economies, Societies and Nations, Little, Brown, 2004.[36] P. Tynjala, “Towards expert knowledge? A comparison between a constructivist and a traditional
learning environment in the university”, International Journal of Education Research. 31, 1999, 357-442.
[37] J.P. Shim, M. Warkentin, J.F. Courteny, D.J. Power, R. Sharda, and C. Carlsson, “Past, Present, and Future of Decision Support Technology”, Decision Support Systems, 33, 2002, 111-126.
[38] S. Weiss and C. A. Kulikowski, A Practical Guide to Designing Expert Systems, Chapman and Hall Ltd, 1984.
[39] S. E. White, J. E. Dittrich and J. R. Lang, “The Effects of Group Decision Making Process and Problem Situation Complexity on Implementation Attempts”, Administration Science Quarterly, 25(2), 1980, 428-440.
[40] J. Wolfers and E. Zitzewitz, “Prediction Markets in Theory and Practice”, In S. Durlauf and L. Blume (eds) The Palgrave Dictionary of Economics, McMillan, 2nd ed, 2006
[41] M. Woodridge and N.R. Jennings, “Intelligent Agents: Theory and Practice”, The Knowledge Engineering Review, 10(2), 1995, 115-152.