Download - Raford PhD defense final
Large-Scale Participatory Futures Systems
A Comparative Study of Online Scenario Planning Approaches
Michael Flaxman (Chair)Assistant Professor, Urban Information Systems Group, MIT
Joseph FerreiraProfessor of Urban Planning and Operations Research, Associate Department Head and Head of Urban Information Systems Group, MIT
Andres SevtsukLecturer, Department of Urban Studies and Planning, MIT
Noah RafordPhD Candidate, Urban Information Systems Group, City Design and Development Group, Department of Urban Studies and Planning, MIT
Candidate:
Committee:
Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches
Outline1. Introduction
2. Review & Synthesis of the Literature
3. Study Design & Methodology
4. Findings & Discussion
5. Conclusion
2
Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches
Introduction
3
Qualitative Scenario Planning
Humans have important shortcomings that limit our ability to make effective decisions under conditions of dynamic uncertainty (Dorner, 1997)
“A disciplined methodology for imaging possible futures in which organizational decisions may be played out” (Shoemaker, 1995)
“Tools for foresight discussions... whose purpose is not a prediction or a plan, but a change in the mindset of the people who use them” (de Gues, 1997)
ID Issues
Generate key themes
ID driving forces
Rank factors
Develop draft scenario logic
Create draft final scenarios
Finalise scenarios
Consider implications
Identify indicators
Meetings, conversations
Expert interviews, brainstorm with client, desktop research
Extract key themes, create trends and timelines, key events
Select key uncertainties and forces, list by uncertainty / impact, predetermined drivers
Create scenario snippets, draft systems diagrams, mix and match trends, 2x2 grids
Integrate themes from draft scenarios, create headlines and scenario narratives
Get client feedback, refine, detail, elaborate narrative to final form
Identify key strategic themes, reflect on strategic questions in the context of each scenario
ID key indicators in each scenario for strategic concerns
Client defines key questions through initial conversations & meetings
F2F & phone interviews
Group workshop
Consultant report
Group workshop
Consultant report
Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches
Introduction
4
Scenarios
Increased learning
More accuratemental models
Betterdecisions
Improved performance
Reduced individual and group decision bias (Tetlock, 2006)
Gain appreciation of different stakeholders’ positions and attitudes (Chermack, 2003)
Enhanced awareness of environmental change, future risks & opportunities (Weick, 1999)
Greater !exibility and better decision-making (Schwartz, 1997)
Purported Bene!ts
(Chermack, 2003)
Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches
Introduction
5
Challenges to Scenario Planning in the Public Realm
Labor intensive & expensive
Bene"ts poorly documented (no veri"cation or reputation systems)
Limited participation (time, space & numbers)
Predominance of senior decision-making elite (participant bias)
Highly dependent on facilitator skills & consultant synthesis (facilitator & author bias)
Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches
IntroductionResearch Questions
Do web-based participatory approaches add value to the traditional scenario planning process? If so, where and in what ways?
If not, where do they fall short, in what ways, and why?
6
Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches
Outline1. Introduction
2. Review & Synthesis of the Literature
3. Study Design & Methodology
4. Findings & Discussion
5. Conclusion
7
Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches
Literature Review & Synthesis
8
Urban Planning& Policy Policy
Scenario Planning
ICT Platforms& Web 2.0
Role of the FuturePlanning SupportSystems (PSS)
?
Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches
Literature Review & SynthesisUrban Planning & Public Policy
“The future orientation of planning is unique to the field's identity... The very substance of urban planning is founded in time'' (Myers and Kitsuse, 2000)
Four planning traditions (Freidman, 1987):
• Social Reform• Policy Analysis (Simon, 1945; Forrester, 1968; Stokey and Zeckhauser, 1978)
• Social Learning (Majone, 1989; Scott, 1998; Schon, 1983)
• Social Mobilization (Davidoff, 1965; Forester, 1989; Castells, 1977; Healey, 1992; Innes, 1996)
Growing demand for public participation (Arnstein, 1969; Hulchanski, 1977; APA, 1990)
“Urban planning has retreated from strategic, future-oriented topics to become absorbed in operational and managerial activities characterized by short time horizons and value choices likely to be equally short-sighted and ad hoc” (Coucelis, 2005)
9
Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches
Literature Review & SynthesisICT Platforms
Planning Support Systems (PSS) “Loosely coupled assemblages of computer-based techniques”, forming a mixed toolbox of techniques to help decision-makers in their daily tasks (Britton Harris, 1989; Brail and Klosterman, 2001; Batty, 2003)
• PPGIS (Warnecke, Beatie, & Lyday, 1998; Craig & Elwood, 1998; Geertman & Stillwell, 2003)
• Alternative Futures Analysis (Steinitz, 2003; Lagigno & Reed, 2003; Hopkins & Zapata, 2007)
• Participatory Agent Based Modeling (Bousquet & Le Page, 2004; Barnaud et al., 2007; Castella et al, 2005)
“Modelling as negotiation” (Guhathakurta, 1993)
“Complicated, convoluted, time-consuming, and intimidating... that do not achieve genuine participation in planning or other decisions” (Innes & Booher, 2004; Cooke & Kotari, 2001)
10
Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches
Literature Review & SynthesisCrowdsourcing & Web 2.0
Web 2.0 (O’Rielly; 2005; Anderson; 2007)
• Crowdsourcing (Howe, 2006)
• Collective Intelligence (Levy, 1994; Por, 2008; Malone et al., 2010)
• Human Computation (Quinn and Bederson, 2010; Sakamoto et al., 2010)
“The creation, aggregation and interpretation of strategically relevant information for decision-making through distributed means” (Por, 2008)
Wikipedia, Innocentive, Threadless, CrowdFlower, IdeaScale, Reddit, etc.
Have been studied but rarely used as research instruments themselves (Malone, 2010)
11
Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches
Literature Review & SynthesisScenario Planning
Creative, narrative, group-based processes for engaging with uncertainty and change (Wack, 1985; Van der Heijden, 1997)
• Double loop organizational learning (Argys & Schon; 1974)
• Constructivist & social learning theory (Piaget, 1977)
• Sensemaking & organizational awareness (Weick, 1979; Kleine,1999)
• Activity- & practice-based strategizing (Jarzabkowski, 2005; Orlikowski, 1992)
• Competitive advantages of perception management (Boyd, 1976)
Labor intensive & expensive, bene"ts poorly documented (no veri"cation or reputation systems), limited participation (time, space & numbers), predominance of senior decision-making elite (participant bias), dependent on facilitator skills & consultant synthesis (facilitator & author bias)
12
Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches
Introduction
13
Contribution of This Study
1. Operational: Help to understand the role that online systems might play in enhancing multi-stakeholder policy creation, speci"cally in the context of the challenges of future-focused, public planning initiatives
2. Methodological: Help to generate new analytical frameworks that can improve our understanding of how such systems may be used for measurement instruments and data analysis platforms
Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches
Outline1. Introduction
2. Review & Synthesis of the Literature
3. Study Design & Methodology
4. Findings & Discussion
5. Conclusion
14
Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches
Study Design & MethodologyResearch Questions• Do web-based participatory approaches add value to the traditional scenario
planning process? If so, where and in what ways?• If not, where do they fall short, in what ways, and why?
Participation• The number and type of
participants involved, and in what phases?
• The geographic scope of participation enabled?
• The range of expert professional disciplines consulted?
Interaction• The number of variables and
opinions incorporated?• The mechanism of analysis,
ranking and clustering?• The time spent on data
collection and analysis?• The amount of user debate
and re!ection?
15
Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches
An Exploratory Case Study Approach
A three-tiered, mixed method, case-study based approach, including:
Study Design & Methodology
• Informant interviews to identify key themes and constructs (n=46)
• Creation of two novel, prototypical data generation platforms and application on in-depth cases
• Pair-wise comparison of case studies to a base case
• Evaluation of three additional comparative examples from secondary sources
Interviews
In-depth cases
Base case
Comparativeexamples
16
Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches
Study Design & MethodologyCase 1: Futurescaper: The Impact of Climate Change Impacts on the UK
17
186 drivers, ranked, analyzed and visualized as system maps
Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches
Study Design & MethodologyCase 2: SenseMaker Scenarios: Future of Public Services Under Financial Uncertainty
18
• 265 participants, micro-scenarios
• Aggregated to three sketch scenarios based on pre-de"ned archetypes
Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches
Study Design & MethodologyBase Case: Future of a Northern Region in Spain
19
• Face-to-face scenario method
• Expert scenario consultancy
• 15 in-depth interviews
• Two day workshop, 20 participants
• Four regional scenarios
Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches
Study Design & MethodologyComparative Examples
Institute for the Future’s Foresight Engine
• 700 participants• 81 countries• 5,000 submissions in 24 hours
WikiStrat Collaborative Strategy Platform
• 30 teams• 13 countries• ~35,000 words of high-quality content created in 4
weeks
The Future of Facebook Project• 25 video interviews• 109 Quora interactions• ~50 Facebook participants
20
Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches
Study Design & MethodologyData Constructs Measured
21
Participant Characteristics• Degree of public openness (including promotion & recruitment efforts)
• Amount of preparation required• The number of participants involved• Reasons for participation• Degree of user anonymity• Type of participants involved
• Level of Education• Professional Experience• Professional Discipline• Age• Geographic Origin
Interaction Characteristics• Tasks performed• Amount and types of input considered
• Amount and types of visualization tools used
• Amount and types of analytical tools used
• Amount and kinds of socialization enabled
• Amount and kinds of feedback provided
• Supplementary interviews
Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches
Study Design & MethodologyChallenges
1) The relevant categories and variables for measurement were unknown in advance
2) There was little empirical evidence for, or agreement on, the key outcome variables for scenario planning
3) There were no standard measurement instruments or protocols available that could be readily applied
Both dependent and independent variables were unknown and no standard method of comparison could be established.
An exploratory, or “revelatory” case study design (Yin, 1994) was appropriate.
22
Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches
Outline1. Introduction
2. Review & Synthesis of the Literature
3. Study Design & Methodology
4. Findings & Discussion
5. Conclusion
23
Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches
Findings & DiscussionFinding 1: Greater Number and Diversity of Participants
24
More participants were involved
0
175
350
525
700
125150
700
265
35
Number of Participants
Base CaseSenseMakerForesight EngineWikiStratFoFB
(166)
Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches
Findings & DiscussionFinding 1: Greater Number and Diversity of Participants
25
More participants were involved
From more diverse locations
0
23
45
68
90
30
82
18
5
Number of Countries Represented
Base CaseSenseMakerForesight EngineWikiStrat
Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches
Findings & DiscussionFinding 1: Greater Number and Diversity of Participants
26
Wider range of experts & professional disciplines
More participants were involved
From more diverse locations
Base Case: ~20 different disciplines
Case 1: 35 different disciplines
Case 2: Signi"cant experience
WikiStrat: Mixed teams of highly trained inter-disciplinary contributors
Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches
Findings & DiscussionFinding 2: Most Participation Was Light, Skewed Towards a Few Heavy Users
27
Base Case: ~4.5 contributions per user, more extensive involvement & conversation through-out workshop
Case 1: ~ 1 contribution per user
Case 2: ~ 1 contribution per user
IFTF: ~6 contributions per user (1.5 original contributions, 4.5 responses to others), 20% of users = 70% of content
WikiStrat: Intensive contribution through-out process, ~7,000 words per team
Name Location Occupation
# of Cards Played
Forecasting Points
# SI Awards
!"#$%&$"'(")*'+ ,-./-. 012.3*' 456 447857 8 9*2&*.:*'(';"/";& <"&12.($-.=+
>??-;#@$"$2-."A+B2-A-(2&$
C5 5DE6 7 F*G.;".
&*!'*$*.(2.**' ?-A-'"/- H.(2.**' ID4 47J7 7 9*2&*.:*'(0H>+0"A3
;"$1#@.3 K-'$A"./=+LM N"$1*;"$2!2".I46 E84 7 N"!O'$1@'9*2&*.:*'(
$1*#A-$P$12!3*.&B?=+?"."/" HA*!$'2!"A+H.(2.**'
II5 5II 7
Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches
Findings & DiscussionFinding 3: Rapid Driver Generation & Exploration
28
Driver Generation:
Base Case: 80 hours + 120 minutes in workshop (5 hours per driver)Case 1: ~5 minutes per driverCase 2: ~10 minutes per driverIFTF: ~90 seconds per driver
Clustering & Ranking:
Base Case: ~2 hours in workshop, “not enough time to discuss”Case 1: Instantly sortable along number of dimensionsCase 2: Instantlysortable along number of dimensionsIFTF: N/AWikiStrat: N/AFoFB: Unknown, but “signi"cant and more than we thought”
Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches
Finding 4: Most In"uential at Early Stages
Findings & Discussion
29
ID Issues
Generate key themes
ID driving forces
Rank factors
Develop draft scenario logic
Create draft final scenarios
Finalise scenarios
Consider implications
Identify indicators
Case 1:Futurescaper
Case 2:SenseMaker
ForesightEngine
WikiStrat OpenForesight
ScenarioPlanning
Steps
Detailed Case Studies
ComparativeExamples
Increases the likelihood that a wide variety of forces and factors will be included
Increases likelihood that a diversity of perspectives will be achieved
Implies that individual and group biases may be less dominant at the early drivers exploration stage
Strong scaling potential
Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches
Findings & DiscussionFinding 5: “The Hourglass Effect”
Tension between structured / unstructured interfaces and analysis approaches
30
Trade off between ease of use & level of participation
More data = greater analytical burden
Case 1: Highly structured interface, open-ended analysisCase 2: Open-ended interface, highly structured analysis
IFTF: Largest number of drivers and social interaction, but very dif"cult tomake sense ofFoFB: “None of us had any idea it would take this long to complete.”
IFTF: Simple, game-like engaging interface, very light analytic powerWikiStrat: High barrier of entry, rich analytic input and deep participation
Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches
Findings & DiscussionFinding 5: “The Hourglass Effect”
Tension between structured / unstructured interfaces and analysis approaches
30
Trade off between ease of use & level of participation
More data = greater analytical burden
Case 1: Highly structured interface, open-ended analysisCase 2: Open-ended interface, highly structured analysis
IFTF: Largest number of drivers and social interaction, but very dif"cult tomake sense ofFoFB: “None of us had any idea it would take this long to complete.”
IFTF: Simple, game-like engaging interface, very light analytic powerWikiStrat: High barrier of entry, rich analytic input and deep participation
“People enter these activities with little background experience. Part of your job is to help model the thinking process that they should undergo.”
Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches
Findings & DiscussionFinding 6: Role of Visuals & Multimedia
31
Decreasing water qualityDecreasing water availability
Increasing toxic algal blooms
Increasing improved water and sanitation
Increasing diarrhea
Decreasing agricultural productivity
Increasing pollution
Decreasing deaths from cold temperatures
Increasing migration
Increasing water shortages
Increasing flooding
Increasing contamination of water supplyIncreasing droughts
Increasing air pollution
Increasing market
Increasing demand
Decreasing sustainablilty of crop production
Decreasing crop yields
Increasing population displacement
Increasing food prices
Increasing hardships for women
Increasing malaria
Increasing hydrological imbalance
Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches
Findings & DiscussionFinding 6: Role of Visuals & Multimedia
31
Decreasing water quality
Decreasing water availability
Increasing toxic algal blooms
Increasing improved water and sanitation
Increasing diarrhea
Decreasing agricultural productivity
Increasing pollution
Decreasing deaths from cold temperatures
Increasing migrationIncreasing water shortages
Increasing flooding
Increasing contamination of water supply
Increasing droughts
Increasing air pollution
Increasing market
Increasing demand
Decreasing sustainablilty of crop production
Decreasing crop yields
Increasing population displacement
Increasing food prices
Increasing potency of airborne diseases
Increasing malaria
Increasing hardships for women
Increasing hydrological imbalance
Increasing uncertainty in food production
Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches
Findings & DiscussionFinding 6: Role of Visuals & Multimedia
31
Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches
Findings & DiscussionFinding 6: Role of Visuals & Multimedia
31
Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches
Findings & DiscussionFinding 7: Social Experience of Online Scenario Building
32
Base Case was far more effective at producing active socialization and interaction between participants
“People need feedback in order to stay involved. You can provide automated feedback, but other people are the best kind of feedback you can possibly ask for.”
Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches
Findings & DiscussionFinding 7: Social Experience of Online Scenario Building
32
Base Case was far more effective at producing active socialization and interaction between participants
“People need feedback in order to stay involved. You can provide automated feedback, but other people are the best kind of feedback you can possibly ask for.”
Different kinds of experience were possible with IFTF and WikiStrat
• Ranks and Roles• “Coopetition”
Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches
Findings & DiscussionSpeculative Finding 1: Better Outcomes?
The evidence suggests that the use of such systems on their own will not produce the desired outcome of the scenario process
Augment early-stages
• Transparency• Speed• Ef"ciency• Larger scale engagement
Suggests may be effective analytically, but is it psychologically? A hybrid approach is worth exploring to get the full bene"ts
33
Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches
Findings & DiscussionSpeculative Finding 2: Impact on Professional Standards
Greater transparency could facilitate reputation systems (eBay, Amazon)
“The futures profession is decentralized, eclectic and intellectually varied: there are no schools that train its elite, few barriers to entry, no certi!cation or regulatory body.” (Pang, 2009)
Commoditize the scenarios market, split between “fast & cheap” or “slow & bespoke”
Trade-off between quality (qualitative) aspects & quantity / speed
34
Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches
Findings & DiscussionSpeculative Finding 3: Impact on Scholarly Method
Continuous, self-re!ective and emergent
Allow for user re!ection on, and modi"cation of, research constructs
“Moderators... sometimes have the feeling that they’re barely holding on for dear life, because sometimes the carriage tries to run away without them.”
Requires post-hoc and real-time evaluation, dif"cult to determine what to study in advance
Signi"cantly enhanced potential for creativity, but signi"cant challenges for research design and rigor
35
Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches
Outline1. Introduction
2. Review & Synthesis of the Literature
3. Study Design & Methodology
4. Findings & Discussion
5. Conclusion
36
Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches
Conclusion
37
Contribution
1) Creating under-explored connections between urban planning, public participation, online tools and scenario planning
2) The creation and evaluation of two unique online platforms for participatory scenario planning in urban planning and public policy
2) The creation of an intellectual framework for measuring and evaluating their role in the qualitative scenario planning process.
Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches
Conclusion
38
Limitations
Lack of a more rigorous experimental design, more controlled cases or a peer-reviewed evaluation framework
Lack of a controlled, standardized recruitment process for participation
Differences in de"nitions, processes and goals between cases and comparative examples
Strongly dissenting views and participants self-selected out of being interviewed, thereby biasing the results and discussion towards those available and interested in the subject
Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches
Conclusion
39
Possible Evolution of These Approaches
Personal Futures Systems
Real-time Horizon Scanning & Scenario Generation Systems
Crowdsourced Think Tank Policy Review
Mass Media Speculation Engines
Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches
Conclusion
40
Areas for Future Research
Continue to develop more rigorous measures for evaluating the scenario process and its outcomes
Conduct more controlled research on the impacts of speci"c design and interaction features
Explore the impact of various forms of socialization systems (chat, commenting, voting, etc.) on the process and outcomes
Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches
Thank You
41
Noah RafordPhD Candidate, Urban Information Systems Group, City Design and Development Group, Department of Urban Studies and Planning, MIT
August 29, 2011
Questions?