meghna babbar-sebens - the participatory web –a medium for human-computer collaborative design of...
Upload: consortium-of-universities-for-the-advancement-of-hydrologic-science-inc
Post on 15-Dec-2015
9 views
DESCRIPTION
2015 CUAHSI Hydroinformatics ConferenceTRANSCRIPT
-
The Participatory Web A medium for human-computer collaborative design of watershed
management solutions
Meghna Babbar-Sebens, Ph.D., Oregon State University
3rd CUAHSI Conference on HydroInformatics, July 17th 2015, Tuscaloosa, AL
1
-
Outline
1. Acknowledgements2. Introduction Need for participatory design in watershed planning Human-centered design on the Participatory Web
3. Research Goals4. Methodology5. Results Lessons in experiments with students and stakeholders
1. User learning and engagement2. User models for detecting revealed preferences3. Interactive search algorithms
6. Conclusions
2
-
1. ACKNOWLEDGEMENTS
Collaborators and Students Snehasis Mukhopadhyay, E Jane Luzar, Kristen Macuga, Edna Loehman,
Adriana D. Piemonti, Vidya B. Singh, Jon Eynon, Jill Hoffman, and the various student participants
Funding Agencies (NSF Award IDs ESE:1014693/ 1332385; NOAA/CPO/SARP Award ID NA14OAR4310253; ISDA Award ID A337-9-PSC-002)
Indiana Universitys FutureSystems (old name FutureGrid) (NSF Grant # 0910812)
Stakeholder participants and partners
3
-
Restoration of the hydrologic cycle in degraded watersheds need a network of
distributed conservation and storage practices Stakeholder engagement is
even more critical now!
acceptable choices will depend on the local site and stakeholder conditions and preferences
Image source: http://cdn.phys.org/
2. INTRODUCTION
References: Hey et al. 2004; Mitsch & Day Jr., 2006; Heisel, 2009
Filter Strips
No-till
Grassed Waterways
Wetlands
4
-
Participatory Methods for Engaging Stakeholders
Integrated Watershed & Water Resources Management (Schramm, G. 1980; Viessman, W. 1996; Hooper, 2007)
Shared Vision Modeling and Planning (Hamlet, A. et al. 1996; Palmer et al., 1995)
subjectivity and preferences
5
-
Participatory Design of Alternatives
Crowd-sourcing the design process to the community Bottom-up and democratic
problem-solving process Community building and
empowerment Emergence of a new human-
based computing paradigm, where human-computer interactions play an important role in design algorithms
References: von Ahn, 2009; Fraternali et al., 2012
6
-
Opportunities on the Participatory Web 2.0 ( and the Web 3.0 in the future)
The Internet provides opportunities for coordination of efforts of billions of humans Combined power of humans and computers can now be
harnessed to solve complex problems!
Consideration of issues that arise when human are included in the loop (Ipeirotis and Paritosh, 2011) Human factors Quality of human contribution Market, ethical, and legal aspects
7
-
The Participatory Optimization Concept
Optimization Algorithm Operations
Numerical Evaluation
Optimal Solutions
Initial Feed of Designs
Non-interactive Optimization Algorithm Approach
New Designs
(Interactive Optimization/ Human-guided search/ Human-Centered Optimization)8
-
User Web Interface
Feedback(Information
Gathering and
Decision Making)
Feedback(Likert Scale User Rating and UsabailityData)
Designs/Alternatives
OptimizationAlgorithm (IGAMII) Operations
Numerical Evaluation
Optimal Solutions
Initial Feed of Designs
SubjectiveEvaluation New Designs
Designs/Alternatives
Babbar-Sebens et al. (2015)
Online User Preference Models
The Participatory Optimization Concept(Interactive Optimization/ Human-guided search/ Human-Centered Search)
Interactive Optimization Algorithm Approach
9
-
3. RESEARCH GOALS
Represent stakeholders qualitative/unquantified knowledge (wisdom) within numerical optimization process through interaction
Participatory optimization algorithms and tools1. How do users engage with and learn from such
participatory optimization tools?2. Can we model users nonstationary and noisy feedback
provided through the interfaces to detect underlying revealed preferences?
3. How does the human-centered search process affect the design of watershed plans?
10
-
4. METHODOLOGYWRESTORE (http://wrestore.iupui.edu)11
-
WRESTOREs Client-Server Architecture
Arrows indicate data flow and workflow
HTTP
(Thin clients)
Web Interface
12
-
13
Participatory Optimization Introspection Loop to support Users Cognitive Learning
Participatory optimization/Human-
guided search sessions (Human or Simulated
User)
Introspection sessions (Human User)
Im sessionsHSn_m sessions
Where,n = number of iterations in optimization algorithmm = number of introspection sessions
n
m loops
Participatory optimization algorithm: IGAMII (Babbar-Sebens et al., 2015)
13
-
Restoring upland storage in Eagle Creek Watershed (HUC 11), Central Indiana
Conservation practices modeled using Soil and Water Assessment Tool (Neitsch et al., 2005), for a 2005-2008 simulation period.
Impact of practices estimated by comparing economic costs, peakflows, sediments, and nitrateswith the baseline scenario that does not contain any new practice.
Study site14
-
20 participants: Students and Stakeholders Decision variables: cover crops (0/1) and
filter strips (0-5m) in 108 sub-basins Quantitative objectives (Numerical
Evaluation): Economic Costs, Peak flow reduction, Nitrate reduction, and Sediment reduction at watershed scale
Qualitative objective (Subjective Evaluation): User Rating of alternatives based on users subjective assessment of suitability of practices in their local sub-basins
5. RESULTS
2 Loops
6
15
-
Q1. How do users engage with and learn from such participatory optimization tools?
a) Overall response times
Stakeholders exhibited more variability from DeJongs power learning curve in HSn_m sessions Small sample size, statistical detection of outliers, stakeholders might have a different
learning model structure
Im sessions HSn_m sessionsWhere,n = 1 to 6m = 1 to 3
- By tracking usability metrics (Tullis and Albert (2013))16
-
Q1. How do users engage with and learn from such participatory optimization tools?
b) Information Gathering versus Decision Making Stakeholders spent 16% more time on Information Gathering than students Stakeholders clicked 19% more in information gathering areas than students
I1 HS_1 I2 HS_2 I3
Students
Stakeholders52%
19%
29%45%
20%
35%
51%
12%
37% 46%
25%
29% 38%
5%
57%
35%
23%
42% 33%
25%
42%27%
13%60%
25%
32%
43% 32%
22%
46%
Mean percentage of response times
I1 HS_1 I2 HS_2 I3
Students
Stakeholders
39%
39%
22% 30%
51%
19%29%
30%
41%25%
56%
19%32%
40%
28%
55%30%
15%
46%
39%
15%
53%23%
24%48%
37%
15%
48%
7%
45%
Mean percentage of clicking events
Students
Stakeholders
Students
Stakeholders
17
-
Q1. How do users engage with and learn from such participatory optimization tools?
c) Response time vs. number of clicks per session per participant and based on trends in averages of self-reported confidence levels
y = 18.359x0.9401R = 0.6727
y = 10.023x0.8133R = 0.5581
y = 7.7771xR = 0.6645
0
50
100
150
200
250
300
0.00 5.00 10.00 15.00 20.00 25.00 30.00 35.00
Num
ber o
f Clic
ks
Time (min)
Information Gathering
y = 28.126x0.5678R = 0.6363
y = 29.703x0.6113R = 0.6488
y = -3.4978x2 + 26.858xR = 0.5815
0.00 5.00 10.00 15.00 20.00 25.00 30.00 35.00Time (min)
Decision Making
Positive trend Negative trend No trend
When users spend more time and make more mouse clicks in information gathering areas, their confidence levels are likely to increase over time
18
-
Q2. Can we model users nonstationary and noisy feedbacks provided through the interfaces to detect underlying revealed preferences?
Simulated Users (Avatars) To enable extensive search where individual users qualitative criteria is represented via user models
Which Machine Learning Algorithm to use?
User modeling in Environmental Problems is still in infancy! Lessons can be learned from HCI, Intelligent Interfaces, Adaptive Interfaces,
Cognitive Engineering, Intelligent Information Retrieval, Intelligent Tutoring, Expert Systems, etc.
Challenges:
High Number of input parameters
Varying size of input parameters
Limited Feedback data
Skewed training data
Online User Preference Models
19
-
Q2. Can we model users nonstationary and noisy feedbacks provided through the interfaces to detect underlying revealed preferences?
X axis: SDMs at end of introspection sessions1: I1 data = Epoch 12: HS_1 & I2 data = Epoch 23: HS_2 & I3 data = Epoch 34: Epoch 1 & 2 data5: Epoch 1, 2, & 3 data
Erro
r in
pre
dict
ed u
ser
ratin
g
Stakeholder SDM
Deep Learning tended to perform much better than any other machine learning algorithm we used
m loops20
-
Q3. How does the user-guided search affect the design of watershed plans?
10
15
20
25
PF
R (
cms)
Participant1
-2.6 -2.4 -2.2 -2 -1.8 -1.6x 107
4.5
5
5.5
x 106
Cost ($/Watershed)
NR
(kg
/Wat
ersh
ed)
PARTICIPANT 1
10
15
20
25P
FR (c
ms)
Participant2
-2.6 -2.4 -2.2 -2 -1.8 -1.6 -1.4x 107
4.5
5
5.5
x 106
Cost ($/Watershed)
NR
(kg/
Wat
ersh
ed)
PARTICIPANT 2
In Objective Space, differences and similarities can be detected in the acceptable or unacceptable alternatives found by the users
21
-
Q3. How does the user-guided search affect the design of watershed plans?
In Decision Space, Cover crops: Across all participants, 45% of sub-basins have low variability in
probability of cover crops (i.e. standard deviation< 0.13) Filter widths: Across all participants, 69% of the sub-basins have low variability in
modes (i.e. < 1.2 m)
Cover crops across participants
Higher average
probability across
participants
Filter Strip width across participants
Standard deviation
I like it Alternatives
Higher mode
-
Interface usability is important! Participants can have the same view for different
reasons. Cues are interpreted differently when they occur
Participants can have different views based on the same information different interpretations
User modeling needs to be adaptive to a humans own learning process
Putting stakeholders-in-the-loop enables identification of design features in alternatives that are satisficing from the humans perspective
6. CONCLUSIONS
-
Paradigm shift: Optimization based on human-computer collaboration via interaction. The participatory web provides an enabling framework. Knowledge and preferences of community included. Supports learning process of decision makers Need for collaborations across multiple disciplines
Foundation laid for further investigation in interactive and participatory design methodologies How will cognitive machines solve complex participatory design
problems for watershed groups in the future?
6. CONCLUSIONS24
-
Questions?
mathematical formulas alone do not produce consistently relevant results. Human intelligence is still a very important part of the process.
(Jimmy Wales, Founder of Wikipedia. Source - Businessweek.com, 12/2006)
25
The Participatory Web A medium for human-computer collaborative design of watershed management solutionsOutline1. ACKNOWLEDGEMENTSSlide Number 4Participatory Methods for Engaging StakeholdersParticipatory Design of AlternativesOpportunities on the Participatory Web 2.0 ( and the Web 3.0 in the future)The Participatory Optimization ConceptThe Participatory Optimization Concept3. RESEARCH GOALSSlide Number 11WRESTOREs Client-Server ArchitectureSlide Number 13Study siteSlide Number 15Q1. How do users engage with and learn from such participatory optimization tools?Q1. How do users engage with and learn from such participatory optimization tools?Q1. How do users engage with and learn from such participatory optimization tools?Q2. Can we model users nonstationary and noisy feedbacks provided through the interfaces to detect underlying revealed preferences?Q2. Can we model users nonstationary and noisy feedbacks provided through the interfaces to detect underlying revealed preferences?Q3. How does the user-guided search affect the design of watershed plans?Q3. How does the user-guided search affect the design of watershed plans?6. CONCLUSIONS6. CONCLUSIONSSlide Number 25