making representations matter: understanding practitioner experience in participatory sensemaking
DESCRIPTION
TRANSCRIPT
Making Representations Matter Understanding Practitioner Experience in Participatory Sensemaking
Al Selvin
Knowledge Media Institute The Open UniversityMilton Keynes, UKandVerizon Telecom ITValhalla, NY USA
http://people.kmi.open.ac.uk/selvin
8 June 2011, The Open University
What is practitioner sensemaking in participatory representations?
Other kinds of participatory representational experiences
3
Everyday participatory representational experiences
4:47
Background and motivation
• Development of the Compendium software, methodology, and practice
• Ten years of practice experience in business, research, and community settings
• Desire to find research that talked about the experience of such practice
� What practitioners encounter in the heat of the moment
� How they make sense of anomalies and shape representations of value to their participants
� Characterizing the practice as articulation work
Research context
Research questions
• RQ1: How to characterize and compare the interactions of specific representational situations and practitioner actions?
• RQ2: What kinds of obstacles, breaches, discontinuities, and anomalies occur that interfere with a representation's coherence, engagement, or usefulness?
• RQ3: How do practitioner actions at sensemaking moments serve to restore coherence, engagement, and usefulness?
• RQ4: What are the specific practices involved in making the hypermedia aspects of the representation coherent, engaging, and useful?
Conceptual framework
8
Conceptual framework
9
Conceptual framework
10
Conceptual framework
11
Other forms of participatory representation also map on
12
Initial literature
review
Grid and sensemaking
moment analyses of
two expert in situ sessions
Analysis: Need ways to characterize whole session and context
Subjects: Need to contrast with non-expert, non-in situ
sessions
Data: Need skill and experience profiles of
practitioners
1 2
3
4
5
Round 1: Pilot study
Exploratory qualitative approach
14
Initial literature
review
Grid and sensemaking
moment analyses of
two expert in situ sessions
Analysis: Need ways to characterize whole session and context
Subjects: Need to contrast with non-expert, non-in situ
sessions
Data: Need skill and experience profiles of
practitioners
1 2
3
4
5
Round 1: Pilot study
Develop CEU and Shaping analysis
tools
Conduct Ames and
Rutgers sessions
Develop subject questionnaire
Second literature
reviewAnalysis: Need way to
characterize ‘experience’ dimensions
6
7
8
9
10
Round 2: Expanded study
Exploratory qualitative approach
15
Initial literature
review
Grid and sensemaking
moment analyses of
two expert in situ sessions
Analysis: Need ways to characterize whole session and context
Subjects: Need to contrast with non-expert, non-in situ
sessions
Data: Need skill and experience profiles of
practitioners
1 2
3
4
5
Round 1: Pilot study
Develop CEU and Shaping analysis
tools
Conduct Ames and
Rutgers sessions
Develop subject questionnaire
Second literature
reviewAnalysis: Need way to
characterize ‘experience’ dimensions
6
7
8
9
10
Round 2: Expanded study
Develop Framing Model
analysis tool
Apply all five
analysis tools to all
eight sessions
Conduct comparative
analysis across
sessions11
12
13
Round 3: Comparative study
Exploratory qualitative approach
Practice /Experience Dimension
Computing research
Practitioner studies
Participatory design
Facilitation & mediation
Arts-based practices
Aesthetics
Ethics
Narrative
Sensemaking
Improvisation
Related work
16
Practice /Experience Dimension
Computing research
Practitioner studies
Participatory design
Facilitation & mediation
Arts-based practices
Aesthetics
Ethics
Narrative
Sensemaking
Improvisation
Related work
17
• Visual representations in communication and group work
• Engagement with such representations
• The importance of situation and context in studying practice
• Analysis at the move-by-move level
• Limitations of research focused on tools, methods, and outcomes
Research settings – Mobile Agents
Hab Crew
Remote Science Team
18
Research settings – Ames
Ames Group 1 Ames Group 2
Ames Group 4Ames Group 3 19
Research settings – Rutgers
Rutgers Group 1
Rutgers Group 2
20
Characteristics of practitioners
21
22
12
3
4
5
67
8
9
10
11
12
0
5
10
Ames Group 11
2
3
4
5
67
8
9
10
11
12
-4
1
6
Ames Group 21
2
3
4
5
67
8
9
10
11
12
-4.0
1.0
6.0
Ames Group 3
12
3
4
5
67
8
9
10
11
12
-4
1
6
Ames Group 41
2
3
4
5
67
8
9
10
11
12
0
5
10
Rutgers Group 11
2
3
4
5
67
8
9
10
11
12
0
5
10
Rutgers Group 2
12
3
4
5
67
8
9
10
11
12
0
5
10
RST1
2
3
4
5
67
8
9
10
11
12
0
5
10
Hab Crew
Practitioner skills and experience
Larger plot = greater levels of self-reported skill and experience
1How long have you been using Compendium?
2How long have you acted as a facilitator of groups?
3How long have you acted as a facilitator of groups using any kind of software?
4How long have you acted as a facilitator of groups using Compendium?
5How many times or sessions have you acted as a facilitator?
6How many times or sessions have you acted as a facilitator of groups using any kind of software?
7How many times or sessions have you acted as a facilitator of groups using Compendium?
8How would you describe your skill level with knowledge mapping / concept mapping software?
9How would you describe your skill level with Compendium?
10How would you describe your skill level as a group facilitator?
11How would you describe your level of technical proficiency with software?
12How familiar are you with hypermedia and hypertext concepts?
Shaping form
CEU analysis
Narrative description
Grid analysis
Framing analysis
Characterizing the representational characterof the whole session
What kind of shaping took place?
Analytical tools
23
Shaping form
CEU analysis
Narrative description
Grid analysis
Framing analysis
Characterizing the representational characterof the whole session
What kind of shaping took place?
Analytical tools
24
AG4 Example
Shaping form
CEU analysis
Narrative description
Grid analysis
Framing analysis
Mapping the coherence, engagement, and usefulness dimensions of each timeslot to build up a signature for the session
Aids in identifying sensemaking episodes
Analytical tools
25
Shaping form
CEU analysis
Narrative description
Grid analysis
Framing analysis
Mapping the coherence, engagement, and usefulness dimensions of each timeslot to build up a signature for the session
Aids in identifying sensemaking episodes
Analytical tools
26
AG4 Example
Shaping form
CEU analysis
Narrative description
Grid analysis
Framing analysis
Rich description of sensemaking episode
Analytical tools
27
Shaping form
CEU analysis
Narrative description
Grid analysis
Framing analysis
Rich description of sensemaking episode
Analytical tools
28
AG4 Example
Shaping form
CEU analysis
Narrative description
Grid analysis
Framing analysis
Micro-moment moves and choices during the episode
Analytical tools
29
Shaping form
CEU analysis
Narrative description
Grid analysis
Framing analysis
Micro-moment moves and choices during the episode
Analytical tools
30
AG4 Example
Shaping form
CEU analysis
Narrative description
Grid analysis
Framing analysis
Characterizing the practitioner actions during the episode in aesthetic, ethical, and experiential terms (informed by theoretical framework)
Analytical tools
31
Shaping form
CEU analysis
Narrative description
Grid analysis
Framing analysis
Characterizing the practitioner actions during the episode in aesthetic, ethical, and experiential terms (informed by theoretical framework)
Analytical tools
32
AG4 Example
33
Session
Context
(historical,technical,
social, etc.)
TimeslotChoices/Moves
Choices/Moves
Choices/Moves
Choices/Moves
Choices/Moves
Granularity
Shaping & FramingAnalysis
Sensemaking Moment & Grid Analysis
CEUAnalysis
Analytical tools
Timeslot
Timeslot
Session
TimeslotChoices/Moves
Choices/Moves
Choices/Moves
Choices/Moves
Choices/Moves
Timeslot
Timeslot
Session
TimeslotChoices/Moves
Choices/Moves
Choices/Moves
Choices/Moves
Choices/Moves
Timeslot
Timeslot
34
CEU heat maps showing sensemaking episodes
35
Sensemaking triggers and responses (AG4 example)
Trigger Response
Response type: Holding forward progress until new strategy is in place
Ethical dimension: Direct intervention for purpose of practitioner action
Aesthetic dimension: Creating space for remedial shaping to take place
Pertaining to volume or type of participant input (“Too much too fast”)
Facilitator: “But we had a question that says ‘how can the public become co-creators?’”Mapper creates Question node and facilitator narrates answers from previous discussion
Collaborative navigation to find item of interest
Negotiation/agreement on placement of an item
Direct collaboration between practitioner and participants
36Hab
Triggers (RQ2)
Pertaining to representational structure
Pertaining to volume or type of participant input
Pertaining to information/subject matter
Pertaining to intended process/plan
37
Sensemaking dimensions
Ethical Dimensions
Direct collaboration between practitioners and participants
Direct intervention aimed at participants
Direct intervention for purpose of practitioner action
Indirect intervention
Changing/blurring roles
Non-intervention
Aesthetic Dimensions
Direct contribution to shaping
Intended to help participant shaping
Creating space for remedial shaping to take place
Partially having to do with shaping
No aesthetic dimension
Practitioner responses (RQ1, 3)
Triggers (RQ2)
Pertaining to representational structure
Pertaining to volume or type of participant input
Pertaining to information/subject matter
Pertaining to intended process/plan
38
Sensemaking dimensions
Ethical Dimensions
Direct collaboration between practitioners and participants
Direct intervention aimed at participants
Direct intervention for purpose of practitioner action
Indirect intervention
Changing/blurring roles
Non-intervention
Aesthetic Dimensions
Direct contribution to shaping
Intended to help participant shaping
Creating space for remedial shaping to take place
Partially having to do with shaping
No aesthetic dimension
Practitioner responses (RQ1, 3)
AG4
• Overall context, tone, or characterof a session
• Initial plan and other pre-session factors• Dimensions
• Advance• As-played-out
• Interpersonal interactions and communicative styles
• Dimensions• Regulating• Bringing to the representation• Collaboration (style, force, purpose)
• Physical and conceptual shaping of the representations
• Relating the sessions to the normative model
Shaping and Framing categories
Category A – Conducting (RQ1)
Category B – Planning (RQ1, 3)
Category C – Relating (RQ1, 2, 3)
Category D – Shaping (RQ1, 4)
Category E – Framing (RQ1, 3)
How “good”/successful was the session?Multiplicity/heterogeneity of focus aspectsDegree of expressed participant resistance, disagreementDegree of ‘noise,’ chaos, boisterousness Degree of “meta” discussionSpectrum from “discussion-centric” to “map-centric”
Choice of methodHow much of shaping/process is "emergent" vs. pre-determinedGranularity of the pre-created structure (degree and complexity)Ambitiousness of the planned approachDegree of practitioner adherence to method during the sessionParticipant adherence/faithfulness to the intended plan
Density of practitioner verbal moves (frequent vs. infrequent)Practitioner willingness to interveneHigh practitioner “drive” of the session vs. high participant “drive” Practitioner-asked clarifying questions to participant inputPractitioner-requested validation of changesPractitioner “gating” of participant input Intervention to get participants to look at representationCollaboration between multiple practitioners Collaboration between practitioners and participants
Attention to textual refinement of shapingAttention to visual/spatial refinement of shapingAttention to hypertextual refinement of shapingDegree of ‘finishedness’ of the artifactsDensity of practitioner shaping movesComplexity of the software techniques in useDegree of de-linked interaction with representation
Narrative consistency and usefulness Inclusiveness of the narrative framingEvocativeness of the narrative framing Clarity of artifacts Openness and dialogicity of the mediated objects Resistance from participants and materials Addressing and incorporating participant impulses and desires
Shaping and Framing categories
Category A – Conducting (RQ1)
Category B – Planning (RQ1, 3)
Category C – Relating (RQ1, 2, 3)
Category D – Shaping (RQ1, 4)
Category E – Framing (RQ1, 3)
Comparative method
Using Compendium to rank order the sessions along eachqualitative dimension and capture rationale
(http://people.kmi.open.ac.uk/selvin/analysis)
Comparative method
Using Compendium to rank order the sessions along eachqualitative dimension and capture rationale
(http://people.kmi.open.ac.uk/selvin/analysis)
AG4
Shaping rankings and ratings
RG2
RST
Granularity of pre-created structureHigh Low
RST Hab AG2 RG1 AG4 AG3 AG1 RG2
44
Intervention to get participants to look at the representation
RG2
45
Intervention to get participants to look at the representation
AG4
High LowRST
High
AG3
High
Hab
Med High
RG2
Med
RG1
Med
AG1
Med Low
AG2
Low
AG4
Low
AG3
AG4
Visual/spatial refinement
47AG4
High LowRST
High
AG3
High
RG2
Med High
Hab
Med
AG1
Med Low
RG1
Low
AG4
Low
AG2
Low
RG2
Hab
Hypertextual refinement
48
49
1 23
45
6
7
8
9
1011
12131415
1617
18
19
20
21
22
2324
2526
0
5
10
Ames Group 11 2
34
5
6
7
8
9
1011
12131415
1617
18
19
20
21
22
2324
2526
0
5
10
Ames Group 21 2
34
5
6
7
8
9
1011
12131415
1617
18
19
20
21
22
2324
2526
0
5
10
Ames Group 3
1 23
45
6
7
8
9
1011
12131415
1617
18
19
20
21
22
2324
2526
0
5
10
Ames Group 41 2
34
5
6
7
8
9
1011
12131415
1617
18
19
20
21
22
2324
2526
0
5
10
Rutgers Group 11 2
34
5
6
7
8
9
1011
12131415
1617
18
19
20
21
22
2324
2526
0
5
10
Rutgers Group 2
1 23
45
6
7
8
9
1011
12131415
1617
18
19
20
21
22
2324
2526
0
5
10
RST1 2
34
5
6
7
8
9
1011
12131415
1617
18
19
20
21
22
2324
2526
0
5
10
Hab Crew
It is now possible to compare sessions along the experiential dimensions
Larger plot = generally higher rankings
Shaping and framing dimensions
50
12
3
4
5
67
8
9
10
11
12
0
5
10
Ames Group 11
2
3
4
5
67
8
9
10
11
12
-4
1
6
Ames Group 21
2
3
4
5
67
8
9
10
11
12
-4.0
1.0
6.0
Ames Group 3
12
3
4
5
67
8
9
10
11
12
-4
1
6
Ames Group 41
2
3
4
5
67
8
9
10
11
12
0
5
10
Rutgers Group 11
2
3
4
5
67
8
9
10
11
12
0
5
10
Rutgers Group 2
12
3
4
5
67
8
9
10
11
12
0
5
10
RST1
2
3
4
5
67
8
9
10
11
12
0
5
10
Hab Crew
Practitioner skills and experience
51
1 23
45
6
7
8
9
1011
12131415
1617
18
19
20
21
22
2324
2526
0
5
10
Ames Group 11 2
34
5
6
7
8
9
1011
12131415
1617
18
19
20
21
22
2324
2526
0
5
10
Ames Group 21 2
34
5
6
7
8
9
1011
12131415
1617
18
19
20
21
22
2324
2526
0
5
10
Ames Group 3
1 23
45
6
7
8
9
1011
12131415
1617
18
19
20
21
22
2324
2526
0
5
10
Ames Group 41 2
34
5
6
7
8
9
1011
12131415
1617
18
19
20
21
22
2324
2526
0
5
10
Rutgers Group 11 2
34
5
6
7
8
9
1011
12131415
1617
18
19
20
21
22
2324
2526
0
5
10
Rutgers Group 2
1 23
45
6
7
8
9
1011
12131415
1617
18
19
20
21
22
2324
2526
0
5
10
RST1 2
34
5
6
7
8
9
1011
12131415
1617
18
19
20
21
22
2324
2526
0
5
10
Hab Crew
It is now possible to compare sessions along the experiential dimensions
Larger plot = generally higher rankings
Shaping and framing dimensions
52
Facilitation skills were a stronger predictor than technical skills
SessionShapingIndex
Session goodness
rank
Software proficiency
rank
Facilitation proficiency
rank
Hab 83 1 1 1
RST 78 2 1 2
RG2 70 5 4 3
AG4 66 3 6 5
AG3 55 7 2 6
RG1 54 4 7 5
AG1 41 6 5 4
AG2 18 8 3 7
Shaping/framing dimension
A.1 How “good”/successful was the session?
B.5Practitioner adherence to the intended method during the session
B.6Participant adherence/faithfulness to the intended plan
C.2 Practitioner willingness to intervene
C.4 Practitioner-asked clarifying questions
C.5Practitioner-requested validation of changes to representation
C.7Intervention to get participants to look at the representation
C.8Collaboration between multiple practitioners (if applicable)
C.9Collaboration/co-construction between practitioners and participants
D.1 Attention to textual refinement of shaping
D.2 Attention to visual/spatial refinement of shaping
D.3 Attention to hypertextual refinement of shaping
D.4 Degree of ‘finishedness’ of the artifacts
Generalization to other genres of participatory representations
Graphic facilitation
Informal whiteboarding
54
Contributions
• Offers analytical tools
Contributions
• Offers analytical tools
• Provides a language to characterize and compare instances of representational practice
Contributions
• Offers analytical tools
• Provides a language to characterize and compare instances of representational practice
• Describes the types of sensemaking moments that practitioners encounter
57
Contributions
• Offers analytical tools
• Provides a language to characterize and compare instances of representational practice
• Describes the types of sensemaking moments that practitioners encounter
• Highlights the specific role of a hypermedia technology
58
Contributions
• Offers analytical tools
• Provides a language to characterize and compare instances of representational practice
• Describes the types of sensemaking moments that practitioners encounter
• Highlights the specific role of a hypermedia technology
• Contributes to reflective methods for practitioner and practice development
59
Future work
• Studies� Performing longitudinal studies and action research with
emphasis on artifactual sensemaking� Comparing other practices
• Tools and methods� Developing the analytical tools� Developing training, assessment, and reflective practice
methodologies
• Theory� Exploring the “recursive” nature of the experiential
dimensions as they relate to representational practices (as something you apply vs. something you live within)
60
For more
• Analysis artifacts http://people.kmi.open.ac.uk/selvin/analysis/
• Research blog http://knowledgeart.blogspot.com
• Recent journal articlehttp://oro.open.ac.uk/20948/1/Selvin-HumanTechnology2010.pdf