intentions to guide learning analytics
TRANSCRIPT
![Page 1: Intentions to guide Learning Analytics](https://reader031.vdocuments.site/reader031/viewer/2022020706/61fd0abc07b3d029d83571e9/html5/thumbnails/1.jpg)
Script-Aware Monitoring Model:
Using Teachers’ Pedagogical
Intentions to guide Learning
Analytics
María Jesús Rodríguez-Triana, Alejandra Martínez
Monés, Juan Ignacio Asensio Pérez and Yannis
Dimitriadis
TaPTA Workshop @ ECTEL 2012
18/09/2012
University of Valladolid Spain
![Page 2: Intentions to guide Learning Analytics](https://reader031.vdocuments.site/reader031/viewer/2022020706/61fd0abc07b3d029d83571e9/html5/thumbnails/2.jpg)
Overview Context and motivation
Research goals
Script-aware monitoring model
Conclusions
2
![Page 3: Intentions to guide Learning Analytics](https://reader031.vdocuments.site/reader031/viewer/2022020706/61fd0abc07b3d029d83571e9/html5/thumbnails/3.jpg)
Overview Context and motivation
Research goals
Script-aware monitoring model
Conclusions
3
![Page 4: Intentions to guide Learning Analytics](https://reader031.vdocuments.site/reader031/viewer/2022020706/61fd0abc07b3d029d83571e9/html5/thumbnails/4.jpg)
Context
4
Research on CSCL & TEL Focus on (non-expert) teachers Focus on learning design / CSCL scripts Focus on higher education (but also
primary)
2011 2012 2013
M. J. Rodríguez thesis …
Middle stage
![Page 5: Intentions to guide Learning Analytics](https://reader031.vdocuments.site/reader031/viewer/2022020706/61fd0abc07b3d029d83571e9/html5/thumbnails/5.jpg)
Context Computer Supported Collaborative Learning
5
EVALUATION
ENACTMENT
DESIGN
INSTANTIATION
Teacher-oriented approach
CSCL situation
life-cycle [Gómez-Sánchez et al., 2009]
![Page 6: Intentions to guide Learning Analytics](https://reader031.vdocuments.site/reader031/viewer/2022020706/61fd0abc07b3d029d83571e9/html5/thumbnails/6.jpg)
One of the main concerns:
enhance collaboration
6
EVALUATION
ENACTMENT
DESIGN
INSTANTIATION
Scripting
![Page 7: Intentions to guide Learning Analytics](https://reader031.vdocuments.site/reader031/viewer/2022020706/61fd0abc07b3d029d83571e9/html5/thumbnails/7.jpg)
However, despite using scripting
...
7
… eventualities may happen
Hmm,
we could try with a
brainstorming by
pairs ...
Having a plan is
not enough ...
![Page 8: Intentions to guide Learning Analytics](https://reader031.vdocuments.site/reader031/viewer/2022020706/61fd0abc07b3d029d83571e9/html5/thumbnails/8.jpg)
Monitoring can help to know
what is happening
8
EVALUATION
ENACTMENT
DESIGN
INSTANTIATION
Monitoring
[Soller et al., 2005]
![Page 9: Intentions to guide Learning Analytics](https://reader031.vdocuments.site/reader031/viewer/2022020706/61fd0abc07b3d029d83571e9/html5/thumbnails/9.jpg)
However, monitoring approaches
...
9 … often do not satisfy teachers' needs
Being aware of the participants
behaviour when diverse ICT tools are
being used in the classroom ...
is time consuming and error-prone
Ana
lysi
s ef
fort
2 students
1 tool
Amount of data
1 group
1 tool
whole class
1 tool
whole clase
several tools Teachers need a more
global view about what
is happening / has
happened
![Page 10: Intentions to guide Learning Analytics](https://reader031.vdocuments.site/reader031/viewer/2022020706/61fd0abc07b3d029d83571e9/html5/thumbnails/10.jpg)
Synergies may appear when combining
scripting and monitoring ...
10
EVALUATION
ENACTMENT
DESIGN
INSTANTIATION
Monitoring
Scripting
… however, the integration is not straightforward (Martínez-Monés et al., 2011)
Design problems
Technological problems
![Page 11: Intentions to guide Learning Analytics](https://reader031.vdocuments.site/reader031/viewer/2022020706/61fd0abc07b3d029d83571e9/html5/thumbnails/11.jpg)
Besides … the complexity of current CSCL
scenarios hinders orchestration
11
EVALUATION
ENACTMENT
DESIGN
INSTANTIATION
Distributed Learning
Environments (DLEs)
≠ Locations ≠ Interactivity types
≠ Social levels
(Dillenbourg et al., 2009)
![Page 12: Intentions to guide Learning Analytics](https://reader031.vdocuments.site/reader031/viewer/2022020706/61fd0abc07b3d029d83571e9/html5/thumbnails/12.jpg)
Teachers need monitoring tools for
orchestration of blended, scripted CSCL
scenarios supported by DLEs
12
EVALUATION
ENACTMENT
DESIGN
INSTANTIATION
Monitoring
DLEs
Blended CSCL
scenarios
Scripting
![Page 13: Intentions to guide Learning Analytics](https://reader031.vdocuments.site/reader031/viewer/2022020706/61fd0abc07b3d029d83571e9/html5/thumbnails/13.jpg)
Overview Context and motivation
Research goals and expected contributions
Script-aware monitoring model
Conclusions
13
![Page 14: Intentions to guide Learning Analytics](https://reader031.vdocuments.site/reader031/viewer/2022020706/61fd0abc07b3d029d83571e9/html5/thumbnails/14.jpg)
Main Goal
Provide non-expert teachers
with design and enactment support
capable of linking pedagogical intentions with
monitoring needs
for orchestrating blended CSCL scenarios
in distributed learning environments
14
![Page 15: Intentions to guide Learning Analytics](https://reader031.vdocuments.site/reader031/viewer/2022020706/61fd0abc07b3d029d83571e9/html5/thumbnails/15.jpg)
EVALUATION
ENACTMENT
DESIGN
INSTANTIATION
Monitoring
Scripting
Support non-expert teachers/designers to identify and include the monitoring issues
throughout the design process of CSCL activities. 1
Partial objectives
15
![Page 16: Intentions to guide Learning Analytics](https://reader031.vdocuments.site/reader031/viewer/2022020706/61fd0abc07b3d029d83571e9/html5/thumbnails/16.jpg)
EVALUATION
ENACTMENT
DESIGN
INSTANTIATION
Monitoring
Scripting
Support non-expert teachers/designers to identify and include the monitoring issues
throughout the design process of CSCL activities. 1
Provide teachers with coarse-grained information about the evolution of the CSCL situation, based on
the constraints obtained from the script 2
Partial objectives
16
![Page 17: Intentions to guide Learning Analytics](https://reader031.vdocuments.site/reader031/viewer/2022020706/61fd0abc07b3d029d83571e9/html5/thumbnails/17.jpg)
EVALUATION
ENACTMENT
DESIGN
INSTANTIATION
Monitoring
Scripting
Support non-expert teachers/designers to identify and include the monitoring issues
throughout the design process of CSCL activities. 1
Provide teachers with coarse-grained information about the evolution of the CSCL situation, based on
the constraints obtained from the script 2
Support the automation of the data gathering and integration
tasks in CSCL DLEs 3
Partial objectives
17
![Page 18: Intentions to guide Learning Analytics](https://reader031.vdocuments.site/reader031/viewer/2022020706/61fd0abc07b3d029d83571e9/html5/thumbnails/18.jpg)
Expected contributions
18
EVALUATION
ENACTMENT
DESIGN
INSTANTIATION
Monitoring
Scripting
Monitoring-aware learning design process
Design-aware monitoring process
Architecture for monitoring DLEs
![Page 19: Intentions to guide Learning Analytics](https://reader031.vdocuments.site/reader031/viewer/2022020706/61fd0abc07b3d029d83571e9/html5/thumbnails/19.jpg)
Expected contributions
19
EVALUATION
ENACTMENT
DESIGN
INSTANTIATION
Monitoring
Scripting
Monitoring-aware learning design process
Design-aware monitoring process
Script-aware monitoring model
Architecture for monitoring DLEs
![Page 20: Intentions to guide Learning Analytics](https://reader031.vdocuments.site/reader031/viewer/2022020706/61fd0abc07b3d029d83571e9/html5/thumbnails/20.jpg)
Overview Context and motivation
Research goals
Script-aware monitoring model
Conclusions
20
![Page 21: Intentions to guide Learning Analytics](https://reader031.vdocuments.site/reader031/viewer/2022020706/61fd0abc07b3d029d83571e9/html5/thumbnails/21.jpg)
Script-aware monitoring
model Method followed:
1. Analysis of modelling approaches
Goal: Identify elements relevant to scripting and monitoring
2. Pilot studies
Co-design experiences with teachers in real scenarios
Goals:
Identify attributes needed to answer teacher monitoring interests
Test the usefulness of the overall approach
21
![Page 22: Intentions to guide Learning Analytics](https://reader031.vdocuments.site/reader031/viewer/2022020706/61fd0abc07b3d029d83571e9/html5/thumbnails/22.jpg)
Script-aware monitoring model
1. Review of modelling approaches
22
[Weinberger et al., 2009
Dillenbourg & Tchounikine, 2007
Fischer et al., 2007
Kollar et al., 2006]
Groups
Participants
Roles
Resources
Activities
CSCL scripting
![Page 23: Intentions to guide Learning Analytics](https://reader031.vdocuments.site/reader031/viewer/2022020706/61fd0abc07b3d029d83571e9/html5/thumbnails/23.jpg)
Script-aware monitoring model
1. Review of modelling approaches
23
[Harrer et al., 2009] [Weinberger et al., 2009
Dillenbourg & Tchounikine, 2007
Fischer et al., 2007
Kollar et al., 2006]
Groups
Participants
Roles
Resources
Activities Actions
Computer Supported Collaboration Analysis Models
CSCL scripting
![Page 24: Intentions to guide Learning Analytics](https://reader031.vdocuments.site/reader031/viewer/2022020706/61fd0abc07b3d029d83571e9/html5/thumbnails/24.jpg)
24
Roles Participants Groups
Resources Activities
play belong to
Actions
involve conform
are supported by
dependencies
Script-aware monitoring model
Elements
![Page 25: Intentions to guide Learning Analytics](https://reader031.vdocuments.site/reader031/viewer/2022020706/61fd0abc07b3d029d83571e9/html5/thumbnails/25.jpg)
Script-aware monitoring model
2. Case-study
Co-design with a CSCL-design experienced teacher
Two loops of the CSCL life-cycle were enacted
Research goals:
Identify elements and attributes of the model
Collect initial impressions of the teacher after completing
the lifecycle
25
![Page 26: Intentions to guide Learning Analytics](https://reader031.vdocuments.site/reader031/viewer/2022020706/61fd0abc07b3d029d83571e9/html5/thumbnails/26.jpg)
26
Roles Participants Groups
Resources Activities
play belong to
Actions
involve conform
are supported by
dependencies
Script-aware monitoring model
Attributes
![Page 27: Intentions to guide Learning Analytics](https://reader031.vdocuments.site/reader031/viewer/2022020706/61fd0abc07b3d029d83571e9/html5/thumbnails/27.jpg)
27
Roles Participants Groups
Resources Activities
play belong to
Actions
involve conform
are supported by
dependencies
Script-aware monitoring model
Attributes
![Page 28: Intentions to guide Learning Analytics](https://reader031.vdocuments.site/reader031/viewer/2022020706/61fd0abc07b3d029d83571e9/html5/thumbnails/28.jpg)
28
Roles Participants Groups
Resources Activities
play belong to
Actions
involve conform
are supported by
dependencies
Script-aware monitoring model
Attributes
![Page 29: Intentions to guide Learning Analytics](https://reader031.vdocuments.site/reader031/viewer/2022020706/61fd0abc07b3d029d83571e9/html5/thumbnails/29.jpg)
29
Roles Participants Groups
Resources Activities
play belong to
Actions
involve conform
are supported by
dependencies
Script-aware monitoring model
Attributes
![Page 30: Intentions to guide Learning Analytics](https://reader031.vdocuments.site/reader031/viewer/2022020706/61fd0abc07b3d029d83571e9/html5/thumbnails/30.jpg)
During the period set by the deadlines, and according to the social level, is there any evidence of the specified interactivity type from the monitored actions?
30
Roles Participants Groups
Resources Activities
play belong to
Actions
involve conform
are supported by
dependencies
The model in use
![Page 31: Intentions to guide Learning Analytics](https://reader031.vdocuments.site/reader031/viewer/2022020706/61fd0abc07b3d029d83571e9/html5/thumbnails/31.jpg)
During the period set by the deadlines and according to the monitored actions, does the
evidence show that the participants are using the resources as expected?
31
Roles Participants Groups
Resources Activities
play belong to
Actions
involve conform
are supported by
dependencies
The model in use
![Page 32: Intentions to guide Learning Analytics](https://reader031.vdocuments.site/reader031/viewer/2022020706/61fd0abc07b3d029d83571e9/html5/thumbnails/32.jpg)
Script-aware monitoring model The model in use
Example of feedback given to the teacher in the pilot
study: http://www.gsic.uva.es/~chus/MASUP
32
![Page 33: Intentions to guide Learning Analytics](https://reader031.vdocuments.site/reader031/viewer/2022020706/61fd0abc07b3d029d83571e9/html5/thumbnails/33.jpg)
Script-aware monitoring model
Teachers’ feedback
Preliminary feedback from 3 pilot studies:
Specifying the attributes for monitoring required little extra
effort
The feedback helped to confirm that students were
following properly the script
Unexpected situations could be detected on time
Some “false alarms” were raised by the system, but this
was not considered to be a problem
The monitoring process enabled by the system was useful
and efficient
33
![Page 34: Intentions to guide Learning Analytics](https://reader031.vdocuments.site/reader031/viewer/2022020706/61fd0abc07b3d029d83571e9/html5/thumbnails/34.jpg)
Overview Context and motivation
Research goals
Script-aware monitoring model
Final remarks
34
![Page 35: Intentions to guide Learning Analytics](https://reader031.vdocuments.site/reader031/viewer/2022020706/61fd0abc07b3d029d83571e9/html5/thumbnails/35.jpg)
Monitoring-aware learning design
process
Design-aware monitoring
process
Script-aware monitoring model
Architecture for monitoring DLEs
Methodology
Design-based research
Evaluation
• Evaluation with experts
• Case studies in CSCL scenarios
Summary
![Page 36: Intentions to guide Learning Analytics](https://reader031.vdocuments.site/reader031/viewer/2022020706/61fd0abc07b3d029d83571e9/html5/thumbnails/36.jpg)
16/09/2012 36
Next steps