lak13 supporting action research with learning analytics

Supporting Action Research with Learning Analytics

Anna Lea DyckhoffVlatko LukarovArham MuslimMohamed Amine ChattiUlrik Schroeder

3rd Conference on Learning Analytics and Knowledge (LAK’13)Leuven, Belgium, 8-12 April, 2013

2

What’s the Impact of Learning Analytics?

LAK13 "Learning Analytics and Action Research" Dyckhoff, Lukarov

3

Goals of Learning Analytics


4LAK13 "Learning Analytics and Action Research" Dyckhoff, Lukarov

Useful Tool

University

TeachersStudents

Diverse Scenarios

Different Questions

Action Research

Requirements

Related Indicators

Non-experts

5

Outline

Action Research and Learning Analytics Summary of Prior Research Research Questions and Method Discussion of Findings Recap


6

Action Research vs. Learning Analytics

Active improvement of teaching

Human-driven

Manually

Individual, perfectly fitting

Limited by time-constraints ...

Feedback for awareness & reflection

Data-driven

Automatically

Standardized, approved by experts

Limited by missing data ...



Develop Question

Formulate Research

Plan

Collect Data

Analyze & Reflect

Develop Action Plan

Record in Writing

Share

Action Research

Learning Analytics

Action Research & Learning Analytics

8

Prior Research on teachers’ questions


How do students value specific

learning offerings?

Are there specific learning offerings NOT used at all?

Do native speakers have fewer problems

with learning offerings than non-native speakers?

Are students using specific learning

materials (e.g., lecture recordings) in addition to

attendance or alternatively?

Which didactical acitivities, e.g. email

notifications, facilitate continous

learning?

Is the performance in e-tests

somehow related to exam grades?

Dyckhoff, 2010

9

Research Questions Q1 - Indicator-Question-Mapping:

Which questions cannot be mapped to the available (sets of) indicators?

Which indicators could deliver what kind of enlightenment?

Q2 - Teacher-Data-Indicators: Are there tools that explicitly correlate teacher data with student data?

How should teacher data be correlated?

Q3 - Missing Impact Analysis: How could learning analytics impact teaching?

How could this impact be evaluated?


10

Method

Study is based on results of a qualitative meta-analysis (Dyckhoff, 2011) collection of publications on 27 learning analytics tools


Collected 198

IndicatorsCategorized

Question Indicator Mapping

Analysis and

Discussion


Q1 – How to map a question to indicators?

12

Excerpt of our Mapping Results

Q1

13

Q1 – Mapping Results Questions that can be answered

Quantitative measure

Qualitative measure

Effects on performance

Complex correlation

Different data sources


Missing: Teacher Data Indicators

EASY NOT so EASY

14

Q2 Teacher Data Indicators


Sociogram (Dawson, 2010)

• Current situation

• Possible Scenarios- Forum Participation- Correspondence Indicator- Average assignment grading time

15

Q3 Impact Analysis

Tool Availability

Classroom State

Impact Prediction

Exit Interviews Conclusions


16

Study Limitations


International Context

German teachers

’ question

s

Set of all indicators

International indicators of 27 available

tool

17

Recap

Supporting Action Research with Learning Analytics Several Questions NOT yet answered by indicators Teacher Data Indicators should be researched What’s the Impact of Learning Analytics? Participatory Design for LA Tools/Indicators Inform Tool Design/Learning Technologies Design


Develop Question

Formulate

Research Plan

Collect Data

Analyze & Reflect

Develop Action Plan

Record in

Writing

Share

18

<References>

Dyckhoff, A.L. 2011. Implications for Learning Analytics Tools: A Meta-Analysis of Applied Research Questions. IJCISIM. 3, (2011), 594–601.

Dawson, S. 2010. “Seeing” the learning community: An exploration of the development of a resource for monitoring online student networking. BRIT J EDUC TECHNOL. 41, 5 (Sep. 2010), 736–752.

Dyckhoff, A.L., Lukarov V., Muslim A., Chatti M.A. and Schroeder, U. 2013. Supporting Action Research with Learning Analytics. Proceedings of LAK’13, April 08-12 2013, Leuven, Belgium, 220-229.


19

Name and Snapshot

Description/Use Case

Data Sources

Limitations

Indicator Documentation

lak13 supporting action research with learning analytics

Technology

action research dyckhoff

6lak13 learning analytics

9lak13 learning analytics

8lak13 learning analytics

11lak13 learning analytics

continous learning

analytics impact teaching

missing data