adaptive learning systemsc21u.gatech.edu/sites/default/files/presentations... · university of new...

Adaptive Learning SystemsThe State of the Field

Michael Madaio

What is Adaptive Learning?

● Individualized learning:○ Content difficulty○ Content form (text, video)○ Sequence of content○ Pace

● Adapted based on:○ Student knowledge, performance○ Student learning style, preferences, goals

OverviewComponentsImplications

Brief History

● 1920’s - Personalized learning at scale (Dewey, Montessori)

● 1960’s - Mastery-based learning (Bloom)

● 1970’s / 1980’s - Computer-assisted Instruction, Cognitive Tutors

Koe

ding

er e

t al.,

199

7; M

ödrit

sche

r et a

l., 2

004


Why now?

● Proliferation of learning technologies○ Students, teachers, institution

● Learning mediated through technology○ Flipped classes, MOOC’s○ Learning apps, student collaboration on forums


Why now?

● Volume of learning data generated○ Variety, velocity, granularity

● Improvements in “big” data analytics○ Machine learning○ Hadoop, MapReduce○ Student clustering○ Learning recommendations


Potential

● Improved student engagement

● Improved student performance on learning outcomes

● Improved student retention in courses

● Improved research-based interventions

Mag

nisa

lis, 2

011;

Nat

riel

lo, 2

011;

Poe

lhub

er,

2008


Challenges

● Material:○ Bandwidth○ Laptops, tablets○ Cost to license software

● Infrastructural:○ System setup and support○ Integration with existing LMS’s○ Training and adoption curve


Challenges

● Pedagogical:○ Integration into existing teaching processes○ Course authoring○ Accuracy of recommendations○ Social dynamics of self-paced learning


Current Adaptive Landscape

● We will look at 16 major adaptive learning providers

● These 16 are currently used in several public and private universities, by a variety of public K-12 school districts, and by corporations and individuals


Adaptive Providers

● Major companies○ Knewton ○ Cerego○ CogBooks○ DreamBox

● Others○ SmartSparrow○ Open Learning Initiative○ LoudCloud○ ALEKS


Adaptive ProvidersOverviewComponentsImplications

Sites of Adaptive Learning

● Arizona State University (Knewton)○ Emporium math courses have seen an 18 percent increase in pass rates and 47 percent

drop in student withdrawals, from 2011-2013. ○ ASU leadership estimates that the institution has retained $12m in what would have

been lost tuition revenue, for 2012-2013.

● University of New South Wales (SmartSparrow)○ Developed with 6 other Australian universities to teach threshold concepts in

mechanics courses○ Saw a decline in average course drop-out rate from 31 percent to 14 percent,

even as course enrollments increased by nearly 30 percent, in 2012-2013.


Edu

catio

n G

row

th A

dvis

ors,

201

3

Other Institutions

Carnegie Mellon UniversityNew York UniversityUniversity of New HampshireSouthern New Hampshire UniversityAmerican Public UniversityWestern Governors University


Adaptive System Components

● Domain Model

● Learner Model

● Adaptation Model


Domain Model - Overview

● A conceptual map of the course or domain

● Course elements are assigned to nodes○ Could be videos, articles, assignments, quizzes

● Edges are hierarchical relationships○ Linear or nonlinear○ With prerequisites defined

Nes

bit,

2006

; Che

n, 2

008;

Gra

f & Iv

es, 2

010


Domain Model - Method

● Created and arranged either by the provider or by teachers○ Some adaptive systems use pre-authored content○ Some allow for teacher authoring and arrangement

● Multiple conventions for domain representation and navigation○ Zooming, panning, filtering

Kar

ampi

peris

& S

amps

on, 2

005;

Mag

nisa

lis,

2011

; Chu

ng &

Kim

, 201

2


Domain Model - Risks

● Trade off of teacher autonomy and effectiveness of system○ Issues of consistency and sufficiency of metadata

● Ability for students to view their own “position” in the domain○ Risks of cognitive overload and frustration

Bar

gel e

t al.,

201

2; D

iBito

nto

et a

l., 2

013


Rei

man

n, 2

013

Falm

agne

, 201

1

Domain Model - SystemsOverviewComponentsImplications


● Domain Model

● Learner Model



Learner Model - Overview

● Model of each learner’s current knowledge state

● Either an overlay model or a stereotype model○ Overlay - compared to the overall domain model○ Stereotype - clustered with similar learner models

● Stereotype model is based on assumptions about student similarity


Nitc

hot e

t al.,

201

0; K

nauf

et a

l., 2

010;

Kla

šnja

-M

iliće

vić

et a

l., 2

011

Learner Model - Method

● Data is collected either:○ Statically or dynamically○ Explicitly or implicitly

● Static data○ Cognitive characteristics, background knowledge○ Non-cognitive - topic preference, learning goals

● Explicit data collection methods○ Collected via pre-test, survey, feedback prompts


Kar

ampi

peris

& S

amps

on, 2

005;

Che

n, 2

008;

K

lašn

ja-M

iliće

vić

et a

l., 2

011

Learner Model - Method

● Dynamic data○ Knowledge state○ Learning style (may be static or inferred dynamically)○ Time spent on course element or LMS○ Clickstream data○ Assessment scores ○ Student feedback (prompted explicitly)

● Implicit data collection methods○ Automatically collected through interactions with the

system.


Mor

eno-

Ger

et a

l., 2

007;

Akb

ulut

& C

arda

k,

2012

● Risks for explicit data collection○ Non-completion or inaccuracy○ Interrupts learning process

● Risks for implicit data collection○ Data provenance○ Accuracy of inferences made with data○ Data privacy issues

OverviewComponentsImplicationsLearner Model - Risks

Kla

šnja

-Mili

ćevi

ć et

al.,

201

1; M

agni

salis

et a

l.,

2011

; Lo

et a

l., 2

012

Kla

šnja

-Mili

ćevi

ć, 2

011

Che

n, 2

005

Learner Model - SystemsOverviewComponentsImplications


● Domain Model

● Learner Model



Adaptation Model - Overview

● Unit of Adaptivity - Course element being adapted:○ Difficulty○ Content media○ Sequence (micro or macro)○ Pace

● Method of Adaptation○ Variety of algorithm types used for different purposes○ Direct or indirect presentation of adaptation


Mitr

ovic

& M

artin

, 200

4; K

umar

, 200

6; U

llric

h et

al.,

200

9; S

osno

vsky

, 201

0

http://onlinelibrary.wiley.com/doi/10.1111/j.1365-2729.2010.00365.x/full#b23

● Bayesian network tracing - clustering students into groups

● Hidden Markov models - predicting likelihood of learner success

● Genetic algorithms○ Refine models and construct optimal learning paths

from pre-tests

● Neural networks○ Pattern recognition updated with input (eg: inferring

learning styles from interactions)

OverviewComponentsImplicationsAdaptation Model - Method

Bru

silo

vsky

, 200

1; C

hen,

200

8; M

agni

salis

et

al.,

2011

Adaptation Model - Method

● Direct Adaptation○ Students are given a visible recommendation for the

next course element○ Micro-level - problem feedback, explanations, links○ Macro-level - learning path presented as optional

● Indirect Adaptation○ Link hiding○ Learning path presented as only option


Hau

ger &

Köc

k, 2

007;

Mag

nisa

lis e

t al.,

201

1;

Akb

ulut

& C

arda

k, 2

012

Adaptation Model - Risks

● Risks to student agency if the system controls too much○ Feelings of paternalism

● Systems with learner choice○ Lack of student ability to choose optimal path

● Design of adaptation engine influences the learning process


Bar

gel e

t al.,

201

2; A

kbul

ut &

Car

dak,

201

2;

Kirs

chne

r and

Mie

rren

beer

, 201

3

ww

w.s

mar

tspa

rrow

.com

ww

w.k

new

ton.

com

Adaptation Model - SystemsOverviewComponentsImplications

Implications - Technical

● Must be embedded in a platform○ Either adaptive learning platform○ Integrated with LMS○ Used in a digital textbook

● Need adequate bandwidth, available hardware, technical support for set-up and maintenance


Implications - Pedagogical

● Choose an adaptive provider that fits your goals○ Consider pedagogical goals and mode of instruction○ How much teacher agency involved in authoring the course?

● Need faculty and student buy-in○ Overcoming cultural resistance to new models of learning

● Need departmental or institutional buy-in○ Freedom to experiment○ Competency-based credit



● Mode of instruction○ Online course○ Blended, face-to-face

● Teacher goals○ Address differences in background knowledge○ Appeal to different learning styles, goals



● How does whole-class instruction happen when students are not working on the same content at the same time?○ Challenges for peer learning, mentoring, collaborative

groups, project-based learning

○ Could be an opportunity for changing how we view those experiences


Implications - Research

● Opportunities○ Improving student learning outcomes, retention,

engagement

○ Informing curricular development

○ Connections between cognitive factors and performance (ie: self-regulation, motivation)


Implications - Research

● Challenges○ Data acquisition - greater volume, variety, and

velocity of data than many teachers or researchers may be equipped to deal with

○ Data privacy - involvement of 3rd party adaptive providers raises questions about privacy and security of data


Thank you!

Questions?

Michael [email protected]@mmadaio

adaptive learning systemsc21u.gatech.edu/sites/default/files/presentations... · university of new...

Documents