gauging mooc learners’ adherence to the designed … · 2016-06-29 · mooc enrolled pass chains...
TRANSCRIPT
![Page 1: GAUGING MOOC LEARNERS’ ADHERENCE TO THE DESIGNED … · 2016-06-29 · MOOC ENROLLED PASS CHAINS PASS/NON QUIZ RATE QUESTIONS TRIES VIDEOS ACCESSED FP101x RI101x Frame101x EX101x](https://reader033.vdocuments.site/reader033/viewer/2022042809/5f8eff6a7607e74eea366055/html5/thumbnails/1.jpg)
GAUGING MOOC LEARNERS’ ADHERENCE TO THE DESIGNED LEARNING PATHDAN DAVIS, GUANLIANG CHEN, CLAUDIA HAUFF, GEERT-JAN HOUBEN
TU DELFT WEB INFORMATION SYSTEMS GROUP
![Page 2: GAUGING MOOC LEARNERS’ ADHERENCE TO THE DESIGNED … · 2016-06-29 · MOOC ENROLLED PASS CHAINS PASS/NON QUIZ RATE QUESTIONS TRIES VIDEOS ACCESSED FP101x RI101x Frame101x EX101x](https://reader033.vdocuments.site/reader033/viewer/2022042809/5f8eff6a7607e74eea366055/html5/thumbnails/2.jpg)
WHAT IS A LEARNING PATH ?WHAT IS A LEARNING PATH ?
![Page 3: GAUGING MOOC LEARNERS’ ADHERENCE TO THE DESIGNED … · 2016-06-29 · MOOC ENROLLED PASS CHAINS PASS/NON QUIZ RATE QUESTIONS TRIES VIDEOS ACCESSED FP101x RI101x Frame101x EX101x](https://reader033.vdocuments.site/reader033/viewer/2022042809/5f8eff6a7607e74eea366055/html5/thumbnails/3.jpg)
WHAT IS A LEARNING PATH ?THE SEQUENCE OF EVENTS A STUDENT TAKES TOWARDS A LEARNING OBJECTIVE
![Page 4: GAUGING MOOC LEARNERS’ ADHERENCE TO THE DESIGNED … · 2016-06-29 · MOOC ENROLLED PASS CHAINS PASS/NON QUIZ RATE QUESTIONS TRIES VIDEOS ACCESSED FP101x RI101x Frame101x EX101x](https://reader033.vdocuments.site/reader033/viewer/2022042809/5f8eff6a7607e74eea366055/html5/thumbnails/4.jpg)
RQ to what extent do learners adhere to the designed learning path?
![Page 5: GAUGING MOOC LEARNERS’ ADHERENCE TO THE DESIGNED … · 2016-06-29 · MOOC ENROLLED PASS CHAINS PASS/NON QUIZ RATE QUESTIONS TRIES VIDEOS ACCESSED FP101x RI101x Frame101x EX101x](https://reader033.vdocuments.site/reader033/viewer/2022042809/5f8eff6a7607e74eea366055/html5/thumbnails/5.jpg)
VIDEO QUIZ PROGRESS FORUM
WATCH START VIEW STARTSUBMITEND
SUBMITEND
![Page 6: GAUGING MOOC LEARNERS’ ADHERENCE TO THE DESIGNED … · 2016-06-29 · MOOC ENROLLED PASS CHAINS PASS/NON QUIZ RATE QUESTIONS TRIES VIDEOS ACCESSED FP101x RI101x Frame101x EX101x](https://reader033.vdocuments.site/reader033/viewer/2022042809/5f8eff6a7607e74eea366055/html5/thumbnails/6.jpg)
VIDEO QUIZEND
QUIZSUBMIT
QUIZSTART
VIDEO
VIDEO
VIDEO
QUIZEND
QUIZEND
QUIZSUBMIT
QUIZSTART
QUIZSUBMIT
QUIZSTART
Data Analysis
Functional Programming
Framing
Responsible Innovation
SUBMIT ENDFORUMSTART
SUBMIT ENDFORUMSTART
v Q Q Q Q Q Q Q Q Q Q Q Q Q QQ
Q Q Q Q Q Q Q Q Q Q Q Q Q Q
Q Q Q Q Q Q Q Q Q Q Q Q
Q
Q Q Q Q Q Q
v v
v v v F F F F F F
F F F
F F F F F F F F F F F F F F F
v v v v v v v
v v v v v v v
v v v v v v
Q
DESIGNED LEARNING PATHS
Data Analysis
Functional Programming
Framing
Responsible Innovation
![Page 7: GAUGING MOOC LEARNERS’ ADHERENCE TO THE DESIGNED … · 2016-06-29 · MOOC ENROLLED PASS CHAINS PASS/NON QUIZ RATE QUESTIONS TRIES VIDEOS ACCESSED FP101x RI101x Frame101x EX101x](https://reader033.vdocuments.site/reader033/viewer/2022042809/5f8eff6a7607e74eea366055/html5/thumbnails/7.jpg)
RELATED WORK RELATED WORK
Identifying latent study habits by mining learner behavior patterns in massive open online courses. In CIKM ’14MIAOMIAO WEN & CAROLYN ROSÉ
Demographic differences in how students navigate through MOOCs. In L@S ‘14PHILIP GUO & KATHARINA REINECKE
![Page 8: GAUGING MOOC LEARNERS’ ADHERENCE TO THE DESIGNED … · 2016-06-29 · MOOC ENROLLED PASS CHAINS PASS/NON QUIZ RATE QUESTIONS TRIES VIDEOS ACCESSED FP101x RI101x Frame101x EX101x](https://reader033.vdocuments.site/reader033/viewer/2022042809/5f8eff6a7607e74eea366055/html5/thumbnails/8.jpg)
MOOC ENROLLED PASS CHAINSPASS/NON
QUIZQUESTIONSRATE
TRIES VIDEOSACCESSED
FP101x
RI101x
Frame101x
EX101x
37k
9k
34k
33k
5.3%
4.3%
2.4%
6.5%
1.06M/807K
66k/30k
95k/141k
1.02M/855k
288
75
26
136
1
1—3
2
2
67.5%
49.7%
51%
45%
*
* Guo, P. & Reinecke, K. Demographic differences in how students navigate through MOOCs (2014) : 78%
![Page 9: GAUGING MOOC LEARNERS’ ADHERENCE TO THE DESIGNED … · 2016-06-29 · MOOC ENROLLED PASS CHAINS PASS/NON QUIZ RATE QUESTIONS TRIES VIDEOS ACCESSED FP101x RI101x Frame101x EX101x](https://reader033.vdocuments.site/reader033/viewer/2022042809/5f8eff6a7607e74eea366055/html5/thumbnails/9.jpg)
MOOC ENROLLED PASS CHAINSPASS/NON
QUIZQUESTIONSRATE
TRIES VIDEOSACCESSED
FP101x
RI101x
Frame101x
EX101x
37k
9k
34k
33k
5.3%
4.3%
2.4%
6.5%
1.06M/807K
66k/30k
95k/141k
1.02M/855k
288
75
26
136
1
1—3
2
2
67.5%
49.7%
51%
45%
*
* Guo, P. & Reinecke, K. Demographic differences in how students navigate through MOOCs (2014) : 78%
![Page 10: GAUGING MOOC LEARNERS’ ADHERENCE TO THE DESIGNED … · 2016-06-29 · MOOC ENROLLED PASS CHAINS PASS/NON QUIZ RATE QUESTIONS TRIES VIDEOS ACCESSED FP101x RI101x Frame101x EX101x](https://reader033.vdocuments.site/reader033/viewer/2022042809/5f8eff6a7607e74eea366055/html5/thumbnails/10.jpg)
Frame101x
VIDEO
QUIZEND
FORUMSUBMIT
FORUMEND
PROGRESS
QUIZSTART
FORUMSTART
QUIZSUBMIT
3
3
4
6
36
VIDEO
QUIZEND
FORUMSUBMIT
FORUMEND
PROGRESS
QUIZSTART
FORUMSTART
QUIZSUBMIT
1 1
1
121
VIDEO
QUIZEND
FORUMSUBMIT
FORUMEND
PROGRESS
QUIZSTART
FORUMSTART
QUIZSUBMIT
3
7 7
7
6
VIDEO
QUIZEND
FORUMSUBMIT
FORUMEND
PROGRESS
QUIZSTART
FORUMSTART
QUIZSUBMIT
5 1
1 1
1
1
1
101
FP101x
RI101x
EX101x
![Page 11: GAUGING MOOC LEARNERS’ ADHERENCE TO THE DESIGNED … · 2016-06-29 · MOOC ENROLLED PASS CHAINS PASS/NON QUIZ RATE QUESTIONS TRIES VIDEOS ACCESSED FP101x RI101x Frame101x EX101x](https://reader033.vdocuments.site/reader033/viewer/2022042809/5f8eff6a7607e74eea366055/html5/thumbnails/11.jpg)
Frame101x
VIDEO
QUIZEND
FORUMSUBMIT
FORUMEND
PROGRESS
QUIZSTART
FORUMSTART
QUIZSUBMIT
3
3
4
6
36
VIDEO
QUIZEND
FORUMSUBMIT
FORUMEND
PROGRESS
QUIZSTART
FORUMSTART
QUIZSUBMIT
1 1
1
121
VIDEO
QUIZEND
FORUMSUBMIT
FORUMEND
PROGRESS
QUIZSTART
FORUMSTART
QUIZSUBMIT
3
7 7
7
6
VIDEO
QUIZEND
FORUMSUBMIT
FORUMEND
PROGRESS
QUIZSTART
FORUMSTART
QUIZSUBMIT
5 1
1 1
1
1
1
101
FP101x
RI101x
EX101x
EACH COURSE IS UNIQUE
![Page 12: GAUGING MOOC LEARNERS’ ADHERENCE TO THE DESIGNED … · 2016-06-29 · MOOC ENROLLED PASS CHAINS PASS/NON QUIZ RATE QUESTIONS TRIES VIDEOS ACCESSED FP101x RI101x Frame101x EX101x](https://reader033.vdocuments.site/reader033/viewer/2022042809/5f8eff6a7607e74eea366055/html5/thumbnails/12.jpg)
Frame101x
VIDEO
QUIZEND
FORUMSUBMIT
FORUMEND
PROGRESS
QUIZSTART
FORUMSTART
QUIZSUBMIT
3
3
4
6
36
VIDEO
QUIZEND
FORUMSUBMIT
FORUMEND
PROGRESS
QUIZSTART
FORUMSTART
QUIZSUBMIT
1 1
1
121
VIDEO
QUIZEND
FORUMSUBMIT
FORUMEND
PROGRESS
QUIZSTART
FORUMSTART
QUIZSUBMIT
3
7 7
7
6
VIDEO
QUIZEND
FORUMSUBMIT
FORUMEND
PROGRESS
QUIZSTART
FORUMSTART
QUIZSUBMIT
5 1
1 1
1
1
1
101
FP101x
RI101x
EX101x
EACH COURSE IS UNIQUE
ANALYZE ACCORDINGLY
![Page 13: GAUGING MOOC LEARNERS’ ADHERENCE TO THE DESIGNED … · 2016-06-29 · MOOC ENROLLED PASS CHAINS PASS/NON QUIZ RATE QUESTIONS TRIES VIDEOS ACCESSED FP101x RI101x Frame101x EX101x](https://reader033.vdocuments.site/reader033/viewer/2022042809/5f8eff6a7607e74eea366055/html5/thumbnails/13.jpg)
APPROACH APPROACH 1. VIDEO INTERACTIONS2. BEHAVIOR PATTERN CHAINS3. EVENT TYPE TRANSITIONS
![Page 14: GAUGING MOOC LEARNERS’ ADHERENCE TO THE DESIGNED … · 2016-06-29 · MOOC ENROLLED PASS CHAINS PASS/NON QUIZ RATE QUESTIONS TRIES VIDEOS ACCESSED FP101x RI101x Frame101x EX101x](https://reader033.vdocuments.site/reader033/viewer/2022042809/5f8eff6a7607e74eea366055/html5/thumbnails/14.jpg)
FINDINGSFINDINGS
![Page 15: GAUGING MOOC LEARNERS’ ADHERENCE TO THE DESIGNED … · 2016-06-29 · MOOC ENROLLED PASS CHAINS PASS/NON QUIZ RATE QUESTIONS TRIES VIDEOS ACCESSED FP101x RI101x Frame101x EX101x](https://reader033.vdocuments.site/reader033/viewer/2022042809/5f8eff6a7607e74eea366055/html5/thumbnails/15.jpg)
1. VIDEO INTERACTIONSDO LEARNERS WATCH VIDEO LECTURES IN THE PRESCRIBED ORDER?
![Page 16: GAUGING MOOC LEARNERS’ ADHERENCE TO THE DESIGNED … · 2016-06-29 · MOOC ENROLLED PASS CHAINS PASS/NON QUIZ RATE QUESTIONS TRIES VIDEOS ACCESSED FP101x RI101x Frame101x EX101x](https://reader033.vdocuments.site/reader033/viewer/2022042809/5f8eff6a7607e74eea366055/html5/thumbnails/16.jpg)
VIDEO INTERACTIONS
Non-Passing
Passing
Week 1 Week 2Week 3
FP101xDesigned Lecture Order
Executed Paths
![Page 17: GAUGING MOOC LEARNERS’ ADHERENCE TO THE DESIGNED … · 2016-06-29 · MOOC ENROLLED PASS CHAINS PASS/NON QUIZ RATE QUESTIONS TRIES VIDEOS ACCESSED FP101x RI101x Frame101x EX101x](https://reader033.vdocuments.site/reader033/viewer/2022042809/5f8eff6a7607e74eea366055/html5/thumbnails/17.jpg)
Non-Passing
Passing
Week 1Week 2
Week 3
VIDEO INTERACTIONS
Non-Passing
Passing
Week 1 Week 2Week 3
Non-Passing
Passing
Week 1 Week 2Week 3
Non-Passing
Passing
Week 1Week 2
Week 3
FP101x
RI101x
Frame101x
EX101x
![Page 18: GAUGING MOOC LEARNERS’ ADHERENCE TO THE DESIGNED … · 2016-06-29 · MOOC ENROLLED PASS CHAINS PASS/NON QUIZ RATE QUESTIONS TRIES VIDEOS ACCESSED FP101x RI101x Frame101x EX101x](https://reader033.vdocuments.site/reader033/viewer/2022042809/5f8eff6a7607e74eea366055/html5/thumbnails/18.jpg)
2. BEHAVIOR PATTERN CHAINSWHAT ARE THE MOST COMMON SEQUENCES IN THE EXECUTED LEARNING PATHS?
![Page 19: GAUGING MOOC LEARNERS’ ADHERENCE TO THE DESIGNED … · 2016-06-29 · MOOC ENROLLED PASS CHAINS PASS/NON QUIZ RATE QUESTIONS TRIES VIDEOS ACCESSED FP101x RI101x Frame101x EX101x](https://reader033.vdocuments.site/reader033/viewer/2022042809/5f8eff6a7607e74eea366055/html5/thumbnails/19.jpg)
FORUMEND
EIGHT-STEP CHAINEXAMPLE
LECTUREWATCH QUIZSTART QUIZSUBMIT QUIZSUBMIT
QUIZEND PROGRESS LECTUREWATCH
![Page 20: GAUGING MOOC LEARNERS’ ADHERENCE TO THE DESIGNED … · 2016-06-29 · MOOC ENROLLED PASS CHAINS PASS/NON QUIZ RATE QUESTIONS TRIES VIDEOS ACCESSED FP101x RI101x Frame101x EX101x](https://reader033.vdocuments.site/reader033/viewer/2022042809/5f8eff6a7607e74eea366055/html5/thumbnails/20.jpg)
MOOC ENROLLED PASS CHAINSPASS/NON
QUIZQUESTIONSRATE
TRIES VIDEOSACCESSED
FP101x
RI101x
Frame101x
EX101x
37k
9k
34k
33k
5.3%
4.3%
2.4%
6.5%
1.06M/807K
66k/30k
95k/141k
1.02M/855k
288
75
26
136
1
1—3
2
2
67.5%
49.7%
51%
45%
*
* Guo, P. & Reinecke, K. Demographic differences in how students navigate through MOOCs (2014) : 78%
![Page 21: GAUGING MOOC LEARNERS’ ADHERENCE TO THE DESIGNED … · 2016-06-29 · MOOC ENROLLED PASS CHAINS PASS/NON QUIZ RATE QUESTIONS TRIES VIDEOS ACCESSED FP101x RI101x Frame101x EX101x](https://reader033.vdocuments.site/reader033/viewer/2022042809/5f8eff6a7607e74eea366055/html5/thumbnails/21.jpg)
OPEN CARD SORTINGOPEN CARD SORTING
![Page 22: GAUGING MOOC LEARNERS’ ADHERENCE TO THE DESIGNED … · 2016-06-29 · MOOC ENROLLED PASS CHAINS PASS/NON QUIZ RATE QUESTIONS TRIES VIDEOS ACCESSED FP101x RI101x Frame101x EX101x](https://reader033.vdocuments.site/reader033/viewer/2022042809/5f8eff6a7607e74eea366055/html5/thumbnails/22.jpg)
MOTIFA RECURRING OR REPEATED ELEMENT
![Page 23: GAUGING MOOC LEARNERS’ ADHERENCE TO THE DESIGNED … · 2016-06-29 · MOOC ENROLLED PASS CHAINS PASS/NON QUIZ RATE QUESTIONS TRIES VIDEOS ACCESSED FP101x RI101x Frame101x EX101x](https://reader033.vdocuments.site/reader033/viewer/2022042809/5f8eff6a7607e74eea366055/html5/thumbnails/23.jpg)
MOTIF FREQTOTAL
FREQPASS
FREQFAIL
QUIZ COMPLETE 552,363(29.4%)
328,995(30.8%)
223,368(27.7%)
1.
BINGE WATCHING 149,784(8%)
59,498(5.6%)
90,286(11.2%)
2.
LECTURE -> QUIZ COMPLETE 100,179(5.3%)
50,415(4.7%)
49,764(6.2%)
3.
QUIZ COMPLETE -> FORUM 99,828(5.3%)
67,722(6.3%)
32,106(4%)
4.
FP101x
XQUIZ events only with at least 1 X = SUBMIT
WATCH events only
WATCH event(s) followed by XQUIZ events w/ >1 X = SUBMIT
XQUIZ events w/ >1 X = SUBMIT followed by XFORUM events
![Page 24: GAUGING MOOC LEARNERS’ ADHERENCE TO THE DESIGNED … · 2016-06-29 · MOOC ENROLLED PASS CHAINS PASS/NON QUIZ RATE QUESTIONS TRIES VIDEOS ACCESSED FP101x RI101x Frame101x EX101x](https://reader033.vdocuments.site/reader033/viewer/2022042809/5f8eff6a7607e74eea366055/html5/thumbnails/24.jpg)
3. EVENT TYPE TRANSITIONSHOW DO STUDENTS NAVIGATE BETWEEN CERTAIN EVENT TYPES?
![Page 25: GAUGING MOOC LEARNERS’ ADHERENCE TO THE DESIGNED … · 2016-06-29 · MOOC ENROLLED PASS CHAINS PASS/NON QUIZ RATE QUESTIONS TRIES VIDEOS ACCESSED FP101x RI101x Frame101x EX101x](https://reader033.vdocuments.site/reader033/viewer/2022042809/5f8eff6a7607e74eea366055/html5/thumbnails/25.jpg)
EVENT TYPE TRANSITIONS
Frame101x Non-PassingMarkov ModelState Visualization
(EXECUTED)
![Page 26: GAUGING MOOC LEARNERS’ ADHERENCE TO THE DESIGNED … · 2016-06-29 · MOOC ENROLLED PASS CHAINS PASS/NON QUIZ RATE QUESTIONS TRIES VIDEOS ACCESSED FP101x RI101x Frame101x EX101x](https://reader033.vdocuments.site/reader033/viewer/2022042809/5f8eff6a7607e74eea366055/html5/thumbnails/26.jpg)
EVENT TYPE TRANSITIONS
Frame101x Non-Passing Frame101x Passing
(EXECUTED)
![Page 27: GAUGING MOOC LEARNERS’ ADHERENCE TO THE DESIGNED … · 2016-06-29 · MOOC ENROLLED PASS CHAINS PASS/NON QUIZ RATE QUESTIONS TRIES VIDEOS ACCESSED FP101x RI101x Frame101x EX101x](https://reader033.vdocuments.site/reader033/viewer/2022042809/5f8eff6a7607e74eea366055/html5/thumbnails/27.jpg)
EVENT TYPE TRANSITIONS
Frame101x Non-Passing Frame101x Passing
(EXECUTED)
32%
35%
VIDEO
QUIZEND
QUIZSUBMIT
FORUMSUBMIT
FORUMEND
PROGRESS
FORUMSTART
QUIZSTART VIDEO
QUIZEND QUIZ
SUBMIT
FORUMSUBMIT
FORUMEND
PROGRESS
FORUMSTART
QUIZSTART
21%28%
44%
44%
![Page 28: GAUGING MOOC LEARNERS’ ADHERENCE TO THE DESIGNED … · 2016-06-29 · MOOC ENROLLED PASS CHAINS PASS/NON QUIZ RATE QUESTIONS TRIES VIDEOS ACCESSED FP101x RI101x Frame101x EX101x](https://reader033.vdocuments.site/reader033/viewer/2022042809/5f8eff6a7607e74eea366055/html5/thumbnails/28.jpg)
DAVIS, D., CHEN, G., JIVET, I., HAUFF, C., & HOUBEN, G. J. (2016). ENCOURAGING METACOGNITION & SELF-REGULATION IN MOOCS THROUGH INCREASED LEARNER FEEDBACK. LAK 2016 LAL WORKSHOP
FEEDBACKEXAMPLE
![Page 29: GAUGING MOOC LEARNERS’ ADHERENCE TO THE DESIGNED … · 2016-06-29 · MOOC ENROLLED PASS CHAINS PASS/NON QUIZ RATE QUESTIONS TRIES VIDEOS ACCESSED FP101x RI101x Frame101x EX101x](https://reader033.vdocuments.site/reader033/viewer/2022042809/5f8eff6a7607e74eea366055/html5/thumbnails/29.jpg)
Frame101x PassingFrame101x Non-Passing
RI101x Non-Passing RI101x Passing
EX101x PassingEX101x Non-Passing
FP101x Non-Passing FP101x Passing
![Page 30: GAUGING MOOC LEARNERS’ ADHERENCE TO THE DESIGNED … · 2016-06-29 · MOOC ENROLLED PASS CHAINS PASS/NON QUIZ RATE QUESTIONS TRIES VIDEOS ACCESSED FP101x RI101x Frame101x EX101x](https://reader033.vdocuments.site/reader033/viewer/2022042809/5f8eff6a7607e74eea366055/html5/thumbnails/30.jpg)
APPLICATIONSAPPLICATIONS
![Page 31: GAUGING MOOC LEARNERS’ ADHERENCE TO THE DESIGNED … · 2016-06-29 · MOOC ENROLLED PASS CHAINS PASS/NON QUIZ RATE QUESTIONS TRIES VIDEOS ACCESSED FP101x RI101x Frame101x EX101x](https://reader033.vdocuments.site/reader033/viewer/2022042809/5f8eff6a7607e74eea366055/html5/thumbnails/31.jpg)
1. BEYOND PASS-FAIL SEGMENTATION2. PATTERN MINING & CONFORMANCE CHECKING3. POSITIVE DEVIANCE
![Page 32: GAUGING MOOC LEARNERS’ ADHERENCE TO THE DESIGNED … · 2016-06-29 · MOOC ENROLLED PASS CHAINS PASS/NON QUIZ RATE QUESTIONS TRIES VIDEOS ACCESSED FP101x RI101x Frame101x EX101x](https://reader033.vdocuments.site/reader033/viewer/2022042809/5f8eff6a7607e74eea366055/html5/thumbnails/32.jpg)
THANK YOUTU DELFT WEB INFORMATION SYSTEMS GROUP
DAN DAVIS, GUANLIANG CHEN, CLAUDIA HAUFF, GEERT-JAN HOUBEN
BIT.LY/WIS-LEARNING-ANALYTICS