generating actionable predictive models of academic performance
TRANSCRIPT
Abelardo Pardo, Negin Mirriahi, Roberto Martínez-Maldonado, Jelena Jovanovic, Shane Dawson, Dragan Gašević
Kannan B flickr.com
Generating Actionable Predictive Models of Academic Performance
International Conference on Learning Analytics and Knowledge University of Edinburgh 29 April 2016
Generating Actionable Predictive Models of Academic PerformancePardo, Mirriahi, et al.
The problem
• Detailed data footprints collected
• Sophisticated algorithms applied
• Predictive models created
• How to derive/apply actions?2
Michael Pereckas flickr.com
Generating Actionable Predictive Models of Academic PerformancePardo, Mirriahi, et al.
Retention/Attrition
3
Trevor Huxm
an flickr.com
Predict student abandoning course/institution
E.g., Jayaprakash, S. M., Moody, E. W., Eitel, J. M., Regan, J. R., & Baron, J. D. (2014). Early Alert of Academically At-Risk Students : An Open Source Analytics Initiative. Journal of Learning Analytics, 1, 6-47.
Generating Actionable Predictive Models of Academic PerformancePardo, Mirriahi, et al.
Sophisticated predictive models
4
Kev Lewis flickr.com
Romero, C., López, M.-I., Luna, J.-M., & Ventura, S. (2013). Predicting students' final performance from participation in on-line discussion forums. Computers & Education, 68, 458-472. doi:10.1016/j.compedu.2013.06.009
Classification
• Divide students in groups
• Useful for instructors
• Unclear how to intervene
Generating Actionable Predictive Models of Academic PerformancePardo, Mirriahi, et al. 5
Tanes, Z., Arnold, K. E., King, A. S., & Remnet, M. A. (2011). Using Signals for appropriate feedback: Perceptions and practices. Computers & Education, 57(4), 2414-2422. doi:10.1016/j.compedu.2011.05.016
Course Performance
• Well
• Mediocre
• Poor
Vit Brunner Flickr.com
Generating Actionable Predictive Models of Academic PerformancePardo, Mirriahi, et al.
Disproportionate attention
6
Farrukh Flickr.com
Intervene
Wise, A. F. (2014). Designing pedagogical interventions to support student use of learning analytics. Paper presented at the International Conference on Learning Analytics and Knowledge.
Generating Actionable Predictive Models of Academic PerformancePardo, Mirriahi, et al. 7
Gather data on the state of the student
Identify action to take
Deliver feedback
McKay, T., Miller, K., & Tritz, J. (2012). What to do with actionable intelligence: E2Coach as an intervention engine. Paper presented at the International Conference on Learning Analytics and Knowledge, Vancouver, BC, Canada.
Paul flickr.com
Generating Actionable Predictive Models of Academic PerformancePardo, Mirriahi, et al.
Objective
1. Data indicators close to learning design
2. Predictive model
3. Bridge between model and application
4. Straightforward delivery method
8
Oliver Braubach flickr.com
Generating Actionable Predictive Models of Academic PerformancePardo, Mirriahi, et al.
• Event counts from interactive course material
• Midterm/final exam scores
• Recursive partitioning
• Divide cohort into performance categories
9
Louish Pixel flickr.com
Generating Actionable Predictive Models of Academic PerformancePardo, Mirriahi, et al.
Recursive Partitioning
• Arbitrary magnitudes in factors
• Handle large number of factors
• Handle heterogeneous factos
• Model with intuitive interpretation
• Performance?10
theilr flickr.com
Generating Actionable Predictive Models of Academic PerformancePardo, Mirriahi, et al. 11
William
Murphy flickr.com
• 13 Week first year Engineering • Weekly activities (formative/summative) • Videos, MCQ, Exercises, dashboard • n = 272, Weeks 2-5 and 7-13
Generating Actionable Predictive Models of Academic PerformancePardo, Mirriahi, et al. 12
Data collected
• Indicators are directly connected with learning design
• Data structure shaped by the schedule (weeks)
• Data available in a per-week basis
• What is the expected midterm/final score in week n?
Generating Actionable Predictive Models of Academic PerformancePardo, Mirriahi, et al.
Result Example• Week 10
• Predicted score at leaves (out of 40)
• Conditions at nodes
• If (EXC.in >=22) and (VID.PL < 8.5) then score = 6
13
Generating Actionable Predictive Models of Academic PerformancePardo, Mirriahi, et al.
• Each leaf node represents a group of students with their estimated score.
• Example: 6, 8.3, 8.4, 9.4, 9.9, 10, 15 (out of 40)
• Intervention: suggested work before exam
14
Result interpretation
Generating Actionable Predictive Models of Academic PerformancePardo, Mirriahi, et al. 15
shabnam m
ayet Flickr.com
PerformanceRMSE: Root mean square error, MAE: Mean absolute error
Generating Actionable Predictive Models of Academic PerformancePardo, Mirriahi, et al.
Conclusions and Future Work
Indicators closed to learning design
Hierarchical partition
Student partition respect to midterm/final
Acceptable performance
Immediate actionby instructors
16
Ham
ish Irvine flickr.com
Abelardo Pardo, Negin Mirriahi, Roberto Martínez-Maldonado, Jelena Jovanovic, Shane Dawson, Dragan Gašević
Kannan B flickr.com
Generating Actionable Predictive Models of Academic Performance
International Conference on Learning Analytics and Knowledge University of Edinburgh 29 April 2016