learning analytics conference 2015 presentation
TRANSCRIPT
![Page 1: Learning Analytics Conference 2015 Presentation](https://reader034.vdocuments.site/reader034/viewer/2022051516/55a510ef1a28ab482d8b45fd/html5/thumbnails/1.jpg)
Using Transaction-Level Data to Diagnose
Knowledge Gaps and Misconceptions
Randy Davies, Rob Nyland, John Chapman, Gove Allen
Brigham Young University
LAK15, Marist College, Poughkeepsie, NY
@robnyland @chapmjs
![Page 2: Learning Analytics Conference 2015 Presentation](https://reader034.vdocuments.site/reader034/viewer/2022051516/55a510ef1a28ab482d8b45fd/html5/thumbnails/2.jpg)
Introduction
• Does assessment data give us an
accurate picture of student knowledge?
• Do assessments leave room for possible
student misconceptions?
• What could be the possible problems with
these student misconceptions?
![Page 3: Learning Analytics Conference 2015 Presentation](https://reader034.vdocuments.site/reader034/viewer/2022051516/55a510ef1a28ab482d8b45fd/html5/thumbnails/3.jpg)
Types of Data
System Level Data
• Macro-level data: Data for groups of students, universities. Typically the realm of academic analytics.
Individual Level Data
• Assessment Data: Data about student performance in class activities
Transaction Level Data
• The individual transactions that create an assessment. Step level data.
![Page 4: Learning Analytics Conference 2015 Presentation](https://reader034.vdocuments.site/reader034/viewer/2022051516/55a510ef1a28ab482d8b45fd/html5/thumbnails/4.jpg)
Research Questions
1. How can we identify misconceptions from
student log data?
2. Does student log data tell a different story
than final answer data?
![Page 5: Learning Analytics Conference 2015 Presentation](https://reader034.vdocuments.site/reader034/viewer/2022051516/55a510ef1a28ab482d8b45fd/html5/thumbnails/5.jpg)
Progress Diagram
Individualize
Remediation
Process
Automate
Process
Actionable
Information
Manual Analysis
Phase 1 Phase 2 Phase 3
![Page 6: Learning Analytics Conference 2015 Presentation](https://reader034.vdocuments.site/reader034/viewer/2022051516/55a510ef1a28ab482d8b45fd/html5/thumbnails/6.jpg)
Data Collection
• Data collected from online Intro to Excel
Class
– www.myeducator.com
• Assessments are situated and task-based
• Step-level data for each task is captured
![Page 7: Learning Analytics Conference 2015 Presentation](https://reader034.vdocuments.site/reader034/viewer/2022051516/55a510ef1a28ab482d8b45fd/html5/thumbnails/7.jpg)
Example Student Log
![Page 8: Learning Analytics Conference 2015 Presentation](https://reader034.vdocuments.site/reader034/viewer/2022051516/55a510ef1a28ab482d8b45fd/html5/thumbnails/8.jpg)
Knowledge Components
• Syntax
• Cell Referencing
• Calculation
• Absolute
References ($)
![Page 9: Learning Analytics Conference 2015 Presentation](https://reader034.vdocuments.site/reader034/viewer/2022051516/55a510ef1a28ab482d8b45fd/html5/thumbnails/9.jpg)
Types of Errors
Optimal
SolutionTEXT TEXT TEXT
Used $ when
not needed
=$C11*$C$8
=C11*$C$8
Major Issue
Failed to use
$ when it was
needed
=C11*C8
=$C8*$C11
Used $
incorrectly
=C$11*C8
=C8*C$11
Type in value
to avoid $ use
=C11*0.0675
0.50.05
Error Weighting
Optimal Solution: =C11*C$8
0.5 0.6
Minor Issue
![Page 10: Learning Analytics Conference 2015 Presentation](https://reader034.vdocuments.site/reader034/viewer/2022051516/55a510ef1a28ab482d8b45fd/html5/thumbnails/10.jpg)
Transaction Level Data Example
D11Error
ratingD20
Error
ratin
g
Optimal Solution =C11*C$8 =F19*C$13/12
Step 1 =D11*C8 .5 =(F19*C13)/12 .5
Step 2 =C11*C8 .5=($F$19*$C$13)/1
2.55
Step 3 =C11*$C$8 .05 =(F19*C15)/12 .5
Step 4 =(F19*C13)/12 .5
Step 5 =F19*($C$13/12) .05
Final Solution =C11*$C$8 .05 =F19*($C$13/12) .05
![Page 11: Learning Analytics Conference 2015 Presentation](https://reader034.vdocuments.site/reader034/viewer/2022051516/55a510ef1a28ab482d8b45fd/html5/thumbnails/11.jpg)
Knowledge Gap Analysis Results
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
Solution Process Final Answer
First Attempt
2nd Attempt
Amount of Error Detected
Major Errors Only
![Page 12: Learning Analytics Conference 2015 Presentation](https://reader034.vdocuments.site/reader034/viewer/2022051516/55a510ef1a28ab482d8b45fd/html5/thumbnails/12.jpg)
Knowledge Gap Analysis Results
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
Persisted Resolved Emerged No Error
Solution Process
Final Answer
Only Major Errors
![Page 13: Learning Analytics Conference 2015 Presentation](https://reader034.vdocuments.site/reader034/viewer/2022051516/55a510ef1a28ab482d8b45fd/html5/thumbnails/13.jpg)
Knowledge Gap Analysis Results
0%
10%
20%
30%
40%
50%
60%
70%
80%
Persisted Resolved Emerged No Error
Solution Process
Final Answer
Including Minor Errors
![Page 14: Learning Analytics Conference 2015 Presentation](https://reader034.vdocuments.site/reader034/viewer/2022051516/55a510ef1a28ab482d8b45fd/html5/thumbnails/14.jpg)
Future Work
• Automating the process of discovering
patterns in student answers
• Give feedback to the student based on
their responses