actionable learning analytics: practical lessons for teaching and learning
TRANSCRIPT
![Page 1: Actionable Learning Analytics: Practical Lessons for Teaching and Learning](https://reader033.vdocuments.site/reader033/viewer/2022051318/58eea0621a28ab51128b45f3/html5/thumbnails/1.jpg)
@stevelonn
ActionableLearningAnalytics:PracticalLessonsforTeaching&Learning
StevenLonn,Ph.D.
![Page 2: Actionable Learning Analytics: Practical Lessons for Teaching and Learning](https://reader033.vdocuments.site/reader033/viewer/2022051318/58eea0621a28ab51128b45f3/html5/thumbnails/2.jpg)
@stevelonn
Acknowledgements
• ColleaguesinDEI atMichigan• ColleaguesandstudentsinLEDLab atMichigan
• DevonCancilla andteamatUMKC• StephanieTeasley• TimMcKay• MatthewPistilli• KimArnold• JoeWaddington
2
![Page 3: Actionable Learning Analytics: Practical Lessons for Teaching and Learning](https://reader033.vdocuments.site/reader033/viewer/2022051318/58eea0621a28ab51128b45f3/html5/thumbnails/3.jpg)
@stevelonn
Who(ifanyone)isusingthecontentyoucreatefor
yourcourses?
3
![Page 4: Actionable Learning Analytics: Practical Lessons for Teaching and Learning](https://reader033.vdocuments.site/reader033/viewer/2022051318/58eea0621a28ab51128b45f3/html5/thumbnails/4.jpg)
@stevelonn
IsThis“Actionable?”
TheChronicleofHigherEducationJanuary5,2016http://chronicle.com/article/This-Chart-Shows-the-Promise/234573
4
![Page 5: Actionable Learning Analytics: Practical Lessons for Teaching and Learning](https://reader033.vdocuments.site/reader033/viewer/2022051318/58eea0621a28ab51128b45f3/html5/thumbnails/5.jpg)
@stevelonn
JoannaMirecki Millunchick
• ProfessorofMaterialsScienceandEngineering
• TeacheslargeintroductoryMaterialsScienceandEngineering(MSE)course
5
![Page 6: Actionable Learning Analytics: Practical Lessons for Teaching and Learning](https://reader033.vdocuments.site/reader033/viewer/2022051318/58eea0621a28ab51128b45f3/html5/thumbnails/6.jpg)
@stevelonn
ScreencastsforHWSolutions
6
![Page 7: Actionable Learning Analytics: Practical Lessons for Teaching and Learning](https://reader033.vdocuments.site/reader033/viewer/2022051318/58eea0621a28ab51128b45f3/html5/thumbnails/7.jpg)
@stevelonn
Screencastusagepositivelycorrelatedwithcourseperformance
• Chemicalengineersusedscreencaststheleast,yetreceivedthehighestgrades
• Industrial&operationsengineersusedscreencaststhemost,andreceivedaveragegrades
7
![Page 8: Actionable Learning Analytics: Practical Lessons for Teaching and Learning](https://reader033.vdocuments.site/reader033/viewer/2022051318/58eea0621a28ab51128b45f3/html5/thumbnails/8.jpg)
@stevelonn
What’sgoingoninthecurriculum?Materials Science&Engineering
• Physics ofMaterials
• PrinciplesofEngineeringMaterials
• ThermodynamicsofMaterials
• KineticsandTransport
• StructuresofMaterials
• MaterialsLaboratoryIandII
ChemicalEngineering
• IntrotoMaterialandEnergyBalances
• ChemicalandEngineeringThermodynamics
• FluidMechanics• MassandHeat
Transfer• Separation
Processes• Reaction
EngineeringandDesign
• ChemicalEngineeringLaboratory
AerospaceEngineering
• IntrotoAerospaceEngineering
• IntrotoAerospaceEngin.Systems
• IntrotoSolidMechanicsandAerospaceStructures
• IntrotoGasDynamics
• AircraftandSpacecraftStructures
• Aerodynamics• Propulsion
Industrial&Operations Eng.
• Economic DecisionMaking
• OperationsModeling
• Probability andStatistics
• OptimizationMethods
• MarkovProcesses• Ergonomics• LinearStatistics• DataProcessing
Screencastsshouldhaveagreaterimpactasasupplementaryresourcetoaidstudentlearningforstudentswithlessfamiliaritywiththecoursecontent.
8
![Page 9: Actionable Learning Analytics: Practical Lessons for Teaching and Learning](https://reader033.vdocuments.site/reader033/viewer/2022051318/58eea0621a28ab51128b45f3/html5/thumbnails/9.jpg)
@stevelonn
CaseStudyonExamPerformance
• IfIOEstudentswatchedany screencasts,theyperformedbetteronaparticular question.– r =.425p =.001
• IfIOEstudentswatchedaparticular screencast,therelationshipevenstrongerfortheperformanceonthatparticular question– r =.375p =.003
9
Green,Pinder-Grover&Millunchick,2012
![Page 10: Actionable Learning Analytics: Practical Lessons for Teaching and Learning](https://reader033.vdocuments.site/reader033/viewer/2022051318/58eea0621a28ab51128b45f3/html5/thumbnails/10.jpg)
@stevelonn
Summary• Analytic:LMShitsandperceptionsurveysaboutsupplementaryproblem-setscreencasts
• Finding:Sig.content-performanceimpactforlearnerswithoutpriorMSEknowledge– NoimpactforstudentswithpriorMSEknowledge
• Whatcanyoudo?:Beginbybreakingdownassessmentsbydiscipline/sub-discipline.Unpackanysig.differences– (gender,ethnicity,curriculum,etc.)
10
![Page 11: Actionable Learning Analytics: Practical Lessons for Teaching and Learning](https://reader033.vdocuments.site/reader033/viewer/2022051318/58eea0621a28ab51128b45f3/html5/thumbnails/11.jpg)
@stevelonn
WhatisLearningAnalytics?
11
![Page 12: Actionable Learning Analytics: Practical Lessons for Teaching and Learning](https://reader033.vdocuments.site/reader033/viewer/2022051318/58eea0621a28ab51128b45f3/html5/thumbnails/12.jpg)
@stevelonn
Learninganalyticsisthemeasurement,collection,analysisandreportingofdataaboutlearnersandtheircontexts,forpurposesofunderstandingandoptimisinglearningandtheenvironmentsinwhichitoccurs.
1212
![Page 13: Actionable Learning Analytics: Practical Lessons for Teaching and Learning](https://reader033.vdocuments.site/reader033/viewer/2022051318/58eea0621a28ab51128b45f3/html5/thumbnails/13.jpg)
@stevelonn
measurement,collection,analysisandreporting
ofdata
1313
![Page 14: Actionable Learning Analytics: Practical Lessons for Teaching and Learning](https://reader033.vdocuments.site/reader033/viewer/2022051318/58eea0621a28ab51128b45f3/html5/thumbnails/14.jpg)
@stevelonn
learners andtheircontexts
1414
![Page 15: Actionable Learning Analytics: Practical Lessons for Teaching and Learning](https://reader033.vdocuments.site/reader033/viewer/2022051318/58eea0621a28ab51128b45f3/html5/thumbnails/15.jpg)
@stevelonn
understanding andoptimising learning
1515
![Page 16: Actionable Learning Analytics: Practical Lessons for Teaching and Learning](https://reader033.vdocuments.site/reader033/viewer/2022051318/58eea0621a28ab51128b45f3/html5/thumbnails/16.jpg)
@stevelonn
IsUMKCpreparedtoimplement
learninganalyticstoday?
16
![Page 17: Actionable Learning Analytics: Practical Lessons for Teaching and Learning](https://reader033.vdocuments.site/reader033/viewer/2022051318/58eea0621a28ab51128b45f3/html5/thumbnails/17.jpg)
@stevelonn
WhenisanInstitutionReady toImplementLearningAnalytics?
17
Read�i�ness(ˈred-ē-nis)
noun1 willingness to do something.
2 the state of being fully prepared for something.
3 immediacy, quickness, or promptness.
![Page 18: Actionable Learning Analytics: Practical Lessons for Teaching and Learning](https://reader033.vdocuments.site/reader033/viewer/2022051318/58eea0621a28ab51128b45f3/html5/thumbnails/18.jpg)
@stevelonn 18
TheNeedforReflection
• Comprehensiveunderstanding
• Multipleperspectivesneeded
• Multiplelevelsneeded
TheLearningAnalyticsReadinessInstrument
Arnold, Lonn&Pistilli,2014
![Page 19: Actionable Learning Analytics: Practical Lessons for Teaching and Learning](https://reader033.vdocuments.site/reader033/viewer/2022051318/58eea0621a28ab51128b45f3/html5/thumbnails/19.jpg)
@stevelonn
LARIBetaParticipants
19
Assoc/Pub-R-L,Bac/A&S,&Bac/Diverse
Master’sLarge&ResearchUniversities(HighResearchActivity)
ResearchUniversities(VeryHighResearchActivity)
5institutions129respondents
15institutions337respondents
4institutions84respondents
![Page 20: Actionable Learning Analytics: Practical Lessons for Teaching and Learning](https://reader033.vdocuments.site/reader033/viewer/2022051318/58eea0621a28ab51128b45f3/html5/thumbnails/20.jpg)
@stevelonn
ReadinessFactors
20
Factor # SurveyItems
Example
Culture 22 “Myinstitutionhasaculturethatacceptstheuseofdatatomakedecisions.”
DataManagementExpertise
8 “Myinstitutionhastheabilitytostoreincreasinglylargevolumesofdata.”
DataAnalysisExpertise
6“My Institutionhasprofessionalswithmathematical/statisticalexperienceinmanipulatingandtransforming data.”
Communication&Policy Application
7 “MyInstitutionhasprofessionalswithbusinessacumeninmarketing/publicity.”
Training 3“Myinstitutionhasprofessionalswithcustomer-facingsupport experienceintrainingconstituentsontheuseofnewsystems.”
Oster,Lonn,Pistilli,Brown,2016
![Page 21: Actionable Learning Analytics: Practical Lessons for Teaching and Learning](https://reader033.vdocuments.site/reader033/viewer/2022051318/58eea0621a28ab51128b45f3/html5/thumbnails/21.jpg)
@stevelonn
CultureisPervasive
21
![Page 22: Actionable Learning Analytics: Practical Lessons for Teaching and Learning](https://reader033.vdocuments.site/reader033/viewer/2022051318/58eea0621a28ab51128b45f3/html5/thumbnails/22.jpg)
@stevelonn
GrowinganInstitutionalLearningAnalyticsCommunity
22
“StudentLearningandAnalyticsatMichigan”seminarseries
crlt.umich.edu/slam
Othercomponents:
• LearningAnalyticsfacultytaskforce(3yearcharge)• “ExploringLearningAnalytics”researchprojects• Learninganalyticsfellowsprogram• Streamliningdataaccessandestablishingnewdatasets
![Page 23: Actionable Learning Analytics: Practical Lessons for Teaching and Learning](https://reader033.vdocuments.site/reader033/viewer/2022051318/58eea0621a28ab51128b45f3/html5/thumbnails/23.jpg)
@stevelonn
PracticalLearningAnalyticsMOOC
23
Everyoneineducationhasquestions….PLAisaboutansweringthem.
ReturningSpring2016
![Page 24: Actionable Learning Analytics: Practical Lessons for Teaching and Learning](https://reader033.vdocuments.site/reader033/viewer/2022051318/58eea0621a28ab51128b45f3/html5/thumbnails/24.jpg)
@stevelonn
Summary• Analytic:Institutionalreadinessmeasuredbyinstitutionalreflection(surveyresponses)
• Finding:5factors:Culture,datamanagement,dataanalysis,communication&policyapplication,&trainingallimportantforreadiness
• Whatcanyoudo?:Helpestablishcommunicationaboutlearninganalyticswithcolleagues.– e.g.,Organizeamonthlyseriestodiscussandhavespeakers.
24
![Page 25: Actionable Learning Analytics: Practical Lessons for Teaching and Learning](https://reader033.vdocuments.site/reader033/viewer/2022051318/58eea0621a28ab51128b45f3/html5/thumbnails/25.jpg)
@stevelonn
Aretheredifferentkindsofanalyticsineducation?
25
![Page 26: Actionable Learning Analytics: Practical Lessons for Teaching and Learning](https://reader033.vdocuments.site/reader033/viewer/2022051318/58eea0621a28ab51128b45f3/html5/thumbnails/26.jpg)
@stevelonn
Howlonguntiltheygraduate?
4-yeargraduationrate:22%6-yeargraduation rate:51%
Whichstudentsareaccepted?
Roughly4,300 HSstudentsapplyUMKCaccepts65-68%ofapplicants
Ofthose,about39%enroll(F15 n=1,021)
Whichstudentsstay?
Freshman retentionrate:73%Part-timestudentretentionrate:28%
LearningAnalyticsvs.AcademicAnalytics
26
![Page 27: Actionable Learning Analytics: Practical Lessons for Teaching and Learning](https://reader033.vdocuments.site/reader033/viewer/2022051318/58eea0621a28ab51128b45f3/html5/thumbnails/27.jpg)
@stevelonn
LearningAnalyticsvs.AcademicAnalytics
27
AcademicLoad
EnrollmentbyAcademicLevel
EnrollmentbyDistanceEducation
Sources:irapweb.umkc.edunces.ed.gov
![Page 28: Actionable Learning Analytics: Practical Lessons for Teaching and Learning](https://reader033.vdocuments.site/reader033/viewer/2022051318/58eea0621a28ab51128b45f3/html5/thumbnails/28.jpg)
@stevelonn
HowtoMeasureStudentLearning
• Traditionalmeasures
• Holisticmeasures
• Psychometrics
• 21st CenturySkills
28
![Page 29: Actionable Learning Analytics: Practical Lessons for Teaching and Learning](https://reader033.vdocuments.site/reader033/viewer/2022051318/58eea0621a28ab51128b45f3/html5/thumbnails/29.jpg)
@stevelonn
Data+Tools=NewWaystoMeasureTeachingandLearning
29
Student InformationSystemsLearningManagementSystems
Onlinehomework, peerevaluation,discussions, practicetools,etc.
DashboardsStatisticalanalysistools
MachinelearningSocialnetworkanalysis
Naturallanguageprocessing
![Page 30: Actionable Learning Analytics: Practical Lessons for Teaching and Learning](https://reader033.vdocuments.site/reader033/viewer/2022051318/58eea0621a28ab51128b45f3/html5/thumbnails/30.jpg)
@stevelonn
NewWaysforInstructorstoViewStudentActivity&Performance
30
![Page 31: Actionable Learning Analytics: Practical Lessons for Teaching and Learning](https://reader033.vdocuments.site/reader033/viewer/2022051318/58eea0621a28ab51128b45f3/html5/thumbnails/31.jpg)
@stevelonn
UnpackingCourseResourceAnalytics
31
• NancyKerner• LecturerIV,GeneralChemistry(Emeritus)
• TaughtlargeintroductoryChemistrycourse
![Page 32: Actionable Learning Analytics: Practical Lessons for Teaching and Learning](https://reader033.vdocuments.site/reader033/viewer/2022051318/58eea0621a28ab51128b45f3/html5/thumbnails/32.jpg)
@stevelonn 32
• 10semestersofdata– 8,762students;703,191“hits”
• 4categoriesemerged– Courseinformation– Lecture– Assignments– ExamPreparation
UnpackingCourseResourceAnalytics
![Page 33: Actionable Learning Analytics: Practical Lessons for Teaching and Learning](https://reader033.vdocuments.site/reader033/viewer/2022051318/58eea0621a28ab51128b45f3/html5/thumbnails/33.jpg)
@stevelonn 33
• Raw#of“hits”:Astudents>Bstudents>Cstudents
• 25th à 50th percentileofresourceuse:
UnpackingCourseResourceAnalytics
ResourceCategory ‘A’vs.‘C’ ‘B’vs.‘C’
CourseInformation 0.856 0.907
Lecture 1.228 Not SignificantAssignments Not Significant Not Significant
ExamPreparation 1.811 1.565
Waddington &Nam,2014
![Page 34: Actionable Learning Analytics: Practical Lessons for Teaching and Learning](https://reader033.vdocuments.site/reader033/viewer/2022051318/58eea0621a28ab51128b45f3/html5/thumbnails/34.jpg)
@stevelonn
DoestheModelScale?HowCanitScaleforYou?
34
• Expandedto7largeSTEMcourses– 24Kstudents
• Similartrends– OnlyExamPrepresourcesrelatedtobetterperformancein5of7courses
Waddington, Nam,Lonn&Teasley,InReview
![Page 35: Actionable Learning Analytics: Practical Lessons for Teaching and Learning](https://reader033.vdocuments.site/reader033/viewer/2022051318/58eea0621a28ab51128b45f3/html5/thumbnails/35.jpg)
@stevelonn
Summary• Analytic:LMShitsoncourseresourcesandsemi-manualcategorizationinto4keytypes.
• Finding:Exampreparationmaterials(whenclicked)onlysignificantresourcetyperelatedtoimprovedcourseperformance.
• Whatcanyoudo?:IdentifythetypesofcourseresourcesavailableinyourLMSwebsite– whatcategoriesemerge?– WhatdoestheLMStellyouabouthow,andwho,isusingdifferentcategories?
– Whatistheeffectonlearning?
35
![Page 36: Actionable Learning Analytics: Practical Lessons for Teaching and Learning](https://reader033.vdocuments.site/reader033/viewer/2022051318/58eea0621a28ab51128b45f3/html5/thumbnails/36.jpg)
@stevelonn
Whatlearninganalyticsinnovationsareonthehorizon
atMichiganandbeyond?
36
![Page 37: Actionable Learning Analytics: Practical Lessons for Teaching and Learning](https://reader033.vdocuments.site/reader033/viewer/2022051318/58eea0621a28ab51128b45f3/html5/thumbnails/37.jpg)
@stevelonn
PresentingCourse-RelatedAnalyticstoStudents
37
![Page 38: Actionable Learning Analytics: Practical Lessons for Teaching and Learning](https://reader033.vdocuments.site/reader033/viewer/2022051318/58eea0621a28ab51128b45f3/html5/thumbnails/38.jpg)
@stevelonn
UsingAnalyticstoCoachStudents
38
![Page 39: Actionable Learning Analytics: Practical Lessons for Teaching and Learning](https://reader033.vdocuments.site/reader033/viewer/2022051318/58eea0621a28ab51128b45f3/html5/thumbnails/39.jpg)
@stevelonn
NaturalLanguageProcessing
39
• Outofthebox:Sentiment,bag-of-wordsgroupings,readinglevel• Pairs,triads,sentencestructures• Topicmodeling• Customdictionariesbydiscipline,contentarea,etc.
![Page 40: Actionable Learning Analytics: Practical Lessons for Teaching and Learning](https://reader033.vdocuments.site/reader033/viewer/2022051318/58eea0621a28ab51128b45f3/html5/thumbnails/40.jpg)
@stevelonn
EthicalConsiderations
40
![Page 41: Actionable Learning Analytics: Practical Lessons for Teaching and Learning](https://reader033.vdocuments.site/reader033/viewer/2022051318/58eea0621a28ab51128b45f3/html5/thumbnails/41.jpg)
@stevelonn
AnalyticsforLearners
41
![Page 42: Actionable Learning Analytics: Practical Lessons for Teaching and Learning](https://reader033.vdocuments.site/reader033/viewer/2022051318/58eea0621a28ab51128b45f3/html5/thumbnails/42.jpg)
@stevelonn 42
Knowingis notenough;wemustapply.
Wil l ingis notenough;wemustdo.
―JohannWolfgangvonGoethe
![Page 43: Actionable Learning Analytics: Practical Lessons for Teaching and Learning](https://reader033.vdocuments.site/reader033/viewer/2022051318/58eea0621a28ab51128b45f3/html5/thumbnails/43.jpg)
@stevelonn 43
ThankYou!
StevenLonn,Ph.D.AssistantDirectorforAssessment&EvaluationDigitalEducation&InnovationUniversityofMichigan
[email protected]@stevelonn
digitaleducation.umich.edu