learning analytics: more than data-driven decisions

Learning Analytics:More Than Data-Driven Decisions

Steven LonnResearch Fellow

USE Lab, Digital Media Commonswww.umich.edu/~uselab

1

USE LabUniversity of Michigan

http://umich.edu/~uselab

Acknowledgements

• USE Lab:– Stephanie D. Teasley– Andrew Krumm– R. Joseph Waddington

• John Campbell• John Fritz• Tim McKay• David Wiley

2

USE LabUniversity of Michigan

http://umich.edu/~uselab

What is Analytics?

3

+ +

USE LabUniversity of Michigan

http://umich.edu/~uselab

Analytics in Our Lives

4

USE LabUniversity of Michigan

http://umich.edu/~uselab5

Analytics in Our Lives

USE LabUniversity of Michigan

http://umich.edu/~uselab

Analytics in Our Work

6

USE LabUniversity of Michigan

http://umich.edu/~uselab

Analytics in Our Work

6

USE LabUniversity of Michigan

http://umich.edu/~uselab

Analytics in Our Work

6

What does one DO with all this d

ata?

USE LabUniversity of Michigan

http://umich.edu/~uselab

Data Collected at . .

7

What kind of data is already available those

“in the know?”

USE LabUniversity of Michigan

http://umich.edu/~uselab

• High school GPA• SAT & ACT• Parental education• First generation college student?• Socio-economic status• Admission “rank”• AP tests & scores

8

Admissions

Data Collected at . .

USE LabUniversity of Michigan

http://umich.edu/~uselab

• Gender• Ethnicity• Age• Michigan residency• Country of origin & citizenship• Athlete?

9

Demographics

Data Collected at . .

USE LabUniversity of Michigan

http://umich.edu/~uselab

• Cumulative GPA • Specific course grades• Major / minor• Number of Michigan credits• Number of transfer credits• Credits / grades in subsets (e.g., math courses)

10

Academic Record

Data Collected at . .

USE LabUniversity of Michigan

http://umich.edu/~uselab

• CTools (courses, projects, etc.)• Library (Mirlyn, website, electronic journals)• Wolverine Access• Other UM tools (LectureTools, SiteMaker,

UM.Lessons, MFile, Webmail, etc.)

11

Other Places Data is Gathered...

Data Collected at . .

USE LabUniversity of Michigan

http://umich.edu/~uselab

Current Use of Data...

12

USE LabUniversity of Michigan

http://umich.edu/~uselab

What if...• Identify:

– Who needs the most help– Most successful sequence of courses– Most / least successful portions of a course

• Notify:– Instructors about their students– Students about their performance compared to peers– Academic advisors about students “at risk”– Staff about their resources (e.g., library use)

13

USE LabUniversity of Michigan

http://umich.edu/~uselab

Milestones

• Stage 1: Extraction & reporting of transaction-level data

• Stage 2: Analysis and monitoring of operational performance

• Stage 3: What-if decision support (e.g., scenario building)

• Stage 4: Predictive modeling & simulation

• Stage 5: Automatic triggers of business processes (e.g., alerts)

14

-- Goldstein & Katz, 2005

USE LabUniversity of Michigan

http://umich.edu/~uselab

!"#$%&"#'()#*+,""#'-#.//#&(0&1&02,+#"$20)($"3

USE LabUniversity of Michigan

http://umich.edu/~uselab

Signals

• Purdue University

• System developed in 2007

• Use of analytics for:

– improving retention

– identifying students “at risk” of academic failure

16

USE LabUniversity of Michigan

http://umich.edu/~uselab

Signals

• NBC Nightly News Clip: http://www.msnbc.msn.com/id/21134540/vp/32634348

• Aired August 31, 2009

17

USE LabUniversity of Michigan

http://umich.edu/~uselab

Signals• 6-10% improvement in retention• 58% of students using report seeking help b/c of

Signals use

• Controlled by the instructor• Course-by-course• Does not show students direct comparison with

their peers

19

USE LabUniversity of Michigan

http://umich.edu/~uselab

“Check My Activity” Tool• University of Maryland, Baltimore County

20

USE LabUniversity of Michigan

http://umich.edu/~uselab

“Check My Activity” Tool• University of Maryland, Baltimore County

20

USE LabUniversity of Michigan

http://umich.edu/~uselab

“Check My Activity” Tool• University of Maryland, Baltimore County

20

USE LabUniversity of Michigan

http://umich.edu/~uselab

“Check My Activity” Tool• University of Maryland, Baltimore County

20

• Student-controlled

• Designed to promote student agency & self-regulation

• Low impact for the instructor

USE LabUniversity of Michigan

http://umich.edu/~uselab

Projects

• ITS UM-Data Warehouse– One place where all data can be aggregated and reported

out.– Currently includes:

• Student Dataset• eResearch

• Financial• Human Resources• Payroll

• Physical Resources

21

USE LabUniversity of Michigan

http://umich.edu/~uselab

Projects

• M-STEM Academy & USE Lab– 50 Engineering students per cohort– Use CTools data to better inform

mentor team• When do they need mentoring /

direction to resources?

– How do mentors & students make use of this data?

– How does behavior change?

22

USE LabUniversity of Michigan

http://umich.edu/~uselab

Projects

• M-STEM Academy & USE Lab– 50 Engineering students per cohort– Use CTools data to better inform

mentor team• When do they need mentoring /

direction to resources?

– How do mentors & students make use of this data?

– How does behavior change?

22

!"!!#$

%!"!!#$

&!"!!#$

'!"!!#$

(!"!!#$

)!"!!#$

*!"!!#$

+!"!!#$

,!"!!#$

-!"!!#$

%!!"!!#$

./0$-$ ./0$%*$ 123$&$ 123$-$ 123$%*$

!"#$"%

&'(")!*+%&,)

-'&")

../0)123)/*45+%"6)788)

4567/85$9:;35$

<=2>>$?@/32A/$

USE LabUniversity of Michigan

http://umich.edu/~uselab

Projects

23

Social Network Analysis

USE LabUniversity of Michigan

http://umich.edu/~uselab

Projects

• Tim McKay– Arthur F. Thurnau

Professor of Physics

• Taught into Physics courses for years

• Director: LS&A Honors Program

• Used LS&A ART tool to track student progress.

24

USE LabUniversity of Michigan

http://umich.edu/~uselab

Projects

• Studied nearly 50,000 students over 12 years

• Can predict final grades within 0.5 grade dispersion

• Next project: use an e-coach programmed with analytics data to motivate ALL students

26

USE LabUniversity of Michigan

http://umich.edu/~uselab

Issues to Ponder• Who is the audience?

– Students, Instructors, Advisors, Deans, Staff, Others?

• Who has the control?

– Issues of burden?

• Which views?

• Privacy concerns?

– Is their an institutional obligation?

• Is Learning Analytics just a fad?

• Others?

26

USE LabUniversity of Michigan

http://umich.edu/~uselab

Further Reading• Campbell, J., Deblois, P., & Oblinger, D. (2007). Academic analytics: A new tool for a new era.

EDUCAUSE Review, 42(4), 40−57.

• Fritz, J. (2011). Classroom walls that talk: Using online course activity data of successful students to raise self-awareness of underperforming peers. The Internet and Higher Education, 14(2), 89-97. doi:10.1016/j.iheduc.2010.07.007

• Goldstein, P., & Katz, R. (2005). Academic analytics: The uses of management information and technology in higher education — Key findings (key findings) (pp. 1–12). Educause Center for Applied Research. http://www. educause.edu/ECAR/AcademicAnalyticsTheUsesofMana/156526

• Macfadyen, L. P., & Dawson, S. (2010). Mining LMS data to develop an “early warning system” for educators: A proof of concept. Computers & Education, 54(2), 588−599. doi:10.1016/j.compedu.2009.09.008.

• Morris, L. V., Finnegan, C., & Wu, S. (2005). Tracking student behavior, persistence, and achievement in online courses. The Internet and Higher Education, 8(3), 221−231. doi:10.1016/j.iheduc.2005.06.009.

27!"#$#%&'((%)%*+'((,-./012#3-