learning analytics: realizing their promise in the california state university
DESCRIPTION
TRANSCRIPT
![Page 1: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/1.jpg)
John Whitmer, CSU Office of the Chancellor & CSU ChicoKate Berggren, CSU Northridge
Hillary Kaplowitz, CSU NorthridgeTom Norman, CSU DH
Learner Analytics Realizing their Promise in the CSU
Download slides at: http://bit.ly/HqaHBF
![Page 2: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/2.jpg)
Outline
1. Promise of Learner Analytics
2. Tools & Systems in Practice
3. CSU Case Studies:• Analytics at Work in the Classroom (Hillary)• GISMO & SQL Query Tools (Kate)• Vista in RELS 180 (John)
4. Q & A
![Page 3: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/3.jpg)
1. PROMISE OF LEARNER ANALYTICS
![Page 4: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/4.jpg)
Steve Lohr, NY Times, August 5, 2009
![Page 6: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/6.jpg)
Economist. (2010, 11/4/2010). Augmented business: Smart systems will disrupt lots of industries, and perhaps the entire economy. The Economist.
![Page 7: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/7.jpg)
7
Source: jisc_infonet @ Flickr.com
Source: jisc_infonet @ Flickr.com
![Page 8: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/8.jpg)
8
What’s different with Big Data? 4 V’s:1. Volume2. Variety3. Velocity4. Variability
(IBM & Brian Hopkins, Forrester)
![Page 9: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/9.jpg)
Academic Analytics
“Academic Analytics marries large data sets with statistical techniques and predictive modeling to
improve decision making”
(Campbell and Oblinger 2007, p. 3)
![Page 10: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/10.jpg)
10
Academic Analytics1. Term adopted in 2005 ELI research report
(Goldstein & Katz, 2005)
– Response to widespread adoption ERP systems, desire to use data collected for improved decision making
– 380 respondents; 65% planned to increase capacity in near future
2. Call to move from transactional/operational reporting to what-if analysis, predictive modeling, and alerts
3. LMS identified as potential domain for future growth
![Page 11: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/11.jpg)
DD Screenshot
![Page 12: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/12.jpg)
Learner Analytics:
“ ... measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs.” (Siemens, 2011)
![Page 13: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/13.jpg)
or said plainly:
What are students doing?
Does it matter?
![Page 14: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/14.jpg)
Learner Analytics1. Analyze combinations of data including:
– Frequency of ed tech usage (e.g. clickstream analysis)– Student learning “outputs” (e.g. quiz scores, text answers)– Student background characteristics (e.g. race/ethnicity)– Academic achievement (e.g. grades, retention, graduation)
2. Current rsch: mostly data mining, not hypothesis-driven
3. More complex than Academic Analytics, considering: – Immaturity of ed tech reporting functionality– Translation of usage into meaningful activity– No significant difference: not what technology used, it’s how it’s
used, who uses it, and for what purpose
![Page 15: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/15.jpg)
A few promises of analytics for faculty and students …
1. Provide behavioral data to investigate student performance
2. Inform faculty about students succeeding or at risk of failing a course
3. Warn students that they are likely to fail a course – before it’s too late
4. Help faculty evaluate the effectiveness of practices and course designs
5. Customize content and learning activities (e.g. adaptive learning materials)
![Page 16: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/16.jpg)
What’s the promise of analytics for academic technologists?
1. Decision-making based on actual practices (not just perceptions) and student outcomes
2. Support movement of A.T. into strategic role re: teaching and learning by:– demonstrating the link between technology
and learning– distinguishing our role from a technology
infrastructure provider
![Page 17: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/17.jpg)
Our 2 biggest barriers
Image Source: http://bit.ly/Hq9Cdg
![Page 18: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/18.jpg)
Image Source: Utopian Inc http://bit.ly/Hq9sCq
![Page 19: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/19.jpg)
Image Source: Privacy in the Cloud: http://bit.ly/HrF6zk
![Page 20: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/20.jpg)
2. TOOLS & SYSTEMS IN PRACTICE
![Page 21: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/21.jpg)
SIGNALS
http://www.itap.purdue.edu/studio/signals/Purdue Signals Project
![Page 22: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/22.jpg)
SNAPP
http://www.snappvis.org/SNAPP (Social Networks Adapting Pedagogical Practice)
![Page 23: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/23.jpg)
KHAN
http://www.khanacademy.org/Khan Academy
![Page 24: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/24.jpg)
OLI
http://oli.web.cmu.edu/openlearning/initiative/processCM Open Learning Initiative
![Page 25: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/25.jpg)
PARCHMENT
http://www.parchment.com/c/my-chances/Parchment
![Page 26: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/26.jpg)
3. CSU CASE STUDIES
![Page 27: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/27.jpg)
27
ANALYTICS AT WORK IN THE CLASSROOM (HILLARY)
![Page 28: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/28.jpg)
28
How can data help teachers and students?
Two stories about how data helped students and teachers work better together
![Page 29: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/29.jpg)
29
![Page 30: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/30.jpg)
30
![Page 31: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/31.jpg)
31
“Hey Professor,
I just looked at my assignments and realized that my Chapter 11 summary did not get submitted, which I'm having trouble believing that I didn't submit it... especially because I see that I did it, and I always submit my assignments as soon as I finish them.”
![Page 32: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/32.jpg)
32
Now the hard part….
Do I believe him? If I only I could check…
![Page 33: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/33.jpg)
33
![Page 34: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/34.jpg)
34
![Page 35: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/35.jpg)
35
And it was all his idea…
The student suggested that I check Moodle and if that didn’t work told me how to check the Revision History in GoogleDocs with step-by-step directions!
![Page 36: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/36.jpg)
36
![Page 37: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/37.jpg)
37
Hybrid Course Weekly Structure
1. Watch lectures
2. Read textbook
3. Online chat and tutoring
4. Post questions and take practice
quiz
4. Class meets
5. Aplia quiz
![Page 38: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/38.jpg)
38
“The quiz is unfair”
![Page 39: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/39.jpg)
39
But the story was not that simple…
»Reports on Moodle painted a different picture»Student was watching the lectures at 10:00 p.m.»Then immediately taking quiz
![Page 40: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/40.jpg)
40
Enabled constructive feedback…
1. Advised the student how the structure of the course was designed to enhance learning
2. Student revised their study habits3. Improved grades and thanked the instructor!
![Page 41: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/41.jpg)
41
What we can do with data now
1. Use Reports in Moodle to verify student claims2. Review participant list to see last access time3. Empower students to review their own reports4. Analyze usage and advise students how to study better5. Review quiz results to find common misconceptions
![Page 42: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/42.jpg)
42
And if we had better tools that are easier to use…
1. Let our students see more details about how their habits affect their grades and encourage them to use them
2. Give instructors access to more information and better tools to organize data so they can see patterns of access and time on task and how they relate to outcomes
3. Have tools that red flag students with teacher set criteria 4. Help streamline workflow for instructors by organizing
student information– View all ungraded assignments
![Page 43: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/43.jpg)
43
Could we help improve student learning outcomes if we knew the effect of…
Coffee
Time
Amount
Textbook
LMS AccessLMS Activities
Mobile
Attendance
Friends
![Page 44: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/44.jpg)
44
GISMO & SQL QUERY TOOL (KATE)
![Page 45: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/45.jpg)
45
GISMO – Course Block
![Page 46: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/46.jpg)
46
GISMO – Access Overview
![Page 47: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/47.jpg)
47
GISMO – Access by Student
![Page 48: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/48.jpg)
48
GISMO – Quiz Overview
![Page 49: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/49.jpg)
49
SQL Query Tool
![Page 50: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/50.jpg)
50
List of Contributed Queries
![Page 51: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/51.jpg)
51
Query Example
![Page 52: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/52.jpg)
52
Query Results: Most Active Courses
![Page 53: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/53.jpg)
53
Query: Most Popular Activities
![Page 54: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/54.jpg)
54
Query: Systemwide use of
Activities and Resources
![Page 55: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/55.jpg)
55
Query:Forum Use Count by Type
![Page 56: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/56.jpg)
Learner Analytics
Thomas J. NormanCalifornia State University
Dominguez Hills
![Page 57: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/57.jpg)
eBook
![Page 58: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/58.jpg)
A New Way of Reading
![Page 59: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/59.jpg)
From Textbooks to Apps
![Page 60: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/60.jpg)
![Page 61: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/61.jpg)
![Page 62: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/62.jpg)
![Page 63: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/63.jpg)
Assignments
![Page 64: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/64.jpg)
Grading To Do List
![Page 65: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/65.jpg)
Real Time Metrics
![Page 66: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/66.jpg)
Warnings
![Page 67: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/67.jpg)
LearnSmart Progress
![Page 68: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/68.jpg)
Analysis by AACSB Categories
![Page 69: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/69.jpg)
Bloom’s Taxonomy
![Page 70: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/70.jpg)
Performance by Learning Objective/Difficulty
![Page 71: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/71.jpg)
Ideas? Questions?
• 310-243-2146
![Page 72: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/72.jpg)
72
CSU CHICO VISTA ANALYTICS
![Page 73: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/73.jpg)
73
LMS Learner Analytics @ Chico StateCampus-wide
– How are faculty & students using the LMS? – What meaningful activities are being conducted? – How does that usage vary by student background, by college, by
department?
Course level– What is the relationship between LMS actions, student background
characteristics and student academic achievement? (6 million dollar question)
– Intro to Religious Studies: redesigned in Academy eLearning, increased enrollment from 80 to 327 students first semester
Ultimate goal: provide faculty and administrators with what-if modeling tools to identify promising practices and early alerts
![Page 74: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/74.jpg)
74
Chart from Scott Kodai, Chico State
![Page 75: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/75.jpg)
CSU Practice
![Page 76: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/76.jpg)
INTRODUCTION TO RELIGIOUS STUDIES (RELS 180)
![Page 77: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/77.jpg)
![Page 78: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/78.jpg)
![Page 79: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/79.jpg)
![Page 80: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/80.jpg)
80
CLOSING THOUGHTS
![Page 81: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/81.jpg)
Call to Action
1. Metrics reporting is the foundation for Analytics
2. Don’t need to wait for student performance data; good metrics can inspire access to performance data
3. You’re *not* behind the curve, this is a rapidly emerging area that we can (should) lead ...
4. If there’s any ed tech software folks in the audience, please help us with better reporting!
![Page 82: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/82.jpg)
Want more? Resources on Analytics
Googledoc: http://bit.ly/HrG6Dm
![Page 83: Learning Analytics: Realizing their Promise in the California State University](https://reader033.vdocuments.site/reader033/viewer/2022051513/546aae72af7959664c8b493d/html5/thumbnails/83.jpg)
83
Q&A and Contact Info
Resources Googledoc: http://bit.ly/HrG6Dm
Contact Info: • John Whitmer ([email protected])• Hillary C Kaplowitz ([email protected])• Berggren, Kate E ([email protected])
Download presentation at: http://bit.ly/HqaHBF