diving into data: trends and patterns in mooc analytics
DESCRIPTION
A unique partnership between a computer science and education faculty member has provided an opportunity to create a MOOC for middle and high school teachers designed to provide high quality online professional development in computer science education. The MOOC provided teachers with instruction and support in learning App Inventor, a free web-based application that allows users to learning programming by creating mobile apps. The MOOC was designed around the principles of evidence-based practices in online learning and included design features that addressed social presence and building community with participants. Two unique features of this MOOC included learning communities facilitated by mentor teachers and required Google Hangout sessions. This online workshop, funded by a grant from Google, has allowed the faculty to research learner behavior by quantifying and analyzing analytics in the MOOC. Data was collected through Course Builder, the learning management system, and Google Analytics. Interesting patterns and trends have been identified, that have provided information for online learning in any capacity as well as the development of future MOOCS. For example, the presenters have identified the different ways that certain participants have engaged with course material. The MOOC was designed to allow any participant to have access to the material with accountability measures required of "certificate completers". These participants showed differences in how they accessed and engaged with the material in the course. In this session, the presenters will provide analytics from the MOOC and will ask participants to help engage in analyzing the data for trends. Ideas for using analytics for continuous improvement and course design in online learning will be included in the discussion.TRANSCRIPT
Diving into DataTrends and Patterns from MOOC Analytics
Online Learning Consortium Conference
Orlando, Florida
October, 2014
Chery Takkunen, PhD-School of Education
Jen Rosato, MA -Department of Computer Science/Information Systems
The College of St. Scholasticawww.css.edu
SoTL Commons Conference- 2014- Georgia
College of St. Scholastica, www.css.eduCollege of St. Scholastica, www.css.edu
Goals of the SessionQuestions
Background
Course Design
Data
Implications
College of St. Scholastica, www.css.edu
The College of St. Scholastica
Location
College
Growth Strategy 10% Gr
College of St. Scholastica, www.css.eduCollege of St. Scholastica, www.css.edu
Background
CS + EDU= Unique Partnership
Computer Science Education
Professional Development Workshops
Experience in Online Teaching and Learning
Grants= TAG, Google CS4HS, Local/Regional*New- 2015- National Science Foundation
*New- Certificate in Computer Science Education
College of St. Scholastica, www.css.edu
CS4HS is an annual grant program promoting computer science education worldwide by connecting educators to the skills and resources they need to teach computer science & computational thinking concepts in fun and relevant ways. Traditionally, these have been in-person workshops.
College of St. Scholastica, www.css.edu
MOOC defined *
A Massive Open Online Course (MOOC; English pronunciation: /muːk/) is an online course aimed at unlimited participation and open access via the web. In addition to traditional course materials such as videos, readings, and problem sets, MOOCs provide interactive user format that help build a community for students, professors, and teaching assistants (TAs). MOOCs are a recent development in distance education.
Wikipedia.org: http://en.wikipedia.org/wiki/Massive_open_online_course
College of St. Scholastica, www.css.edu
http:mfeldstein.comemerging_student_patterns_in_moocs_graphical_view/
Phil Hill
College of St. Scholastica, www.css.eduCollege of St. Scholastica, www.css.edu
Conceptual FrameworkCommunity of Inquiry
The Community of Inquiry model. Garrison, R., Anderson, T, Archer, W. and Rourke, L et al. (2007).
College of St. Scholastica, www.css.eduCollege of St. Scholastica, www.css.edu
Teacher Professional Development
1) Long-term and intensive2) Clear outcomes3) Collaboration and community4) Use of online tools for effective PD5) Five core features:
a) content and pedagogyb) consistency with reforms c) coherence with educational goalsd) active learning- reflection and inquirye) aligned with standards
College of St. Scholastica, www.css.eduCollege of St. Scholastica, www.css.edu
Applying Principles to CS4HS Course
MOOC-like DesignLMS- Course BuilderParticipation LevelsBuilding Community
Professional Learning Communities (PLC) Mentors
Google HangoutsGoogle Hangout on Air with guest speakers
Discussion forumsNarrated presentations
College of St. Scholastica, www.css.eduCollege of St. Scholastica, www.css.edu
Participant Levels
Casual Participant
vs.
Certificate Completer
College of St. Scholastica, www.css.eduCollege of St. Scholastica, www.css.edu
App Inventor
Dave WolberProfessor, Computer ScienceUniversity of San Francisco
College of St. Scholastica, www.css.eduCollege of St. Scholastica, www.css.edu
Course
Unit Page Types:1. Objectives2. CS Unplugged*3. Hangout On Air4. App Inventor Tutorial, Part 1*5. App Inventor Tutorial, Part 2*6. Pedagogy*7. Group Hangouts8. Discussion9. Additional Resources
*Included activities (formative assessments)
College of St. Scholastica, www.css.eduCollege of St. Scholastica, www.css.edu
Analytics Data...the discovery and communication of meaningful patterns in data
GA - Google AnalyticsGCB - Google Course Builder
GG - Google Groups
http://en.wikipedia.org/wiki/Analytics
College of St. Scholastica, www.css.edu
Participants
We were planning on 50
Over 400 participants
Over 40 states
Over 40 countries
College of St. Scholastica, www.css.edu
Google Analytics (GA)
Other: Canada, Puerto Rico, Tunisia
90 % of visits from the US
*Kristen Donahue, Kassandra Quick & Alvaro Hernandez-Feris
College of St. Scholastica, www.css.edu
GCB: Registration Data
College of St. Scholastica, www.css.edu
http:mfeldstein.comemerging_student_patterns_in_moocs_graphical_view/
Phil Hill
College of St. Scholastica, www.css.edu
GCB: Student Progress
College of St. Scholastica, www.css.edu
GA: Student Pageviews
What questions does this raise? What else would you
like to know? How would you investigate?
College of St. Scholastica, www.css.edu
GCB: Completion Rates
College of St. Scholastica, www.css.edu
GCB: Assessment Progress
College of St. Scholastica, www.css.edu
GA: Page Types
What questions does this raise? What else would you
like to know? How would you investigate?
College of St. Scholastica, www.css.edu
GG: Discussion Post Data
College of St. Scholastica, www.css.edu
GG: Thread Interactions
Average
Total
Replies
3.9 67
Views 32.5 552
What questions does this raise? What else would you
like to know? How would you investigate?
College of St. Scholastica, www.css.edu
GG: Thread Interactions
College of St. Scholastica, www.css.eduCollege of St. Scholastica, www.css.edu
Summary of Trends
Completion rates follow the typical MOOC behavior (w/higher rates in ours)Combine analytics and registration data for a more complete pictureCompletion rates increase later in course (students are more invested)Assessment participation follows completion ratesDiscussion participation follows completion ratesSome content appears to be more engaging than others
(pedagogy & guest speakers vs unplugged lessons)
College of St. Scholastica, www.css.eduCollege of St. Scholastica, www.css.edu
Continuous Improvement
How have we used the data from summer 2013?
Multiple certificate levelsOnline office hours & Community manager (contact for those not in a PLC)Paid close attention to the Unit 2 drop off
Unit 2 lighter than followingMentors checked in with all participants in their PLC groupsMonitored forum more closely
Page types Kept a separate pedagogy page in unitsQuizly questions AI tutorial pages
College of St. Scholastica, www.css.edu
Recommendations
Analytics Recommendations:Know what data you can get from where
(GA vs LMS and other tools)Learn what the data meansPilot the analytics before the courseSet time limits for collecting data
General Recommendations:Be intentional Give teachers a range of optionsMatch intent with support levelsPlan design around learner behaviorProvide opportunities to informally interact
w/content & each otherRequire discussion - tempers the superposter
phenomenon