inside moocs: student expectations and achievement #aln13

http://tinyurl.com/insidemoocs-aln13

Clark Shah-Nelson, Sr. Instructional Designer

Center for Teaching and Learning [email protected]

Sukon Kanchanaraksa, Director

Center for Teaching and Learning [email protected]

Inside MOOCs: Student Expectations and Achievement

Founded: 1916

Graduate School

Students: 2164 from 87 nations

Faculty: 619 FT; 785 PT

Research: on-going in more than 130 countries

First institution of its kind worldwide Largest school of public health in the world Receives 20 percent of all grants and contracts awarded to the 50 accredited U.S. schools of public health Ranked No. 1 by U.S. News & World Report since 1994

Johns Hopkins Bloomberg School of Public Health

Since 1997…

JHSPH Online Courses

1

1 1

0 1

4 3 1

3 5

5

7 7

13 5

7 6 4

5 7 7

7 4

19

9 9

0

20

40

60

80

100

120

00-‐01 01-‐02 02-‐03 03-‐04 04-‐05 05-‐06 06-‐07 07-‐08 08-‐09 09-‐10 10-‐11 11-‐12 12-‐13

Number of Online Offerings* AY00 to AY12

New

Repeat

Courses

As an important component of the School’s mission, JHSPH publishes the materials used in the teaching of actual courses freely and openly available on the Web. (Since 2005)

Creative Commons license is used with the conditions – attribution, non-commercial and share alike. (creativecommons.org)

No permission is needed. We encourage students, educators, and public health practitioners to use OCW content.

JHSPH OCW

Content from 112 academic courses, trainings, and symposia is published on JHSPH OCW.

In the past 5 years (Oct 28, 2009 – Oct 27, 2013)

- 1.36 million visits - 1 million unique visitors

China is the country with the third most visits (after US and India)

- 48,358 visits - 41,150 unique visitors

JHSPH OCW

2012: CTL Looking to make OCW more interactive, started discussion OCW+

Began relationship with Coursera – moved rapidly to prepare first round of courses

Groundwork in place:

intellectual property & copyright instructors on board openness culture faculty with online teaching experience existing online course development/instructional design/OCW infrastructure

JHSPH OCW -> Coursera

Coursera First Run: Total Enrolled Students

Course Instructor Enrolled Students

Bio Statistics Brian Caffo 16,872

Computing for Data Analysis Roger Peng 55,558

Community Change Bill Brieger 15,386

Data Analysis Jeff Leek 101,747

Food Systems Bob Lawrence, Keeve Nachman 17,164

Obesity Economics Kevin Frick 23,410

Health For All Henry Perry 13,145

Vaccine Trials Karen Charron, Amber Cox 11,546

Total Students 254,828

Coursera First Run: Statements of Achievement (SOA)

SOA = Statement of Achievement

Bio

Sta(s(cs Comp Data

Community Change

Data Analysis

Food Systems

Obesity Economics

Health For All

Vaccine Trials

Total Students Enrolled 16,872 55,558 15,386 101,747 17,164 23,410 13,145 11,546

Total SOA 749 3,940 768 4,260 2,556 1,908 1,033 1,865

Percent 4% 7% 5% 4% 15% 8% 8% 16%

•  Surveys administered at opening/closing of course

•  Multiple surveys with same email address

•  Used “conditional formatting” in Excel to identify entries

•  Color coded surveys to indicate properties of duplicate entries •  Looked at time between entries, number of entries and number of differences

•  Kept first entry to capture participants’ initial reaction

•  Sorted out entries after the first survey submission

The Data

The Survey Data

Total Students: 254, 828

Aggregate of all courses Percent

Total Students earned an SOA 17079 6.70% Total Students who submiNed

Pre-‐Survey 53459 21.00% Total Students who submiNed

Post-‐Survey 10046 3.90% Total Students who submiNed both

surveys 4695 1.80%

How do students intend to participate in our MOOCs?

0.00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00% 70.00% 80.00% 90.00%

Watch all videos

Watch videos of interest

Watch all videos,

complete assignment &

earn SOA

No Response

9.50% 2.40%

87.90%

0.20%

Pre-Survey Intentions Response

http://www.polleverywhere.com/multiple_choice_polls/yLOHh2ekLAqG6DT

http://pollev.com/aln13 TXT code to 37607

Poll on Intent and Achievement 1:

Of those who intended to earn an SOA, how many did?

92%

8%

Students Who Intended to Earn an SOA

Earned SOA Didn't Earn SOA

http://www.polleverywhere.com/multiple_choice_polls/B9EdfZbRhXp15w6

http://pollev.com/aln13 TXT code to 37607

Poll on Intent and Achievement 2:

Of those who did NOT intend to earn an SOA, how many did?

69%

31%

Students Who Didn't Intend to earn an SOA

Earned SOA Didn't earn SOA

What are some of the factors motivating student participation in our MOOCs?

0.00%

5.00%

10.00%

15.00%

20.00%

25.00%

30.00%

35.00%

40.00%

Academic Field Relevant

Job/Career Skills

Earn a Credential

Prestigious University

Fun and Enjoyable

Curiousity about online

courses

Why Students Enrolled

Very Important

Quite Important

Moderately important

Slightly important

Not important

No Response

●  Top 3 reasons for enrollment:

●  The course teaches skills to help participant’s career

●  The course will be enjoyable and fun

●  The course is directly related to the participant’s academic field of study

●  One Exception: Obesity Economics Top 3 Reasons

●  The course will be enjoyable and fun

●  The participant is curious about what it is like to take an online course

●  The course teaches skills to help participant’s career

What are some of the factors motivating student participation in our MOOCs?

How many students submitted one or all assignments?

*Data Analysis omitted

Total Students in 7 courses: 153,081

Aggregate of 7 courses Percentage

SOA's earned 12822 8.40%

Students submiNed all required assignments 13321 8.70%

SubmiNed at least one assignment 32653 21.30%

Forum Participation and SOA’s

Total Students: 158,988

48% of students who participated in the forums earned an SOA

52% of students who participated in the forums did not earn an SOA

Aggregate across

5 courses

Percent of total students

enrolled ParScipated at least

once in forum 6062 3.80% ParScipated at least

once in forum & earned SOA 2935 1.80%

ParScipated in Forum & didn't earn SOA 3125 2.00%

Didn't parScipate in forum & earned SOA 5827 3.70%

White paper:

“Massive Open Online Courses in Public Health” in Frontiers in Public Health

http://tinyurl.com/moocs-in-ph-provisional

Gooding I, Klaas B, Yager JD and Kanchanaraksa S(2013) Massive Open Online Courses in Public Health. 1:59. doi:10.3389/fpubh.2013.00059

http://www.frontiersin.org/Journal/Abstract.aspx?s=503& name=public%20health%20education%20and%20promotion& ART_DOI=10.3389/fpubh.2013.00059

More information

“Funnel of Participation” – 9 JHSPH MOOCs


“Funnel of Participation” 9 JHSPH MOOCs with Peer Assessments


How valuable were each of the following factors in helping students complete the course?

0.00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00%

Regular Assessment Deadlines

Interaction with others

Sustained engagement with videos

Appropriate difficulty level

Appropriate length of course

Related personal background and experience

Interaction with Professor

No Response

Strongly disagree

Disagree

Neutral

Agree

Strongly agree


How satisfied were students with various aspects of the course?

0%

10%

20%

30%

40%

50%

60%

I am happy with

what I learned

Course materials presented

in engaging manner

Want to take a more

advanced course

Found this course

personally fulfilling

Learned what I was hoping to learn in

this course

Will use what I

learned in this course in my job

Strongly agree

Agree

Neutral

Disagree

Strongly disagree

No Response


What would have made students more likely to complete the course?

0% 5% 10% 15% 20% 25% 30% 35% 40% 45%

Reduce Time Commitment

Make Course Material Easier

Make Course Material More Difficult

Credential More Valuable

Make Course Length Shorter

More Direct Access with Instructor

No Response Strongly disagree Disagree Neutral Agree Strongly agree


Did they find the course useful, even if they didn't complete it?

0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50%

Happy with what I learned

Course Materials Engaging

Desire to take a more advanced course

Personally Fulfilling

Met Learning Expectations

Relevant to Job

No Response

Strongly disagree

Disagree

Neutral

Agree

Strongly agree


Enrollment in MOOCs Updated: October 31, 2013

*Total enrollment includes enrollment in planned, in-progress, and completed offerings.

Updated SOA Numbers through 3 Offerings

Course 1st SOA % 2nd SOA % 3rd SOA %

Training & Learning Programs for Volunteer CHWs 7,194 Saving Lives Millions at a Time 7,122

Design & InterpretaSon of Clinical Trials 8,685 StaSsScal Analysis of fMRI 8,552

MathemaScal BiostaSsScs Bootcamp 2 8,514

StaSsScal Reasoning for Public Health 10,393

Major Depression in the PopulaSon 16,406

IntroducSon to the US Food System 17,164 2,556 14.89% 4,171

Case-‐Based IntroducSon to BiostaSsScs 19,631 2,712 13.81% 2,249

Vaccine Trials: Methods and Best PracSces 11,546 1,868 16.18% 9,471 1,033 10.91%

Health for All Through Primary Health Care 13,145 1,033 7.86% 12,706 718 5.65% 7,532 436 5.79%

Principles of Obesity Economics 23,193 1,908 8.23% 6,355

MathemaScal BiostaSsScs Bootcamp 1 16,398 749 4.57% 22,153 1,427 6.44% 18,270 1,109 6.07%

Community Change in Public Health 15,271 767 5.02% 12,046 716 5.94% 1,105

CompuSng for Data Analysis 50,883 3,938 7.74% 44,661 6,271 14.04% 83,441 6,768 8.11% Data Analysis 101,747 4,254 4.18% 72,202

Updated SOA Numbers through 3 Offerings

Course

No. of Completed

Offerings To Date

End of Course Enrollment in Completed Offering(s)

SOAs Issued in Completed Offering(s)

Passing Rate in Completed Offering(s)

IntroducSon to the US Food System 1 17,164 2,556 14.89%

Case-‐Based IntroducSon to BiostaSsScs 1 19,631 2,712 13.81%

Vaccine Trials: Methods and Best PracSces 2 21,017 2,901 13.80%

Health for All Through Primary Health Care 3 33,383 2,187 6.55% Principles of Obesity Economics 1 23,193 1,908 8.23%

MathemaScal BiostaSsScs Bootcamp 1 3 56,821 3,285 5.78% Community Change in Public Health 2 27,317 1,483 5.43% CompuSng for Data Analysis 3 179,015 16,977 9.48% Data Analysis 1 101,747 4,254 4.18%

Total 17 479,288 38,263 7.98%

Thank you! Questions? [email protected]

Additional Credits Instructors: Bill Brieger, Brian Caffo, Karen Charron, Amber Cox, Kevin Frick, Bob Lawrence, Jeff Leek, Keeve Nachman, Roger Peng, Henry Perry JHSPH CTL OWC/MOOC Coordinator: Ira Gooding JHU School of Education Intern: Hilliary Googash http://tinyurl.com/insidemoocs-aln13

inside moocs: student expectations and achievement #aln13

Education

survey datatotal students

jhsph ocw coursera

soa didnt

graduate school students

jhsph ocw content

academic courses

schools of public health

online coursethe course