1

Evaluating a Web-Based Program: Digital Learning Commons

Karl NelsonDigital Learning Commons

karln@learningcommons.org

Paul SternWSU - Social & Economic Sciences Research Center

sternpo@wsu.edu

WERAMarch 27, 2008

2

3

4

Online courses

5

6A brief history…

7

So what?

8

Was this course otherwise available to this student?

• Approximately 76% of the courses were not otherwise available.

9

Would the student have graduated without this course?

• 33% of the students would not have graduated without taking DLC course(s).

10

Would the student be “college eligible” without this course?

• The vast majority of college eligible students met four-year college admission requirements without the DLC courses.

11

Would the student be “college prepared” without this course?

• More than half of the students who were college eligible took DLC classes to become more college prepared.

12

Now what?

13

• What are the characteristics of students who are currently enrolled in DLC courses?

• What are the enrollment and completion patterns of students enrolled in DLC courses?

• What is the relationship between the online course grade earned and overall GPA?

Research Questions

14

Data Sources

• DLC staff provided enrollment records on all students enrolled in at least one course during the 2005-2006 academic year. Records included high school, courses enrolled in, completion status, and grade earned

• Student characteristics were obtained via a match against CSRS to obtain data on ethnicity, socio-economic status, disability status, and cumulative GPA.

15

• What are the characteristics of students who are currently enrolled in DLC courses?

o Are DLC courses being accessed by a diverse range of students or is their use concentrated to particular types of students?

Student Characteristics

16

%

Male %

Female DLC

Enrolled Expected

Males Expected Females

District A 50% 50% 2 1 1 District B 25% 75% 8 2 6 Total 10 3 7

Creating a Comparison GroupAn Example to Consider…

17

Gender

Gender of Students Enrolled in DLC Courses Compared to State

(N= 410)

49%51% 51% 49%48% 52%

0%

10%

20%

30%

40%

50%

60%

Male Female

% o

f St

uden

tsX

XX

Comparison Group

Statewide

DLC Data

18

Ethnicity

Ethnicity of Students Enrolled in DLC CoursesCompared to State

(N= 410)

0%9%7%10%11%

63%

1%11%8%

3% 5%

72%

0%

70%

6%9%

6%10%

0%

20%

40%

60%

80%

Asian NativeAmerican

Black Hispanic White Other

Ethnicity

% o

f St

uden

ts

Comparison Group

Statewide

DLC Data

19

Grade Level

Grade Level of Students Enrolled in DLC Courses Compared to State (N= 410)

26% 24%23%24% 24% 23%27%

26%

37%

31%

20%

10%

0%5%

10%15%20%25%30%35%40%45%50%

Freshmen Sophomores Juniors Seniors

% o

f Stu

dent

sX

XX

Comparison Group

Statewide

DLC Data

20

Grade Point Average (GPA)

Overall GPA of Students Enrolled in DLC Courses Compared to State by Gender

2.0 -2.9 GPA 41%

GPA less than 15%3.0 GPA or

greater44%

21

Overall GPA of Students Enrolled in DLC Courses Compared to State by Gender

20%10%

47%

35%

33%

55%

0%

20%

40%

60%

80%

100%

Male (160) Female (176)

% o

f St

uden

tsX

XX

3.0 GPA or greater

2.0-2.9 GPA

0-1.9 GPA

Student GPA Compared to Student Gender

22

Region

Region of Students Enrolled in DLC Courses Compared to State(N= 410)

5%

25%28%

8%15%

7% 7%5%

9%7%9%6% 4%

11% 12%

43%

0%

10%

20%

30%

40%

50%

PugetSound I-5Corridor

Eastern Northwest Coastal Central Southwest Spokane EasternPuget Sound Regions

% o

f Stu

dent

s

DLC Data

State

23

Special Education Students

DLC Special Education Students(N= 410)

10%

3%

11%

0%

5%

10%

15%

20%

DLC Students Statewide Comparison Group

% o

f St

uden

ts

24

Free and Reduced Price Lunch

Free and Reduced Price Lunch Students(N= 410)

40%

30%35%

0%

10%

20%

30%

40%

50%

DLC Students Statewide Comparison Group

% o

f St

uden

tsXX

XX

X

25

Enrollment and Completion Patterns

What are the enrollment and completion patterns of students enrolled in DLC courses? • What are the characteristics of students who

enroll in one DLC course? In multiple DLC courses?

• Are certain types of students more likely than others to complete an online course that they started?

26

Number of Courses Taken

Number of Courses Taken

2 Course28%

3 or More Courses

4%

1 Course68%

27

Course Completion by Courses Taken

Number of Courses Dropped by Number of Courses Taken

80%

41%

23%

0%

20%

40%

60%

80%

100%

1 course (279) 2 courses (116) 3 or more courses (15)

Number of Courses Taken

% o

f St

uden

tsX

XX

28

Course Completion by Region

Proportion of Students who Dropped at Least One Course by Region

37%

57% 57%

19%21%

44%42%

11%

0%

20%

40%

60%

80%

100%

% o

f Stu

dent

s

29

Course Completion by Ethnicity

Students Who Dropped at Least One Course by Ethnicity

29%

50%

32%

11%

27%

0%

20%

40%

60%

80%

100%

Asian(33)

Black(37)

Hispanic(22)

NativeAmerican

(24)

White(270)

Ethnicity

% o

f St

uden

tsXX

X

30

Course Level

DLC Course Enrollees Course Level(Total Enrollees Course Level)

Credit Recovery22%

AP/Honors12%

Standard66%

31

Course Level by Region

84% 81%

97%

9%9%

7%

7%16% 19%

21%

52%

89%89%

73%

3%

32%

3%27%

2%

72%

8%

0%

20%

40%

60%

80%

100%

Central (33) Coastal (60) Eastern (155) Northwest(90)

EasternPuget Sound

(21)

Southwest(46)

Spokane (30) Puget SoundI-5 Corridor

(122)

% o

f Cou

rse

Enr

olle

es

CreditRecovery

AP/ Honors

Standard

Course Level by Region

32

Course Level by Number of Courses Taken

41% 40%

81%

48% 55%

16%

11% 5% 3%

0%

20%

40%

60%

80%

100%

Standard AP/ Honors Credit Recovery

% o

f Stu

dent

s

3 or MoreCourses2 Courses

1 Course

Course Level by Number of Courses Taken

33

DLC Course Enrollee Course Level by GPA

12%25%

36%

11%

63%

52%

89%

12%

0%

20%

40%

60%

80%

100%

Standard (368) AP/ Honors (67) Credit Recovery(122)

% o

f Cou

rse

Enr

olle

esX

XX

3.0 and higher

Between 2.0and 2.9

Less than 2.0

Course Level by GPA

34

Course Subject

Course Subject

25%

19% 19% 17%

6% 5%3% 3% 3%

0%

10%

20%

30%

40%

50%

% o

f C

ours

e E

nrol

lees

35

Gender of Course Enrollees by Subject

79%61% 52% 51% 48% 46%

36% 35%25%

21%39% 48% 49% 52% 54% 64% 65% 75%

0%

20%

40%

60%

80%

100%

% o

f C

ours

e E

nrol

lees

Female

Male

Gender of Course Enrollees by Subject

36

Course Grade

Course Grade for Course Enrollees

D5%

F25%

C26%

B21%

A23%

37

Course Grade by Gender of Course Enrollees

28%

6%

30%

21%

15%

31%

24%

4%

20%21%

0%

10%

20%

30%

40%

50%

A B C D F

Grade

% o

f C

ours

e E

nrol

leesX

XX

Male (253)

Female (304)

Course Grade by Gender of Course Enrollees

38

Grades by Number of Courses Taken in a Single Term

19%

31%

2%

23%25%

6%

22% 21%25%26%

0%

10%

20%

30%

40%

50%

A B C D F

% C

ours

e E

nrol

lees

XX

X

2 or more coursessimultaneously

One course

Grades by Number of Courses Taken in a Single Term

39

Dropped Courses by Subject

15%11%

22%27%32%32%

42%50%50%

0%

20%

40%

60%

80%

100%

% o

f C

ours

es

Dropped Courses by Subject

40

Relationship Between Online Course and Overall GPA

What is the relationship between the online course grade earned and overall GPA?• Do specific types of students have better

relative grades in online courses than others? • In which subjects do students get the highest

and lowest grades online?

41

Overall GPA Compared to DLC GPA

45%40%

15%30% 30%

39%

0%

20%

40%

60%

80%

100%

0 - 1.9 2.0 - 2.9 3.0 and greater

% S

tude

ntsX

XX

Overall GPA

DLC GPA

Overall GPA Compared to DLC GPA

42

Grades Earned in DLC Courses by Students' Overall GPA

49% 40%16%

41%37%

21%

10%23%

63%

0%

20%

40%

60%

80%

100%

0 - 1.9(N=41)

2.0 - 2.9(N=111)

3.0 +(N=124)

Overall GPA

% o

f St

uden

tsXX

X

A or B

C

D or F

Grades Earned in DLC Courses by Students’ Overall GPA

43

Comparison of DLC and Cumulative GPA of Students by Course Level

3.63.1

2.3

1.3

2.3

2.7

0.0

1.0

2.0

3.0

4.0

DLCGPA

OverallGPA

DLCGPA

OverallGPA

DLCGPA

OverallGPA

Standard AP/ Honors Credit Recovery

Ave

rage

GPA

Comparison of DLC and Cumulative GPA by Course Level

44

Comparison of DLC and Cumulative GPA of Students by Subject

1.8

2.8

1.9

2.6

2.02.42.4

1.7

3.3 3.12.7 2.73.4

2.6

3.1

2.6

3.43.4

0.0

1.0

2.0

3.0

4.0

Arts

Busines

s

Foreign L

anguag

e

Languag

e Arts

Lifesk

ills-H

ealth

Math

Science

Social S

tudies

Technolo

gy

Ave

rage

GPA

DLC GPA

Overall GPA

Comparison of DLC and Cumulative GPA by Subject

45

Outcomes

46

47

Who should take courses?

48

Comparison of DLC and Cumulative GPA of Students by Course Level

3.63.1

2.3

1.3

2.3

2.7

0.0

1.0

2.0

3.0

4.0

DLCGPA

OverallGPA

DLCGPA

OverallGPA

DLCGPA

OverallGPA

Standard AP/ Honors Credit Recovery

Ave

rage

GPAHmmm….

49

Next steps…

50

Report

The report is available on the DLC website:

www.learningcommons.org/about/dlc_archives.php(“2006-2007 External Evaluation - Online Courses”)