0

10

20

30

40

50

60

70

80

90

100

1996 1997 1998 1999

Homework Project FinalMeasures (grades)

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

1996 1997 1998 1999

UG Honors GradCombined class makeup

Initial evaluation (Analysis)• results– homework (and overall) grades improved substantially– distribution of grades also narrowed substantially

• comments– may partially result from an observed improvement in the

overall performance of our undergraduate students– the addition of the problem-solving sessions was valuable

(and was one of the goals of the framework)– change is dramatic enough, however, to indicate that the

framework was effective

Initial evaluation (1997-98)• on-campus students – primarily used MANIC materials for review and while working on

homework assignments– supported by our analysis of the MANIC logs

• 45 minutes per MANIC session• 50% used the text-only features (not the audio track)• the average time on play-out was 7 minutes

– the search/index capability of MANIC particularly useful in locating materials related to specific assignments

– online audio material useful in “filling in the gaps”– had load and bandwidth problems

• off-campus– MANIC presentations user-friendlier than videotapes because of

the index/search capability– had problems with security and firewalls

anecdotal evidence and MANIC log analysis

Initial evaluation (1999 Questionnaire) • Utilization– 82% used text, “coursepacks” and copies of the overhead

transparencies• 29% found the text, “coursepacks” useful• 88-89% found overhead copies useful

– 59% used MANIC units• 90% found MANIC useful• 60-70% used MANIC for review and homework

– but preferring to look first at printed material– MANIC superior to the printed and on-line Power Point slides, because of

the detail added by the audio track

• Satisfaction– students were quite satisfied with the courses– too much material– liked the “problem solving” sessions– 24% said that class attendance was affected

• but it provided a level of comfort, not a temptation

Attended class Most of the time Once/week Infrequently MANIC affect?59% 24% 18% 24%

Used Text/handout MANIC Videotape82% 53% 59% 18%

Useful Not useful % usedText/course-pack 29% 65% 47%Handouts 88% 12% 100%Online Power Point 89% 11% 53%MANIC 90% 10% 59%Videotape 67% 33% 18%

Replace class Review Homework SupplementalUsed MANIC for 30% 70% 60% 30%

Office hours email Web MANICFor questions, used: 50% 44% 65% 53%

Questionnaire