inline tutorial self assessments: improving satisfaction (and possibly learning) robert schudy dan...

Inline Tutorial Self Assessments: Improving Satisfaction

(and possibly learning)

Robert SchudyDan Hillman

Problem Statement

Conventional web pages and “talking at them” lectures have serious pedagogic weaknesses:– When the presentation is done students don’t know

what they learned and didn’t, so they are unsure about their preparation for subsequent topics.

– Most assessments (feedback) are done on a weekly, non-immediate basis, delaying guidance and slowing learning.

– If the particular presentation didn’t work for a student an alternative presentation is not automatically presented.

The Test Yourself Solution

• We defined and implemented, using JavaScript, the ability to introduce interactive tutorial self assessments directly in lecture web pages.

• Dan named these “Test Yourself.”• All Test Yourself formats include the ability to

incorporate tutorial material, often per answer.• We have used these in several courses in the MSCIS

and Admin Sciences online and on-campus programs, and they have been well received by students.

Test Yourself Styles

1. Single answer “show answer”2. Single answer “radio buttons”3. Multiple answer “check boxes”

1. Single answer “show answer”

Note answerand detailedmathematicalproblemsolution

2. Single answer“radio buttons”

Note indicationsof student answer,correct and incorrectanswers, and tutorialmaterial for eachanswer.

3. Multiple answer“check boxes”

Note thetutorial materialfor each answer,which helps studentsfocus their studyon the answersthey missed,without needingto search foranything.

Evidence that students love Test Yourself

• In-course surveys were very positive.• Students have praised Test Yourself in the student

evaluations.• When courses didn’t have Test Yourself, students

who experienced them in earlier courses requested them during the course and in the evaluations.

• When Test Yourself was in some but not all pages of a course, students requested them in all pages.

Evidence that suggests that Test Yourself Improves Learning

• Students reported that Test Yourself improves the time efficiency of their learning.

• There is some evidence that where we put Test Yourself items we have improved quiz scores; we need to assess this more formally.

• It would be interesting to conduct a formal controlled learning experiment, but we may need to involve the human subjects institutional review board because we have enough evidence that it helps learning that we would risk disadvantaging the control group.

Disclaimers and Realities

• We have not conducted a formal research evaluation of how effective this is in promoting learning.

• Developing the content requires about one hour of faculty time per Test Yourself.

• We don’t have the infrastructure to collect discriminant analysis, usage, or other data.

• Implementation requires a little knowledge of HTML. This is easy online, where we have professional instructional designers, but may be problematic without.

Lessons Learned

“What did I put for my answer?”

“Do the checks mean statementsare true or that my answers are correct?”

Make it obvious.

Correct/incorrectfeedback

Make it obvious.

Correct/incorrectfeedback

Studentanswer

Make it obvious.

Correct/incorrectfeedback

Studentanswer

True or false prefaces

Future Work

• This is a small step in making our web sites more interactive and supportive, and adapting the learning to the student, but much more is possible.