Assessment in Online Courses: Practical Examples

Dr. Roger Von HolzenMs. Darla RunyonDr. Phillip HeelerNorthwest Missouri State University

2

Heard in the Halls“If we are to be required to assess educational quality and learning by virtue of how long a student sits in a seat, we have focused on the wrong end of the student.” Laura Palmer Noone

3

Heard in the Halls“Every person putting the same amount of time into the same subject matter, regardless of the learner’s previous experience, aptitude for that subject, or ability to learn, no longer makes sense.” William Draves

4

“How do you do online exams?”

Question based on notion that online assessment must follow assessment methods used on campus Not necessarily true

In an online environment Radical change to the role of the instructor

Shift from the deliverer of content to student mentor

Function of assessment techniques must also change*

5

Online Assessment: An Interactive Mentoring Opportunity

Need to move beyond the rhetoric that assessment should be utilized as a teaching tool and not as an evaluation mechanismUse quizzes and tests as interactive mentoring opportunities Enable students to evaluate their own

progress through the course materials Provide feedback on course content areas

that need further enhancement and/or development*

6

Beyond the RhetoricQuizzes and tests should be viewed as means of promoting learning Open book and extensive testing time

Should be only a small component of the overall assessment strategy for the online course*

7

Beyond the RhetoricEvolution from seat-time/credit hours to outcomes-based education as a measure of learning Acknowledging present reality

What matters is whether the student has actually learned something*

8

The New Role of Assessment

Assessment techniques should be based on desired learning outcomesAssessment results should be used by students to evaluate progress through course materials Provide the instructor with: evidence of effectiveness of course

materials indications of content areas that need

further enhancement and/or development*

9

Assessment StrategyCumulative processAids in forming student assessment profiles snapshot of student understanding

Profile constructed by: building learning outcomes based on

critical course content use of applicable assessment methods to

determine student’s understanding of learning outcomes*

10

Assessment StrategyProvides guidance to further develop conceptual frameworkContinuous process (formative)Should guide the student to mastery of the learning outcomesAssessment strategy becomes foundation for developing the instructional design of the online course*

11

Assessment StrategyLearning outcomes: should be measurable through an

applicable assessment of that outcome

should provide evidence of mastery of the learning outcome through student performance*

12

Learning OutcomesDetermine critical course content Discern what the students should know

or accomplish based on the critical content What must the student know in order to

function in authentic situations? Decide what evidence is acceptable as

proof of knowledge or accomplishment of the learning outcome Selected student performance must furnish

the method of assessment of critical content*

13

Communication of Learning Outcomes

Include in syllabusList in course introductory moduleList for each individual module or unitConvey in related activities and assignments*

14

Assessment Strategy Steps1.Assist faculty in integrating new

assessment techniques and developing overall assessment strategy

2.Administer pre-assessment Provides guidance in the development

of appropriate learning activities3.Present critical content through

interactive, instructional concepts and activities*

15

Assessment Strategy Steps4.Punctuate course with short

assessment opportunities Provide student with performance feedback on learning concepts and activities

Provide a diverse array of assessment methods to reflect student understanding of the learning outcomes

Provide opportunities for relearning and reassessment*

16

Assessment Strategy Steps5.Develop a post-assessment

(summative) Provides evaluation of the overall student performance

Indicates ultimate mastery of critical content and ability to incorporate content into appropriate situations*

17

Additional Assessment Results

Provide instructor feedback on content delivery methods and techniques Feedback directs the instructional

redesign of the course and the instructor’s role*

18

Online AssessmentProvides an organized and systematic approach to assessment Digital exam building features

Variety of traditional testing methods are available

Multiple Choice True and False Fill-in-the-Blank Multiple Answer Ordering Matching Short Answer/Essay

Options to pool questions and control the delivery of the material*

19

Online AssessmentTraditional methods should only be a small component of the overall assessment strategy Learning outcomes should be assessed using applicable assessment techniquesOnline delivery provides an environment conducive to incorporating a diverse array of assessment techniques strategies that may be employed across a

variety of course subject areas*

20

Online AssessmentFlexibility of delivery allows for a more student-centered approach to assessment and feedback Proctored exams Some situations may require on-site

examinations Expenses and effort involved must be

considered Traditional methods should only be a small component of the overall assessment strategy *

21

Issues of Academic Dishonesty

Work closely with faculty as they discuss and develop new perspectives on assessment Academic dishonesty and conduct in an online course should be examined as faculty design and develop online assessment strategies*

22

Issues of Academic Dishonesty

Academic dishonesty and honor code policies should be clearly stated early in the course Include in the course syllabus*

23

Issues of Academic Dishonesty

Policy examples Students are responsible for

submitting their own work Students who cooperate on

examinations or other work without authorization share the responsibility for violation of academic principles and are subject to disciplinary action*

24

Dealing with PlagiarismWeb tracking services www.turnitin.com www.plagiarism.com

Instructional design of course site, assignments and examsCommunicate with students*

25

Practical ExamplesOne-Sentence Summary Challenges students to answer the

questions "Who does what to whom, when, where, how, and why?" about a given topic, and then to synthesize those answers into a single informative, grammatical, and long summary sentence.*

26

Practical ExamplesMinute Paper The instructor asks students to

submit comments related to the following two questions: "What was the most important thing you learned from this lesson?" and "What important question remains unanswered?"

Students then submit their responses by e-mail or in a threaded discussion*

27

Practical ExamplesPunctuated Lectures Requires students to go through five steps:

listen, stop, reflect, write, and give feedback Students begin by listening/viewing a lecture or

demonstration. Then, after a portion of the presentation has been

completed, it is stopped. The students are asked to reflect on the lecture or

demonstration. They then write down any insights they have gained. Finally, they submit feedback to the instructor in the

form of short notes.*

28

Practical ExamplesConcept Maps Drawings or diagrams showing the

mental connections that students make between a major concept the instructor focuses on and other concepts they have learned.

Students are asked to sketch the important features of the geography around major concepts such as democracy, racism, art, or free trade.*

29

Practical Examples Paper or Project Prospectus Paper Prospectus--a brief, structured first-

draft plan for a term paper or term project. Prompts students to think through elements of

the assignment, such as the topic, purpose, intended audience, major questions to be answered, basic organization, and time and resources required.

Project Prospectus--focuses on the tasks to be accomplished, skills to be improved, and products to be developed.*

30

Practical ExamplesAnalytic Memos Requires students to write a one- or

two-page analysis of a specific problem or issue

The person for whom the memo is being written is usually identified as an employer, a client, or a stakeholder who needs the student's analysis to inform decision making*

31

Practical ExamplesElectronic Mail Feedback The instructor poses a question to the

class, via e-mail, about his or her teaching, and invites student responses

Student respond to the e-mail question with a personal message sent to the instructor's e-mail account*

32

Practical ExamplesExam Evaluations Allow instructors to examine both what

students think they are learning from exams and tests and students' evaluations of the fairness, appropriateness, usefulness, and quality of tests or exams

May help provide verification as to the authorship of exam answers*

33

ReferencesAngelo, T, & Cross, P. Classroom Assessment Techniques: A Handbook for College Teachers (Second Edition). 1993.

Boettcher, J, & Conrad, R. Faculty Guide for Moving Teaching and Learning to the Web. 1999.

Draves, W. Teaching Online. 2000.

Kaczmarczyk, L. Accreditation and Student Assessment in Distance Education: Why We All Need to Pay Attention. Proceedings of the 6th Annual Conference on Innovation and Technology in Computer Science Education. 2001.

34

Dr. Roger Von Holzen, DirectorCenter for Information Technology in Educationrvh@mail.nwmissouri.edu

Ms. Darla RunyonAssistant Director/Curriculum Design SpecialistCenter for Information Technology in Educationdrunyon@mail.nwmissouri.edu

Dr. Phillip Heeler, ChairmanComputer Science/Information Systems Departmentpheeler@mail.nwmissouri.edu