modeling and assessing online discussions for faculty learning and technology development janet de...

Modeling and AssessingOnline Discussionsfor Faculty Learningand Technology Development

Janet de Vry and George WatsonUniversity of Delaware

Barbara FreyUniversity of Pittsburgh

Copyright de Vry, Frey, Watson, 2006. This work is the intellectual property of the author. Permission is granted for this material to be shared for non-commercial, educational purposes, provided that this copyright statement appears on the reproduced materials and notice is given that the copying is by permission of the author. To disseminate otherwise or to republish requires written permission from the author.

Agenda

• Faculty Development Class Janet de Vry

• Problem Based Learning ContextGeorge Watson

• Assessment Barbara Frey

• Application to Your Campus

Workshop Goal:Experience Learning in an Online

Discussion

• Participate in collaborative discussions

• Engage in critical thinking

• Interact at a high level

“The principal idea behind PBL is that the starting point for learning should be a problem, a query, or a puzzle that the learner wishes to solve.”

—Boud (1985)

What is Problem-Based Learning?

“…careful inspection of methods which are permanently successful in formal education…will reveal that they depend for their efficiency upon the fact that they go back to the type of situation which causes reflection out of school in ordinary life. They give pupils something to do, not something to learn; and if the doing is of such a nature as to demand thinking, or the intentional noting of connections; learning naturally results.”

—John Dewey (1916)

What is Problem-Based Learning?

PBL is a learning approach thatchallenges students to “learn to learn,”working cooperatively in groups,to seek solutions to real world problems.

What is Problem-Based Learning?

What is Problem-Based Learning?

PBL prepares studentsto think critically and analytically,to find and use appropriate learning resources,to communicate effectively, orally and in writing,to work well as members of a team.

What are the Common Featuresof PBL?

• Learning is initiated by a problem.

• Problems are based on complex, real-world situations.

• All information needed to solve problem is not initially given.

• Students identify, find, and use appropriate resources.

• Students work in permanent groups.

Problem-Based Learning CycleOverview/

Assessment Problem, Project, or Assignment

Group Discussion

Research

Group Discussion

Preparation ofGroup “Product”

Whole Class Discussion

Mini-lecture(only when needed!)

A Typical Day in a PBL Course

Good PBL Problems…• Relate to real world, motivate students• Require decision-making or judgments• Are multi-page, multi-stage• Are designed for group-solving• Pose open-ended initial questions that

encourage discussion• Incorporate course content objectives,

higher order thinking, other skills

Workshop Design Issues

• Developing problem

• Gathering authentic data

• Simulating asynchronous world in a synchronous setting

• Developing questions, activities, staging

• Sequencing activities; both online and face to face

The Problem in 5 Stages

1. Identify problems with your students’ discussion

2. Formulate strategies for solutions

3. Post best solutions for entire class

4. Get expert advice

5. Develop an individual action plan

Initial Online Instructions

AuthenticData—

DiscussionTopics

Subject: War in IraqI think that if you don't support the war, you don't support our troops.

You are DISLOYAL!------------------------------------------------------------

There are veterans and families who don't support the war. What do youthink about that?------------------------------------------------------------

>I think that if you don't support the war, you don't support our troops.>You are DISLOYAL!

I agree!!!! ------------------------------------------------------------

You are just wrong, you idiot!!!!!

Authentic Data—Sample Student Dialogue

We are in the classroom.

Talking is permitted.

We are in the “Online World.”

Please refrain from talking.

Activities

Online Face to Face

1. Pre-class to analyze problem

1. Warm up exercise

2. Group to generate & evaluate strategies

2. Group to compare online experience to actual class

3. Individual to pose comments and develop questions for experts

3. Mini lecture on rubrics; presentation of action plan

Ongoing Activities

• Chat with experts• Submit Resources• Individual action plan

Asynchronous Discussions

• Strengths– Student interaction– Flexible format– Reflection time to

develop thoughts– Permanent record of

discussion

• Weaknesses– Time consuming – Multiple simultaneous

threads – Linear presentation of

postings– Late contributions are

not developed– Single-pass reading

for unread messages

Small Group Discussion Map

Discussion Question:Develop a list of at least 10 different strategies to improve the quality of online discussions. Share your top 3 strategies with the whole class.

Analyzing Discussions

• Consider trigger or initial questions

• Review whole thread

• Consider depth vs. breadth of discussions

• Assess levels of thinking (Bloom’s Taxonomy)

• Reflect on unanswered questions and undeveloped topics

• Consider timeliness of responses

Evaluation

Synthesis

Analysis

Application

Comprehension

Knowledge

Bloom’s Levels of Cognitive Skill

http://www.umuc.edu/ugp/ewp/bloomtax.html

Small Group Discussion Map

Advantages & Disadvantagesof Discussion Map

• Easy to visualize where a posting is located within the structure of a thread

• Easy to see the complexity or simplicity of a thread

• Provides a mechanism for reviewing the postings – code by level of thinking, date or participant

• Time consuming to develop

Observations• Postings were on-topic• Subject lines did not change to reflect content

or intent of posting• Thread initiators usually responded to thread

contributors• Type of trigger question affected cognitive skill

demonstrated in postings more than level/depth within thread

• Expectations woven into some discussion questions

• Some threads ended with unanswered question or comment

Recommendations

Basic training and guidelines for participants should include guidelines to:– Change Subject Lines which act as advanced

organizers– Take “ownership” of initiated threads; if

appropriate, respond to questions and be responsible for summarizing the thread

Recommendations

Facilitators should set clear guidelines and expectations. These should include:– Design good initiator or trigger questions– Keep topics specific and focused– Establish specific time frame– Provide models of well composed subject

lines and postings – Encourage participants to summarize

threads – Consider the facilitator’s role in discussions

Feedback from Participants

What will be useful in your teaching?

– “Managing online discussions”– “Rubrics for online discussion”– “How to moderate discussion”– “How to keep students on topic”

3 months later—what was the impact of workshop?

• “Defining topics is as important as setting guidelines”

• “Experience with what can go wrong with discussions”

• “Concrete ideas for implementation and evaluation”

• “I still had trouble figuring out how to use discussions.”

Questions?

Workshop website

www.udel.edu/janet/MARC2006