The Asynchronous The Asynchronous Discussion Board as an Discussion Board as an

Assessment Tool: A Assessment Tool: A Critical AppraisalCritical Appraisal

Dr Jeremy Williams

Universitas 21 Global

8th CAA ConferenceLoughborough University

6-7 July 2004

Overview

1. The context2. Learning through discussion3. Criteria for assessing discussion4. The hazards of assessing

discussion5. Authentic discussion board

assignments6. Preliminary findings

1. The context1. The context

Universitas 21 Global

The global nature of U21G

Online graduate business school, headquartered in Singapore

Around 300 students enrolled from more than 20 countries, in many different time zones

Adjunct faculty from Europe, Australasia, Asia and North America

Pedagogy

Instructor-led, student centredProblem-based learningExtensive use of Harvard Business

school casesA commitment to authentic

assessmentModus operandi = asynchronous

discussion => how discussion boards are used critically important

2. Learning 2. Learning through through

discussiondiscussion

Communities of practice

‘With the expansion of online learning … tremendous opportunities are becoming available to teachers and learners to foster peer relationships, team skills, collaboration, group problem solving and debate, as well as less formal or structured communications between students and their wider professional communities’

Morgan & O’Reilly (1999, p. 86)

Peer learning

‘In the online classroom, it is the relationships and interactions among people through which knowledge is primarily generated’

Palloff and Pratt (1999, p. 15) Collaboration between students from far-

flung institutions around the globe can considerably enrich the experience and broaden the contextual perspectives of each participant

Day (1998)

Dialogue and greater learner autonomy

Enhancing dialogue is a way of promoting independence and autonomy in learners, and challenges power relationships in teaching and learning

Evans & Nation (1989) ‘The digital learning environment will

probably be the most efficacious “enabler” of independent and self-determined learning’

Peters (2000, p. 16)

Laurillard’s Conversational Framework

‘… studies show that a collaborative discussion environment is highly valued by students …’ ‘Students have access to an expert whom they

can question to clarify the expert’s description Students can articulate and re-articulate their

descriptions of the topic in response to others’ ideas and comments

Students can reflect on the discussion to clarify their own understanding’

Laurillard (2002, p. 148)

Encouraging online discussion

Grades – the currency that students deal in

Swan et al (2000)Needs to be well-integrated into

the subject, and will also be of greater purpose to students if it is assessed

Day (1998) cited in Morgan & O’Reilly (1999, p. 86)

3. Criteria for 3. Criteria for assessing assessing

discussiondiscussion

MacKinnon (2000)

Four ‘categories of interaction’:1. Challenging a point of view2. Forwarding a new perspective3. Relating the theory to one’s

experience4. Offering support for a position

based on the literature.

Sabin et al (2000)

P is for participationA is for additional

commentaryC is for constructive criticismE is for encouraging

Meyer (2004)

Proffers 4 different analytical frameworks:

Two were developmental models: King and Kitchener’s Reflective Judgment

Model Perry’s model of intellectual and ethical

development

Two captured levels of thinking: Garrison’s four-stage critical-thinking model Bloom’s taxonomy of educational objectives

4. The hazards of 4. The hazards of assessing assessing

discussiondiscussion

“You can take a horse to water, but …”

‘Collaborative learning is undeniably important, and the communicative media are powerful enablers that match what is needed for discussion and collaboration, but to what extent do they succeed in enabling learning?’

‘… the properties of a medium do not determine the quality of learning that takes place…’

Diana Laurillard (2002) Rethinking University Teaching, p. 148

Assessing discussion: Counter-intuitive?

Harlen and Deakin Crick (2003) – the motivation for learning can be discouraged unwittingly by assessment and testing practices

Early experience at U21G suggests that when discussion contributions are assessed, discussion becomes a little contrived

A case of Goodhart’s Law applied to assessment?

5. Authentic 5. Authentic discussion board discussion board

assignmentsassignments

Discussion boards for discussion

Discrete small-scale problems abandoned in favour of discussion board assignments integrated with case study assignments.

Small assessment weighting for participation remains

The main incentive to contribute – the grounding it will provide for the submission of a case study assignment with a larger assessment weighting.

Assessment ‘in disguise’

Trials now taking place:1. All discussion board activity is

evaluated with an overall mark awarded

2. Students nominate selected discussion board contributions for assessment (e.g. their 5 best)

3. Discussion boards will be considered by the instructor in validating peer assessments of student performance

6. Preliminary 6. Preliminary findings findings

Student feedback

This prompted actual threads of discussion rather than what I've found in other modules where one makes rather artificial comments

on other student's postings as the class attempts to get marks for making exactly the

same points in slightly different words.

Student feedback

I thought the discussion boards were useful in preparing for the assignments because they focused on the content of those assignments so you were able to get tips from your fellow

students and the tutor. In other modules, discussion boards were separate to the

written assignment and so represented an additional rather than complementary task.

Summary

Appropriately structured, discussion boards can provide a useful vehicle for learner assessment

The act of assessing need not have a stultifying effect on the quality of debate and the depth of learning

Future work

A comparative analysis of the qualitative substance of discussion board postings before and after the change in discussion board format