participants guide - learning and teaching...

Assessment Alignment &

Building Rubrics

Participants Guide

©Copyright RMIT – Academic Development Group – College of Business 2015

i

Table of contents

Introduction ..................................................................................................................... iv

Assessment Rubrics ......................................................................................................... 1

Purpose ............................................................................................................................ 1

Section outcomes ............................................................................................................ 1

Rubrics Unpacked ................................................................................................................ 2

Benefits for Educators .......................................................................................................... 2

Benefits for Students ............................................................................................................ 2

Supporting the feedback process ......................................................................................... 2

Use of rubrics in Blackboard GradeMark ............................................................................... 3

Constructive Alignment .................................................................................................... 4

Purpose ............................................................................................................................ 4


PLOs and CLOs Unpacked .................................................................................................. 5

Consistency is the KEY ......................................................................................................... 5

Summarising the PLOs ......................................................................................................... 5

Summarising the CLOs ......................................................................................................... 6

Checking the Assessment Criteria ........................................................................................ 7

Building Rubrics ............................................................................................................... 8

Purpose ............................................................................................................................ 8


Finding the Right Words ....................................................................................................... 9

Constructing Sentences ....................................................................................................... 9

Rubric Example - Essay ...................................................................................................... 10

The Australian Qualifications Framework (AQF) - It’s Role in Language ................................. 11

Blooms Taxonomy and Verbs ............................................................................................. 12

Bloom’s Cognitive Matrix .................................................................................................... 13

Things to Consider… ...................................................................................................... 16

References ..................................................................................................................... 17


ii

About this workbook

This workbook has been designed using adult learning concepts to support a variety of learning styles.

Prompts used in this workbook

Summary

Caution

Activity

Key Concept

Notes

Reference


iii

Introduction

Welcome The aim of this session is to provide you with the skills and knowledge to effectively write clear assessment criteria and develop assessment rubrics. You will be provided with an opportunity to gain practical experience in aligning Program Learning Outcomes (PLOs) and Course Learning Outcomes (CLOs) with assessment criteria while supporting students by building an assessment rubric.

Outcomes By the end of this training session, you should be able to:

Constructively align PLOs with CLOs to inform correct use of Rubric language

Align assessment criteria to CLOs to cover the PLOs entirely

Map assessment types used across courses and where they support the assessment outcome.

Pre-requisites No pre-requisite.

Target audience Course Coordinators and nominated teaching staff

Structure You will be provided with an overview of Assessment Alignment, including developing rubrics and its associated policies and processes.

The learning environment encourages interaction by all participants and is facilitated by a training specialist.

You will have the opportunity to participate in activities to further enhance learning and understanding of the session content.

©Copyright RMIT – Academic Development Group – College of Business 2015 iv

Assessment Rubrics

Purpose To highlight the value of using assessment rubrics for both teaching staff and students.

Section outcomes Areas covered in this section

• Rubrics unpacked

• Benefits of using assessment rubrics for staff

• Benefits of using assessment rubrics for students

• How assessment rubrics can support the feedback process

• Use of rubrics in Blackboard GradeMark


1

Rubrics Unpacked

Assessment rubrics list the criteria set for an assessment task and articulate the quality of performance in each criterion. They are valuable for both academic teaching staff and also students. Including assessment rubrics for each assessment in each course assists students to identify clearly the expected standards for all assessment tasks.

Key features of including rubrics in teaching can include:

• Rubrics are a powerful tool for both teaching and assessment and provide clarity for both academics and students

• Broadly, rubrics underpin the requirements and expectations of how students need to approach their assessments to achieve CLOs

• Rubrics guide feedback about progress towards the standards that have been set for the assessment criteria

Benefits for Educators

For educators, using rubrics can:

• Provide the foundation of feedback to students • Improve marking objectivity, consistency and efficiency, both across different standards

and across student cohorts • Decrease queries from students about assessments

Benefits for Students

For students, using rubrics can:

• Provide explicit criteria and standards • Provide clarity in expectations of what is required • Provide guidance in identifying the performance gap between their submitted work and

the desired standard • Assist students to develop understanding and make judgments about the quality of

their work

Supporting the feedback process

Regular feedback is the most powerful way to enhance students’ learning. Students need frequent opportunities to articulate what they know and receive suggestions for improvement. Feedback that occurs during learning is active and supports students in realising their progress and respond to improve their achievement. Good timely feedback:

• Clarifies goals, criteria, expected standards that are aligned to the course learning outcomes and the assessment rubrics

• Enables the development of self-assessment through reflection • Encourages teacher and student conversations about learning • Encourages positive motivational beliefs and self esteem

©Copyright RMIT – Academic Development Group – College of Business 2015 2

Use of rubrics in Blackboard GradeMark

GradeMark enables instructors to grade student papers online by evaluating against qualitative or quantitative rubrics and adding comments and feedback. Rubrics can be attached in GradeMark to improve efficiency of marking. Guidance regarding the use of rubrics in Blackboard GradeMark is available from your College Academic Development or Learning and Teaching Group. Training and guides on how to use and develop rubrics in GradeMark and Turnitin are available via developME Blackboard professional development site https://sites.google.com/a/rmit.edu.au/blackboard-training/courseware/3-assessments/turnitin NOTES:


3

https://sites.google.com/a/rmit.edu.au/blackboard-training/courseware/3-assessments/turnitin

Constructive Alignment

Purpose To identify the key factors that influence learning and program objectives for the alignment process for building assessment rubrics.

Section outcomes Areas covered in this section:

• The importance of mapping PLOs with CLOs to ensure they are aligned

• Summarising PLOs

• Summarising CLOs

• Checking the Assessment Criteria


PLOs and CLOs Unpacked

Program Learning Outcomes (PLOs) and Course Learning Outcomes (CLOs) need to be mapped and aligned carefully to ensure the student learning experience is being offered as expected.

• Program Learning Outcomes describe the knowledge, skills and its application, that students will be able to do when they graduate by completing the required courses of a program.

• Course Learning Outcomes clearly describe the learning to be attained by students.

Consistency is the KEY

Consistent use and the quality of rubrics used as an assessment tool/ guide can contribute to a range of outcomes and ensure students expectations are met through all courses in a program. Some of these outcomes (as described in the book: Introduction to Rubrics - Dannelle D. Stevens and Antonia J.Levi) can include:

• Less students at risk • Less confusion • Less enquiries • Less grievances.

Summarising the PLOs

Summarising the PLOs is the first step to the alignment phase. By summarising down to the key concepts, the alignment process becomes streamlined and easer to work with.

Identifying the key concepts/ words in the PLOs will help simplify the outcomes for this program. See Example-1 below.

EXAMPLE-1

PLO statement and outcomes extract:

Juris Doctor graduates will have the foundational legal knowledge required for a career as a legal practitioner or legal/non-legal roles in a variety of law-related fields You will also possess the ability to analyse a problem involving a question or questions of law and, through research, to provide a solution to it. You will possess the skills needed to practice law in a variety of fields, industry sectors and locations; and you will develop skills in effective writing, drafting, advocacy, interviewing and advising, negotiation, team work and presentations. You will have an understanding of the application of ethical standards; be able to demonstrate judgment and strategic reasoning; and be effective in self-management and be lifelong learners. Possessing strong self- management skills, you will be able to work well individually and as members of teams. You will have sound commercial judgment and an awareness of the ethical and justice issues affecting all stakeholders in the Australian legal system.

On the successful completion of this program, you will be able to:

1. Apply an advanced and integrated understanding of a complex body of knowledge in professional practice.

2. Demonstrate the intellectual and practical skills needed to justify and interpret theoretical propositions, legal methodologies, conclusions and professional decisions.


5

3. Identify, research, evaluate and synthesise relevant factual, legal and policy issues.

4. Practice law in a variety of fields, industry sectors and locations.

5. Apply and integrate professional theory with practice in authentic Work Integrated Learning (WIL) contexts, professional contexts and industry settings.

PLOs summarised:

• Foundational - Application and legal knowledge • Effective – Research and thinking analysis skills • Clear – Communication and collaboration • Sound – Commercial Judgement • Strong – Management Skills • Awareness – Ethical and Justice Issues

Summarising the CLOs

Summarising the CLOs is the second step to the alignment phase. Extracting important keywords that will relate to assessment provides easy reference to outcomes, so that alignment can be undertaken easily.

EXAMPLE -2

LAW2528 CLOs Extracted:

At the end of this course, students will have developed:

1. Knowledge of a wide range of legal concepts, values, principles and rules of Australian contract law;

2. A sound understanding of the grounds on which a contract may be deemed void, rescinded and/or terminated and the ability to identify the various forms of relief and remedies which may be available in these circumstances;

3. An appreciation of the commercial context in which contractual, equitable and statutory obligations interact and operate;

4. The ability to work collaboratively and communicate effectively to a range of different audiences;

5. The skills to identify the relevant principles in cases and statutes to support decision-making and advice;

6. The capacity to articulate and apply those principles to specific fact scenarios and come to a well-reasoned conclusion on the relevant legal obligations and likely outcomes.

CLOs summarised:

• Knowledge of Legal Concepts • Sound understanding of contract law • An appreciation of commercial context • Work collaboratively and communicate effectively to different audiences • Able to identify relevant principles in cases and status • Capacity to articulate principles to scenarios and come to a well-reasoned conclusion.


IMPORTANT: If you feel that some of your CLOs are not aligning, remember constructive alignment should not take place in isolation. It is recommended that teaching teams collaborate and share ideas and suggestions when making the alignment between the PLO and the CLOs

Checking the Assessment Criteria To ensure alignment of assessment is correct, you need to make sure that each of the assessment criteria in each assessment aligns back to some or all of the CLOs. For example: If you have three assessments in your course, you need to make sure that collectively all of the assessment criteria across the three assessments have been aligned back to all the CLOs.

CLOs Assessment Criteria

CLO-1 _______________________ CLO-2_______________________ CLO-3_______________________ CLO-4_______________________ CLO-5_______________________ CLO-6_______________________

Assessment 1

• Criteria 1 • Criteria 2 • Criteria 3

Assessment 2

• Criteria 1 • Criteria 2 • Criteria 3 • Criteria 4

Assessment 3

• Criteria 1 • Criteria 2 • Criteria 3

NOTES:


7

Building Rubrics

Purpose To ensure a uniform approach across all teaching teams within a program, including the language within a rubric and the dialogue with the students is consistent.

Section outcomes Areas covered in this section:

• Building verbs for specific rubrics

• Using language to support the expectations

• The Impact AQF can have on building rubric language

• Blooms Taxonomy and assessment types

• Tips for building sentences in rubrics


Finding the Right Words

When identifying the right verbs for the type of assessment criteria it is important to ensure the language is tied back to the PLO and CLOs using the language that is related to the outcomes. As a teaching team you need to question the verbs chosen before building the language around the verbs. The verbs need to be appropriate to the assessment type. For example, you probably would not use the term ‘dynamic’ to describe an essay. A better word could be ‘awe inspiring’ However, the word ‘dynamic’ might be best used to describe a presentation. CAUTION: It is very important to identify the right verbs that fit the assessment criteria. Be very careful when copying existing rubrics!

Constructing Sentences

Reflection plays a vital role in identifying how to articulate what you require from students and what they must achieve. You need to prepare your students to reach their optimum goal/mark by describing what the expectations are to achieve this successfully.

To get started on building sentences it’s important to do some reflection. Ask yourself the following two questions; what it is you are asking the students to do, and why did you create the assignment and what are your expectations of the students? Below is an example of how this was answered for a Juris Doctor program:

EXAMPLE- 3

Juris Doctor Reflection

Q1. What it is you are asking the students to do?

A. We are asking students to prepare a 2000 word brief to council.

Q2. Why did you create the assignment and what are your expectations of the students?

A. We want to see what they know about legal concepts; how well they can identify relevant principles and that they have the capacity to articulate with clarity and produce well-reasoned conclusions.

Use the Rubric Development Reflection tool below to deepen the reflection process. (See Figure 1 Rubric Development Reflection)


9

Figure 1. Rubric Development Reflection - Adapted from (Stevens & Levi 2005)

After the reflection, have a look at the learning objectives listed. Based on your reflection you should be able to write up the highest expectation first, followed by the lowest e.g. HD and NN (see Figure. 2).

Rubric Example - Essay Learning

Objectives HD DN CR PP NN

10% Consistent use of legal citation and referencing (AGLC) and correct use of terminology

Excellent and highly detailed citation and referencing and consistently correct terminology.

Very good citation and referencing with mostly correct use of terminology

Good citation and referencing with generally correct terminology

Fair citation and referencing and room for improvement in use of terminology

Failure to use the correct legal citation and referencing and incorrect use of terminology.

40% Thoroughness of research, and quality of synthesis of legal materials and information relevant to the topic for the written assessment task.

Relevant research materials thoroughly synthesised and discussed in a way that demonstrates integrated understanding of a range of perspectives.

Widely researched with evidence of very good comprehension of diverse perspectives.

Well researched with some diversity of reading and good synthesis of materials.

Reasonable reading and synthesis with room for improvement in depth and breadth of reading and synthesis.

Inadequate research with little or poor synthesis of materials.

Rubric Development Refection

1 Why did you create the

assessment? 2

Have you given something

simiar before?

3 How does it

relate to everything else

you are teaching?

4 What skills are

to be developed to successfully complete the assignment?

5 What exactly is

the task assigned?

6 What evidence do the students

need to provide?

7 What are the

highest expectations?

8 What are the

worst expectations?


50% Clear articulation of issues, applicable law and legal reasoning; a well-structured discussion and presentation of arguments; critical assessment of legal issues and law reform (as appropriate).

Excellence in structure and well-articulated issues, using sophisticated reasoning; excellent analysis and critique of appropriate issues.

Very good structure and reasoning with extensive analysis and critique of appropriate issues.

Good structure and reasoning with some analysis and critique of appropriate issues.

Reasonable structure and reasoning with basic analysis and critique of appropriate issues.

Poor or inadequate structure and reasoning and a failure to address and analyse critical issues.

Figure 2. Rubric Example - Juris Doctor Essay

The Australian Qualifications Framework (AQF) - It’s Role in Language

Australian Qualifications Framework (AQF) underpins Australia’s national education system of qualifications. It draws upon current industry practices to inform what level of capability Industry needs from their graduates. By asking what the current industry skill set requirements are, we can determine what level a qualification should be awarded. (See Figure 3.)

The Australian Qualifications Framework Council outlines the Criteria and Qualification Type Descriptors for each AQF level. We can use this resource to link the knowledge, skills and application to the appropriate course level. The criteria and descriptors draw upon language from an industry’s current needs, then filters down to the course level outcomes which directly impact the assessment level and language used to describe that level.

Students develop the specific capabilities outlined in the AQF/Program level to prepare them for industry expectations and in turn build the industry’s confidence in the qualification.

Figure 3. Impact on Education

What Industry Needs

AQF (Level of

Qualification/ program)

Course Level

Assessment Level

(Rubric)

Student Capabilities


11

Blooms Taxonomy and Verbs

Blooms taxonomy was developed in 1948 by psychologist Benjamin Bloom. Although the original taxonomy focused on three domains; cognitive, affective, and psychomotor, the most prominent use of the taxonomy is the Cognitive domain. The Cognitive domain provides a framework for acquiring knowledge, and can assist teachers when designing learning activities and assessments.

Within the Cognitive domain there are six developmental categories: Creating, Evaluating, Analysing, Applying, Understanding and Remembering. These categories support the development of a student’s intellectual aptitudes and skills (see Figure. 4)

Figure 4. Blooms Taxonomy –Revised

Source: http://ww2.odu.edu/educ/roverbau/Bloom/blooms_taxonomy.htm

We can use Blooms Taxonomy to help identify what type of assessment is appropriate to support the cognitive level of development in students. By using a cognitive matrix (see Figure. 5) to map the cognitive level to the action; activities and assignments can be better defined and target specific levels of development more accurately.

Using the Blooms Cognitive Matrix in conjunction with the Blooms Taxonomy Verb Wheel (see Figure. 6), you can easily and quickly find the verbs that are appropriate to the student outcome/products.

As part of the reflection process when choosing and developing language around the rubric verbs, you can draw on the verbs from the Blooms Taxonomy and match the cognitive level with the rubric grading scale e.g. a verb used in the Evaluating or Creating domain could help support a High Distinction (HD) or Distinction (D) grade.


Bloom’s Cognitive Matrix

Cognitive Level This includes (student

products)…

That can be identified by… Using tools such as…

CREATING Compose, construct, design,

manage, prepare, arrange,

produce

Digital Analogue • PowerPoint

• Wikispaces

• Google Blogger/

Communities

• Blackboard

• Virtual Music Composer

• Windows Movie Maker

• Photostory

• video production

• Podcast

• Blog

• Wiki

• Song

• Digital stories

• Professional presentation

EVALUATING Estimate, compare, argue,

critique, predict, assess, judge,

measure

• Video Reflections

• Online Group

discussions - debating

• Online Peer editing

• Case-studies

• Appraisals

• Projects

• Self- evaluating

• Google Communities

• Blackboard Discussion

Boards

• Google Hangouts

• Google Docs

ANALYSING Compare, debate, question,

differentiate, test, solve,

experiment, relate

• Online surveys

• Digital graphs

• Online Peer review

• Mashing

• Linking

• Tests/quiz

• Class discussions

• Google Form

• Survey Monkey

• Microsoft Excel

• Marple


13

APPLYING Implementing, illustrate,

demonstrate, discuss, use

• Online Simulations

• Online role-plays

• Info graphics

• Online Interview

• Digital Photographing

• Digital Drawing

• demonstrations • Keynote

• Microsoft Publisher

• Blackboard

• Skype

• Google Hangouts

• Captivate

• Adobe InDesign

• Adobe Photoshop

UNDERSTANDING Describe, report, identify,

translate, summarise

• Mind Map

• Blog Journal

• Discussion board

comments

• Advanced Web

searching

• Tweeting

• Tagging

• Review

• Test

• Writing

• Verbal Presentation

• Google Blogger

• Free Mind (Mind Mapping)

• Blackboard Discussion

board

• Google Advanced Search

• Twitter

REMEMBERING Repeat, memorise, recall,

recognise, describe

• Word Processing

• Social Networking

• Basic Web searching

• Examples

• Illustrations

• Bullet pointing

• Worksheet

• Microsoft Word

• Facebook

• Google search

Figure 5. Blooms Cognitive Matrix


Figure 6. Blooms Taxonomy Verb Wheel Source: http://www.greenchameleon.com/


15

source:%20http://www.greenchameleon.com/

Things to Consider…

When aligning assessment rubrics to PLOs and CLOs, it is important to remember that the process is truly characteristic of the course within their program. It should be continually reviewed and maintained to ensure it is current to industry and the assessments are representative of that. The following are some additional factors to consider:

• Rubrics should be used as a learning tool not just as an assessment tool, don’t rely on a rubric to provide a comprehensive guide to the assessment

• Rubrics are best developed within a teaching team as it is important for consistency and to develop deeper perspectives

• Consider developing rubrics with your students to enable them to take ownership of how the assessment will be critiqued

• Clarify the rubric at the beginning of the assessment with your students. Don’t presume all of your students will understand the rubrics as well as you do

• Use your rubrics to help facilitate feedback to your students. Rubrics can act as a qualitative and consistent feedback tool

• You can’t capture everything! The criterion is about measuring explicit outcomes; therefore it would be inconceivable to measure every aspect of the assessment performance.

NOTES:


16

Program Learning Objectives (PLO)

Program: Course: Assessment Alignment Template

Course Learning Objectives (CLO)

Assessment 1Criteria



Program: Course: Assessment Alignment Template

Program Learning Objectives (PLO)

Course Learning Objectives (CLO)

Assessment Criteria HD DI CR PA NN

References

Dabbagh, N. (2000) Online-protocols. Retrieved January 28, 2015, from http://mason.gmu.edu/~ndabbagh/wblg/online-protocol.html

Sadler, D. R. (2005) ‘Interpretations of criteria-based assessment and grading in higher education’, Assessment and Evaluation in Higher Education, 30(2), 175-194.

Stevens, DD & Levi, A 2005, Introduction to Rubrics: an assessment tool to save grading time, covey effective feedback, and promote student learning, Stylus Publishing, Canada. Overbaugh, R. & Schultz, L. “Bloom’s Taxonomy.” Date Accessed: 14 September 2015. http://ww2.odu.edu/educ/roverbau/Bloom/blooms_taxonomy.htm


17

http://ww2.odu.edu/educ/roverbau/Bloom/blooms_taxonomy.htm

participants guide - learning and teaching...

Documents