rubrics and online assessment

@t_eloise Dr. Eloise Tan, Learning Innovation Unit, Dublin City University

/

http://softchalkconnect.com/lesson/files/QARNsW98cnVGeh/Rubrics_Lesson_print.html

@t_eloise Dr. Eloise Tan, Learning Innovation Unit, Dublin City University

LI502: Assessment and Feedback in the Online Environment

@t_eloise Dr. Eloise Tan, Learning Innovation Unit, Dublin City University

Rubric Design and Deep Learning

@t_eloise Dr. Eloise Tan, Learning Innovation Unit, Dublin City University

OverviewRubrics: definition, components (based on

(Meeusen, 2011) http://softchalkconnect.com/lesson/QARNsW98cnVGeh

Rubrics in the online environmentRubrics for online assessments: online

discussions, wikis, blogs Class admin

What is a rubric?

Do you use them for your assessments?

@t_eloise Dr. Eloise Tan, Learning Innovation Unit, Dublin City University

Definitions of rubrics from (Meeusen, 2011) http://softchalkconnect.com/lesson/QARNsW98cnVGeh

Criterion –referenced tools

“used to evaluate student performance against a standard of mastery, not just to compare students”

“a scoring guide for evaluating student work”

@t_eloise Dr. Eloise Tan, Learning Innovation Unit, Dublin City University

What rubrics can do (Palloff & Pratt, p.33)

• Way to provide feedback• Define characteristics of a high quality

assignment• Help student understand expectations• Establish range of performance

categories• Provide a concrete way of evaluating

their own performance (self-assessment & reflection)

• Takes guesswork out of grading for lecturers

• Aims to achieve reliability, validity, and transparency within marking process

• Helps students to manage time• Helps lecturers clarify what is

important to them in the assessment: “What do I want students to learn from this assessment?”

• Opens up discussion on marking criteria, student learning

@t_eloise Dr. Eloise Tan, Learning Innovation Unit, Dublin City University

Objectives of rubrics

Validity(Assesses

what it says it will assess)

Transparency

(Makes clear to students

the criteria)

Reliability(Assesses

consistently across

submissions)

@t_eloise Dr. Eloise Tan, Learning Innovation Unit, Dublin City University

What could be some challenges or unintended consequences of using rubrics? Enter into chat or raise hand to use Talk

button

Chat room comments: Lead to mechanical approach to

assessmentsOpens up room for conflict relating to

assessmentsCould constrain student creativityMarking is subjective, hard to pin down

criteria Takes a lot of time to design

@t_eloise Dr. Eloise Tan, Learning Innovation Unit, Dublin City University

Responses to comments Lead to mechanical approach to assessments

We know students already are strategic about their assessments, rubrics help focus their time/effort.

Opens up room for conflict relating to assessmentsRubrics provide a co-constructed tool that can be used by students

to dispute their grades. This could be a positive because they are relating their performance to pre-determined criteria and not making subjective arguments about a grade.

Could constrain student creativityYes, and rubrics could also be seen as a framework through which

students can strategically approach their assessment, instead of hoping that their strategy works.

Marking is subjective, hard to pin down criteria Yes, that is the point of rubrics. To make marking valid, reliable, and

transparent. Takes a lot of time to design

Definitely! Many tools to help though as we will see…

@t_eloise Dr. Eloise Tan, Learning Innovation Unit, Dublin City University

Holistic v. Analytic RubricsHolistic rubrics

provide a single score based on an overall impression of a student’s performance on a task.

Analytic rubrics provide specific feedback along several dimensions

http://www.schreyerinstitute.psu.edu/pdf/rubricbasics.pdf

@t_eloise Dr. Eloise Tan, Learning Innovation Unit, Dublin City University

Example of Holistic v. Analytic Webtourhttp://softchalkconnect.com/lesson/files/QA

RNsW98cnVGeh/Rubrics_Lesson_print.html

@t_eloise Dr. Eloise Tan, Learning Innovation Unit, Dublin City University

Remember

What you do not include in a rubric is just as important as what you put in.

@t_eloise Dr. Eloise Tan, Learning Innovation Unit, Dublin City University

Parts of a rubric (Meeusen, 2011) Scale: the scale of points to be assigned in

scoring a piece of work on a continuum of quality. Higher points are typically assigned for the best work.

Descriptors: the descriptions for each level of performance that contain criteria and standards by which the performance will be judged

Criteria: Criteria describe the conditions that any performance must meet to be successful. Criteria should describe both strengths and errors.

Standards: standards specify how well criteria must be met.

@t_eloise Dr. Eloise Tan, Learning Innovation Unit, Dublin City University

Rubrics and online assessment Tools to design rubrics: Rubistar http://rubistar.4teachers.org (free) Rubrics in Moodle 2.2 http://

www.youtube.com/watch?v=KXavtUhDINAPage compiling online rubric tools:

http://www.teach-nology.com/web_tools/rubrics/

Open Educational Resources: Foundations for rubrics can be found online. Example : http://school.discoveryeducation.com/schrockguide/assess.html

@t_eloise Dr. Eloise Tan, Learning Innovation Unit, Dublin City University

Self, peer assessment and rubricsCould I involve students in setting the

rubrics? Could students use the rubric to self assess

/ peer assess? (Having rubrics readily available promotes self-assessment and reflection)

http://tellio.blogspot.com/

@t_eloise Dr. Eloise Tan, Learning Innovation Unit, Dublin City University

Designing rubrics for online assessmentsAm I clear on what I am assessing in the

activity? What is my learning objective for the assessment?

Will technical skills / challenges affect students’ ability to carry out the assessment or meet successful criteria?

Careful of: Tendency to quantifyLack of time / preparation/ scaffolding of

technical skills and/or new ways of writing for students

@t_eloise Dr. Eloise Tan, Learning Innovation Unit, Dublin City University

Evaluating your rubric – some tips1. Does it assess what you want it to?

For an online discussion forum – are you assessing for critical thought displayed in the post? Or are you assessing for number of contributions?

2. Do students have the skills necessary / or the opportunity to gain the skills necessary to achieve success through your rubric? Have you provided tutorials for how to post, use a wiki, etc?

3. Have you paid careful attention to what goes into and what stays out of your rubric?

Is it important to allocate marks for style, presentation, referencing for the assessment? Might be different for different levels.

4. Do the marks allocated for criteria reasonably correlate to the amount of time students should spend on that criterion? 5. Have you shared your rubric with students in advance? Or better

yet involved them in the design? 6. Have you included room in your rubric for feedback

@t_eloise Dr. Eloise Tan, Learning Innovation Unit, Dublin City University

Sites to help you evaluate your rubricUseful set of questions from the Office of

Provost at Rensselaer Polytechnic Institutehttp://provost.rpi.edu/node/32

@t_eloise Dr. Eloise Tan, Learning Innovation Unit, Dublin City University

Webquest and DiscussionSearch for rubrics online for 1) online

discussions, 2) wikis , 3) blogs, or 4) any online assessment you would like to discuss

Come back and share/discuss/evaluate rubrics

@t_eloise Dr. Eloise Tan, Learning Innovation Unit, Dublin City University

Online discussion forum rubricsRubric that combines frequency with

qualityhttp://

www.ankeqiang.org/IEP/Online-Eval.pdf

Rubric that details the different types of posts for forums (responses, initial post, content contribution)

http://www.udel.edu/janet/MARC2006/rubric.html

Also available in our core texts: Crisp, Palloff and Pratt

@t_eloise Dr. Eloise Tan, Learning Innovation Unit, Dublin City University

Wikis (can be used for group work)Rubric for self / peer assessment of team

members in a grouphttps://

wiki.umn.edu/pub/TeachingWithWikis/AssessingWikis/team_rubric.pdf

Rubric for the collaboration processhttps://wiki.umn.edu/pub/TeachingWithWiki

s/AssessingWikis/wiki_rubric.pdf

@t_eloise Dr. Eloise Tan, Learning Innovation Unit, Dublin City University

Related to last webinar: Rubrics as self-

assessment and as plagiarism prevention

@t_eloise Dr. Eloise Tan, Learning Innovation Unit, Dublin City University

Current Featured Resource on Moodle

http://www.docstoc.com/docs/3647772/Tips-For-Effective-Rubric-Design-How-to-design-a-rubric

@t_eloise Dr. Eloise Tan, Learning Innovation Unit, Dublin City University

Questions?