christine doherty lydia li karen tsao stanford university (aka tests & quizzes) s akai a...

104
Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O

Upload: milton-wells

Post on 16-Dec-2015

219 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O

Christine DohertyLydia Li

Karen Tsao

Stanford University

(aka Tests & Quizzes)

S akaiA ssessment M anager IGO

Page 2: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O

Agenda

• Feature Overview• Demo of Features• 2.5 Highlights• 2.6 Roadmap• Quick Create Demo• Q&A

Page 3: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O

What are SAMigo’s features:

• Assessment Authoring• Assessment Publishing• Assessment-Taking Management• Assessment Grading• Question Pools Management• Assessment-Type Management

Page 4: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O

Pedagogical Flexibility

• Online test• Self-study questions• Homework, problem-sets• Paper, code, or project submission• Language drills• Quick knowledge probes• Surveys• Others???

Page 5: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O

Organize & Publish Assessments• Core (pending) Assessments for a course

– Starting page for assessment authoring– Can be exported/imported in QTI XML format– When ready, published via Settings

• Published Assessments for student assessment-taking– When students respond, Scores link appears– Can change release, due, retract dates, plus

some feedback and grading settings– Can remove published assessments * New in 2.5

Page 6: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 7: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O

Question Types

Auto-scored• Multiple Choice (single and multiple correct)• Survey (multiple choice, different rating scales

available)• Fill in the Blank• Numeric Response• Matching• True/False

Manually scored• Short Answer/Essay• Audio Recording• File Upload

Page 8: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 9: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 10: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O

Rich-Text & Attachments

• At all levels of assessment authoring:– Assessment/Part Headers – Questions– Selections– Feedback

Page 11: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O

Create an Assessment

Page 12: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 13: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 14: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 15: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 16: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 17: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 18: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 19: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 20: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 21: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 22: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 23: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 24: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 25: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 26: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O

* New in 2.5

Page 27: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 28: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 29: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O

* New in 2.5

Page 30: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 31: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 32: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O

* New in 2.5

Page 33: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 34: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 35: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 36: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 37: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 38: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 39: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O

Import/Export

• Supports QTI v 1.2• Export types: IMS QTI-compliant XML or IMS

Content Packaging (for use with attachments)• Useful for

– Making copies of core assessment in same site– Copying assessment to different site

Page 40: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 41: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 42: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 43: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 44: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 45: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 46: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O

Settings• Configures delivery of assessments (how they

appear to students)

• Options include: – display/organization

– late handling

– number of submissions

– feedback options

– high security and time limit for high stakes testing

• Most settings are not editable after publishing, so it is very important for instructors to review these carefully before publishing

Page 47: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 48: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 49: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 50: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O

* New in 2.5

Page 51: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O

Save Settings and Publish - to release to students

Save Settings - to make changes to core assessment settings,

can publish assessement at later time

Page 52: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 53: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O

Publishing

• Release assessment to students• Copy of core assessment -- core assessment

can be modified without affecting published version (and released with new title)

• Can set Available Date in future -- useful for instructors who want to do bulk of setup early

• Can retract any time, making assessment unavailable to all students

Page 54: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 55: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 56: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O

Published Inactive Assessments

• Retracted assessments will not appear to any students

• Assessments that are past the due date will not appear to students, with following exception:– When late submission is allowed, students who

have not submitted will get “one last chance” to submit

Page 57: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 58: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O

Taking an Assessment (Students)

• Take an Assessment section– List determined by available, due, and retract dates– Submit for Grading and Save for Later options– Table of Contents and Mark for Review features

available in non-linear assessment organization

• Submitted Assessments section– Provides submission confirmation with timestamp– Depending on settings, students can review scores,

feedback, grader’s comments, statistics

Page 59: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 60: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 61: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 62: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 63: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 64: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 65: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 66: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 67: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O

Grading Workflows

• Ability to grade all questions for one student or grade all responses to one question

• Manual adjustments can be made to individual questions or entire assessment

• Comments can be made for individual questions or entire assessment

• Display of scores can be filtered by section

Page 68: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O

Other Grading Features

• Email students from Scores page• Allow Retake for individual student• Download All function for file upload

questions -- generates zip file of all student submissions

• Statistics and histograms• Export students' responses to Excel file (New

in 2.5)

Page 69: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O

Grading Settings

• Anonymous grading OR show student identities

• Send scores to Gradebook -- this can be toggled on and off after assessment is published

• Record highest score OR last score

Page 70: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 71: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 72: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 73: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 74: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 75: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 76: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 77: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O

* New in 2.5

Page 78: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 79: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 80: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 81: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 82: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 83: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 84: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 85: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O

Question Pools

• Organize questions into multiple subpools • Move/copy/remove multiple questions to

other pool * New in 2.5• Author questions from within Pool Manager• Import questions from IMS QTI-compliant

XML file• Randomly draw from pools for an assessment• IMPORTANT NOTE: Question pools belong

to an individual user only, available across all user’s sites

Page 86: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 87: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 88: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O

Assessment Types

• Create assessment “types” for different disciplines, different uses

• Hide complexity by selecting which settings are visible to the instructor.

• Pre-set defaults by making setting choices for the type (e.g., Self-study always has immediate feedback)

Page 89: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O

Assessment Creation Workflow

Page 90: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 91: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 92: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 93: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 94: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 95: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O

2.5 Highlights• Adjust score in edit assessment screen *• Remove published assessments• Allow feedback to be viewed right after an assessment is

submitted• Export students' responses to Excel file **• Reset point values of questions when a random draw

from pool part is created *• Move/copy/remove multiple questions to other pool *• Copy all questions in a part to a selected pool *• Event logging for user operations

* Contributed by Rutgers University** Partly based on contributions from Rutgers University

Page 96: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O

2.6 Roadmap• Edit a published assessment

– Includes editing content of existing questions and most settings

– Does not include editing parts, reordering parts or questions, adding or removing questions

– Ability to regrade auto-scored questions for existing submissions

• Copy a (core) pending assessment• Share question pools *• Release assessment to one or more groups/sections **• Statistics for item analysis **

* Based on contributions from Universidad Politecnica de Valencia** Contributed by University of Cape Town

Page 97: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O

Quick Create• Contributed by Claremont McKenna College and

UC Davis• Disabled by default, but can be turned on in sakai

properties• Efficiency created by reducing number of clicks,

pages loading and saving• Useful for transferring assessments created in other

applications or LMS (e.g., MS Word, WebCT), no need to paste into different fields

• Rich text and attachments can be added later, through edit assessment screen

Page 98: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 99: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 100: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 101: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 102: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O
Page 103: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O

Questions?

Page 104: Christine Doherty Lydia Li Karen Tsao Stanford University (aka Tests & Quizzes) S akai A ssessment M anager I G O

Contacts

Questions and feeback: [email protected]

Lydia Li, Stanford University Project Manager [email protected]

Homepage:http://confluence.sakaiproject.org/confluence/display/SAM