tests and quizzes ( samigo) 2009

Tests & Quizzes (aka SAMigo)

Christine Doherty, Stanford University

• Feature Overview• Demo of Features• 2.6 Highlights• 2.7 Roadmap• Q&A

Agenda

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 2

• Assessment Authoring• Assessment Publishing• Assessment-Taking Management• Assessment Grading• Question Pools Management• Assessment-Type Management

What are SAMigo’s Features:

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 3

• Online test• Self-study questions• Homework, problem-sets• Paper, code, or project submission• Language drills• Quick knowledge probes• Surveys• Others???

Pedagogical Flexibility

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 4

• Pending* Assessments• Starting page for assessment authoring• Quick Create feature available OOTB *• Can be exported/imported in QTI XML format• Can create copies easily *• When ready, published via Settings

• Published Assessments for student assessment-taking• When students respond, Scores link appears• Can edit published assessments and some settings *• Can remove published assessments • Last Modified column with name/date/time *

Organize & Publish Assessments

* New in 2.6

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 5

* New in 2.6

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 8

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 9

Auto-scored• Multiple Choice (single and multiple correct)• Survey (multiple choice, different rating scales available)• Fill in the Blank• Numeric Response• Matching• True/False

Manually scored• Short Answer/Essay• Audio Recording• File Upload

Question Types

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 10

xx

What’s New in Question Types

• Multiple Choice: Multiple Correct, Single Selection• Full credit if student picks any of the possible correct answers

• Multiple Choice/Single Correct, True/False: Negative Point Value for Incorrect Answer Selection• Subtracts designated point value from total score

• Discourages random guessing

• Reset Selection option in delivery for students to clear radio button selection

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 13

What’s New in Question Types

• Audio Recording: Improved Recording Applet• Ability to display multiple audio recording questions on a page

without performance problems; link pops up applet in separate window

• Save/Close button in applet for improved usability

• Improved recording length reliably saved (300 sec)

• Improved processing time to save recording

• Volume meter bar to provide live feedback

• Aesthetic enhancements (applet size, text size)

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 14

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 15

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 16

• At all levels of assessment authoring:• Assessment/Part Headers • Questions• Selections• Feedback

Rich Text & Attachments

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 17

Create an Assessment

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 18

* New in 2.6

Quick Create

• Contributed by Claremont McKenna College and UC Davis• Available in 2.6 OOTB• Efficiency created by reducing number of clicks, pages

loading and saving• Useful for transferring assessments created in other

applications or LMS (e.g., MS Word, WebCT), no need to paste into different fields

• Supports Multiple Choice, Fill in the Blank, Short Answer/Essay, and True/False question types

• Rich text, attachments, and additional questions can be added later, through assessment authoring page

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 47

Import/Export

• Supports QTI v 1.2

• Export types: IMS QTI-compliant XML or IMS Content Packaging (for use with attachments)

• Useful for:• Copying assessment to different site• Archiving assessments offline

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 53

Settings• Configures delivery of assessments (how they appear

to students)

• Options include: • display/organization• late handling• number of submissions • feedback/grading options • high security and time limit for high stakes testing

• Some settings are not editable after publishing, so it is very important for instructors to review these carefully before publishing

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 60

What’s New in Settings

• Option to release assessment to one or more groups/sections • groups/sections need to be defined in Section Info or Site

Info > Manage Groups

• Mark for Review is configurable per assessment; deselected by default

• Option to auto-submit saved assessments after due date • needs to be enabled in sakai.properties

(samigo.autoSubmit.enabled=true)• quartz job needs to be created in Administration Workspace

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 64

* New in 2.6

* New in 2.6

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 67

* New in 2.6

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 68

Save Settings and Publish - to release to students

Save Settings - to make changes to pending assessment settings,

can publish assessment at later time

Publishing• Release assessment to students

• Copy of pending assessment – pending assessment can be modified without affecting published version

• Can set Available Date in future -- useful for instructors who want to do bulk of setup early

• Can retract any time, making assessment unavailable to all students

• Can remove published assessment

• Can edit published assessment *New in 2.6

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 71

Edit a Published Assessment (OOTB)

• OOTB 2.6 allows editing a published assessment ONLY IF no students have started taking it

• Includes editing content of existing questions and point values

• Does not include editing parts, reordering parts or questions, adding or removing questions

• More editable settings than 2.5: Delivery Dates, Timed Assessment, Assessment Organization, Submissions, Feedback, Grading

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 72

Edit a Published Assessment (full)• Option to edit a published assessment AFTER

students have started taking it• needs to be enabled in sakai.properties

(samigo.editPubAssessment.restricted=false)

• Ability to regrade auto-scored questions for existing submissions

• Ability to allow students to update their latest submission

• Warning messages for students to notify them of modified assessments

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 73

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 75

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 77

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 78

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 79

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 80

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 81

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 82

Published Inactive Assessments

• Retracted assessments will not appear to any students

• Assessments that are past the due date will not appear to students, with following exception:

• When late submission is allowed, students who have not submitted will get “one last chance” to submit

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 83

Taking an Assessment (Students)

• Take an Assessment section• List determined by available, due, and retract dates,

plus number of allowed submissions• Submit for Grading and Save for Later options• Table of Contents and Mark for Review features

available in non-linear assessment organization

• Submitted Assessments section• Provides submission confirmation with timestamp• Depending on settings, students can review scores,

feedback, grader’s comments, statistics

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 85

What’s New in Assessment Delivery

• Save for Later confirmation screen

• Submit for Grading confirmation screen

• Show anonymous users their test results immediately after they submit assessment

• Disable Save for Later button for timed assessment

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 86

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 92

* New in 2.6

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 95

* New in 2.6

Grading Workflows

• Ability to grade all questions for one student or grade all responses to one question

• Manual score adjustments can be made to individual questions or entire assessment

• Comments can be made for individual questions or entire assessment

• Display of scores can be filtered by section

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 99

Other Grading Features

• Email students from Scores page

• Allow Retake for individual student

• Download All function for file upload questions -- generates zip file of all student submissions

• Statistics and histograms

• Export students' responses to Excel file

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 100

What’s New in Grading Features

• Download All function for file upload questions – download by section/group

• Improved statistics – Item Analysis

• Export part scores as additional columns in Excel spreadsheet

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 101

Grading Settings

• Anonymous grading OR show student identities

• Send scores to Gradebook OR not

• Record highest score OR last score

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 102

xxxxxxxxxxxxxx

xxxxxxxxxxxxxx

xxxxxxxxxxxxxx

xxxxxxxxxxxxxx

* New in 2.6

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 116

* New in 2.6

Question Pools

• Organize questions into multiple subpools

• Move/copy/remove multiple questions at a time

• Author questions from within Pool Manager

• Import questions from IMS QTI-compliant XML file

• Randomly draw from pools for an assessment

• Question pools belong to an individual user, available across all user’s sites

• Ability to share question pools with other site members *New in 2.6

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 122

* New in 2.6

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 126

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 127

Assessment Types

• Create assessment “types” for different disciplines, different uses

• Hide complexity by selecting which settings are visible to the instructor.

• Pre-set defaults by making setting choices for the type (e.g., Self-study always has immediate feedback)

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 128

Assessment Creation Workflow

July 2009 10th Sakai Conference - Boston, MA, U.S.A.

2.6 Highlights• Edit a published assessment

• Includes editing content of existing questions and most settings• Does not include editing parts, reordering parts or questions, adding or

removing questions• Ability to regrade auto-scored questions for existing submissions

• Copy a pending assessment• Question Type enhancements:

• Multiple Choice/Multiple Correct/Single Selection• Negative Point Value for Incorrect Answer Selection*• Improved Recording Applet

• Submit for Grading*** and Save for Later confirmation screens• Share question pools *• Release assessment to one or more groups/sections **• Statistics for item analysis / Export part scores **

*Contributed by Universidad Politecnica de Valencia** Contributed by University of Cape Town *** Contributed by Rutgers University

2.7 Roadmap

• Auto-Save and Save (on demand)*• Auto-save interval configurable by system admin• Save button allows students to save work without leaving page• Significantly reduces loss of data

• Reconfiguration/Renaming of navigation buttons in delivery• Submit for Grading button position changed• Save for Later changed to Exit

• Email Notification when publishing/republishing assessments

• Respondus integration

• Print assessment**

* Based on contributions from Rutgers University** Based on contributions from Universidad Politecnica de Valencia

Questions?

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 137

Contacts

Questions and feedback: samigo-dev@lists.stanford.edu

Lydia Li, Stanford University Project Manager lydial@stanford.edu

Homepage:http://confluence.sakaiproject.org/confluence/display/SAM

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 138