quality matters : inter-institutional quality assurance in online learning
Post on 02-Jan-2016
42 Views
Preview:
DESCRIPTION
TRANSCRIPT
Quality MattersQuality Matters: : Inter-Institutional Quality Assurance Inter-Institutional Quality Assurance
in Online Learningin Online Learning
Teaching with New Technologies
May 2007
Session Goals
• Provide an overview of the critical materials, processes, and administrative elements of the Quality Matters online course quality assurance program.
• Describe the Peer Course Review process.
• Explore lessons learned in creating a quality assurance program and applications of the QM process.
About About Quality Matters Quality Matters
Peer Course Peer Course ReviewReview
FeedbackFeedback
CourseCourse
Course Meets Course Meets Quality ExpectationsQuality Expectations
Course Course RevisionRevision
Instructional Designers
InstitutionsFaculty Course Developers National Standards &
Research Literature
Rubric
Faculty ReviewersTraining
Quality Matters:Quality Matters: Peer Course Peer Course
Review ProcessReview Process
Factors that Affect Factors that Affect Course QualityCourse Quality
Factors Affecting Course Quality
• Course design QM REVIEWS THIS
• Course delivery (i.e. teaching, faculty performance)
• Course content• Course management system• Institutional infrastructure• Faculty training and readiness• Student engagement and readiness
Underlying Principles of QM• The QM toolset and process are:
– based in national standards of best practice, the research literature and instructional design principles
– designed to promote student learning
– part of a faculty-driven, peer review process
• Process designed to ensure all reviewed courses will eventually meet expectations
• Collegial review process, not an evaluation process
For Our Purposes, Quality Is…
• More than average; more than “good enough”
• An attempt to capture what’s expected in an effective online course at about an 85% level
• Based on research and widely accepted standards
85 %
QM Process Provides
• Institutional toolset and process to meet quality expectations:– Online course design– Student learning– Improved instruction– Assessment and feedback loops– Professional development
What’s In It For Institutions …• External validation
• Strengthen accreditation package
• Raise QA as a priority activity
• Access to a sustainable, replicable, scalable QA process
• Inform online course training & practices
• Provide professional development activities
What’s In It For Faculty …• Improve online courses
• External quality assurance
• Review other courses and gain new ideas for own course; expand professional community
• Participation useful for professional development plan and portfolio
What this process is NOT• Not about an individual instructor
(it’s about the course)
• Not about faculty evaluation
(it’s about course quality)
• Not about “win/lose” or “pass/fail”
(it’s about continuous improvement in a supportive environment)
Design vs. Delivery
The faculty member is integral to both design and delivery.
Course Design … is the forethought and planning that a faculty member puts into the course.
Course Delivery …is the actual teaching of the course, the implementation of the design.
QM is about DESIGN - not delivery or faculty performance
Quality MattersQuality MattersToolset Toolset
The InstructorThe Instructor Worksheet Worksheet
Instructor Worksheet
• Includes info such as:– Institutionally mandated objectives,
materials, practices, policies– Materials outside course pages– Types of interaction used & instructor’s
statement on the appropriateness of interaction in the course
– Additional items that may require review
The Rubric The Rubric
The Rubric
• Eight standards: – Course Overview and Introduction– Learning Objectives – Assessment and Measurement– Resources and Materials– Learner Interaction– Course Technology– Learner Support– Accessibility
Key components must align.
What is Alignment?
Critical course elements
work together to ensure
that students achieve
the desired learning outcomes.
The RubricThe Rubric Essential Standards Essential Standards
(14) (14)
General Standard I:Course Overview and Introduction
1.1: Navigational instructions make the
organization of the course easy to
understand.
1.2: A statement introduces the student to the
course and to the structure of the student
learning, and, in the case of a hybrid
course, clarifies the relationship between
the face-to-face and online components.
General Standard II:Learning Objectives (Competencies)
II.I: The course learning objectives
describe outcomes that are
measurable.
II.2: The module/unit learning objectives
describe outcomes that are measurable and consistent with the
course-level objectives.
General Standard III: Assessment and Measurement
III.1: The types of assessments selected
measure the stated learning objectives
and are consistent with course activities
and resources.
III.2: The course grading policy is stated clearly.
III.3: Specific and descriptive criteria are
provided for the evaluation of students’
work and participation.
General Standard IV: Resources and Materials
IV.I: The instructional materials support
the stated learning objectives.
IV.2: The instructional materials have
sufficient breadth, depth, and
currency for the student to learn the
subject.
General Standard V:Learner Interaction
V.1: The learning activities promote theachievement of stated learning objectives.
V.2: Learning activities foster instructor-student, content-student, and if
appropriate to this course, student-studentinteraction.
V.3: Clear standards are set for instructorresponse and availability (turn-aroundtime for email, grade posting, etc.)
General Standard VI:Course Technology
VI.I: The tools and media support the
learning objectives and are
appropriately chosen to deliver the
content of the course.
General Standard VII: Learner Support
• No 3-point elements
This standard has no “essential” 3-point elements because it’s primarily concerned with academic support services, student support services and technical support services….usually thought to be the primary responsibility of the institution and not the individual instructor.
General Standard VIII:Accessibility
VIII.1: The course acknowledges the importance ofADA requirements.
To meet this standard, the course must include BOTHof these elements:
• The course must be offered using software that is accepted as “ADA compliant.”.
AND• The course should include a brief statement that
clearly tells students how to access ADA services at the institution.
The RubricThe Rubric Scoring Scoring
Rubric ScoringStandards Points Relative Value
14 3 Essential
12 2 Very Important
14 1 Important
TOTALS
40 80
• Team of three (3) reviewers
• One score per standard based on team majority
• Assigned point value; not sliding scale
To Meet Expectations…
A course must achieve:
• “Yes” on all 14 of the 3-point “essential” standards.
• A minimum of 68 out of 80 points
68/80 = 85%
Online & Hybrid Courses
• Rubric designed for application to fully online and hybrid/blended courses
• Same set of standards apply to both
• How we achieve the standards may differ
• For hybrids, focus on pedagogical integration of online and F2F components
RecommendationsRecommendations
Peer Course Review Peer Course Review ProcessProcess
The Peer Review Team
• 3 faculty peer reviewers:– must be experienced online instructors– must attend QM training– one MUST be external to the course’s originating
institution– there must be a subject matter expert (SME)
on the team. NOTE: The SME could also be the external reviewer.
AND • Faculty course developer:
– access to rubric prior to review– involved in pre-review discussions– consulted during review
About the Course
• QM is designed to review “mature” courses (taught at least two semesters)
• QM logo indicates year course met expectations
• Triggers for subsequent reviews:• Faculty request• More than 3 years since original review• New textbook or instructor• Professional or accreditation review pending
About the Review
• On average, a course review takes 7-10 hours; active review period approximately 3 weeks
• Factors affecting review time include
• Reviewer familiarity with the discipline• Reviewer familiarity with the CMS• Reviewer familiarity with the QM review
process• Organization of the course
Review Outcome• If meets expectations:
– Recognized by Quality Matters– Notifications distributed– ID support provided if requested
• If does not yet meet expectations:– ID support provided if requested– Team Chair and ID approve revisions– Course meets expectations
Roles and Responsibilities
Faculty DeveloperPeer Reviewers
Team Chair
QM To DateQM To Date
QM in Transition
• 2003 – August 2006– QM project funded by FIPSE grant money– materials and some services freely available
• August 2006 and beyond– QM project funded by MarylandOnline– Some limited materials will be freely available– Other materials available to individuals and
institutions at nominal fees– Institutional membership affords full access to
materials and services
QM to Date
• Overall Participation– Individuals & programs from 150
institutions across 30 states
• Course Reviews
• Peer Reviewer Rubric Training– 900+ trained
Quality MattersQuality MattersApplicationsApplications
Multiple Uses of QM
Reported Uses of QM System:• Guidelines for initial online course development
• Quality assurance of existing courses
• Ongoing faculty professional development
• Institutional reaccredidation packages
• Formation of distance learning policies & steering committees
Thanks to YOU…Thanks to YOU…Quality Matters! Quality Matters!
top related