overcoming the teacher’s dilemma (breadth vs. depth)
DESCRIPTION
Overcoming the Teacher’s Dilemma (Breadth vs. Depth). The American River College Model of College-Wide Course-Level SLO Assessment ***********. Mr. John Aubert SLO Assessment Coordinator, Geography Dr. Yuj Shimizu Faculty Researcher, Psychology. Our Goals Today. Provide background - PowerPoint PPT PresentationTRANSCRIPT
Overcoming the Teacher’s Overcoming the Teacher’s DilemmaDilemma
(Breadth vs. Depth)(Breadth vs. Depth)The American River College Model The American River College Model of College-Wide Course-Level SLO of College-Wide Course-Level SLO
AssessmentAssessment
**********************Mr. John AubertSLO Assessment Coordinator, Geography
Dr. Yuj ShimizuFaculty Researcher, Psychology
Provide backgroundProvide background Our collegeOur college Our transition to new standardsOur transition to new standards
Share our course-level assessment Share our course-level assessment programprogram Development and implementationDevelopment and implementation Integration with institutional planningIntegration with institutional planning Breadth vs Depth approachBreadth vs Depth approach
Discuss preliminary resultsDiscuss preliminary results
Our Goals TodayOur Goals Today
American River American River College College Background Background
and Contextand Context Located in Sacramento, CA Located in Sacramento, CA (one of four (one of four colleges in the Los Rios CCD)colleges in the Los Rios CCD)
36,646 unduplicated students36,646 unduplicated students 65 academic departments65 academic departments 2,197 individual courses2,197 individual courses 3,473 sections taught (Fall 2009)3,473 sections taught (Fall 2009) 13,318 course level SLOs defined13,318 course level SLOs defined Accreditation site team arrives inAccreditation site team arrives in 5 5
days…days…
The SLO/Assessment The SLO/Assessment Index…Index…
3131The number of times
that the words “assess” and “assessment” appear in the accreditation standards*…
The number of times that the word
“outcome” appears in the
accreditation standards*…
2424
*2006, Accrediting Commission for Community and Junior Colleges
Accrediting
Commission
for
Community
and Junior
Colleges
Accreditation Workshop Today!
AUBERT
American River American River College College Transition to Transition to
New StandardsNew Standards Late 1990s: Blooms taxonomy Late 1990s: Blooms taxonomy utilized in writing course objectivesutilized in writing course objectives
2004: SLO creation/revision 2004: SLO creation/revision embedded in curriculum processembedded in curriculum process
The Outcomes vs Objectives The Outcomes vs Objectives distinction treated as a continuum…distinction treated as a continuum…
OUTCOMES
OBJECTIVESbroad,
overarchingspecific, discrete
American River American River College College Transition to Transition to
New StandardsNew Standardscontinued…continued…
2004-2006: SLO creation/revision continues2004-2006: SLO creation/revision continues 2006-2007: Focus shifts toward assessment2006-2007: Focus shifts toward assessment Spring 2007: Reassigned time allocated to Spring 2007: Reassigned time allocated to
develop assessment programdevelop assessment program Fall 2007: SLO Assessment Coordinator Fall 2007: SLO Assessment Coordinator
appointed (20-40%)appointed (20-40%)
Our Task…Our Task… To develop an SLO Assessment To develop an SLO Assessment
program which would:program which would: Improve teaching and learningImprove teaching and learning Address the needs of a large collegeAddress the needs of a large college Be consistent with our campus Be consistent with our campus
cultureculture Meet accreditation standardsMeet accreditation standards
To survive the process…To survive the process…
Our Guiding Our Guiding Principles…Principles…■Respect faculty workloadRespect faculty workload
■Rely on faculty expertiseRely on faculty expertise■Be flexibleBe flexible■Integrate with existing Integrate with existing processesprocesses■Practice shared governancePractice shared governance■Meet or exceed accreditation Meet or exceed accreditation standards standards
What do they want What do they want from us?from us?■Pervasive, widespread dialoguePervasive, widespread dialogue
■Assessment which is:Assessment which is:■ institutionalized and integratedinstitutionalized and integrated■ systematic and continuous (not systematic and continuous (not
episodic)episodic)
■Data analysis which leads to Data analysis which leads to changechange■Change which improves student Change which improves student learninglearning■Documentation!Documentation!
Our solution…Our solution…■Comprehensive assessment Comprehensive assessment program adopted September program adopted September 20072007
■New standing committee New standing committee formedformed
■Program review streamlinedProgram review streamlined■65 departments divided into 3 65 departments divided into 3
cohortscohorts■Cohort 1 entered 1Cohort 1 entered 1stst year year
(three year cycle)(three year cycle)
■Substantial outreach and Substantial outreach and training !training !
The American River CollegeSix Year
Institutional Planning Cycle
SLO Assessment CohortsSLO Assessment Cohorts
American River CollegeTwo-Part SLO Assessment for
Courses<<< Three-Year Cycle for Departments >>>
Part I
Student Self Assessment
(every course assessed)
Part II
Faculty Designed
Assessment(one course assessed)
YEAR 1
Action Plan
developand
implement
YEAR 2
Action Plan
continue implementatio
n
YEAR 3
Assessment Cycle Assessment Cycle Version 1.0Version 1.0
DENIAL
DEPRESSION
ACCEPTANCE
ANGER
BARGAINING
The ProblemThe Problem: Are students learning : Are students learning what we expect them to?what we expect them to?
The ChallengeThe Challenge: How we can measure : How we can measure this in a this in a sensiblesensible way? way?
SLOs and SLOs and AssessmentAssessment
Our solution: Our solution: Proposed Proposed Model of AssessmentModel of Assessment
To develop a process that would:To develop a process that would: minimize workload minimize workload gather data broadlygather data broadly gather data systematicallygather data systematically be specific to each course’s SLOsbe specific to each course’s SLOs be transparentbe transparent improve teaching and learningimprove teaching and learning
Course Outline of Record
Course SLO’s
Student Self Assessment-Survey Pilot Template
Course SLO’s
Student Self Assessment-Survey Pilot
Positive Ratings!!!
Student Self Assessment Student Self Assessment SurveySurvey Survey piloted Spring 2007.Survey piloted Spring 2007.
Fast and SimpleFast and Simple Results matched teacher predictionsResults matched teacher predictions
2007-2008 (12007-2008 (1stst Cohort): Full term, face Cohort): Full term, face to face courses (43% Return Rate)to face courses (43% Return Rate)
2008-2009(22008-2009(2ndnd Cohort): All courses Cohort): All courses (63% Return Rate)(63% Return Rate)
2009-2010 (32009-2010 (3rdrd Cohort): Now in Cohort): Now in progress (Return Rate ???)progress (Return Rate ???)
Student Self Assessment Student Self Assessment Survey Survey
Adopted Class Climate Software by Adopted Class Climate Software by Scantron to administer surveys, Scantron to administer surveys, store survey data, and to report store survey data, and to report results (centralized approach)results (centralized approach)
Our approach can be easily modified Our approach can be easily modified for a decentralized / departmental for a decentralized / departmental approach approach
Sample Student Self Assessment Survey Sample Student Self Assessment Survey
Survey ResultsSurvey Results
Self assessments/Student ratings Self assessments/Student ratings can be both reliable (can be both reliable (Ross, Rolheiser, Ross, Rolheiser, & Hogaboam-Gray, 2002)& Hogaboam-Gray, 2002) and valid and valid (Cohen, 1981; Fox & Dinur, 2006; Ross, (Cohen, 1981; Fox & Dinur, 2006; Ross,
2006)2006)
Criticisms of Student Self Criticisms of Student Self Report DataReport Data
Known biases in self, course and instructor Known biases in self, course and instructor ratings (ratings (Dunning, Heath, & Suls, in press;Dunning, Heath, & Suls, in press; Frye, Frye, 2005)2005) Self presentation concerns, overconfidenceSelf presentation concerns, overconfidence Class sizeClass size Class format (Lecture vs. Seminar)Class format (Lecture vs. Seminar) Class purpose (elective, major, GE requirement)Class purpose (elective, major, GE requirement) Class level (intro vs. 2Class level (intro vs. 2ndnd year course) year course) Subject matter (Behavioral Science, Humanities, Subject matter (Behavioral Science, Humanities,
Sciences)Sciences) Difficulty of instructor grading Difficulty of instructor grading Liking for instructor Liking for instructor “ “Hotness” and “Ease” (Coladarci & Kornfield, 2007)Hotness” and “Ease” (Coladarci & Kornfield, 2007)
Interpreting Results CorrectlyInterpreting Results Correctly
Look at Relative Ratings, Not Absolute Ratings
SLO 1 →
SLO 2 →
SLO 3 →
SLO 4 →
SLO 5 →
SLO 6 →
SLO 7 →
Typical Instructor Reaction to Typical Instructor Reaction to Student Self Assessment Student Self Assessment
ResultsResults Instructors typically agree and can Instructors typically agree and can
make quick sense of the data:make quick sense of the data: Aligns with time spent on each SLO, Aligns with time spent on each SLO, Realize that they do not teach that SLO Realize that they do not teach that SLO
anymore, anymore, The SLO is at the end and they never get to The SLO is at the end and they never get to
it, it, Realize that the SLO is the most difficult Realize that the SLO is the most difficult
part of the course, part of the course, Written in a way that students would not Written in a way that students would not
understandunderstand
Yeah but…Yeah but…
QuickQuick Broad and systematicBroad and systematic Sensitive to outliers (produced actionable Sensitive to outliers (produced actionable
data)data)
Thus, survey providedThus, survey provided
IndirectIndirect Lacks flexibilityLacks flexibility Does not utilize faculty expertiseDoes not utilize faculty expertise Lacks depthLacks depth
Screening Device
Airport Security
Part 2: Faculty Designed Part 2: Faculty Designed Assessment (Direct Assessment (Direct
Assessment)Assessment)Customized assessment documented on a Customized assessment documented on a
common templatecommon template What course and why?What course and why? What SLO(s)?What SLO(s)? When? When? How broadly? How broadly? How will you assess? (describe tool or rubric)How will you assess? (describe tool or rubric) What is the criteria for successful SLO What is the criteria for successful SLO
achievement? achievement? Who will be administering / scoring?Who will be administering / scoring? Submitted to the SLO Assessment CommitteeSubmitted to the SLO Assessment Committee
American River CollegeTwo-Part SLO Assessment for Courses
Three-Year Cycle for Departments
Part I
Student Self Assessment
(every course assessed)
Part II
Faculty Designed
Assessment(one course assessed)
Action Plan
developand
implement
Action Plan
continue implementatio
n
YEAR 1 YEAR 2 YEAR 3
Year 2: SLO Action PlanYear 2: SLO Action PlanDepartments respond to both parts of assessmentDepartments respond to both parts of assessment
Part 1: Respond to the Student Self-AssessmentPart 1: Respond to the Student Self-Assessment Part 2: Respond to the Faculty Designed AssessmentPart 2: Respond to the Faculty Designed Assessment
Year 2: SLO Action PlanYear 2: SLO Action PlanDetailed InstructionsDetailed Instructions
Action Plan and Instructions approved Action Plan and Instructions approved through an 8 month processthrough an 8 month process
Where we are nowWhere we are now
2/3’s have participated in the college 2/3’s have participated in the college wide SLO Assessment processwide SLO Assessment process
1/ 3 have completed Action Plans1/ 3 have completed Action Plans 63 Separate Actions taken/planned as a 63 Separate Actions taken/planned as a
result of SLO assessmentresult of SLO assessment
Of the 65 Academic departments…
ConclusionConclusion
Thank you!Thank you!
John Aubert, SLO Coordinator, John Aubert, SLO Coordinator, GeographyGeography
[email protected] or 916-484-8637 or 916-484-8637
Yuj Shimizu, Faculty Researcher, Yuj Shimizu, Faculty Researcher, PsychologyPsychology
[email protected] or 916-484-8149 or 916-484-8149
http://inside.arc.losrios.edu/~slo/