Assessment CycleAssessment Cycle

California Lutheran UniversityCalifornia Lutheran UniversityDeans’ CouncilDeans’ Council

February 6, 2006February 6, 2006

Assessment Cycle

Strategic Plan 2014

Assessment Symposium

Program Review

CaliforniaCalifornia Lutheran UniversityLutheran University Dedicated to educating leaders for a

global society who are strong in character and judgment, confident in their identity and vocation, and committed to service and justice.

Core ValuesCore ValuesExcellence Service Diversity Respect

Responsibility Courage Pursuit of Truth

Assessment Symposium

Assessment Cycle

Strategic Plan 2014

Assessment Symposium

Program Review

PurposePurpose

Report Report assessment evidenceassessment evidence and ways and ways

in which the evidence in which the evidence was and will be was and will be

usedused to to improve student learningimprove student learning and and

the the quality of our service.quality of our service.

Assessment Symposium

Guiding PrinciplesGuiding Principles

SimpleSimple MeaningfulMeaningful

Assessment Symposium

BenefitsBenefits

Sharing of research and learnings Sharing of research and learnings

(e.g. Program Review)(e.g. Program Review) Engages various members of the Engages various members of the

University CommunityUniversity Community

Listen, learn, and appreciate othersListen, learn, and appreciate others

Assessment Symposium

BenefitsBenefits

Supportive environment via Supportive environment via roundtable format roundtable format

Encourages assessment to become Encourages assessment to become part of our practicepart of our practice

Research in Action!Research in Action!

Assessment Symposium

FeaturesFeatures

5 step model5 step model 5 – 10 minute presentation 5 – 10 minute presentation One goalOne goal Standardized PowerPoint formatStandardized PowerPoint format Review and practice sessionsReview and practice sessions

Assessment Symposium

5 Step Model5 Step Model

1.1. What What goalgoal was set? was set?

2.2. What What research designresearch design / measure was / measure was used?used?

3.3. What What datadata were collected? were collected?

4.4. What emerged from data (What emerged from data (analysisanalysis)?)?

5.5. What What recommendations recommendations are made?are made? Measurable goals set?Measurable goals set?

Assessment Symposium

ExamplesExamples

AcademicsAcademics Program Review including External Program Review including External

ReviewReview External Review onlyExternal Review only

OperationsOperations

Assessment Symposium

External Review Assessment GoalExternal Review Assessment Goal Review of Foundations Courses and Review of Foundations Courses and

Student Teaching Seminar conducted by Student Teaching Seminar conducted by an external university team an external university team

Purpose: to investigatePurpose: to investigate• Repetition of conceptsRepetition of concepts• If “diversity” met NCATE standard 4If “diversity” met NCATE standard 4• If field experiences met NCATE standard 4If field experiences met NCATE standard 4• If Foundation learning outcomes aligned with If Foundation learning outcomes aligned with

Student TeachingStudent Teaching• If exit survey provided data for improvementIf exit survey provided data for improvement

Research Design and Data Research Design and Data CollectionCollection

Content analysis of syllabi including Content analysis of syllabi including candidate learning outcomes and anchor candidate learning outcomes and anchor activitiesactivities

Analysis of exit surveyAnalysis of exit survey Interviews with Dean and Foundations Interviews with Dean and Foundations

CoordinatorCoordinator

Analysis of DataAnalysis of Data

FindingsFindings Syllabi content analysis shows appropriate Syllabi content analysis shows appropriate

contentcontent ““Diversity” depth is sufficientDiversity” depth is sufficient Field experiences appropriate for Foundations Field experiences appropriate for Foundations

coursescourses Appropriate alignment of Foundations and Appropriate alignment of Foundations and

Student Teaching Seminar learning outcomesStudent Teaching Seminar learning outcomes Exit survey should be strengthenedExit survey should be strengthened

Future GoalsFuture Goals

Goals for upcoming year:Goals for upcoming year: Continue to align online and onsite syllabi and Continue to align online and onsite syllabi and

anchor activitiesanchor activities Distribute specific TPE responsibility between Distribute specific TPE responsibility between

Foundation coursesFoundation courses Specify school demographics for Field Specify school demographics for Field

Activities to ensure experience in diversityActivities to ensure experience in diversity Gather exit portfolio data to help guide Gather exit portfolio data to help guide

program improvement decisionsprogram improvement decisions

Operations Operations Assessment GoalAssessment Goal

Reduce the number of complaints received Reduce the number of complaints received

from credential studentsfrom credential students

Research DesignResearch Design Measures Used to Collect Data Measures Used to Collect Data

Toward Goal ProgressToward Goal Progress Established categories for complaints Established categories for complaints

and recorded data dailyand recorded data daily

Resulting from previous Data Analysis:Resulting from previous Data Analysis: Created position of Student Service Created position of Student Service

CoordinatorCoordinator Provided additional training in customer Provided additional training in customer

service and technical areasservice and technical areas Implemented the development of staff Implemented the development of staff

professional portfoliosprofessional portfolios

Data Collection ResultsData Collection Results

Credential Complaints

0

20

40

60

80

100

120

140

160

180

FY02 FY03 FY04

Series1

Analysis of DataAnalysis of Data Fewer complaints received in areas with closer Fewer complaints received in areas with closer

proximity of a credential supervisor.proximity of a credential supervisor. Mis-advisement has the most room for Mis-advisement has the most room for

improvement; most related to changes or improvement; most related to changes or confusion about state or university regulations or confusion about state or university regulations or policies.policies.

Fewer complaints were received in locations Fewer complaints were received in locations where professional development was regularly where professional development was regularly utilized.utilized.

Greater number of student-centered initiatives Greater number of student-centered initiatives and fewer complaints received from areas where and fewer complaints received from areas where staff had developed professional portfolios.staff had developed professional portfolios.

Future GoalsFuture Goals Develop, implement and measure Develop, implement and measure

additional ways to communicate policies additional ways to communicate policies in order to decrease “mis-advisement” in order to decrease “mis-advisement” complaints.complaints.

Determine the relationship between Determine the relationship between complaints and participation in complaints and participation in professional development and proximity professional development and proximity of credential supervision provided.of credential supervision provided.

Design and implement staff professional Design and implement staff professional portfolios state-wide.portfolios state-wide.

Assessment SymposiumAssessment Symposium May 2006, a.m.May 2006, a.m. FormatFormat

Slide 1: IntroductionSlide 1: IntroductionSlide 2Slide 2 Slide 3Slide 3 Slide 4Slide 4 Slide 5Slide 5 Slide 6Slide 6

2005-06 Goal2005-06 Goal Assessment Assessment DesignDesign

Data Data CollectionCollection

Analysis of Analysis of DataData

2006 - 072006 - 07GoalGoal

What goal was What goal was set? What set? What aspects of aspects of student student learning / learning / service will be service will be assessed? assessed?

Identify Identify measures measures (instrument / (instrument / tool) used to tool) used to provide data on provide data on progress progress toward goal. toward goal. What process What process was used?was used?

What data What data were were collected? collected?

What themes What themes emerged? emerged? What was What was learned? learned?

Recommendations Recommendations for growth based for growth based on analysis on analysis What measurable What measurable goal is set for goal is set for 2006 - 07? 2006 - 07?

Program Review

Assessment Cycle

Strategic Plan 2014

Assessment Symposium

Program Review

Program ReviewProgram Review In Progress:In Progress:

5 reviews completed Spring 20065 reviews completed Spring 2006 Revised Template, incorporating Assessment Revised Template, incorporating Assessment

Plan into Program ReviewPlan into Program Review Program Review HandbookProgram Review Handbook March Workshops for upcoming ReviewsMarch Workshops for upcoming Reviews

External ReviewersExternal Reviewers Internal ReviewersInternal Reviewers

External ReviewersExternal Reviewers

CredentialsCredentials External perspectiveExternal perspective

Lutheran link

Aspiring link

California

National

Study components of program through a Study components of program through a series of specific review objectives or answer series of specific review objectives or answer questions as prepared by program faculty. questions as prepared by program faculty.

Program Review

External ReviewersExternal Reviewers Receive copies of the Program Review Report Receive copies of the Program Review Report

data in advance data in advance

May be asked to respond to questions such as:May be asked to respond to questions such as: Do references in syllabi adequately cover current Do references in syllabi adequately cover current

knowledge in the field?knowledge in the field? Do CVs of faculty demonstrate appropriate Do CVs of faculty demonstrate appropriate

qualifications for teaching in this program?qualifications for teaching in this program? Does the curriculum demonstrate an undergraduate Does the curriculum demonstrate an undergraduate

/ graduate level of instruction?/ graduate level of instruction? Are assessments appropriate?Are assessments appropriate?

Program Review

External ReviewersExternal Reviewers

Complete a site visit and conduct Complete a site visit and conduct

interviews with administrators, interviews with administrators,

faculty, students, employers, and faculty, students, employers, and

alumni, as well as review facilities. alumni, as well as review facilities.

Program Review

External Reviewers’ ReportExternal Reviewers’ Report Three to five pagesThree to five pages

Prepare preliminary report on-sitePrepare preliminary report on-site

Responds to the objectives or questions Responds to the objectives or questions

Includes three parts: general comments, singles out Includes three parts: general comments, singles out

features of the program that merit commendations, features of the program that merit commendations,

and makes recommendations where there is room and makes recommendations where there is room

for improvement for improvement

Program Review

Internal ReviewersInternal Reviewers

California Lutheran University faculty California Lutheran University faculty

members from outside the college or members from outside the college or

school of the program under reviewschool of the program under review

Assessment Committee, Assessment Assessment Committee, Assessment

Director, Associate Provost, Provost, Director, Associate Provost, Provost,

GPEC, …GPEC, …

Program Review

Assessment Cycle

Strategic Plan 2014

Assessment Symposium

Program Review

CaliforniaCalifornia Lutheran Lutheran UniversityUniversity

Dedicated to educating leaders for a global society who are strong in

character and judgment, confident in their identity and vocation, and

committed to service and justice.

Core Core ValuesValues

Excellence Service Diversity Respect

Responsibility Courage Pursuit of Truth

Assessment CycleAssessment Cycle

California Lutheran UniversityCalifornia Lutheran UniversityDeans’ CouncilDeans’ Council

February 6, 2006February 6, 2006

Halyna KornutaHalyna KornutaDirector of Assessment and Educational EffectivenessDirector of Assessment and Educational Effectiveness