program review for integrated planning

11

Accrediting Commission for

Community and Junior Colleges

22

Elements of an Effective Elements of an Effective Program Review for Integrated Program Review for Integrated Planning, Learning Outcomes, Planning, Learning Outcomes,

and Assessmentand Assessment

Dr. Barbara Beno, PresidentDr. Barbara Beno, PresidentDr. Steve Maradian, Vice PresidentDr. Steve Maradian, Vice President

Accrediting Commission for Community and Junior CollegesAccrediting Commission for Community and Junior CollegesWASCWASC

Academic Resource ConferenceAcademic Resource Conference21 April 200921 April 2009

33

Workshop Purpose: learn toWorkshop Purpose: learn to

Construct an effective, integrated system of Construct an effective, integrated system of program review, planning, and resource program review, planning, and resource allocationallocation

Enable the institution to continually assess its Enable the institution to continually assess its effectivenesseffectiveness

Identify and understand the component parts of Identify and understand the component parts of a viable program review processa viable program review process

ContinuedContinued

44

Workshop PurposesWorkshop Purposes ContinuedContinued

Embed the process at all levels of the institutionEmbed the process at all levels of the institution

Create a culture of informed decisionCreate a culture of informed decision--making making and sustainable continuous quality improvementand sustainable continuous quality improvement

Use results of this assessment to advance Use results of this assessment to advance effectiveness and educational quality of an effectiveness and educational quality of an institutional programinstitutional program

55

Elements of Effective Elements of Effective Program ReviewProgram Review

66

Institutional Mission

Programmatic Mission

Outcomes

Inputs

Process

AlignResourcesImplement

ProgramAssessment

Analysis ofOutcomes

IdentifyGaps

DesignProgramChanges

AllocateNeeded

Resources

District Mission and

Expectations

77



Outcomes

Inputs

Process


ProgramAssessment

Analysis ofOutcomes

IdentifyGaps


AllocateNeeded

Resources

District Mission and

Expectations

88

Link between district/system (d/s) and college mission(s)Functional map identifies roles and responsibilities between college and d/s

Review at campus and district level(s)

All data used to drive college improvements and d/s understanding/support

College planning is linked with the d/s in support of college mission(s) and activities

System Mission and

Expectations

CollegeMission

99



Outcomes

Inputs

Process

AlignResources

ImplementProgramAssessment

Analysis ofOutcomes

IdentifyGaps


AllocateNeeded

Resources

1010

Institutional Mission DefinesInstitutional Mission DefinesInstitution’s Broad Purpose

The Intended Student PopulationA Commitment to Student Learning

(the institutional mission flows from the system mission)

All programs and services must align with this missionThe mission is regularly reviewed and revised as neededThe mission is central to planning and decision making

1111



Outcomes

Inputs

Process


ProgramMeasure

Outcomes

Analysis ofOutcomes

IdentifyGaps


AllocateNeeded

Resources

1212

Programmatic MissionProgrammatic MissionProgram’s Purposes:

(E.g.,workforce training, transfer, general education, pre-collegiate education, baccalaureate education, etc.)

Define Programmatic Student Learning Outcomes

The program’s mission must align with institutional missionThe mission is reviewed regularly and revised as needed

The mission is central to programmatic planning and decision making

1313



Outcomes

Inputs

Process

AlignResources


Analysis ofOutcomes

IdentifyGaps


AllocateNeeded

Resources

1414

Student LearningStudent Achievement

OutcomesOutcomes

1515

How are outcomes analyzed?How are outcomes analyzed?

Analysis requires a judgment about whether the Analysis requires a judgment about whether the outcomes are outcomes are ““good enoughgood enough””–– Benchmarks of performance of similar collegesBenchmarks of performance of similar colleges–– Benchmark of past institutional performanceBenchmark of past institutional performance

(e.g., 5% increase over previous year(e.g., 5% increase over previous year’’s) s)

–– Externally imposed benchmarks Externally imposed benchmarks –– state governments, state governments, professional associations, employers, etc. professional associations, employers, etc.

–– Targets or goals established by the institutionTargets or goals established by the institution–– Others? Others?

Analysis requires valid data or informationAnalysis requires valid data or information

1616

Student Achievement OutcomesStudent Achievement Outcomes• Course completion• Retention term to term• Progression to next course/level• Program completion • Degree/certificate completion • Transfer• Success/Scores on licensure exams• Job placement

All data collected must be analyzed

1717

Student Learning OutcomesStudent Learning Outcomes• Established by faculty at the course, program, degree, and certificate levels;

• Faculty, administrators, and trustees play a role

• Authentic assessment designed to determine what students actually learn;

Assessment data are collected and analyzed.

1818

Relationship Among Course, Program, Relationship Among Course, Program, and Institutional SLOsand Institutional SLOs

Institutional Student Learning Outcomes: there is a broad set ofInstitutional Student Learning Outcomes: there is a broad set of SLOs which SLOs which describe outcomes students have achieved when they complete theidescribe outcomes students have achieved when they complete their r degree/certificate/programdegree/certificate/program

Program Student Learning Outcomes: a set of SLOs students have aProgram Student Learning Outcomes: a set of SLOs students have achieved chieved upon completion of a sequence or cluster of courses in a programupon completion of a sequence or cluster of courses in a program. These . These contribute to institutional SLOscontribute to institutional SLOs

Course Student Learning Outcomes: these are agreedCourse Student Learning Outcomes: these are agreed--upon outcomes upon outcomes students must achieve to complete the course and these contributstudents must achieve to complete the course and these contribute to program e to program and/or institutional SLOsand/or institutional SLOs

1919

Development of Student Learning Development of Student Learning OutcomesOutcomes

Program Level: programmatic faculty in the disciplines and external

clients (employers, transfer institutions, & the public)

Institutional Level:Faculty, Academic Administrators

TrusteesThrough the

Mission, Values, andCommitment

Course Level:Discipline Faculty

2020

SLO AssessmentSLO Assessment

Planning is critical for Planning is critical for an onan on--going systemic going systemic efforteffortDocument and use of Document and use of assessment data for assessment data for improvementimprovementInformation sharing Information sharing across disciplines to across disciplines to share best practicesshare best practices

Commitment for all Commitment for all organizational levelsorganizational levelsCelebrate Celebrate accomplishments and accomplishments and improve student improve student learninglearningDonDon’’t delay; itt delay; it’’s not a s not a fadfad

2121



Outcomes

Inputs

Process

AlignResources

ImplementProgram

Assessment

Analysis ofOutcomes

IdentifyGaps


AllocateNeeded

Resources

2222

StudentsExisting Resources

InputsInputs

2323

Existing ResourcesStaff

(number and capacity)

Facilities

Equipment

Funding

All continually aligned to course/program

2424

StudentsWho are they?

How well prepared are they?What are their educational goals?

What are their program goals?What are their support needs?

2525


Programmatic MissionOutcomes

Inputs

Process


ProgramAssessment

Analysis ofOutcomes

IdentifyGaps


AllocateNeeded

Resources

2626

Process:Pedagogy and Support Services

Course outlines

Course content, intended SLOs

Strategies for assessing student learning

Instructional support and services for students

Variable delivery modes & scheduling, etc.

2727



Outcomes

Inputs

Process


ProgramAssessment

Analysis ofOutcomes

IdentifyGaps


AllocateNeeded

Resources

2828

Ongoing Alignment of Resources

(Human, Physical, Technology, & Financial)

• Sufficient and appropriate resources are provided to meet program and institutional needs

• Resource allocation through established processesis integrated with and informed by

institutional evaluation and planning

District/system support necessary to ensure that program review leads to meaningful

improvements at the college

2929



Outcomes

Inputs

Process

AlignResources

ImplementProgram

Assessment

Analysis ofOutcomes

IdentifyGaps


AllocateNeeded

Resources

3030

Program Implementation

Scheduling and sequence of courses

Alignment with general education courses

3131



Outcomes

Inputs

Process

AlignResources


Analysis ofOutcomes

IdentifyGaps


AllocateNeeded

Resources

3232

AssessmentGather meaningful student achievement data

Measure attainment of student learning outcomes

Data are qualitative and quantitative

Data are longitudinal where appropriate

Outcomes data are continually collected and analyzed

3333



Outcomes

Inputs

Process

AlignResources


Analysis ofOutcomes

IdentifyGaps


AllocateNeeded

Resources

3434

Analysis of OutcomesSee Slide 15

• Understanding the meaning of the data collected• Judgments about what is good enough must be made

• Institutional and system governance groups must be informed

Some colleges collect data through the district/system; others collect data independently

3535



Inputs

Process

AlignResources


Analysis ofOutcomes

IdentifyGaps


AllocateNeeded

Resources

3636

Gap Analysis

Comparison of actual outcomes with intendedor targeted outcomes.

What worked to attain intended outcomes?

What part(s) of the program needs to be changedto attain intended outcomes?

3737



Inputs

Process

AlignResources


Analysis ofOutcomes

IdentifyGaps


AllocateNeeded

Resources

3838

Design Program ChangesTo program and course outcomes

To inputs and processes

To human, physical, technological and financial resources

Programmatic changes must be in line with mission

Planned changes must inform the processof budget allocation

System consultation is pivotal

3939



Outcomes

Inputs

Process


ProgramAssessment

Analysis ofOutcomes

IdentifyGaps


AllocateNeeded

ResourcesSystem role

4040

(Re)Allocate Needed Resources• Results of program review are clearly and

consistently linked to the institutional planningand resource (re)allocation processes

• Constituent groups on campus areinvolved in the decision-making processes

• Board Governance decisions reflect institutional priorities identified through assessment

District/system support necessary to ensure that program review leads to meaningful

improvements at the college

4141

Reallocate Needed Resources

•Program review and planning inform/directresource reallocation

District/system support necessary to ensure that program review leads

to meaningful improvements at the college

4242

CourseLevel

StudentLearning

Outcomes

Program Level Student Learning Outcomeswhich meet Program goals

and intentions

Institutional Level Student Learning Outcomesthese are agreed-upon outcomes students must

achieve to complete the degree or certificate.

4343

The Sustainable Continuous The Sustainable Continuous Quality Improvement Level of Quality Improvement Level of

Program ReviewProgram ReviewProgram review processes are ongoing, systematic, Program review processes are ongoing, systematic, and used to assess and improve student learning and and used to assess and improve student learning and achievement; the Board is informed of successes and achievement; the Board is informed of successes and needed improvements needed improvements

The institution reviews and refines the program The institution reviews and refines the program review process to improve institutional effectivenessreview process to improve institutional effectiveness

The results of program review are used to continually The results of program review are used to continually refine and improve program practices resulting in refine and improve program practices resulting in improvements in student achievement and learningimprovements in student achievement and learning

4444

A Culture of AssessmentA Culture of Assessment

Institutional Institutional CommitmentCommitment–– Program reviewProgram review–– Institutional Institutional

effectivenesseffectiveness–– Information sharingInformation sharing–– Systemic assessment Systemic assessment

planningplanning–– Common assessment Common assessment

terminologyterminology

–– Professional Professional developmentdevelopment

–– Planning and resource Planning and resource allocationallocation

–– Involvement and Involvement and support from support from

BoardBoardPresidentPresidentFaculty Faculty StaffStaffInformation technologyInformation technology

4545



Inputs

Process

AlignResources

ImplementProgram

Assessment

Analysis ofOutcome

IdentifyGaps


AllocateNeeded

Resources

System role??

4646

How Did We Get Here?How Did We Get Here?

Accreditation Standards have required program Accreditation Standards have required program review and integrated planning since the 1990sreview and integrated planning since the 1990s

To those requirements, there is an expectation that To those requirements, there is an expectation that Student Achievement and SLO data become a part of Student Achievement and SLO data become a part of the program review and planning processesthe program review and planning processes

4747

What is Expected Now?What is Expected Now?

Accredited colleges are expected to be at the Accredited colleges are expected to be at the Sustainable Continuous Quality ImprovementSustainable Continuous Quality Improvement Level Level on the Commissionon the Commission’’s Rubric for Evaluating s Rubric for Evaluating Institutional Effectiveness Institutional Effectiveness –– Parts I and II (Program Parts I and II (Program Review and Planning)Review and Planning)

4848

What is Expected in the What is Expected in the Future?Future?

Accredited colleges are expected to Accredited colleges are expected to remainremain at the at the Sustainable Continuous Quality ImprovementSustainable Continuous Quality Improvement Level Level on the Commissionon the Commission’’s Rubric for Evaluating s Rubric for Evaluating Institutional Effectiveness Institutional Effectiveness –– Parts I and II (Program Parts I and II (Program Review and Planning)Review and Planning)

4949

What is Expected in the What is Expected in the Future?Future?

By 2012, colleges are expected to be at the By 2012, colleges are expected to be at the ProficiencyProficiency Level on the CommissionLevel on the Commission’’s Rubric Part III s Rubric Part III –– Student Learning OutcomesStudent Learning OutcomesACCJC to release a more detailed ACCJC to release a more detailed ““proficiencyproficiency”” description in Fall 2010 description in Fall 2010

Over time at this level, colleges will achieve the Over time at this level, colleges will achieve the fourth level (fourth level (Sustainable Continuous Quality Sustainable Continuous Quality ImprovementImprovement))

5050

Thoughts?Thoughts?

5151

puzzle

clearer?pieces

Are all

5252

An ExerciseAn ExerciseUsing Program Review Data to Using Program Review Data to

Plan ImprovementsPlan Improvements

5353

Thank YouThank You

program review for integrated planning

Education