A New Comprehensive Program Review Framework for L.A. City College:Using Data to Drive PlanningDr. Edward PaiDean, Institutional EffectivenessPhillip BriggsResearch analyst

L.A. City College Accreditation Recommendation• College Recommendation 1: Program ReviewIn order to increase institutional effectiveness and meet standards, the team recommends that the college more systematically use the results of program/unit reviews to continually refine and improve program practices resulting in appropriate improvements in student achievement, learning, support services, and institutional processes. (I.B.1-7; II.A.2; II.B.4; II.C.2; III.B.2; III.C.2; III.D.3; IV.A.5).

Revised CPR Framework

• Prior Comprehensive Program Review (CPR) Framework:

• Relied solely on open-ended questions

• Example – Describe enrollment trends in your program.

• Responses and validations were very subjective and inconsistent

• Very long – approximately 30 pages

• Didn’t require use of data

• Results were inconsistent at multiple levels of the process

Revised CPR Framework

• Need identified for a more structured and streamlined CPR

• End Goals:

• Systematically review College performance

• Develop unit plans based on College Goals and Priorities

• Assess the Implementation of Previous Strategic Master Plan

• Develop New Strategic Master Plan

• Preparation for 2015 ACCJC Self-Evaluation

Revised CPR Framework

• College Strategic Master Plan provides framework for institutional outcomes

• 4 Strategic Master Plan Goals:

1. Expand Student Access (A)

2. Increase Student Success and Academic Excellence (S)

3. Enhance Resources and Accountability (A)

4. Expand Community Partnerships (P)

• Revised CPR Process – Assesses how well a program implemented these 4 goals

Revised CPR Framework

• Proposed CPR Process:

1. Programs (disciplines) are given data based on the 4 goals of the Strategic Master Plan

2. Dept. chair and faculty provide an explanation for the data based on evaluation framework and a plan for improving data

3. Data and responses are sent to Dean and Validation Teams

4. Validation team provides program with commendations and recommendations using an evaluation rubric

5. Recommendations and program-identified improvement plans become planning goals for the next year and the basis for funding requests

CPR Data Analysis – Program Results• Instructional programs:

• Receive program-specific data on 15 measures chosen to address the 4 goals of the Strategic Master Plan

• For each measure, the program receives:

• 1) Their program-specific data

• 2) How their program-specific data compare to the rest of the college

• Comparison data – median-based quartiles

• Referred to as rubric categories

CPR Data Analysis – Program Results• Rubric Categories (with a few exceptions):

• 4 = 75th percentile to 100th percentile of college

• 3 = 50th percentile to 74th percentile of college

• 2 = 25th percentile to 49th percentile of college

• 1 = 1st percentile to 24th percentile of college

CPR Data Analysis: College-wide Results

CPR Data Analysis – Program Results• Student Services:

• Receive data on different measures that are based on ASAP

• Comparison data – College average

Program Response• Programs provide an explanation for the data and

plans for improvement

• College focuses on programs in categories 1 and 2

• Outcomes:

1. Systematic review of data by each program

2. Data-based improvement plans

Validation Process• Dean and Validation Team review the data and

responses

• 5-question rubric is used by both to evaluate the responses

• Program receives commendations and recommendations based on rubric scores

• Results compiled for use in annual and strategic planning processes

Planning Outcomes• Program-created improvement plans and Validation

Team recommendations become future plans/goals and source of funding requests

• LACC’s existing online planning system tracks progress

• Development of new Strategic Master Plan in Fall 2013 fueled by:

• Program data analysis and improvement plans

• College-level analysis of Previous Strategic Master Plan

Unexpected Outcomes• Training of dept. chairs, program managers, and validation team

members resulted in systematic campus-wide dialog about data and Strategic Master Plan

• Campus receptivity to performance-based evaluation framework

• Setting of college standards – college median

• Improvement in data analysis and planning skills

• Improvement in technology skills

• CPR Website: http://sharepoint.lacitycollege.edu/sgc/program_review/cpr_2012/default.aspx