Evolution of Program Review at UCF Using SAS Business Intelligence Tools to move Towards

Optimization

Patricia Ramsey & Carlos Piemonti2013 FAIR Conference

January 31, 2013

Task: Examine the quality and productivity of programs - Mandated by state statute - 7 year cycle - Approximately 30 programs per year

Goal is to determine which programs to: Enhance, Reduce, Maintain, Eliminate, or Study Further

Process requires a self-study, external consultant visit, and committee recommendations to the Provost, BOT & BOG

Program Review Process

Program Review Data Value

Wide range of data is required to support the process during all steps

Departments and programs - self-study prep External Consultants - better understand the

program prior to site visit Program Review Committee members and the

Provost - to identify opportunities or weaknesses

Program Review Data Value

Each program review data document includes the following:

Series of data points Trend information Comparison of program to overall department

and college

Program Review Data Requirements

Cycle Data Generation

Data Output Data Delivery

2000 Manual MS Word Tables Emailed2003 Programmatic MS Access summary

tables Web display via .asp

2006 Programmatic SAS® Enterprise Guide (EG) to MS Excel pivot tables

Emailed & Web (.pdf)

2008 Programmatic EG – developed report format to .pdf

Web (.pdf)

2012 Programmatic EG and WRS – report surfaced on web

Web

Program Review History

2006SAS EG

2012SAS BI

Progress Through the Years

2000MS Word

2007

2011

2012

Recent Advances

• Surface the data on the web via SAS BI tools – Web Report Studio (WRS) (IR waited for cascading parameters to be available!)

• Provide data for all programs, not just programs under review

• Auto-populate the self-study templates Graphs and charts are now developed in Web Report

Studio not in EG

Old Way

New Way

Update filters to reflect new reporting cycle

The Magic Behind the Scenes

Official data is loaded to Data Warehouse each term

SAS Information Maps create a view of the data designed for a purpose

SAS Web Report Studio graphically surfaces the data on the web

New Maintenance Routine

1. Data prep: run an EG project once a year and Roll forward one cohort in each report by updating filters (Feb)

2. Request refreshed data to UAPS (Projections – May), UAPS does it by replacing table

3. Generate reports: preliminary (Apr); final (Aug) for programs under review

New Program Review CyclePreparationUpdate ‘Year’

filters for next cycle – in each tab in each report

SAS Web Report Studio

Feb

Update source data to include for next reporting year – Automatically Info Maps are refreshed

SAS Enterprise Guide

Feb

University Community has access now!1H

New Program Review CyclePDF Report Generation

Respond to prompts and print reports as PDFs (PMP)Edit PDFs manually

Combine PDFs into 1 report per ‘program area’ and update bookmarks

Days 1-2

Day 3

Day 4

Program Review 2012-2013:12 department reports were created twice (preliminary and final versions) in less than 10 workdays.

Pegasus Mine Portal

https://my.ucf.edu/

College

Department

Undergraduate

Graduate

Report Contents

Sample

Prompts Page on the Portal

Dynamically generate values&

Cascading parameters

Cascading Parameters

Prompts Sample

Report Display in the Portal

Update Filters

External Access

web

Program Review 2012-2013by Department/Program Area

Communication

Dean's Office UGST

Health Mgmt. & Informatics

FL Interactive Entertainment Academy

Modern Languages

Legal Studies

Physical Therapy

Public Administration

Optics

Political Science

Office of Interdisciplinary Studies

Health Professions

Document Layout Key

1

12 For a total of 26 programs!

Limitations

Security - Program Review in the portal is limited to select users. Consultants must have access to data outside the portal. IR must make .pdf versions and post to a location outside this firewall so they have easy access

Design - SAS has not yet designed prompts that apply to the entire report. The user must respond to prompts on each tab