Dashboards as a Data Dashboards as a Data Management ToolManagement Tool

Dr. David C. OehlerDirector of Assessment, Information and Analysis

Northwest Missouri State University

06/21/2006 2

Presentation Overview Facilitating decision-making and

driving improvement Measurement impact on focus of

institutional activities and processes

How we developed our system and integrated it into information systems

Building a House of Quality

MANAGEMENT BY FACT

CONTINUOUSIMPROVEMENT

CU

ST

OM

ER

FO

CU

S

HIG

H E

XP

EC

TA

TIO

NS

INV

OL

VE

ME

NT

AS

SE

SS

ME

NT

&F

EE

DB

AC

K

TRUST

SHARED VALUES AND VISION

06/21/2006 4

Facilitating Decision-making and Driving Improvement

Importance of institutional culture Importance of internal

communication Alignment with mission and vision Alignment with strategic planning

06/21/2006 5

“All work is process.”

“Every process is perfectly designed to create the outcomes it produces.”

“If you can’t define it, you can’t measure it; if you don’t measure it,

you won’t improve it.”

Institutional CultureCQI Philosophy

06/21/2006 6

Communication – Northwest’s Planning Model

Since 1993, all units at Northwest use a Seven-Step Planning Process (the Hubbard Cycle) to focus and define their work.

This process is based on five underlying questions:

06/21/2006 7

CQI Underlying Questions

1. Who are your customers/clients? 2. What are their expectations?3. How do you “deliver” quality? 4. How do you know how well you

are doing? 5. What performance improvement

targets are reasonable?

06/21/2006 8

Key Quality Indicators “Key” - identifying a few very

important things to watch. “Quality” - a perception held by

students, stakeholders, and each other.

“Indicators” - evaluative tools that indirectly measure aspects of quality; multiple indicators are needed where direct measures are not available.

06/21/2006 9

1. What are your key decision areas? (as defined in your job description)

2. What kinds of decisions do you make?3. What information do you need to make

good decisions?4. What data do you need to create good

information?5. What is the best way to obtain that

data?

Focusing on Peoples’ Information Needs

06/21/2006 10

Communication and Systems Alignment

Annual Report Format Five-year Academic Department

Review (Coordinating Board for Higher Education)

Annual departmental visits (Deans’ executive summaries)

Phase 1:STRATEGICCONTEXT

Phase 2:STRATEGY

IDENTIFICATION& REFINEMENT

Phase 3:7-STEP TACTICAL

PLAN DEVELOPMENT &

DEPLOYMENT

Phase 4:BALDRIGE

SELF-ASSESSMENT

MISSION/VISION/VALUES

DECISIONDRIVERS

ENVIRON-MENTAL

SCAN

INTERNALANALYSIS

STRATEGICKQIs

TACTICAL KQIs

OBJECTIVES

ASSESSMENT

ACTION PLANS

RESULTS

PROCESSMANAGEMENT

FACULTY/STAFF FOCUS

PLANNING

STUDENTFOCUS

INFORMATION& ANALYSIS

LEADERSHIP

NORTHWEST QUALITY SYSTEMS MODEL

TRENDS

IMPROVEMENTS

STRETCH GOALS

STRATEGICINITIATIVES

SICHAMPIONS

DEPLOYMENTPLAN

MEASUREMENTPLAN

COMMUNI-CATIONS

PLAN

06/21/2006 13

Presentation Overview Facilitating decision-making and

driving improvement Measurement impact on focus of

institutional activities and processes

How we developed our system and integrated it into information systems

What is measured gets noticed

What is noticed gets acted on

What is acted on

gets improved

This and the following slide were adapted from a Dee W.

Hook presentation.

Phenomenon of Measurement

06/21/2006 15

What is a Dashboard? Definition/use:

Both a process and a tool Looking for unfavorable trends or patterns and

focusing energy on improving priority areas A (diagnostic) means for monitoring

performance to ascertain what is working well and where additional attention is needed

A few (4-6) sets of indicators, representing the most central areas related to high performance

06/21/2006 16

How Does a Dashboard Focus Activities and Processes?

Requires clear definition of outcomes Focuses on a manageable (small) set of key

outcomes (results) Encourages cross-functional communication Requires fact-based decision processes

Data reporting structures Process improvement orientation Layering of detail (summative vs. formative)

06/21/2006 17

Dashboards help you know what’s important

Dashboards focus on actions that make a difference

Collect data to create information you can use, then use it

Time is Increasingly a Precious Resource

06/21/2006 18

Assessment needs to answer questions

Systems to collect, analyze, and report information need to be developed to support the specific information requirements

Data for Decision-making

06/21/2006 19

Assessment System Design

1. Data Collection Centralized measures Decentralized

measures

3. Reporting Systems Summative

Information Formative Information

2. Data Processing Disaggregation system Aggregation system No transformation

4. Analysis/Decision-making

Cabinet, Deans, Directors,

Department Chairs Department Chairs,

Faculty

Data Collection

Reporting Systems

Analysis and Decision-making

Centralized

Decentralized

Disaggregation System

Aggregation System

Summative Information (Dashboards,

Profiles)

Formative Information (Operational)

Cabinet, Governing Board, External

Audiences

Directors, Department Chairs,

Faculty, Staff

No Transformation

Deans, Directors, Department Chairs

No Transformation

Directors, Department Chairs

Much Little

Detail Detail

Data Processing

06/21/2006 21

What Metrics are in a Dashboard?

The Dashboard includes a balanced view of an organization Learning and Growth (student academic progress;

faculty and staff development, scholarship and research

productivity) Customer Relations (students, faculty, staff, alumni,

parent satisfaction and involvement) Internal Processes (functional area performance –

accuracy, timeliness, friendliness) Financial Measures (fiscal health and viability)

Sample Traditional Balanced Scorecard

Measure/Index title 10 9 8 7 6 5 4 3 2 1 Wgt

Index 1 - Student Academic Progress 20%

Index 2 - Faculty & Staff Development 10%

Index 3 - Scholarship & Research Productivity 5%

Index 4 - Student Satisfaction & Involvement 12%

Index 5 - Faculty & Staff Satisfaction & Involvement 4%

Index 6 - Alumni & Parent Satisfaction & Involvement 4%

Index 7 - Internal offices process accuracy 10%

Index 8 - Internal offices process timeliness 5%

Index 9 - Internal offices process friendliness 5%

Index 10 - Fund balance 10%

Index 11 - Endowment 10%

Index 12 - Expenditures 5%

Lea

rnin

g &

G

row

thC

ust

om

er

Rel

atio

ns

Inte

rnal

P

roce

sses

Fin

anci

al

Mea

sure

s

06/21/2006 26

Northwest’s Dashboard Model

Our model includes several types of information/report presentations Dashboards – single screen current status Trend charts – key data element trends over

time Data tables –key data detail trends over time

Majors, minors, advisees, degrees, SCH, financials Special interest charts/tables

06/21/2006 27

Features of the Northwest Balanced Scorecard System

Dashboard “lights” to indicate current status

Hyperlinks to navigate through workbooks Hyperlinks to “drill down” to detail Comparative data links for setting targets Real-time data updates Accommodates various data sources Modular design to facilitate upgrading Automated updating of modules

President’s DashboardPresident’s Dashboard

Provost’s DashboardProvost’s Dashboard

VP Student Affairs’ DashboardVP Student Affairs’ Dashboard

Additional Detail

06/21/2006 32

Comparative Data In order to judge how good your

performance is, results should be put into some context Trends over time Comparisons to other internal units Comparisons with peer groups Comparisons outside of the education

sector

Academic Workload - SCH Production

50%

60%

70%

80%

90%

100%

110%

120%

130%

140%

150%

SCH Production (SU Total) % 100.0% 96.0% 97.9% 105.2% 103.9% 131.7% 142.9% 123.3% 122.5% 121.6%

SCH Production (FA Total) % 100.0% 100.0% 100.1% 101.6% 100.3% 102.3% 104.3% 106.2% 104.9% 103.3%

SCH Production (SP Total) % 100.0% 101.2% 101.7% 103.8% 102.6% 106.2% 104.3% 107.7% 104.2% 101.5%

SCH Production (Total) # 12,602 73,708 67,404 12,095 73,704 68,193 12,339 73,770 68,564 13,259 74,908 69,945 13,089 73,906 69,143 16,601 75,386 71,599 18,011 76,857 70,282 15,533 78,312 72,580 15,442 77,356 70,213 15,328 76,177 68,421

0 0 0 Su 94 F 94 Sp 95 Su 95 F 95 Sp 96 Su 96 F 96 Sp 97 Su 97 F 97 Sp 98 Su 98 F 98 Sp 99 Su 99 F 99 Sp 00 Su 00 F 00 Sp 01 Su 01 F 01 Sp 02 Su 02 F 02 Sp 03 Su 03 F 03 Sp 04

Student Satisfaction

1.0

1.5

2.0

2.5

3.0

3.5

4.0

Instructor made objectives clear 1.5 1.5 1.5 1.5 1.5 1.5 1.4 1.4 1.5

Instructor helped me achieve them 1.8 1.7 1.7 1.7 1.7 1.6 1.6 1.6 1.6

Instructor provided regular feedback 1.8 1.8 1.8 1.8 1.8 1.7 1.6 1.6 1.6

Instructor accessible during office hrs 1.7 1.6 1.6 1.6 1.5 1.4 1.4 1.4 1.4

Alumni - overall satisfaction rating 1.6 1.8 1.8 1.7 1.4

Alumni 'N'

93-94 94-95 95-96 96-97 97-98 98-99 99-00 00-01 01-02 02-03 03-04

Graduation Rate (CBHE)

30%

40%

50%

60%

70%p

erce

nt

(0-1

00)

Northwest 44.0% 44.0% 43.0% 48.0% 53.0% 57.3% 59.0%

Best competitor 45.0% 53.0% 54.0% 55.2% 56.0%

sp 95 data sp 96 data sp 97 data sp 98 data sp 99 data sp 00 data sp 01 data sp 02 data sp 03 data sp 04 data sp 05 data

Positive Referrals - Alumni Outcomes Survey

70%

80%

90%

100%

Per

cen

t P

osi

tive

(0-

100)

choose national 77.8% 81.2% 81.7%

choose cohort 75.7% 80.3% 79.1%

recommend national 94.3% 97.0% 97.0%

recommend cohort 94.6% 98.0% 97.5%

if beginning again, choose this school NW 85.0% 92.9% 94.1%

recommend to others NW 97.0% 99.5% 99.4%

2000 2001 2002

06/21/2006 40

Interpreting Dashboard Indicators

To follow up on indicators of interest, use hyperlinks to access increasing levels of detail

Student satisfaction as an example President’s dashboard to Provost’s dashboard to Noel-Levitz Student Satisfaction

Inventory data trends

Praxis andC-BASEresultsmodule

MajorField test

resultsmodule

AcademicProfileresultsmodule

Undergrad majors and

minors, Graduate majors,

Degrees, Advisees module

Studentopinionnairesof teaching

module

GeneralEducation

localmodule

ProgramSCH

generationmodule

Financialdata

module

Placementdata

module

EMSASmodule

(freshman success)

Alumnisatisfaction

module

Major fieldlocal/senior

capstonemodule

Department ‘A’ Profile andDashboard

College ‘A’Profile andDashboard

Department ‘etc.’ Profile andDashboard

College ‘etc.’Profile andDashboard

Provost’sProfile andDashboard

Service unit ‘A’ Profile and Dashboard

Service unit ‘B’Profile and Dashboard

Service unit ‘C’Profile and Dashboard

Service unit ‘etc.’Profile and Dashboard

President’sDashboard

Provost’s Dashboard Metrics Architecture

July 28, 2002

Comparativedata for targets

Studentsatisfaction

module

Current Status

Satisfaction 1.160 highest 1.160Students 1.074 overall 1.074 Link to

0.756 lowest 0.756 run chart

Item Perf WgtRatio

Calculation annualFr. Noel-Levitz Academic Advising 0.350 1.8% 1.160 1.160 show me!Fr. Noel-Levitz Safety and Security 1.360 1.4% 0.756 0.756 show me!Fr. N-L Instructional Effectiveness 0.640 5.0% 1.044 1.044 show me!Fr. Noel-Levitz Student Centeredness 0.490 2.2% 1.104 1.104 show me!Fr. Noel-Levitz Campus Climate 0.530 6.1% 1.088 1.088 show me!Fr. Noel-Levitz Concern for Individual 0.580 2.2% 1.068 1.068 show me!Fr. N-L Registration Effectiveness 0.700 1.8% 1.020 1.020 show me!Fr. N-L Campus Support Services 0.420 2.5% 1.132 1.132 show me!Fr. Noel-Levitz Recruitment/Fin. Aid 0.690 2.2% 1.024 1.024 show me!Fr. Noel-Levitz Service Excellence 0.590 2.9% 1.064 1.064 show me!Fr. Noel-Levitz Campus Life 0.600 5.4% 1.060 1.060 show me!

06/21/2006 46

Presentation Overview Facilitating decision-making and

driving improvement Measurement impact on focus of

institutional activities and processes

How we developed our system and integrated it into information systems

Praxis andC-BASEresultsmodule

MajorField test

resultsmodule

AcademicProfileresultsmodule

Undergradmajors andminors,

Graduate majors,

Degrees, Advisees module

Studentopinionnairesof teaching

module

GeneralEducation

localmodule

ProgramSCH

generationmodule

Financialdata

module

Placementdata

module

EMSASmodule

(freshman success)

Alumnisatisfaction

module

Major fieldlocal/senior

capstonemodule

Department ‘A’ Profile andDashboard

College ‘A’Profile andDashboard

Department ‘etc.’ Profile andDashboard

College ‘etc.’Profile andDashboard

Provost’sProfile andDashboard

Service unit ‘A’Profile and Dashboard

Service unit ‘B’Profile and Dashboard

Service unit ‘C’Profile and Dashboard

Service unit ‘etc.’Profile and Dashboard

President’sDashboard

Provost’s Dashboard Metrics Architecture

July 28, 2002

Comparativedata for targets

Studentsatisfaction

modules

Alignment of Dashboards and Profiles

President’s Dashboard

Provost’s Dashboard

College Profiles (3) Department Profiles (11)

Department Profiles (5)

Department Profiles (5) VP Student Affairs Dashboard Unit Profiles (8)

Unit Profiles (8)

VP Information Systems Dashboard

VP Finance and Support Services Dashboard

VP Communication and Marketing Dashboard

VP Institutional Advancement Dashboard

Director of Athletics Dashboard

Exec. Assistant to the President Dashboard

Unit Profiles (6)

Unit Profiles (6)

Unit Profiles (3)

Unit Profiles (3)

Unit Profiles (4)

Primary Source Modules

Phase 1:STRATEGICCONTEXT

Phase 2:STRATEGY

IDENTIFICATION& REFINEMENT

Phase 3:7-STEP PLANNING

PROCESS DEVELOPMENT &

DEPLOYMENT

Phase 4:BALDRIGE

SELF-ASSESSMENT

MISSION/VALUES

DECISIONDRIVERS

ENVIRON-MENTAL

SCAN

INTERNALANALYSIS

STRATEGICKQIs

TACTICAL KQIs

GOALS ANDOBJECTIVES

ASSESSMENT

ACTION PLAN

RESULTS

PROCESSMANAGEMENT

FACULTY/STAFF FOCUS

PLANNING

STUDENTFOCUS

INFORMATION& ANALYSIS

LEADERSHIP

NORTHWEST QUALITY SYSTEMS MODEL

TRENDS

IMPROVEMENTS

STRETCH GOALS

STRATEGICINITIATIVES

CHAMPIONS

HRPLANS

MEASURE-MENT PLANS

DEPLOYMENTPLANVISION

COMMUNI-CATIONS

PLAN

06/21/2006 51

Northwest Dashboard Information and Sharing Days

Contact information: Dave Oehler Northwest Missouri State University 800 University Drive 660.562.1527 (voice) oehler@nwmissouri.edu