data driven decision making: an essential tool for these critical times steve gillard naada...

Data Driven Decision Making: An Essential Tool for These Critical Times

Steve Gillard NAADA Conference

June 16, 2009

1

Main Themes

• How to demonstrate alignment with strategy

• Where we have been where are we going

• A framework for driving informed decisions

• Building capacity to use and manage data

• How to address the need for information

• Integrating data with decision making

2

Key Take Aways

• A framework for data driven decision making

• A model for addressing data needs and the capacity to make it real

• An outline for a data plan with action steps

• Useful tools and approaches

• Resources to share and leverage

3

Data Driven Decision Making (DDDM)What it is and what it does

• Using data to inform or support decisions

• Having the right data in the right format at the right time for the right purpose

• Enables a learning organization

• Is a key element of transparency and accountability

• Helps remove barriers to organizational change

4

Guiding Questions

• What is meant by a culture that supports data-driven decision making (DDDM)?

• What things need to exist that promote the effective use of data?

• What barriers exist to building and sustaining a DDDM culture?

• What are some measureable benefits of DDDM?

5

Adapted from CoSN Annual Conference, March 10-12, 2009

Decision Making Model by Rich Howard

-History

-Priorities -Mission

Integrated Knowledge and data

-Information -Facts -Opinion

Organizational Knowledge

Data

Data Informed Decision (Intelligence)

Decision

Making

Assessment

Decision Making Process

6

Metrics and Management Framework

7

8

Key Performance IndicatorsGrowth Indicators Quality Indicators Efficiency Indicators

Instruction • Entering students• Enrollment• Enrollment per major• Scholarship expenditures• SCH/FYE• Degrees awarded• Instructional tuition

• ACT and GRE scores• NHS top 10%• Ugrd. GPA of grads• 1st year retention• Degree completion• Dean’s List• Course Satisfaction• Time-to-degree• Graduate fellowships

• Instructional funds FYE• Faculty workload• SCH per faculty• Instructional finds FTE• Instructional revenue vs. Cost• Instructional cost• Instructional revenue• Support $ FTE

Research • # proposals submitted• # grants awarded• Indirect cost recovery• Size of awards• Award $ per faculty FTE• Grad expenditures• Sponsored funds• Type of proposals/awards

• Award sponsorship• Grants received vs. submitted• Trend in ICR generated• Refereed journal articles• NRC Ranking• Essential Science Indicators

• MAES vs. sponsored funding• MAES per FTE• Sponsored expenditures• Core support to sponsored

expenditures• Return on input

Extension • Annual revenue growth• Number of teaching units• Revenue/State support• Event offerings• Product sales• Access opportunities

• Integrated grant success• Refereed journal articles• Value-based fee structure• Business plans• Evaluation projects

• MAES vs. sponsored funding• Sponsored proposals vs. awards• Sponsored expenditures• Core support to sponsored

expenditures• Support $ FTE

Instructional KPI Example

9

Integrated Data Example

10

Integrated Data Example

11

Dashboard Example

12

Dashboard Example

13

Source: Performance Management Scorecards and Dashboards for IT Operations Data, Rex Parker Microsoft 2008

Dashboard Example

14

Source: Microsoft BI project 2008

Operational vs Strategic

• Most data at the U of MN originates in transactional systems (generally PeopleSoft)

• Operational and strategic decisions, however, require the data to be in different forms and follow different rules

15

Operational Questions

• What’s happening right now

• What action needs to be taken

• Choices generally pre-determined and “triggered” by data

• Interest is usually in individual cases and current values

• The closer to “real time” the better

16

Strategic Questions

• What is the trend?• What plans should we make?• Choices generally open for discussion

and debate• Interest is generally in aggregates and

trends• Data needs to remain unchanged over

time

17

Beyond List Reporting

• What if analysis

• Forecasting

• Statistical analysis

• Predictive modeling

• Alerts and triggers

• Dashboards and scorecards

• Actionable insight

18

Translating data into action

• Define the decision to be made or question to be answered

• Identify data requirements and availability

• Gather, format and consolidate data

• Analyze to generate conclusions

• Take appropriate actions

19

Building Capacity Enabling Data Driven Decision Making

• Engaged, supportive leadership

• Effective staffing

• Training and professional development

• Common language

• Collaborative environment

• Modest IT infrastructure

20

21

Reporting and Analytics(where we have been where we are going)

22

Audience and Complexity

Pre-defined Reporting

User-defined Reporting

Ad-hoc

(OLAP)Adv. Analytical

Mining

UserUser BaseBase

CCoommpplleexxiittyy

Analysis and recommendations for a specialized audience

Basic facts for a general audience

23

ReportsData Warehouse

Reporting and Analysis ServicesTurning data into useful information

TrainingLearning how to use data to make informed decisions.

State and Federal ReportingMeeting reporting compliance

DisseminationSharing data with the community (ie: report cards)

School Interoperability Framework & IMS

Components of a Data Based Decision Making System

SIS

Finance

Assessment

Instruction

Applications

Personalized Instruction

Source: US Department of Education, 2003.

Example of Data, Reporting and Analysis Architecture

25

Source: Technical Evaluation of Business Intelligence-Envisioning the Future White Paper, Knowledge Integrity, Inc 2008

END USER TOOLS & PERFORMANCE MANAGEMENT APPS

ExcelExcel PerformancePoint PerformancePoint ServerServer

BI PLATFORM

SQL Server SQL Server Reporting ServicesReporting Services

SQL Server SQL Server Analysis ServicesAnalysis Services

SQL Server DBMSSQL Server DBMS

SQL Server Integration ServicesSQL Server Integration Services

SharePoint ServerSharePoint Server

DELIVERY

ReportsReports DashboardsDashboards Excel Excel WorkbooksWorkbooks

AnalyticAnalyticViewsViews ScorecardsScorecards PlansPlans

Mind Mapping

“A mind map is a diagram used to represent words, ideas, tasks, or other items linked to and arranged around a central key word or idea. Mind maps are used to generate, visualize, structure, and classify ideas, and as an aid in study, organization, problem solving, decision making, and writing.”

27Source: http://en.wikipedia.org/wiki/Mind_Mapping

Mind mappingCustomer Service

28

Mind mappingAttacking Problems

29

Mind Mapping Activity

Outline key elements or components of effective data driven decision making– Identify key elements and how they link– Feeds your data plan– Work independently or in groups

30Source: http://en.wikipedia.org/wiki/Mind_Mapping

31

Guiding Principles for Implementation (from PEL)

• Strategic Communication

• Collaboration

• Staff Development

• Transparency

• Criteria for Decision Making

• Reflection and Feedback

Source: Implementing Administrative Metrics, June, 30, 2008, PEL 2008 cohort32

A Framework for Success

Source: Implementing Administrative Metrics, June, 30, 2008, PEL 2008 cohort

33

Collaborative Organizational Structure

Source: Implementing Administrative Metrics, June, 30, 2008, PEL 2008 cohort

34

Embed in Existing Management Processes

Source: Implementing Administrative Metrics, June, 30, 2008, PEL 2008 cohort

35

36

37

38

Critical Factors for Successful Implementation

39

Alignment

• Being more strategic in initiatives and foster greater “alignment”

• Alignment means having a stronger connection between the purposes and functions of units and initiatives and the strategic goals of the organization

• Funding is directed toward initiatives that promise progress on strategic goals

40

41

Values

Mission

Vision

Strategic perspectives

Strategic themes and results

OBJECTIVES

Strategy map

Performance measures & targets

Strategic initiatives

Org Assessment(SWOT, Those We Serve)

30,000

25,000

15,000

Ground

Strategic Altitude What we stand for / our code of conduct

Who we are / why we exist / key features of our organization

Emotionally inspiring picture of future success for our organization

Lenses to ensure balance of our efforts

Logic of a Strategic Management System Utilizing Balanced ScorecardLogic of a Strategic Management System Utilizing Balanced Scorecard

Main areas of focus to best serve “Those We Serve”

Strategic effort areas to achieve success /

get results

Map of how objectives will achieve

success (organized into perspectives)

How we will know if we are

achieving desired results

Specific projects that

contribute to desired results

42

STRATEGY MAPDefine & communicate priorities and focus

BALANCED SCORECARDMeasure performance against priorities

MISSIONWhy we exist

VALUESWhat’s important to us

VISIONWhat we want to be

STRATEGYOur game plan

EMPOWERMENT / PERSONAL OBJECTIVESMotivate employees

INITIATIVE and PLANNING PROCESSManage actions and resources to drive change

STRATEGIC OUTCOMES

SatisfiedSHAREHOLDERS

Delighted CUSTOMERS

Efficient and EffectivePROCESSES

Motivated & PreparedWORKFORCE

The Strategy Map and the Balanced Scorecard bridged the strategy implementation gap. They link leadership at the top to management of initiatives, process improvements and

employees’ everyday actions.

43

A commitment to excellence

44

Exceptional Students

Strategic Objectives:

• Make the University of Minnesota a destination of choice for diverse students who reflect the diversity of our community and world, and are sought after because of their unique talents, skills and experiences

• Educate and support all of our students to assume positions of leadership in the community, state, nation and the world

• Provide our students with the most advanced, sophisticated, and comprehensive technology tools to enhance their learning experience

• Globalize our students’ experience, recruit students from around the world and provide an education to prepare students to become global citizens and leaders

Strategic Result:

Recruit, educate,

challenge, and

graduate outstanding

students who become

highly motivated

lifelong learners,

leaders, and global

citizens

45

Exceptional Faculty and Staff

Strategic Objectives:

• Recruit identify, support and reward stars on the rise

• Create a robust culture of collaboration that encourages and rewards boldness, imagination, and innovation.

• Hire, develop and place diverse faculty and staff in positions which match their skills and abilities with organizational needs

• Strengthen the performance evaluation and reward systems to fully engage, motivate and challenge faculty and staff

• Significantly increase the number of faculty receiving awards of distinction

Strategic Result:

Recruit, mentor,

reward, and retain

world-class faculty and

staff who are

innovative, energetic,

and dedicated to the

highest standards of

excellence

46

Exceptional Organization

Strategic Objectives:

• Adopt best practices and embrace enterprise standard business practices processes and technology to achieve efficient, effective and productive operations

• Promote nimble decision-making using data, information, research and analysis

• Achieve a shared services administrative structure

• Align resources to support strategic priorities

• Commit to service and results that are best among peers

Strategic Result:

Be responsible

stewards of resources,

focused on service,

driven by performance,

and known as best

among our peers

47

Exceptional Innovation

Strategic Objectives:

•Foster an environment of creativity that encourages evolution of dynamic fields of inquiry

•Invest in strong core disciplines while supporting cross disciplinary, collaborative inquiry

•Fully leverage our academic, research and community partnerships and alliances to provide leadership in a global context

•Develop innovative strategies to accelerate the efficient and effective transfer and utilization of knowledge for the public good

Strategic Result:

Inspire exploration of

new ideas and

breakthrough

discoveries that address

the critical problems

and needs of the

University, state, nation,

and world

48

Organizational Alignment ExampleOffice of Information Technology (1st)

49

Organizational Alignment ExampleOffice of Information Technology (2nd)

50

Organizational Alignment ExampleOffice of Information Technology (3rd)

51

Organizational Alignment ExampleOffice of Information Technology (4th)

52

Organizational Alignment ExampleOffice of Information Technology (5th)

53

Perspectives for BalancePerspectives for Balance

Objectives Should Be Balanced Among Four Perspectives

(1) The People We Serve

How can we best meet the needs/wants of “Those We Serve”?

(2) Organizational Resource Productivity

How can we maximize the efficiency and effectiveness of our resources?

(3) Mission Driven Processes

How can we improve the way we do our work to increase productivity or improve service?

(4) People and Capacity

How can we engage our employees and increase our capacities (IT, etc.) in order to prepare our

organization to achieve excellence?

Externally facing / Outputs

Internally facing / Inputs

54

Relationships Ensure Comprehensive MapsRelationships Ensure Comprehensive Maps

• Objectives in perspectives should show a cause-effect relationship

• Reading top to bottom, answers the question: “What do we have to do (objective) to achieve this desired result (objective)?

• Reading bottom to top, answers the question: “If we do this (objective), then we can achieve that (objective)

• Arrows are utilized to show the inter-relationship between objectives

Note: the University will use different “perspective” names and order!

55

Barriers to Effectively Using Data for Decision Making

• Trust of the data

• Resources

• Communications

• Training

• Time

56

Major Barriers for Using Data for Decision Making

• Lack of training: 50% • Interoperability—systems that are unable to share or exchange

data: 42% • Lack of understanding of what to do with the data: 39% • Absence of clear priorities on what data should be collected:

36% • Failure to collect data in a uniform manner: 35% • Outdated technology/legacy systems: 31% • Low quality data – inaccurate or incomplete: 24% • Timing of data collection: 24% • User interface is too complicated to understand reports: 22%

Source: A survey conducted by Grunwald & Associates on behalf of CoSN in 2004

57

Developing a Data Plan of Action

58

Tools to Leverage

• Before and after

• Mind mapping

• Critical factors for success

• Force field analysis

• SWOT

• Threat / opportunity matrix

• Stakeholder map

59

Force Field Analysis

60

Source: http://accel-team.com/techniques/force_field_analysis.html

Threat: What is the impact of not adopting DDDM in our group or college?

Opportunity: Identify the opportunities for adopting DDDM

Threat Opportunity

Short Term

Threat / Opportunity Matrix

61

Long Term

Balanced Scorecard

Source: http://images.google.com/

62

Baldrige National Award

Source: Baldrige National Quality Program 2009-2010

63

The Big Picture

64

65

Six Sigma Basic Steps

• Define – What is the problem

• Measure – Acquire data

• Analyze – Analyze data and determine what the data

• Improve – Incorporate process improvements

• Control – Continue to monitor progress and adjust

Project BenefitsOpportunity Statement

Goal Statement(s) Project Scope

Project Plan Team Selection

66

Elements of a Project Charter

Books of Interest Include• Competing on Analytics by Davenport and

Harris• Five Key Principles of Corporate

Performance Management by Paladino• Business Intelligence Competency Centers

by Miller, Brautigam and Gerlach• Transforming Performance Measurement

by Spitzer• The Profit Impact of Business Intelligence

by Williams and Williams

67

Books of Interest Include(Continued)

• Key Performance Indicators by Parwmenter• Collapse of Distinction by McKain• Advancing Campus Efficiencies by

Johnstone• Moneyball by Lewis• People, Processes and Managing Data by

McLaughlin and Howard

68