data driven decision making: an essential tool for these critical times steve gillard naada...
TRANSCRIPT
Data Driven Decision Making: An Essential Tool for These Critical Times
Steve Gillard NAADA Conference
June 16, 2009
1
Main Themes
• How to demonstrate alignment with strategy
• Where we have been where are we going
• A framework for driving informed decisions
• Building capacity to use and manage data
• How to address the need for information
• Integrating data with decision making
2
Key Take Aways
• A framework for data driven decision making
• A model for addressing data needs and the capacity to make it real
• An outline for a data plan with action steps
• Useful tools and approaches
• Resources to share and leverage
3
Data Driven Decision Making (DDDM)What it is and what it does
• Using data to inform or support decisions
• Having the right data in the right format at the right time for the right purpose
• Enables a learning organization
• Is a key element of transparency and accountability
• Helps remove barriers to organizational change
4
Guiding Questions
• What is meant by a culture that supports data-driven decision making (DDDM)?
• What things need to exist that promote the effective use of data?
• What barriers exist to building and sustaining a DDDM culture?
• What are some measureable benefits of DDDM?
5
Adapted from CoSN Annual Conference, March 10-12, 2009
Decision Making Model by Rich Howard
-History
-Priorities -Mission
Integrated Knowledge and data
-Information -Facts -Opinion
Organizational Knowledge
Data
Data Informed Decision (Intelligence)
Decision
Making
Assessment
Decision Making Process
6
Metrics and Management Framework
7
8
Key Performance IndicatorsGrowth Indicators Quality Indicators Efficiency Indicators
Instruction • Entering students• Enrollment• Enrollment per major• Scholarship expenditures• SCH/FYE• Degrees awarded• Instructional tuition
• ACT and GRE scores• NHS top 10%• Ugrd. GPA of grads• 1st year retention• Degree completion• Dean’s List• Course Satisfaction• Time-to-degree• Graduate fellowships
• Instructional funds FYE• Faculty workload• SCH per faculty• Instructional finds FTE• Instructional revenue vs. Cost• Instructional cost• Instructional revenue• Support $ FTE
Research • # proposals submitted• # grants awarded• Indirect cost recovery• Size of awards• Award $ per faculty FTE• Grad expenditures• Sponsored funds• Type of proposals/awards
• Award sponsorship• Grants received vs. submitted• Trend in ICR generated• Refereed journal articles• NRC Ranking• Essential Science Indicators
• MAES vs. sponsored funding• MAES per FTE• Sponsored expenditures• Core support to sponsored
expenditures• Return on input
Extension • Annual revenue growth• Number of teaching units• Revenue/State support• Event offerings• Product sales• Access opportunities
• Integrated grant success• Refereed journal articles• Value-based fee structure• Business plans• Evaluation projects
• MAES vs. sponsored funding• Sponsored proposals vs. awards• Sponsored expenditures• Core support to sponsored
expenditures• Support $ FTE
Instructional KPI Example
9
Integrated Data Example
10
Integrated Data Example
11
Dashboard Example
12
Dashboard Example
13
Source: Performance Management Scorecards and Dashboards for IT Operations Data, Rex Parker Microsoft 2008
Dashboard Example
14
Source: Microsoft BI project 2008
Operational vs Strategic
• Most data at the U of MN originates in transactional systems (generally PeopleSoft)
• Operational and strategic decisions, however, require the data to be in different forms and follow different rules
15
Operational Questions
• What’s happening right now
• What action needs to be taken
• Choices generally pre-determined and “triggered” by data
• Interest is usually in individual cases and current values
• The closer to “real time” the better
16
Strategic Questions
• What is the trend?• What plans should we make?• Choices generally open for discussion
and debate• Interest is generally in aggregates and
trends• Data needs to remain unchanged over
time
17
Beyond List Reporting
• What if analysis
• Forecasting
• Statistical analysis
• Predictive modeling
• Alerts and triggers
• Dashboards and scorecards
• Actionable insight
18
Translating data into action
• Define the decision to be made or question to be answered
• Identify data requirements and availability
• Gather, format and consolidate data
• Analyze to generate conclusions
• Take appropriate actions
19
Building Capacity Enabling Data Driven Decision Making
• Engaged, supportive leadership
• Effective staffing
• Training and professional development
• Common language
• Collaborative environment
• Modest IT infrastructure
20
21
Reporting and Analytics(where we have been where we are going)
22
Audience and Complexity
Pre-defined Reporting
User-defined Reporting
Ad-hoc
(OLAP)Adv. Analytical
Mining
UserUser BaseBase
CCoommpplleexxiittyy
Analysis and recommendations for a specialized audience
Basic facts for a general audience
23
ReportsData Warehouse
Reporting and Analysis ServicesTurning data into useful information
TrainingLearning how to use data to make informed decisions.
State and Federal ReportingMeeting reporting compliance
DisseminationSharing data with the community (ie: report cards)
School Interoperability Framework & IMS
Components of a Data Based Decision Making System
SIS
Finance
Assessment
Instruction
Applications
Personalized Instruction
Source: US Department of Education, 2003.
Example of Data, Reporting and Analysis Architecture
25
Source: Technical Evaluation of Business Intelligence-Envisioning the Future White Paper, Knowledge Integrity, Inc 2008
END USER TOOLS & PERFORMANCE MANAGEMENT APPS
ExcelExcel PerformancePoint PerformancePoint ServerServer
BI PLATFORM
SQL Server SQL Server Reporting ServicesReporting Services
SQL Server SQL Server Analysis ServicesAnalysis Services
SQL Server DBMSSQL Server DBMS
SQL Server Integration ServicesSQL Server Integration Services
SharePoint ServerSharePoint Server
DELIVERY
ReportsReports DashboardsDashboards Excel Excel WorkbooksWorkbooks
AnalyticAnalyticViewsViews ScorecardsScorecards PlansPlans
Mind Mapping
“A mind map is a diagram used to represent words, ideas, tasks, or other items linked to and arranged around a central key word or idea. Mind maps are used to generate, visualize, structure, and classify ideas, and as an aid in study, organization, problem solving, decision making, and writing.”
27Source: http://en.wikipedia.org/wiki/Mind_Mapping
Mind mappingCustomer Service
28
Mind mappingAttacking Problems
29
Mind Mapping Activity
Outline key elements or components of effective data driven decision making– Identify key elements and how they link– Feeds your data plan– Work independently or in groups
30Source: http://en.wikipedia.org/wiki/Mind_Mapping
31
Guiding Principles for Implementation (from PEL)
• Strategic Communication
• Collaboration
• Staff Development
• Transparency
• Criteria for Decision Making
• Reflection and Feedback
Source: Implementing Administrative Metrics, June, 30, 2008, PEL 2008 cohort32
A Framework for Success
Source: Implementing Administrative Metrics, June, 30, 2008, PEL 2008 cohort
33
Collaborative Organizational Structure
Source: Implementing Administrative Metrics, June, 30, 2008, PEL 2008 cohort
34
Embed in Existing Management Processes
Source: Implementing Administrative Metrics, June, 30, 2008, PEL 2008 cohort
35
36
37
38
Critical Factors for Successful Implementation
39
Alignment
• Being more strategic in initiatives and foster greater “alignment”
• Alignment means having a stronger connection between the purposes and functions of units and initiatives and the strategic goals of the organization
• Funding is directed toward initiatives that promise progress on strategic goals
40
41
Values
Mission
Vision
Strategic perspectives
Strategic themes and results
OBJECTIVES
Strategy map
Performance measures & targets
Strategic initiatives
Org Assessment(SWOT, Those We Serve)
30,000
25,000
15,000
Ground
Strategic Altitude What we stand for / our code of conduct
Who we are / why we exist / key features of our organization
Emotionally inspiring picture of future success for our organization
Lenses to ensure balance of our efforts
Logic of a Strategic Management System Utilizing Balanced ScorecardLogic of a Strategic Management System Utilizing Balanced Scorecard
Main areas of focus to best serve “Those We Serve”
Strategic effort areas to achieve success /
get results
Map of how objectives will achieve
success (organized into perspectives)
How we will know if we are
achieving desired results
Specific projects that
contribute to desired results
42
STRATEGY MAPDefine & communicate priorities and focus
BALANCED SCORECARDMeasure performance against priorities
MISSIONWhy we exist
VALUESWhat’s important to us
VISIONWhat we want to be
STRATEGYOur game plan
EMPOWERMENT / PERSONAL OBJECTIVESMotivate employees
INITIATIVE and PLANNING PROCESSManage actions and resources to drive change
STRATEGIC OUTCOMES
SatisfiedSHAREHOLDERS
Delighted CUSTOMERS
Efficient and EffectivePROCESSES
Motivated & PreparedWORKFORCE
The Strategy Map and the Balanced Scorecard bridged the strategy implementation gap. They link leadership at the top to management of initiatives, process improvements and
employees’ everyday actions.
43
A commitment to excellence
44
Exceptional Students
Strategic Objectives:
• Make the University of Minnesota a destination of choice for diverse students who reflect the diversity of our community and world, and are sought after because of their unique talents, skills and experiences
• Educate and support all of our students to assume positions of leadership in the community, state, nation and the world
• Provide our students with the most advanced, sophisticated, and comprehensive technology tools to enhance their learning experience
• Globalize our students’ experience, recruit students from around the world and provide an education to prepare students to become global citizens and leaders
Strategic Result:
Recruit, educate,
challenge, and
graduate outstanding
students who become
highly motivated
lifelong learners,
leaders, and global
citizens
45
Exceptional Faculty and Staff
Strategic Objectives:
• Recruit identify, support and reward stars on the rise
• Create a robust culture of collaboration that encourages and rewards boldness, imagination, and innovation.
• Hire, develop and place diverse faculty and staff in positions which match their skills and abilities with organizational needs
• Strengthen the performance evaluation and reward systems to fully engage, motivate and challenge faculty and staff
• Significantly increase the number of faculty receiving awards of distinction
Strategic Result:
Recruit, mentor,
reward, and retain
world-class faculty and
staff who are
innovative, energetic,
and dedicated to the
highest standards of
excellence
46
Exceptional Organization
Strategic Objectives:
• Adopt best practices and embrace enterprise standard business practices processes and technology to achieve efficient, effective and productive operations
• Promote nimble decision-making using data, information, research and analysis
• Achieve a shared services administrative structure
• Align resources to support strategic priorities
• Commit to service and results that are best among peers
Strategic Result:
Be responsible
stewards of resources,
focused on service,
driven by performance,
and known as best
among our peers
47
Exceptional Innovation
Strategic Objectives:
•Foster an environment of creativity that encourages evolution of dynamic fields of inquiry
•Invest in strong core disciplines while supporting cross disciplinary, collaborative inquiry
•Fully leverage our academic, research and community partnerships and alliances to provide leadership in a global context
•Develop innovative strategies to accelerate the efficient and effective transfer and utilization of knowledge for the public good
Strategic Result:
Inspire exploration of
new ideas and
breakthrough
discoveries that address
the critical problems
and needs of the
University, state, nation,
and world
48
Organizational Alignment ExampleOffice of Information Technology (1st)
49
Organizational Alignment ExampleOffice of Information Technology (2nd)
50
Organizational Alignment ExampleOffice of Information Technology (3rd)
51
Organizational Alignment ExampleOffice of Information Technology (4th)
52
Organizational Alignment ExampleOffice of Information Technology (5th)
53
Perspectives for BalancePerspectives for Balance
Objectives Should Be Balanced Among Four Perspectives
(1) The People We Serve
How can we best meet the needs/wants of “Those We Serve”?
(2) Organizational Resource Productivity
How can we maximize the efficiency and effectiveness of our resources?
(3) Mission Driven Processes
How can we improve the way we do our work to increase productivity or improve service?
(4) People and Capacity
How can we engage our employees and increase our capacities (IT, etc.) in order to prepare our
organization to achieve excellence?
Externally facing / Outputs
Internally facing / Inputs
54
Relationships Ensure Comprehensive MapsRelationships Ensure Comprehensive Maps
• Objectives in perspectives should show a cause-effect relationship
• Reading top to bottom, answers the question: “What do we have to do (objective) to achieve this desired result (objective)?
• Reading bottom to top, answers the question: “If we do this (objective), then we can achieve that (objective)
• Arrows are utilized to show the inter-relationship between objectives
Note: the University will use different “perspective” names and order!
55
Barriers to Effectively Using Data for Decision Making
• Trust of the data
• Resources
• Communications
• Training
• Time
56
Major Barriers for Using Data for Decision Making
• Lack of training: 50% • Interoperability—systems that are unable to share or exchange
data: 42% • Lack of understanding of what to do with the data: 39% • Absence of clear priorities on what data should be collected:
36% • Failure to collect data in a uniform manner: 35% • Outdated technology/legacy systems: 31% • Low quality data – inaccurate or incomplete: 24% • Timing of data collection: 24% • User interface is too complicated to understand reports: 22%
Source: A survey conducted by Grunwald & Associates on behalf of CoSN in 2004
57
Developing a Data Plan of Action
58
Tools to Leverage
• Before and after
• Mind mapping
• Critical factors for success
• Force field analysis
• SWOT
• Threat / opportunity matrix
• Stakeholder map
59
Force Field Analysis
60
Source: http://accel-team.com/techniques/force_field_analysis.html
Threat: What is the impact of not adopting DDDM in our group or college?
Opportunity: Identify the opportunities for adopting DDDM
Threat Opportunity
Short Term
Threat / Opportunity Matrix
61
Long Term
Balanced Scorecard
Source: http://images.google.com/
62
Baldrige National Award
Source: Baldrige National Quality Program 2009-2010
63
The Big Picture
64
65
Six Sigma Basic Steps
• Define – What is the problem
• Measure – Acquire data
• Analyze – Analyze data and determine what the data
• Improve – Incorporate process improvements
• Control – Continue to monitor progress and adjust
Project BenefitsOpportunity Statement
Goal Statement(s) Project Scope
Project Plan Team Selection
66
Elements of a Project Charter
Books of Interest Include• Competing on Analytics by Davenport and
Harris• Five Key Principles of Corporate
Performance Management by Paladino• Business Intelligence Competency Centers
by Miller, Brautigam and Gerlach• Transforming Performance Measurement
by Spitzer• The Profit Impact of Business Intelligence
by Williams and Williams
67
Books of Interest Include(Continued)
• Key Performance Indicators by Parwmenter• Collapse of Distinction by McKain• Advancing Campus Efficiencies by
Johnstone• Moneyball by Lewis• People, Processes and Managing Data by
McLaughlin and Howard
68