1 analytic techniques for decision support asmc – pdi orlando, fl june 3, 2010 ron maccaroni...
TRANSCRIPT
![Page 2: 1 Analytic Techniques For Decision Support ASMC – PDI Orlando, FL June 3, 2010 Ron Maccaroni rmaccaroni@cox.net](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649e9e5503460f94ba0147/html5/thumbnails/2.jpg)
2
Decision Support
Definition
Enabling decision makers to make informed decisions that wisely and better utilize resources, and improve program and
operational effectiveness.
Why is this important?
![Page 3: 1 Analytic Techniques For Decision Support ASMC – PDI Orlando, FL June 3, 2010 Ron Maccaroni rmaccaroni@cox.net](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649e9e5503460f94ba0147/html5/thumbnails/3.jpg)
3
Why Should We Care About Good Decision Support?
Huge annual deficits contributing to a $13+ trillion, and growing, federal debt.
Trillions in unfunded liabilities. Medicare, Medicaid, and Social Security
costs will increase dramatically. Uncertain world climate can lead to future
unexpected events that increase financial burden on Federal budget/programs.
Growing financial burden on the States and Localities for services and social needs.
![Page 4: 1 Analytic Techniques For Decision Support ASMC – PDI Orlando, FL June 3, 2010 Ron Maccaroni rmaccaroni@cox.net](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649e9e5503460f94ba0147/html5/thumbnails/4.jpg)
4
What It Means To YOU!
Today there are challenges in all areas of government.
But!
There are also opportunities for those who are prepared.
![Page 5: 1 Analytic Techniques For Decision Support ASMC – PDI Orlando, FL June 3, 2010 Ron Maccaroni rmaccaroni@cox.net](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649e9e5503460f94ba0147/html5/thumbnails/5.jpg)
5
Objectives Of Presentation Emphasize the need for relevant,
data-driven analysis to support decision making.
Demonstrate the need and benefits of using a structured approach to analysis.
Discuss an example of a structured analytical approach.
![Page 6: 1 Analytic Techniques For Decision Support ASMC – PDI Orlando, FL June 3, 2010 Ron Maccaroni rmaccaroni@cox.net](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649e9e5503460f94ba0147/html5/thumbnails/6.jpg)
6
The “Watchwords” Accountability and Transparency
GAO changed its name from “General Accounting Office” to “Government Accountability Office.”
Accounting and auditing professions are now referred to as the “accountability professions.”
Many politicians, academicians, private sector leaders, and “citizen intermediaries” refer to both terms.
Laws passed that call for either or both.
![Page 7: 1 Analytic Techniques For Decision Support ASMC – PDI Orlando, FL June 3, 2010 Ron Maccaroni rmaccaroni@cox.net](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649e9e5503460f94ba0147/html5/thumbnails/7.jpg)
7
What’s Different Today? The New Playing Field
Focus on achieving and measuring results!
Agencies must demonstrate they are getting results when preparing budgets and reporting annually.
Citizens are more interested in seeing results from public expenditures and meeting their expectations.
Greater use of the Internet for public access to information on government operations.
![Page 8: 1 Analytic Techniques For Decision Support ASMC – PDI Orlando, FL June 3, 2010 Ron Maccaroni rmaccaroni@cox.net](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649e9e5503460f94ba0147/html5/thumbnails/8.jpg)
8
Working For The People!
The American people deserve a Federal Government that: Respects their tax dollars, and uses them
effectively and efficiently. Is transparent, fair, and responsive. Is constantly looking to streamline what works
and to eliminate what does not.
Budget of the U.S. GovernmentFiscal Year 2011
![Page 9: 1 Analytic Techniques For Decision Support ASMC – PDI Orlando, FL June 3, 2010 Ron Maccaroni rmaccaroni@cox.net](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649e9e5503460f94ba0147/html5/thumbnails/9.jpg)
9
Drivers Of Stronger Analysis To Support Decision Making
Management Reform Legislation
RESULTS
• A Broader Role for the CFO
NEW PLAYING FIELDDRIVERS
Higher Leadership and Accountability Expectations
The Management Process Has Changed
• Emphasis on More Timely and Better
Accounting for Resources
• Focus on Performance and Results--Tied To Budget Decisions
• More Difficult Decisions
Presidential Initiatives
• More and Better Data Available
• Need and Opportunity for Stronger Analysis
Ahead: Large and Growing Federal Budget Deficits
![Page 10: 1 Analytic Techniques For Decision Support ASMC – PDI Orlando, FL June 3, 2010 Ron Maccaroni rmaccaroni@cox.net](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649e9e5503460f94ba0147/html5/thumbnails/10.jpg)
10
What Is The Focus OfDecision Making
Starts The Mission . . What should / needs to be done? For whom? Why is it important – what is the desired /
expected impact?
Which Drives Subsequent Decisions . . How should it be done? By whom? When? At what cost?
![Page 11: 1 Analytic Techniques For Decision Support ASMC – PDI Orlando, FL June 3, 2010 Ron Maccaroni rmaccaroni@cox.net](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649e9e5503460f94ba0147/html5/thumbnails/11.jpg)
11
. . . Leading to Performance Management Decisions
Execute
Goal/Objective
Programs Outcomes – Results
Customers Outputs – Products/Services Activities Inputs – Resources
Budget
Key questions: Is resource use having intended results/impact? Is the investment worth the results? How good is the management process?
![Page 12: 1 Analytic Techniques For Decision Support ASMC – PDI Orlando, FL June 3, 2010 Ron Maccaroni rmaccaroni@cox.net](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649e9e5503460f94ba0147/html5/thumbnails/12.jpg)
12
Typical Types Of Analysis Supporting Decisions
Problem/IssueBudgetary ResourcesFinancial ConditionBusiness ProcessesCost / BenefitPolicy / Program Design AlternativesProgram Operations or Impact
![Page 13: 1 Analytic Techniques For Decision Support ASMC – PDI Orlando, FL June 3, 2010 Ron Maccaroni rmaccaroni@cox.net](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649e9e5503460f94ba0147/html5/thumbnails/13.jpg)
13
How Was The Analysis Done?
A decision-maker should always know how an analysis was done in order to:
Know how solid it is.
Be comfortable with and be able to defend any decision made based on the analysis.
![Page 14: 1 Analytic Techniques For Decision Support ASMC – PDI Orlando, FL June 3, 2010 Ron Maccaroni rmaccaroni@cox.net](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649e9e5503460f94ba0147/html5/thumbnails/14.jpg)
14
So . . . What’s In YOUR Analysis?
In conducting analysis, how do you know: What question / issue / concern / need
you should focus on? How reliable, verifiable, relevant,
sufficient, and consistent the data you depend on are?
The best type of analysis to perform? Are you at risk if you don’t know?
![Page 15: 1 Analytic Techniques For Decision Support ASMC – PDI Orlando, FL June 3, 2010 Ron Maccaroni rmaccaroni@cox.net](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649e9e5503460f94ba0147/html5/thumbnails/15.jpg)
15
Basic Guidelines For Decision Support
Ask questions; be willing to wonder.Define the problem / concern / question /
need.Examine the evidence -- data.Analyze assumptions and biases.Avoid emotional reasoning.Don’t use either / or thinking, or over
generalize.Consider other interpretations.Tolerate uncertainty / questions.
![Page 16: 1 Analytic Techniques For Decision Support ASMC – PDI Orlando, FL June 3, 2010 Ron Maccaroni rmaccaroni@cox.net](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649e9e5503460f94ba0147/html5/thumbnails/16.jpg)
16
Structured Approach To Conducting Analysis
Objective and systematic way of collecting, examining / analyzing and reporting information to support decision-making.Conceptual way of thinking.Applies to all aspects of managing
programs, finances, and operations.
![Page 17: 1 Analytic Techniques For Decision Support ASMC – PDI Orlando, FL June 3, 2010 Ron Maccaroni rmaccaroni@cox.net](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649e9e5503460f94ba0147/html5/thumbnails/17.jpg)
17
Structured Approach To Conducting Analysis (cont’d)
To be effective, analysts must: Understand organization’s business. Understand how analytic “need” relates to the
business. Apply sound, thorough analytical methods
specifically tied to the “need.” Provide useful information to support decision-
making. Analysis for decision-making
leads to continual line of questioning.
![Page 18: 1 Analytic Techniques For Decision Support ASMC – PDI Orlando, FL June 3, 2010 Ron Maccaroni rmaccaroni@cox.net](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649e9e5503460f94ba0147/html5/thumbnails/18.jpg)
18
Five Step Structured Analytic Approach
Step 5 – Present Results
Step 4 – Conduct Analysis
Step 3 - Collect Data
Step 2 - Identify Data Sources
Step 1 - Define Questions
![Page 19: 1 Analytic Techniques For Decision Support ASMC – PDI Orlando, FL June 3, 2010 Ron Maccaroni rmaccaroni@cox.net](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649e9e5503460f94ba0147/html5/thumbnails/19.jpg)
19
Step 1Defining Questions
Critical starting point -- Question will drive the entire analysis.
Analysis must zero in on user’s need. Critical considerations:
The user’s business and role in it.What the user needs / wants to
know and why.How the results will be used for
decision-making. Thoroughness requires asking 6 key
questions = “what,” “where,” “when,” “who,” “how,” and “why”
![Page 20: 1 Analytic Techniques For Decision Support ASMC – PDI Orlando, FL June 3, 2010 Ron Maccaroni rmaccaroni@cox.net](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649e9e5503460f94ba0147/html5/thumbnails/20.jpg)
20
What Is The Question?
How effective is the highway patrol at increasing safety?
![Page 21: 1 Analytic Techniques For Decision Support ASMC – PDI Orlando, FL June 3, 2010 Ron Maccaroni rmaccaroni@cox.net](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649e9e5503460f94ba0147/html5/thumbnails/21.jpg)
21
What Are The RealReal Questions? How effective is the highway patrol . . .
Do resources used give adequate security? Are fatalities per 100,000 miles going down? Is ratio of convictions/violations going up?
. . . at increasing safety? Is the number of accidents going down? Are there fewer safety-related incidents? Is there less crime on the roads?
![Page 22: 1 Analytic Techniques For Decision Support ASMC – PDI Orlando, FL June 3, 2010 Ron Maccaroni rmaccaroni@cox.net](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649e9e5503460f94ba0147/html5/thumbnails/22.jpg)
22
Step 2Identify Data Required
Key question = “What data do you need to answer the question?”
Data collected from:Primary sources = you collectSecondary sources = others collected
Sources and quantity of data gather depend on:The questions to be answered.The availability of data.The time allowed for the analysis.
If data doesn’t exist, revisit questions.
![Page 23: 1 Analytic Techniques For Decision Support ASMC – PDI Orlando, FL June 3, 2010 Ron Maccaroni rmaccaroni@cox.net](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649e9e5503460f94ba0147/html5/thumbnails/23.jpg)
23
Consider Three Data Categories Budgetary
Prospective Retrospective
Program Strategic and annual plans and performance Program design, operation, and results
Financial Cost of operations and financial position
(assets, liabilities, net worth, revenue, expenses)
Cost data by office, program, function, output (product or service), or outcome (result)
![Page 24: 1 Analytic Techniques For Decision Support ASMC – PDI Orlando, FL June 3, 2010 Ron Maccaroni rmaccaroni@cox.net](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649e9e5503460f94ba0147/html5/thumbnails/24.jpg)
24
The Data Quality AxiomAssumption = All decision are data
driven.
The quality of the decisioncan be no better than
the quality of the data supporting the decision!
![Page 25: 1 Analytic Techniques For Decision Support ASMC – PDI Orlando, FL June 3, 2010 Ron Maccaroni rmaccaroni@cox.net](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649e9e5503460f94ba0147/html5/thumbnails/25.jpg)
25
Step 3Collecting The Data
(Sound like the easy step?) Key questions in deciding how to collect data:
Where and how are you going to get the data?Will it enable you to answer the questions?
Common data sources:Records = reports, manual/automated files.People = within and outside the organization.
Time available key to source and amount. Verification important = reliability, completeness,
accuracy, relevancy.
![Page 26: 1 Analytic Techniques For Decision Support ASMC – PDI Orlando, FL June 3, 2010 Ron Maccaroni rmaccaroni@cox.net](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649e9e5503460f94ba0147/html5/thumbnails/26.jpg)
26
Data Collection Hierarchy TIME REQUIREMENT
Standard Agency Reports Low
Automated Systems
Manual Files
Personal Interviews
Focus Groups
Surveys High
![Page 27: 1 Analytic Techniques For Decision Support ASMC – PDI Orlando, FL June 3, 2010 Ron Maccaroni rmaccaroni@cox.net](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649e9e5503460f94ba0147/html5/thumbnails/27.jpg)
27
Step 4Analyzing The Data
Analytic method = steps taken to understand, display, or interpret data.
Methodology dependent on question(s).Methods numerous and varied.
Important to know purpose / function of each and when to use them.
Review results of analysis:Did it address question(s)?How sensitive is it to internal / external
influences?
![Page 28: 1 Analytic Techniques For Decision Support ASMC – PDI Orlando, FL June 3, 2010 Ron Maccaroni rmaccaroni@cox.net](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649e9e5503460f94ba0147/html5/thumbnails/28.jpg)
28
Analysis Axiom
The rigor of the analysis must reflect the risk / significance of
the potential impact of the decision!
![Page 29: 1 Analytic Techniques For Decision Support ASMC – PDI Orlando, FL June 3, 2010 Ron Maccaroni rmaccaroni@cox.net](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649e9e5503460f94ba0147/html5/thumbnails/29.jpg)
29
Conduct Analysis Methods Screening
There Categories - Which fits question(s)? Descriptive = What exists?
How was the high-tech training program implemented?
Normative = What exists compared to what was expected? Were job-placement goals of the high-tech training
program met?
Impact = Did the initiative achieve the intended results? Did the high-tech training program have an effect on
job-placement rates?
![Page 30: 1 Analytic Techniques For Decision Support ASMC – PDI Orlando, FL June 3, 2010 Ron Maccaroni rmaccaroni@cox.net](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649e9e5503460f94ba0147/html5/thumbnails/30.jpg)
30
Determining The Type Of Analysis
Primary Data Analysis You collect, assemble, and analyze data.
Secondary Data Analysis You use data gathered and in some cases
analyzed prior to your own analysis. Evaluation Synthesis
You use the combined results from two or more previous analysis.
![Page 31: 1 Analytic Techniques For Decision Support ASMC – PDI Orlando, FL June 3, 2010 Ron Maccaroni rmaccaroni@cox.net](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649e9e5503460f94ba0147/html5/thumbnails/31.jpg)
31
Analytical Methods And Their Uses
Category Content Trend
Statisti-cal
Case Study
Bench-marking
Before-After
Field Experiment
Problem/Issue X X X X X - -
Financial Condition and Operations X X X X X - -
Budgetary Resources X X X X - - -
Business Processes X X X X X X -
Cost-Benefit and Cost-Effectiveness X X X X X X -
Policy and Program Design Alternatives X X X X X X X
Program Operations X X X X X - -
Program Impact X - X X - X X
![Page 32: 1 Analytic Techniques For Decision Support ASMC – PDI Orlando, FL June 3, 2010 Ron Maccaroni rmaccaroni@cox.net](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649e9e5503460f94ba0147/html5/thumbnails/32.jpg)
32
Putting The Steps Together - Example
Step 1Should $5M be added to inoculation
program advertising for face-to-face contact to enhance its effectiveness?Is current approach meeting its goals?
If not, why and what can/needs to be done?If yes, can more aggressive goals be
set/met?What additional benefits are expected
from face-to-face approach?
![Page 33: 1 Analytic Techniques For Decision Support ASMC – PDI Orlando, FL June 3, 2010 Ron Maccaroni rmaccaroni@cox.net](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649e9e5503460f94ba0147/html5/thumbnails/33.jpg)
33
Putting The Steps Together – Example (cont’d)
Step 2 Program, budgetary, and financial
Step 3 Reports, systems, files, and interviews
Step 4 Normative and impact Primary, secondary, and evaluation synthesis Content, trend, statistical, case study,
benchmarking, before-after, field experiment
![Page 34: 1 Analytic Techniques For Decision Support ASMC – PDI Orlando, FL June 3, 2010 Ron Maccaroni rmaccaroni@cox.net](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649e9e5503460f94ba0147/html5/thumbnails/34.jpg)
34
Step 5Reporting Analytic Results
Important phase of process.Impact decision-making; bring about change.
Reporting methods (written vs. oral vs. video) have different advantages.
Goal = convince audience that work done and results reported is reasonable, appropriate, and actionable.
Answering analytic objective, i.e. question, is most critical goal.
Remember what drove the analysis – customer’s need / interest / role, and planned use of results.
![Page 35: 1 Analytic Techniques For Decision Support ASMC – PDI Orlando, FL June 3, 2010 Ron Maccaroni rmaccaroni@cox.net](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649e9e5503460f94ba0147/html5/thumbnails/35.jpg)
35
Structured Approach Steps Interactive And Overlapping
![Page 36: 1 Analytic Techniques For Decision Support ASMC – PDI Orlando, FL June 3, 2010 Ron Maccaroni rmaccaroni@cox.net](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649e9e5503460f94ba0147/html5/thumbnails/36.jpg)
36
Revalidating Questions(Continuous throughout analysis!)
DefineQuestions
IdentifyData
Sources
CollectData
Conduct Analysis
ReportResults
Regularly re-check and validate questions. Questions may need to be modified, dropped, redefined or added.
Analysis Feedback Loop
![Page 37: 1 Analytic Techniques For Decision Support ASMC – PDI Orlando, FL June 3, 2010 Ron Maccaroni rmaccaroni@cox.net](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649e9e5503460f94ba0147/html5/thumbnails/37.jpg)
37
Key Points to Remember Accountability expectations drive need for “hard”
decisions, and therefore critical analysis. Structured approach = objective/systematic way
of collecting, examining, and reporting information.
Five step structured approach: Identify question(s) Identify data Collect data Analyze data Report results
Steps highly interrelated – drives need to revisit results of each step throughout process.
![Page 38: 1 Analytic Techniques For Decision Support ASMC – PDI Orlando, FL June 3, 2010 Ron Maccaroni rmaccaroni@cox.net](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649e9e5503460f94ba0147/html5/thumbnails/38.jpg)
38
Summary
AnalysisGrowing need = support difficult decisions.Growing importance = support
performance and financial accountability.Best if:
Follows a structured approach.Based on user’s/customer’s
questions/need.Impacts decision-making/results.
![Page 39: 1 Analytic Techniques For Decision Support ASMC – PDI Orlando, FL June 3, 2010 Ron Maccaroni rmaccaroni@cox.net](https://reader035.vdocuments.site/reader035/viewer/2022062423/56649e9e5503460f94ba0147/html5/thumbnails/39.jpg)
39
Remember . . .
African Proverb Every morning in Africa, a gazelle wakes
up. It knows it must run faster than the fastest lion or it will be killed. Every morning a lion wakes up. It knows it must outrun the slowest gazelle, or it will starve to death. It doesn’t matter whether you are a lion or a gazelle. When the sun comes up, you’d better start running!