1

Decision SupportBuilding Analytical

Skills

ASMC – PDIOrlando, FLJune 3, 2010

Ron Maccaronirmaccaroni@cox.net

2

Decision Support

Definition

Enabling decision makers to make informed decisions that wisely and better utilize resources, and improve program and

operational effectiveness.

Why is this important?

3

Why Should We Care About Better Decision Support?

Huge annual deficits contributing to an $13+ trillion debt.

Trillions in unfunded liabilities. Medicare, Medicaid, and Social Security

costs will increase dramatically. Uncertain world climate can lead to future

unexpected events that increase financial burden on Federal budget/programs.

Growing financial burden on the States and Localities.

4

Objectives of Presentation Emphasize the need for relevant,

data-driven analysis to support decision making.

Demonstrate the need and benefits of using a structured approach to analysis.

Discuss an example of a structured analytical approach.

5

The New Watchwords Accountability and Transparency

GAO changed its name from “General Accounting Office” to “Government Accountability Office.”

Accounting and auditing professions are now referred to as the “accountability professions.”

Many politicians, academicians, private sector leaders, and “citizen intermediaries” refer to both terms.

Laws passed that call for either or both.

6

The New Playing Field What’s Different Today?

Focus on achieving and measuring results!

Agencies must demonstrate they are getting results when preparing budgets and reporting annually.

Citizens are more interested in seeing results from public expenditures and meeting their expectations.

Greater use of the Internet for public access to information on government operations.

7

Working for the People!

The American people deserve a Federal Government that: Respects their tax dollars, and uses them

effectively and efficiently. Is transparent, fair, and responsive. Is constantly looking to streamline what works

and to eliminate what does not.

Budget of the U.S. GovernmentFiscal Year 2011

8

Drivers Of Stronger Analysis To Support Decision-making

Management Reform Legislation

RESULTS

• A Broader Role for the CFO

NEW PLAYING FIELDDRIVERS

Higher Leadership and Accountability Expectations

The Management Process Has Changed

• Emphasis on More Timely and Better

Accounting for Resources

• Focus on Performance and Results--Tied To Budget Decisions

• More Difficult Decisions

Presidential Initiatives

• More and Better Data Available

• Need and Opportunity for Stronger Analysis

Ahead: Large and Growing Federal Budget Deficits

9

Federal Management Process Has Changed

PLANNING: GPRA

BUDGETING: GPRA

ASSESSING RESULTS:Various

REPORTING: CFO Act +,

GPRA, FMFIA

10

The MissionShould Drive Decision Making

Starting Decisions . . . What should / do I need to do? For whom? Why is it important – what impact do I

want to have?

Drive Subsequent Decisions . . . How will I do it? When? At what cost?

11

Decision Making and Performance Management

Execute

Goal/Objective

Programs Outcomes – Results

Customers Outputs – Products/Services Activities Inputs – Resources

Budget

Key questions: Is the resource use having the intended results/impact? Is the investment worth the results? How good is the management process?

12

Where Analysis is Needed to Support Decisions

Problem/IssueBudgetary ResourcesFinancial ConditionBusiness ProcessesCost/BenefitPolicy/Program Delivery AlternativesProgram Operations and Impact

13

How Was the Analysis Done?

A decision-maker should always know how an analysis was done in order to:

Know how solid it is.

Be comfortable with and be able to defend any decision made based on the analysis.

14

So . . . What’s in YOUR Analysis?

Questions on which your future may depend: In conducting analysis, how do you

know: What question/issue/concern/need you

should focus on? How reliable, verifiable, relevant, sufficient,

and consistent the data you depend on are? The best type of analysis to perform?

Are you at risk if you don’t know?

15

Keys to Decision Support Critical Thinking Creativity Broad Perspective Analysis

Qualitative Quantitative

Communications Skills

16

GuidelinesAsk questions; be willing to wonder.Define the problem / concern / question /

need.Examine the evidence.Analyze assumptions and biases.Avoid emotional reasoning.Don’t use either/or thinking, or over

generalize.Consider other interpretations.Tolerate uncertainty/questions.

17

What it Means to YOU!

Today there are challenges in all areas of government.

But!

There are also opportunities for those who are prepared.

18

Structured Approach to Conducting Analysis

Objective and systematic way of collecting, examining and reporting information to support decision-making.Conceptual way of thinking.Applies to all aspects of managing

programs, finances, and operations.

19

Structured Approach to Conducting Analysis (cont’d) To be effective, analysts must:

Understand organization’s business. Understand how analytic “need” relates to the

business. Apply sound, thorough analytical methods

specifically tied to the “need.” Provide useful information to support decision-

making. Analysis for decision-making

leads to continual line of questioning.

20

Five Step Structured Analytic Approach

Step 5 – Present Results

Step 4 – Conduct Analysis

Step 3 - Collect Data

Step 2 - Identify Data Sources

Step 1 - Define Questions

21

Step 1Defining Questions

Question will drive the entire analysis.

Analysis must zero in on user’s need. Critical considerations:

The user’s business and role in it.What the user needs to know and

why.How the results will be used for

decision-making. Thoroughness requires asking 6 key

questions = “what,” “where,” “when,” “who,” “how,” and “why”

22

Getting the Question Right

How effective is the highway patrol at increasing safety?

23

Getting at the Real Question How effective is the highway patrol . . .

Do resources used give adequate security? Are fatalities per 100,000 miles going down? Is ratio of convictions/violations going up?

. . . at increasing safety? Is the number of accidents going down? Are there fewer safety-related incidents? Is there less crime on the roads?

24

Step 2Identifying Data Required

Key question = “What data do you need to answer the question?”

Data collected from:Primary sources = you collectSecondary sources = others collected

Sources and quantity of data gather depend on:The questions to be answered.The availability of data.The time allowed for the analysis.

If data doesn’t exist, revisit questions.

25

Consider Three Data Categories Budgetary

Prospective Retrospective

Program Strategic and annual plans Program design, operation, and results

Financial Cost of operation and financial position (assets,

liabilities, net worth, revenue, expenses) Cost data by office, program, function, output,

or outcome

26

Assessing Data QualityAssumption = All decision are data

driven.

The Quality Axiom

The quality of the decisioncan be no better than

the quality of the data supporting the decision!

27

Step 3Collecting the Data

(Sounds like the easy step?) Key questions in deciding how to collect data:

Where and how are you going to get the data?How will you use it to answer the questions?

Common data sources:Records = reports, manual/automated files.People = within and outside the organization.

Time available key to source and amount. Verification important = reliability, completeness,

accuracy, relevancy.

28

Data Collection Hierarchy TIME REQUIREMENT

Standard Agency Reports Low

Automated Systems

Manual Files

Personal Interviews

Focus Groups

Surveys High

29

Step 4Analyzing the Data

Analytic method = steps taken to understand, display, or interpret data.

Methodology dependent on question(s).Methods numerous and varied.

Important to know purpose/function of each and when to use them.

Review results of analysis:Did it address question(s)?How sensitive is it to internal/external

influences?

30

Step 4 Axiom

The rigor of the analysis must reflect the risk of the potential

impact of the decision!

31

Conduct Analysis Methods Screening

There Categories - Which fits question(s)? Descriptive = What exists?

How was the high-tech training program implemented?

Normative = What exists compared to what was expected? Were job-placement goals of the high-tech training

program met?

Impact = Did the initiative achieve the intended results? Did the high-tech training program have an effect on

job-placement rates?

32

Determining the Type of Analysis Primary Data Analysis

You collect, assemble, and analyze data. Secondary Data Analysis

You use data gathered and in some cases analyzed prior to your own analysis.

Evaluation Synthesis You use the combined results from two or more

previous analysis.

33

Analytical Methods and Their Uses

Category Content Trend

Statisti-cal

Case Study

Bench-marking

Before-After

Field Experiment

Problem/Issue X X X X X - -

Financial Condition and Operations X X X X X - -

Budgetary Resources X X X X - - -

Business Processes X X X X X X -

Cost-Benefit and Cost-Effectiveness X X X X X X -

Policy and Program Design Alternatives X X X X X X X

Program Operations X X X X X - -

Program Impact X - X X - X X

34

Step 5Reporting Analytic Results

Important phase of process. Impact decision-making; bring about change.

Reporting methods (written vs. oral vs. video) have different advantages.

Goal = convince audience that work done and results reported is reasonable, appropriate, and actionable. Answering analytic objective, i.e. question, is most critical

goal. Remember what drove the analysis:

Who is the user/customer and what is their role? In what subject/issue is the user interested? What does the user want to know? What does the user plan to do with the results?

35

Structured Approach Steps Interactive and Overlapping

Each step dependent upon the successful completion of previous step.

“Recheck” step plans for and results of each step to ensure synchronization with previous steps.

As complexity/comprehensiveness of one step grows, so grows complexity/comprehensiveness of succeeding steps, and time required.

36

Structured Approach Steps Interactive and Overlapping (cont’d)

37

Revalidating Questions(Continuous throughout analysis!)

DefineQuestions

IdentifyData

Sources

CollectData

Conduct Analysis

ReportResults

Regularly re-check and validate questions. Questions may need to be modified, dropped, redefined or added.

Analysis Feedback Loop

38

Key Points to Remember

Accountability expectations drive need for critical analysis.

Structured approach = objective/systematic way of collecting, examining, and reporting information.

Five step structured approach: Identify question(s) Identify data Collect data Analyze data Report results

Steps highly interrelated – drives need to revisit results of each step throughout process.

39

Summary

AnalysisGrowing need = support difficult decisions.Growing importance = support

performance and financial accountability.Best if:

Follows a structured approach.Based on user’s/customer’s questions/need. Impacts decision-making/results.

Ron Maccaroni -- rmaccaroni@cox.net

40

Remember . . .

African Proverb Every morning in Africa, a gazelle wakes up. It

knows it must run faster than the fastest lion or it will be killed. Every morning a lion wakes up. It knows it must outrun the slowest gazelle, or it will starve to death. It doesn’t matter whether you are a lion or a gazelle. When the sun comes up, you’d better start running!