ellig 1st scorecard fiscal 1999
DESCRIPTION
TRANSCRIPT
Preparing Performance Reports:Lessons from the First Mercatus Scorecard
Jerry Ellig
Senior Research Fellow
Mercatus Center
George Mason University
Arlington, VA
www.mercatus.org
What is the Scorecard?
• Annual ranking, first published in May 2000
• Evaluation of 24 CFO Act agencies’ reports
• Assessed quality of reporting, not results
• Examined 12 criteria related to transparency, public benefits, and forward-looking leadership
Transparency
• Is the report accessible and easy to find?
• Can a layperson read and understand it?
• Are data reliable, credible, and verifiable?
• Are baseline and trend data included?
Public Benefits
• Are goals stated as results?
• Do measures focus on results, or activities?
• Is causation clear?
• Are goals and results linked to costs?
Forward-Looking Leadership
• Does agency explain how it improves our quality of life?
• Does report explain failures to meet targets?
• Does it identify major management challenges?
• Does it describe changes to improve performance next year?
Easy for a layperson to read?
• Plain English
• Avoid acronyms!
• Headings, graphics facilitate quick scan
Role model: USAID
Credible, reliable, verifiable data
• Validation procedures
• Limitations of data
• External review/verification
Role model: Education’s frank discussion
Baseline and trend data
• Are they included?
• Can the reader easily track progress?
DOD: 3 years of actual data, plus goals for next 3 years
Goals/measures stated as results
Result: Desired benefit produced (or harm avoided) due to agency’s programs
Examples:• Cleaner air vs. enforcement actions• Safer roads vs. seat belt use• Reduced disease vs. increased access to care
DOT’s strategic goals
• Safety
• Mobility
• Economic growth
• Environment
• National security
DOT measures focused on results
Safety Fatality and injury rates
Mobility Congestion, condition of pavement and runways
Economic growth Openness of international aviation markets
Environment Aircraft noise exposure
National security Sealift capacity, military readiness
Causation
• Correlation is not causation
NSF assumes funding causes breakthroughs
• Multiple causes: Identify portion of improvement caused by the agency
USAID: multiple causes of economic growth
Goals and results linked to costs?
VA juxtaposes costs and performance measures
• Cost per patient in Veterans Health
• Cost per pension claim
• Admin. Costs per housing loan
• Included unfunded liabilities
Quality of life
• Focus on results makes this easier to explain (e.g., DOT)
• “What the department is doing” vs. “What citizens are getting.”
• How does avoidance of nuclear accidents make the U.S. a better place to live?
Explanation of failure
• First step is admitting and reporting failures!
• Don’t move the goalposts after the game
Role model: USAID
Major management challenges
• Identified by GAO, IG, agency’s own management
• Most reports are silent on this
Role models: USAID, DOT, Veterans
Changes for next year
• GPRA is a management tool, not just a reporting requirement
DOT combined performance report with 2001 strategic plan
What won’t work
Going through the motions
Recitation of activities with no plot
Salesmanship and hype
Dishonesty
What will work
• Honesty, candor, and humility
• Willingness to recognize reality
• Willingness to learn from experience
• Commitment, rather than compliance • Use of GPRA as a management tool
For more information…
The complete Mercatus Performance Report Scorecard can be reached through our home page ---
www.mercatus.org