nchrp report 666 (nchrp 8-70) - transportation...

NCHRP Report 666

(NCHRP 8-70)

For: TRB WebinarFebruary 23, 2011

Click to edit Master title style

1. Viewer Window

2. Webinar Control Panel

TYPE HERE

Our webinar presentation slides appear here on your computer screen

GoToWebinar Interface

Click to edit Master title styleCommonly asked question: Can I receive a copy of the presentation slides?

Please view your webinar reminder email for a link to the presenter’s slides.

After the webinar, you will receive a link to the recording of today’s session.

Click to edit Master title stylePDH Certificate Information

This webinar is valued at 1 Professional Development Hours.

You must register and attend the webinar as an individual.

For groups, only the person that registers and attends the session is eligible to receive a certificate.

Email Reggie Gillum: [email protected]

The Transportation Research Board has met the standards and requirements of the Registered Continuing Education Providers Program. Credit earned on completion of this program will be reported to RCEPP. A certificate of completion will be issued to each participant. As such, it does not include content that may be deemed or construed to be an approval or endorsement by NCEES or RCEPP.

Click to edit Master title styleCambridge Systematics Presentation Team

Randy Halvorson: 952-920-8141, [email protected]

Erik Cempel: 312-665-0223, [email protected]

Anita Vandervalk: 850-219-6388, [email protected]

Kim Hajek: 512-691-8509, [email protected]

Kelsey Ahern: 512-691-8506, [email protected]

6

Click to edit Master title styleAgenda

2:00 – 2:10 pm EST Purpose – Randy Halvorson

Research Approach – Randy Halvorson

2:10 – 2:45 pm EST Guide to Target Setting – Randy Halvorson/Erik Cempel

2:45 – 3:25 pm EST Guide to Data Management – Anita Vandervalk/Kim Hajek

3:25 – 3:30 pm EST Concluding Remarks and Additional Research Underway – Randy Halvorson

*Question and Answer Moderator – Kelsey Ahern

7

Click to edit Master title stylePurpose of Research (Targets)

Describe Framework for Performance-Based Resource Allocation (PBRA)

Identify the role of performance measure targets within PBRA

Outline the steps for setting performance targets

8

Click to edit Master title stylePurpose of Research (Data)

To provide a comprehensive description of the following components, which are needed to support (PBRA) decision-making

Data management

Information systems

Institutional arrangements

9

Click to edit Master title styleProject Approach

Interviewed 25+ public and private sector agencies on the use of performance measures and targets and their supporting data systems

Results compiled and documented in case studies

Lessons learned and best practices published in Guide for Target-Setting and Data Management

10

Click to edit Master title stylePerformance Management Framework

PBRA takes place within an overall Management Framework

11

Goals/Objectives

Performance Measures

Target Setting

Evaluate Programs and Projects

Allocate Resources

Budget and Staff

Measure and Report Results

Actual Performance Achieved

Qu

alit

y D

ata

Click to edit Master title styleInterrelationship Among Measures, Targets, Data

12

Performance

measures and

targets are integral

elements in PBRA

Data

management

programs are

used to

organize data

support

functions

Measures and

targets are

supported by

data and

information

systems

Click to edit Master title stylePerformance Measure Target

Definition:

A specific value for a measure that an agency would like to achieve.

These values set a performance level for each organization measure.

14

Click to edit Master title stylePerformance Measure Target

Example:

Goal Area – System Preservation

Performance Measure – International Roughness Index (IRI)

Target – 75% pavement good or better condition

15

Click to edit Master title styleSteps For Target Setting

Establish a Performance

Management Framework

Evaluate the Factors

Influencing

Target Setting

Select the Appropriate

Approach to

Target Setting16

Making the choice – Part 1


Providing the context


Establish Methods

for Achieving Targets

Track Progress

Towards Targets

Adjust Targets

Over Time17

Tracking investmentimpacts

Meeting the targets/making trade-offs

Refining the process


Establish a Performance

Management Framework

18

Goals/Objectives


Target Setting

Evaluate Programs and Projects

Allocate Resources

Budget and Staff

Measure and Report Results

Actual Performance Achieved

Qu

alit

yD

ata

Providing the context


Evaluate the Factors Influencing

Target Setting

19

Agency experience

Data availability

Financial resources

Stakeholder and public involvement

Political/legislative influence

Timeframe




Approach

to Target Setting

20

Edict

Expert Opinion

Benchmarking

Modeling

Hybrid/Collaboration

Making the choice –Part 2

Preferred Approach



Approach

to Target Setting

21

Hybrid/Collaboration - Preferred approach for final

selection

Target definition – specific value for a measure that an agency would like to achieve

Data- based classification of performance condition –develop a performance threshold table

Link performance condition to investment levels

Select specific target levels in collaboration with others

Cont’d

Click to edit Master title styleExample Application of Target Setting to Pavement

Goal Area – System Preservation

Performance Measure – International Roughness Index (IRI)

Target – 75% pavement good or better condition

22

Click to edit Master title stylePavement target selection process (Step 3A)

Establish a data-based classification of performance condition (Performance Threshold Table)

23

Click to edit Master title stylePavement condition target selection process (Step 3B)

Establish a relationship between performance measure and annual funding levels

24

Source: Highway Economic Requirements System- 2008 FHWA C & P

Click to edit Master title stylePavement condition target selection (step 3C)

Develop a chart showing annual pavement condition

25

0

10

20

30

40

50

60

70

80

90

100

2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015

Sy

ste

m H

igh

wa

y M

ile

s (%

)

Year

Target “% "Good”

Click to edit Master title stylePavement condition target selection (Step 3D)

Select specific target using collaborative process

Senior Agency Staff (i.e. Commissioner/Secretary and selected Division Heads)

Agency performance area experts (i.e. state and district pavement engineers)

Stakeholders (i.e. selected state and local elected officials, federal officials, freight haulers and the driving public)

26


Establish Methods


27

Integration into daily routines

Incentives

Personnel performance appraisals

Quality control and support

Trade-off analysis

Meeting the targets/making trade-offs


Establish Methods


28

Trade-off analysis

Define measures of effectiveness

Assess current performance

Develop funding scenarios

Format results for comparison

Cont’d


Track Progress

Towards Targets

29

Annual Attainment Reports

Annual “Snapshot”

Tracking investment impacts


Adjust Targets Over Time

30

Funding adjustments

Policy changes

Trade-offs over time

Refining the process

Click to edit Master title styleMn/DOT as Process Example for Implementing Targets within PBRA

31

Step 1

Identify Investment Needs

Step 2

Project Future Revenues

Step 3

Set Investment Goals

Step 4

Develop Investment Plan

Step 5

Prioritize Unfunded Investment Needs

20-year Highway Investment Plan

STIP 2009-2012

Mid-Range HIP 2013-2018

Long-Range 2019-2028

Total Unfunded

Statewide High Priority

Investment Goals:Balanced Program

Investments to Meet Performance Targets

Regional and Community Improvement Priorites

Revenue Outlook

Legislative Direction (Chapter 152)

System Performance

Stakeholder Input

Highway Investment Plan Development Process

Click to edit Master title styleTarget Setting to Support Performance-Based Resource Allocation

32

Click to edit Master title styleData Management Programs

Data Management Programs are used to:

Strengthen ability of data programs to support core business functions

Improve data quality throughout the organization

Protect data as an asset of the agency

Limit risks associated with loss of data and information

34

Click to edit Master title styleData Management Programs Strengthen ability of data programs to support business functions -Alaska

35

Click to edit Master title styleData Management Programs Link Business Functions to Data Programs - Virginia

36

Click to edit Master title styleSteps For Implementing/Strengthening Data Management Programs

Establish Need for Data Management/

Data Governance

Establish Goals for

Data Management

Assess Current State of

Data Programs37

Click to edit Master title styleSteps For Implementing/Strengthening Data Management Programs

Establish Data Governance

Programs

Use Technology to

Support Data Management

Link Data to Planning,

Performance Measures, &

Target-Setting Process 38

Click to edit Master title styleStep 1: Establish Need for Data Management/Data Governance

Establish need for data management/data governance

Demonstrate Return on Investment (ROI)

Support Performance Measurement and Target Setting

Governance supports data management from policy, practical, technical perspectives

Data Management Maturity Model assesses progression from un-governed to a governed state

39

“A strong Data Management program improves data quality and limits potential risks to the agency regarding the loss of critical data and information”


Relationship of Data Management, Governance, Stewardship

40

Source: Modified from Figure 1 Data Governance Team, The Data Governance Maturity Model, White Paper, RCGInformation Technology, 2008.


41

Figure 1.2 Data Governance Maturity Model

Reward

Low

High

Think Locally,

Act Locally

Think Globally,

Act Locally

Think Globally,

Act Collectively

Think Globally,

Risk

High

Low

Act Globally

Undisciplined Reactive Proactive Governed

People, Policies, Technology Adoption

Source: Data Governance Maturity Model, DataFlux Corporation, 2007.

Data Governance Maturity Model

Click to edit Master title style

42

Step 1: Establish Need for Data Management/Data GovernanceMaturity Model Matrix

Click to edit Master title styleStep 1: Establish Need for Data Management/Data Governance – Example

Washington State DOT

Data Council - set standards for data architecture, data modeling, etc.

Data Stewardship Council – address data issues across the department

Two types of data stewardship

Business stewardship – executives, managers, business operations

Technical stewardship – system architects, database administrators

Partnership of business and technical stewards strengthens data governance

43

Click to edit Master title styleStep 2: Establish Goals for Data Management

Establish goals for data management

Use brainstorming sessions with data program stakeholders

Identify business objectives of the agency

Identify business functions/services that support business objectives

Identify which business functions are supported by which data programs

Establish data collection and use policies

Establish data action plans to address needs and gaps in data and information

Establish risk management program to protect valuable data program assets

44

Click to edit Master title styleStep 2: Establish Goals for Data Management

Success factors

Start small with achievable goals

Use a data business plan

Manage expectations

Use Business-Use Case Models

Identify champions

45

Click to edit Master title styleStep 2: Establish Goals for Data ManagementExample - MNDOT

46

Mn/DOTData Business Plan

Framework

SELECTED EMPHASIS AREAS & OBJECTIVES

MISSION

STRATEGIC DIRECTIONS

POLICIES

BUSINESS PROCESSES

Provide the highest quality, dependable, multi-modal transportation system through ingenuity, integrity,

alliance, and accountability.

1. Traveler Safety

2. Infrastructure Preservation

6. & 7. Mobility

Plan ProduceOperate / Maintain

Support

Click to edit Master title styleStep 3: Assess Current State of Data Programs

Assess current state of data programs

Health Assessment

Highlight critical deficiencies and develop strategies to address the deficiencies

Surveys

Focus Groups

Workshops

Research Studies

Risk Management identifies and prioritizes the risks associated with current data programs

Risk prioritization matrix

47


Risk prioritization matrix

48

Data Program

Value Ranking

(Essential, Helpful, Not

Needed)

Addresses Key


Used To Meet

Federal Mandate

Used To Meet State

Mandate

Used to Support One or More

Defined Business

Emphasis Areas

Risk Level

Associated With Data Program

Program A

Program B

Program C

…

Program Z


Evaluating Data Programs

Accuracy

Timeliness

Completeness

Validity

Coverage

Accessibility

49

Click to edit Master title styleStep 3: Assess Current State of Data ProgramsExample -

Minnesota DOT (MNDOT)

Surveys - Used to assess strengths/weaknesses of current data programs

Focus groups – Used to prioritize needs/gaps identified in surveys

Data Program Workshops – Used to propose solutions to address needs/gaps in data programs that support investments and resource allocation

50

Click to edit Master title styleStep 4: Establish Data Governance Programs

Develop a Data Governance Model

Data Governance Framework

Data Governance Participants

51

Click to edit Master title styleStep 4: Establish Data Governance Programs -General Data Governance Framework

Data Governance Framework Data

Governance Board

Strategic Vision, Mission, Goals

For Data

Agency Data Programs

Data Users and

Stakeholders

Data Steward and

Custodians

Division(s) Mission(s) and

Goals


Define roles and responsibilities of Data Governance participants:

Stewards

Business Owners

Stakeholders

Communities of Interest

Custodians (IT)

53


Develop Data Governance Handbook or Manual

Data Governance Charter

Data management policy

Data governance framework

Roles of data governance participants

Glossary

54


Develop Data Catalog

List of data programs

List of business data owners

List of data stewards

Access to data standards and definitions

55


Develop Business Terms Glossary

Standard definitions of commonly used business terms

56

Click to edit Master title styleStep 4: Establish Data Governance Programs –Alaska Data Governance Framework

57

Alaska DOT&PF Data Governance Framework, Overview

IT SERVICES BUSINESS UNITS

ADOT&PF

1. ASD/IS Data Custodians

2. Program Dev. Div. Programmers

• State Enterprise Technology Services (ETS)

• State Data Standards

• State Data Governance Structure

Agency Goals

(Performance Measures

and Targets)

Data Governance Council

(Division Director(s),

Section Directors)

Program Development Division Goals

Core Data Systems

Core Business Programs

Business Data

Owners

(Section Directors, staff)

COIs and

Working Groups

Click to edit Master title styleStep 5: Use Technology to Support Data Management

Use of Business Intelligence tools support executive decision-making and data sharing within the agency

Geographic Information System (GIS)

Dashboards and Scorecards

Knowledge Management Systems

Bidding, Auctions, and Cost Management Solutions

Invest in staff through training opportunities to increase ROI

58

Click to edit Master title styleStep 5: Use Technology to Support Data Management - Example Hennepin County, Minnesota

Strong executive support from County Administrator

Dashboards

Provide visual display of performance for county programs based on targets and goals

Balanced scorecards

Track Key Performance Indicators (KPIs) for county programs

Align staff around common goals and strategies

Serve as essential decision-making tool

59

Click to edit Master title styleStep 5: Use Technology to Support Data Management – Balanced Scorecard

60

Click to edit Master title styleStep 6: Link Data to Planning, Performance Measures, and Target-Setting Processes

Fully integrate data management processes with agency performance measures and target-setting processes by doing the following:

Do not use one size fits all approach

Link performance measures and targets to budget allocations

Include objectives regarding resource allocation in the agency Business Plan

Use external data sharing agreements to obtain data for performance measures that the agency does not collect internally

61

Click to edit Master title styleStep 6: Link Data to Planning, Performance Measures, and Target-Setting Processes

Revise or stop using targets if data not available to support the target(s)

Arrange performance measures (PMs) in a hierarchical order (i.e., federal, state, local PMs)

Incorporate customer satisfaction as a measure in setting performance targets

Reward business areas that consistently meet targets and goals

62

Click to edit Master title styleStep 6: Example Alaska Department of Transportation and Public Facilities

Planning – Support Planning mission and goals through use of a data business plan

Performance Measures – Develop performance measures using data systems identified in the data business plan

Target-setting – Identify data needed to develop targets and performance measures

“by 2020 to reduce the 3-year moving average of traffic fatalities per 100 million vehicle miles traveled (VMT) by 3% per year”

Link Planning, Performance Measures, Target-setting to support budget and resource allocation decision-making at ADOT&PF

63

Click to edit Master title styleData Management to Support Performance-Based Resource Allocation

64

Click to edit Master title style.

Risk Management

Information Technology

65

Additional Research Underway

Click to edit Master title styleAdditional Research Underway for NCHRP 8-70 – Risk Management

Literature review and case studies

Identify elements of risk management

Describe application to PBRA

Produce a primer which recommends best approaches for use of risk management in PBRA

66

Click to edit Master title styleAdditional Research Underway for NCHRP 8-70 – Data Management

Literature review and case studies

Identify Information Technology (IT) issues affecting data sharing

Discuss how these issues negatively or positively affect data sharing

Produce a primer which recommends best IT approaches for data sharing and access to support Performance-based resource allocation in transportation agencies

67

Click to edit Master title style.

68

Concluding Remarks and Final Questions

Click to edit Master title styleQuestions and Answers

Please type your questions into your webinar control panel

We will ask as many of your questions as time allows.

nchrp report 666 (nchrp 8-70) - transportation...

Documents