program performance: but at what price? · pdf fileprogram performance: but at what price? ......

Strategy and Performance Consulting GroupISO 9001:2000

CommitmentExcellence Dedication

InvolvementRespect and

confidenceTeam work

Tangible results

Program Performance: But at What Price?

2009 Annual CES Conference

Ottawa, Ontario

Raymond Chabot Grant ThorntonStrategy and Performance Consulting Group

Outline of the Presentation

ObjectivesDemonstrate the value of activity-based information to program evaluators and how it exposes the internal structure of the Black BoxPresent storyboarding to participants and show its effectivenessand rapidity in building and analyzing the logic model

OutlineApplicability of activity-based information to program evaluation: How does it fitDemystifying the Black Box: Activity-Based Management/ABCSimulation Model: Activity-Based PlanningStoryboarding


Applicability of activity-based information to program evaluation: How does it fit

Early in the program evaluation processActivity-Based information helps to better understand the organizational context of a program, its activities, the individuals who participate in the processes, the human and financial resources consumed by activities, and the link of activities to outputs and in turn, to outcomes

Continued relevance: Some key questionsAre activities and outputs aligned with the program mandate and intended results and outcomes? Do they provide added value to the program?If not, what is the cost of those not aligned?What is the impact on resources if we expand the program?

Understanding the programEvaluation work should avoid treating a program as a “black box” that automatically transforms inputs into outputs and impacts. This view leaves a huge gap in our understanding of why programs succeed or fail.

3rd Edition of Program Evaluation Methods: Measurement and Attribution of Program Results


Demystifying the Black Box: ABC

Dispute Resolution Activities

- Inquiries and Intake- Early Resolution- Mediations- Investigations- Referred to HR Tribunal

Inputs- 64 FTE- $8.1M

Activity Based Costing Model


Simulation Model: Activity-Based Planning

ACTIVITIES

RECEPTION OF CLAIMS & INQUIRIES

CLOSE FILES

REPRESENT BOARD AT HEARING

PREPARE HEARING

INVESTIGATE CLAIMS

MEDIATE CONFLICTS

COMPLETE FORMS

43,000

1,230

330

396

100

30

1,230

Activity Drivers

ULE: .25 HOURSINTAKE OFFICER

ULE: 1 HOURINTAKE OFFICER

ULE: 7 HOURSMEDIATOR

ULE: 35 HOURSINVESTIGATOR

ULE: 70 HOURSLAWYERS

ULE: 130 HOURSLAWYERS

ULE: 1 HOURINTAKE OFFICER

Resource Drivers

ACTIVITIES

RECEPTION OF CLAIMS & INQUIRIES

8.7 FTE

CLOSE FILES1 FTE

REPRESENT BOARD AT HEARING

3.2 FTE

PREPARE HEARING5.7FTE

INVESTIGATE CLAIMS11.2 FTE

MEDIATE CONFLICTS1.9 FTE

COMPLETE FORMS1.0 FTE

AC

TIVI

TY P

LAN

NIN

G A

ND

BU

DG

ETIN

G D

ATA


Storyboarding

Storyboarding: What is it?First used by Disney to develop storylinesStructured brainstorming Based on a team approach: Participation of personnel (sample group) performing the activities to deliver the program

UsefulnessHelps build the program delivery and ABM model in a short period of timeEnables the identification of potential improvements, both from an effectiveness and efficiency perspective Rapid results

Two-day workshop to build DFAIT’s diplomatic corps model

Equipment required


Storyboarding

Organization of workshopsHomogeneous teams :

Participants are incumbents of similar positions e.g. Commercialofficers responsible for approving applications for contributions

Multidisciplinary teams :Though they are all involved in program delivery, participants may be from different units and hold different positions. Workshops may be organized by program activity or major processes e.g. Commercial officers approving contributions and those assisting applicants in developing their projects

Selecting the participants: They must know the program, its environment and activities performed to deliver the programThey must be open minded, creative, cooperative and capable of expressing themselves in a group format


Information gathered during these session

• Brainstorming on activities performed

• Organize in groupings and ensure completeness

• Identify participants and their levels of effort for each activity

• Identify activity-drivers: Volume that affects resource consumption

Build the Delivery Model

• Identify outputs and link to outcomes

• Identify KPIs• Identify activities that do not create

value to the program: no alignment• Identify potential improvements

Achieve your purpose

• DFAIT’s diplomatic corps• Consular Affairs• Economic mission in Mexico• Supply and Demand: IM/IT

How did we use it for program evaluation


DFAIT Storyboard: Diplomatic Corps

Attend public events

Attend career fair

Represent Canada at

conferences

Prepare speeches/lectures

Give speeches

Give lectures

Plan the event

Schedule the event

Identify purpose of event

Give interviews

Define messages

Adapt local need

Deal with media

Write press release

Respond to press inquiries

Give interviews

Write report

Obtain approval

Reply to correspondence

Write briefing notes




Identify speakers

Arrange for clearance

Rent space

Arrange for audio/video

Prepare guest list

Arrange for food & beverages

Write report

Review & approve invoices

Send invitation

Obtain funding (dept./OGD/

business


Set-up Canadian study program

Write & publish news-letters

Organize events for Canadians

Pass on news from Canada

Identify their needsMeet with

potential foreign student to Canada

Brainstorming:What work do we do?



Attend public events

Participate in public events

Attend career fair

Represent Canada at

conferences

Maintain relations with

media

Prepare speeches/lectures

Public speaking Organize public events

Give speeches

Give lectures

Plan the event

Schedule the event

Connect with Canadians

abroad

Identify purpose of event

Give interviews

Define messages

Adapt local need

Deal with media

Write press release

Respond to press inquiries

Give interviews

Write report

Obtain approval






Identify speakers

Arrange for clearance

Rent space

Arrange for audio/video

Prepare guest list

Arrange for food & beverages

Write report

Review & approve invoices

Send invitation

Obtain funding (dept./OGD/

business


Set-up Canadian study program

Write & publish news-letters

Organize events for Canadians

Pass on news from Canada

Identify their needs

Meet with potential foreign

student to Canada

Maintain Web site

Organize and Group



Adapt to your particular study needs


Raymond Chabot Grant Thornton (RCGT)

Thank YouDaniel Dubé, M.Sc., FCMA819-595-9144 extension 26

[email protected]

program performance: but at what price? · pdf fileprogram performance: but at what price? ......

Documents