1

Prepared for:Canadian Evaluation Society

National Capital ChapterJune 2, 2008

byJohn Harrison

Managing PartnerBMB Consulting Services Inc.

The Program Activity Architecture and theManagement Accountability Framework –

How can Evaluation Contribute?

2

Overview of Presentation…

• Evolving Management Context• What’s Different• Components of the Management, Resources

and Results Structure (MRRS) and theManagement Accountability Framework (MAF)

• “Operationalizing” the MRRS/MAF• The Role of Evaluation• Questions and Answers

3

Evolving Management Context…

• From “planning, organizing, leading, controlling”• To “planning, coordinating, directing, defining

objectives, supporting the work of others,reporting on performance”

• From “doing things right”• To “doing the right things”• From “solving daily problems, putting out fires”• To “thinking through what is generic and

strategic”

4

What’s Different…

• Increased sophistication and rigour of modernmanagement tools (Program ActivityArchitecture (PAA), MAF, Governments ofCanada Strategic Reference Model (GSRM))and technology (data warehouses, businessintelligence tools, scorecards/dashboards) cannow support this evolution

• An increasing recognition of the value ofperformance measurement and programevaluation in supporting this evolution

• To ease the burden on managers, however, theimplementation must be carried out in anintegrated way – and the complexity managedcentrally

5

Policyrequirements foreach programelement include:

Program titleand description

Expectedresults(outcomes)

Outputs (atlowest level)

Performanceindicators

Planned andactual spending

Targets andperformanceagainst targets

Governance

Strategic Outcome

Program ActivityLevel

Program Sub-Activity Level

Program Sub-Sub Activity Level

Lowest Levelprograms

MRRS Policy Components

6

MAF – The “Coxswain” in the Boat

7

Internal Services

Program Delivery Sustaining Agenda

Program Delivery Change Agenda

ManagementDeliverySustainingAgenda

Management DeliveryChange Agenda

$/FTEs

Linking Resources ($ and FTEs) to Program Elements in the PAA

8

GovernanceElements

Architecture/Models/

Maps

EnablingFrameworks

AccountabilityInstruments

•Mission (SustainingAgenda)

•Vision (ChangeAgenda)

•Values Statements•Committee andDecision-MakingStructures

•Delegations

•PAA•ProgramLogic/ReferenceModels

•Chart of Accounts•InformationArchitecture

•Solution Architecture•TechnologyArchitecture

•Business ActivityStructureClassificationSystem (BASCS)

•PerformanceFrameworks

•EvaluationFrameworks

•Integrated Risk ManagementFrameworks

•Control Framework•MAF•Program Results-based Managementand AccountabilityFrameworks (RMAFs)

•Strategic Plans(Change Agenda)

•Business Plans(Sustaining Agenda)

•Strategic CapitalInvestment Plans(HR, IM/IT, RealProperty)

•Program Plans•Project Plans•PerformanceManagementAgreements (PMAs)

Implementing the MRRS…

9

PerformanceStories aretold atMultipleLevels…

Organization

Program

Project

Individual

FoundationalInstruments

CyclicalReporting

Instruments

CyclicalPlanning

Instruments

Logic Model(RMAF)

Project Charter

PositionDescription

PAA/MAF BusinessPlan, RPP

AnnualReport, DPR

(against commitments in RPP)

PerformanceReview

Indicators associated with ongoing and key commitments

PMA/Performance

Contract

DetailedProject

Plan

Project Status Report,

Project-related Indicators

Level/Scopeof Planning, Monitoring

and Reporting

Program Plan

Program Indicators, Periodic

Evaluation

10

…And Across Multiple Dimensions/Perspectives

• Some of the potential dimensions/perspectives are:

– Program (PAA)/Management (MAF) Delivery

– Regional/Geographic

– By Client/Target Group (e.g., Industry Sector,Educational Institution, Not for Profit Organization,“Toronto Seniors”, “Wild Pacific Salmon”)

– Sustaining/Change Agendas

– By Performance Driver Category

– Horizontal Initiatives that cut across the PAA

11

Control Framework

Planning and Reporting Framework

How the Key Frameworks FitTogether

Risk ManagementFramework

Performance MeasurementFramework

Expectations

ActionsResults

CommitmentsPlanning

Directing

Monitoring

Reporting

Relationship Management Framework

12

The Planning and Reporting Cycle

Priorities

VISION

Inputs toStrategic

PlanReporting

Delivery

Monitoring

PerformanceManagementAgreements

BusinessPlans

OngoingAgenda

DPR

RPP

Change Agenda

13

The Six Performance MeasurementSteps

Step I – Educate the Organization

Step II – Document the Program Logic

Step III – Find the Right Things to Measure

Step IV – Build a Reporting Strategy

Step V – Build a Performance Report

Step VI – Refine and Adjust

IndicatorTemplates

Program LogicModel

List of “CriticalFew” Indicators

PerformanceReport

UpdatedProcess

CommonUnderstanding

14

The Results Logic Flow

Inputs

UltimateOutcomes

Activities

Outputs

ImmediateOutcomes

IntermediateOutcomes

So that

So that

So that

So that

Increasing public interest

Increasing departmental/program

influence

So that

15

The Indicator Learning Curve…

16

Definition and Principles ofAccountability

Accountability

• A relationship based on the obligation to demonstrateand take responsibility for performance in light ofagreed expectations

Principles of Accountability

• Clarity of roles and responsibilities• Clarity of performance expectations• Balance of expectations and capacities• Credibility of reporting• Reasonableness of review and adjustment