Quality Management & Improvement TeamMarch, 2014

“Quality is everyone's responsibility.”

Edward Deming

Introduction to QMI

Reviewed on 3/06/14

Agenda & Outcomes

Agenda• Overview of QMI and role of quality• Sources of quality at Rare• Reporting• Rare Lingo

Expected Outcomes• What does QMI do• Understand that quality is a process not a destination• Be able to list sources of quality• Understand the basics of reporting• Familiar with some Rare lingo

QMI is Part of Global Programs

Key

Current Position

Open Position

Bold Text Leadership Team Position

Italic Text Temporary or Contract Position

Latin AmericaKeith Alger, VP

Pride EnglishSteve Watkins, VP

IndonesiaTaufiq Alimi, VP

ChinaJiakai Yuan, VP

Finance & Operations

Oly Bracho, VP

Global ProgramsPaul Butler, Sr. VP

Finance

Information Technology

Program Development

TalentSabina Puppo, VP

Strategy & GrowthMartha Piper, Sr. VP

QMI

Human Resources & Administration

Marketing & Communications

Jason Anderson, VP

Development

COODale Galvin

President & CEOBrett Jenks

Conservation Research and Measurement

RarePlanet

Social Marketing

Training

The Team

Paul Butler

Liz TullyDaniel Hayden

Eddie Tsibulevskiy

Create and manage a set of processes and tools to ensure that Rare is running the most effective conservation projects possible and that Rare has a system to monitor and ensure success by improving current campaign and improving the overall design

process behind campaigns.

Team helps gather information through these tools:• Weekly Flash

Reports• Support Visit

Reports• Partner Surveys• Campaign

Deliverables• Campaign Details

Page and Results

We assess success through the Scorecard, BSC and meta studies. Success measured by:• Constituency• Capacity• Conservation • Bright Spots

We support data-driven decision making• Monthly Regional

Calls• Management &

Leadership Meetings

• Balanced Scorecard

• VP/Director Learning Forum

• Board meetings

Maximize the Impact of Pride

Team Makes Recommendations

• Document and communicate best practices

• Improve Pride through ad hoc changes, training and mentoring

• Initiate and plan major changes

Gather Assess Act Improve

Ensures Information Quality and Accurate Reporting

Quality Management & Improvement Process

Thinking About QualityQuality Management Measures Against the Min and Plans for the Ideal (1)

Dim

ensi

on o

f Qua

lity

High

Low

Time

Minimum Acceptable (Role of QMI)

Ideal

(1) The ideal is subject to the triple constraints: resource, time, scope

Rare’s culture of continuous improvement means expectations are

always increasing

Improvement InitiativesFocus on doing current things better, rather new thingsImproved Pride Scorecard• More focused on aligned to DBSC, aligns to TMT Campaign Framework• Lesser weekly requirements• More action oriented

Cohort Learning Report and Process•Shorter•All data aligns to Impact Reports (lead by CRM)•Geared more towards plus/deltas and generating bright spots•More room for regional customization

Campaign Tracker•Simplifying and consolidating RarePlanet into a new tool: CampaignTracker•Work with DevCom to create streamlined links to Rare.org

Support regions•New alumni survey•Revise partner surveys

Spring 2007 – How many campaigns do we haveSummer 2007 – When will they endFall 2007 How are the campaigns doingWinter 2008: What are the issues being addressedSpring 2008: What does good really look likeSummer 2008: How much do these campaign costsFall 2008: How can we dig in more on these issuesWinter 2009: What does good look likeSpring 2009: How can we learn from this informationSummer 2009: Average population size at siteFall 2010: % of audience changing behaviorSpring 2011: What are realistic SMART goals and tipping pointsWinter 2012: Are we getting better at marine conservation?

The Evolution of Our Questions:Some questions we answer

QMI is often the first

source of information

for the Dev/Com

team

Agenda & Outcomes

Agenda• Overview of QMI and role of quality• Sources of quality at Rare• Reporting• Rare Lingo

Expected Outcomes• What does QMI do• Understand that quality is a process not a destination• Be able to list sources of quality• Understand the basics of reporting• Familiar with some Rare lingo

Quality is conformance to requirements (Crosby).Quality is fitness for useGood quality means a predictable degree of uniformity and dependability with a quality standard suited to the customer. (Deming)Quality is the degree to which performance meets expectations.

So which is highest quality: Lexus, Kia or Ford?

What is Quality?

CPPCountry Ops

PlanCurriculum Open

Standards SOPs BSC/ DBSC

Cohort / Campaign Level Sources Organizational Sources

Sources of Quality at Rare

• Work Plan• Key Milestones• Key Milestones on RarePlanet

Key Perspective: How does compliance/noncompliance:• effect our impact• impact work of other teams / processes• create organizational risk

Sources of Quality: Country Ops Plan

• Defines the vision• Defines the

objectives• Defines the

resources: people, time, budget)

• Details our partners and partner roles

• Defines the milestones

• Defines the monitoring protocol

The Country Operations Plan (formally CPP) is a series of documents that enables Brett to determine whether we

should launch a new cohort.

Sources of Quality: Curriculum

• Globally certified MA program in Communications• Sets high bar for academic rigor and protocols• Sets standard for CM evaluation• Validates Rare’s staff

Conservation Measures

PartnershipOpen Standards

· Develop work plan and timeline

· Develop and refine budget· Implement plans

3. Implement Actions and Monitoring

· Document learning· Share learning· Create learning environment

5. Capture and Share Learning

· Define initial team· Define scope, vision, targets· Identify critical threats· Complete situation analysis

1. Conceptualize

2. Plan Actions and Monitoring

· Develop goals, strategies, assumptions, and objectives

· Develop monitoring plan· Develop operational plan

· Prepare data for analysis· Analyze results· Adapt strategic plan

4. Analyze, Use, Adapt

Sources of Quality: Open Standards for Practice of Conservation

1. Conceptualize – Concept models, threat rating, results chains (ToC)

2. Plan – Strategies, SMART Goals, emphasize on measuring both leading indicators and outcomes

3. Implement – project planning4. Analyze – Final campaign

learning reports5. Share – Campaign learning

reports, cohort learning process, post to RarePlanet, write articlesAmielle is on the CMP Board

Daniel co-leads CMP initiative on Human Wellbeing Targets

The Conservation Measures Partnership (CMP) is a partnership of twenty leading conservation organizations that seek better ways to design,

manage, and measure the impacts of their conservation action.

Sources of Quality: SOPsScope: SOPs Provide List of the Key Must Do Activities

Reasons: Legal compliance, donor requirements, organizational risk mitigation. All SOPs are signed off by Dale, updated by owner of the SOP.

Requires: All SOPs are approved by Dale

SOPS owned (or partly owned) by QMI:• 2nd University Phase Return Process & Approval• Campaign Cancellation• Major Campaign Change• Campaign Review• Campaign Completion & Cohort Closing• Cohort Learning Process

A complete list of SOPs can be found here:M:/Global Programs/QMI/SOPs

Sources of Quality: BSC / DBSCScope: Entire Organization (Programs, HR, Finance, IT, DevCom)

Reasons: • Means to align Rare around

common goals and targets• “Balance” means that there

needs to be tradeoffs

QMI’s role:• QMI Helps ensure a consistent

rollup of department measures to global measures

• Helps with some key processes e.g. alumni surveys, partner surveys, Pride Scorecard

• Helps guide the implementation of measures

Agenda & Outcomes

Agenda• Overview of QMI and role of quality• Sources of quality at Rare• Reporting• Rare Lingo

Expected Outcomes• What does QMI do• Understand that quality is a process not a destination• Be able to list sources of quality• Understand the basics of reporting• Familiar with some Rare lingo

Measure Across the ProjectQMI focuses mostly on operational measures

ProgramGoals

CampaignBaseline

CampaignProcess

CampaignNear-term

ProgramImpact

CampaignSustainabilit

y

• Focus groups• Quantitative

Social Survey• Baseline

Biophysical

• Weekly Flash Reports

• Pride Scorecard

• Milestones

• Partner Surveys

• BSC measures

• Campaign learning reports

• Cohort learning reports

Support alumni survey

• Social goals• Biophysical

goals

1 year 2 years 3- 5 years

QMI supports various measurement systems across the life of the campaign

Weekly Flash ReportsProgress Against 30 Day Work Plan

Frequency: WeeklyCompleted by: PPMAudience: Cohort Team & QMITool: RarePlanet (Campaigntracker starting in March)Goal: • Measure progress

against work plan• Flag compliance issues

Weekly EmailFrequency: WeeklyCompleted by: LizeAudience: Cohort Team Tool: Email Goal: • Summarize key campaign information• Highlight key operational steps• List managerial resources

Pride ScorecardProgress Against 3C Impact (Capacity, Constituency, Conservation)Frequency: MonthlyCompleted by: Cohort TeamsAudience: All Rare, including BoardTool: Google Docs (soon on CampaignTracker)Goal: • Measure progress against impact• Input into the BSC

Komodo: 83Wangi Wangi: 83

Mayalibit: 76

Cohort ObjectiveConserve marine ecosystems through the

implementation of no-take zones and rights-based fisheries.

5/28/2012 - 7/5/2014

2nd half of the Implementation Phase: Month 20

75

Quick Status of Cohort's Top 3 ranked Campaigns

Cohort Status by Score

Current Phase

Cohort Timeline

0

1

2

3

4

5

6

7

8

9

10

11

12

# of Campaigns Achieving a Green, Yellow or Red Pride Scorecard Score Over Time

N/A

red

yellow

green

Monthly UpdatesAligning Dale and the Regions

Frequency: MonthlyCompleted by: Liz and DanielAudience: Dale and Regional VPTool: ExcelGoal: • Align Dale with

regions• Flag critical issues• Summarize

Reports

Cohort RankingShows strategic value of site

Frequency: MonthlyCompleted by: Cohort directorAudience: Dale and Regional VPTool: ExcelGoal: • Show strategic

value of a site• Prioritize our

focus

Campaign Information Leading Indicators

Project Code

Campaign Site IP PPM Rank

Weekly

Flash Score

Pride Scorecard Score over last 3

monthsQMI Flag

Nov. Dec. Jan.

TNKOMO

Wakatobi National Park, South East Sulawesi

Balai Taman National Wakatobi

Yayat 1 ■ 4.7 89à 95  

WAKMPA

Mayalibit Bay Marine Protected Area

Conservation International Indonesia

Katherina 2 ■ 4.1 76ä 85 O

Balanced ScorecardAligning the Regions on Annual and Long-term Plans

Frequency: QuarterlyCompleted by: All department headsAudience: All Rare, including BoardTool: Excel / ClearPointGoal: • Align organization• Track progress

FY11 FY12

(YEAR)ACTUAL

(YEAR)ACTUAL Target Q1 Actual Q2 Actual

1 97% 82% 100% 76% 72%

2 82% 82% 50% 74% 76%

3 30% 14% 30% *24

percentage points

Available Q2

4 8 31 22 7 7

% of Active Campaigns with Documented Plans on track to Monitor Threat Reduction

% of Active Campaigns in Each Cohort that have a Scientifically Rigorous Conservation Result Monitoring Plan on track

% of Primary (Threat-Causing) Target Audiences Adopting New Behaviors

# Pride Trained Graduates in the Current Year

Percent of campaigns in each cohort that have a scientifically rigorous and expert validated CR monitoring plan that measures outcomes at the end of the campaign and at a later date (2-5 years) to be determined by the expert. The measure will require identification of an expert to scientifically validate the CR goals and monitoring plans.

FY13MEASURE DESCRIPTION

The percent of current campaigns that have documented expert validated monitoring plans on track to measure Threat Reduction (TR).

The total number of Campaign Managers (CM's) and Fishery Fellows (FFs) or similar roles who complete the Pride training program and graduate, usually receiving a Masters Degree. (This is cumulative, annual measure).

The average percentage of our campaigns' target audiences that change their behavior after a Pride Campaign.

Key Check in PointsWe target key points to check in on the cohort

Uni 2Final

University Phase

QMI Pre-uni 2

Check in

QMI Onsite Review

QMi/GloPro Pre-uni 3 Check in

Region does

partner survey

Region does

partner survey

Uni 1

Region does

partner survey

Regions can lead cancelation process at any time

Agenda & Outcomes

Agenda• Overview of QMI and role of quality• Sources of quality at Rare• Report• Rare Lingo

Expected Outcomes• What does QMI do• Understand that quality is a process not a destination• Be able to list sources of quality• Understand the basics of reporting• Familiar with some Rare lingo

Rare Lingo: ReportsWeekly Flash ReportA report that Pride Program Managers (PPMs) complete each week on the status of a Pride campaign against that month’s operating plan.

Pride ScorecardA report that a Cohort Team (everyone who supports a campaign such as PPMs, Cohort Director, monitoring specialists, et al) each month against the 3Cs (Capacity, Constituency, Conservation)

Cohort RankingThe ranking of site based on its intrinsic strategic value

Monthly ReportA report that is given to Dale each month that provides a summary of the Weekly Flash Report, Pride Scorecard and other available data on the status of the campaigns.

Balanced ScorecardA management tool that helps us measure our impact (3Cs: capacity, constituency, conservation), and our key internal process around finance, HR, program development)

Rare Lingo: Logic ModelsStrategy MapA tool that helps explain how an organization achieves its mission.

3CsRare’s primary mission is to invest in local Capacity, that creates an engagements & commitment Constituency, to achieve Conservation results.

Theory of ChangeA logic model that explains how a Pride campaign is going to achieve its objectives.

Updating in FY14

Key Resources We SupportRegional Management Manual - key procedures for mentoring a campaign and running a cohort http://www.rareplanet.org/en/resource/guide-managing-pride-campaign

Checklist for a Better Pride - tool to help quality control of key deliverableshttp://www.rareplanet.org/en/search/apachesolr_search/checklist%20for%20a%20better%20Pride

Rare’s SOPs - key procedures for operating at Rarehttp://www.rareplanet.org/en/resource/sop-record-management-0

Data FAQshttp://www.rareplanet.org/en/resource/data-faqs

Rare Balanced ScorecardN:/Global Programs/QMI/Public

Agenda & OutcomesAgenda• Overview of QMI and role of quality• Sources of Quality at Rare• Rare Lingo

Expected Outcomes What does QMI do Understand that Quality is a process not a destination Be able to list sources of quality Understand the basics of reporting Familiar with some Rare lingo

Did We Achieve Our Outcomes?

Questions?