introduction to the quality management and improvement quality management & improvement team...

87
Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not everything that matters is measurable."

Upload: harry-flynn

Post on 25-Dec-2015

216 views

Category:

Documents


1 download

TRANSCRIPT

Page 1: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Introduction to the Quality Management and ImprovementQuality Management & Improvement Team

Staff Retreat 2009

“Not everything measurable matters. Not everything that matters is measurable."

Page 2: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Pending

• Check timing

Page 3: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Agenda

• Introduction to the Concept of Quality Management• Quality Management and Improvement at Rare• How We Keep Score• Resources

Page 4: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Introduction to Quality

Page 5: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

What is Quality?Quality like beauty is easy to recognize, but hard to define

• Formal Definition:“The common element of the business definitions is that the quality of a product or

service refers to the perception of the degree to which the product or service meets the customer's expectations. Quality has no specific meaning unless related to a specific function and/or object. Quality is a perceptual, conditional and somewhat subjective attribute.” Wikipedia.

• Yikes, what does that mean?• What do you think of when you hear quality?• What organizations do you think of?

Page 6: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Process Leverages Best PracticesQMI Process Inspired by Peers, Venture Capital and Consulting

Edwards Deming

Page 7: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Deming CycleThe goal is not constancy of outputs, but constancy of improvements

"Plan: define the goals to be reached and plan how to implement the actions "Do: implement the corrective actions "Check: verify that the set goals are achieved "Act: depending on the results that occurred in the previous step, take preventative measures

Source: http://en.kioskea.net/contents/qualite/qualite-introduction.php3

Page 8: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

QMI ProcessSystematically capture info, use it, improve

Ensures Information Quality and Reporting

Page 9: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Agenda

• Introduction to the Concept of Quality Management

• Quality Management and Improvement at Rare• How We Keep Score• Resources

Page 10: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Knowing Our LimitsTrying to focus on what’s important about quality

Page 11: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Double ChallengeRecord Number of Campaigns and New 2.0 Model

 

Total Number of Campaigns Launched per Fiscal Year(2000 to 2009)

Campaigns at each Phase, by Region(June 2009)

In FY 09 Rare launched a record number of

campaigns …

… most of which are just emerging from 2.0

planning stages.

Sean Please Update

Page 12: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Grade Inflation?Setting StandardsGap between regional standards and central standards

Sean Please Update

Page 13: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Weeding Out the WorstPlan Has Focused on Weeding Out the Bottom

 

Campaigns Cancelled(June 2008 -09)

We have focused on our

weakest performers …

… and exited the worst performers

Campaign StrategicPartner

Reason

Chang Tang Nature Reserve, China

WCS Campaign manager was not from the region or local culture, and did not want to move there. She was eventually dismissed.

Veracruzano National Park, Mexico

CONANP Campaign Manager’s chronic lateness and lack of motivation caused this campaign to be cancelled in September 2008.

Salonga National Park, Democratic Republic of the Congo

WCS The Campaign Manager was fired from the Lead Agency in September 2008. The site was too dangerous to send Rare staff in to support training a new CM.

Marshall Islands TNC The campaign was running extremely late, and in late February 2009, Rare deemed it to be extremely unlikely that any significant conservation results would be achieved.

Number of Campaigns at Each Status Level (1-5)

QMI Score Region’s Score

Campaign Score

Num

ber

of

Cam

paig

ns

Past Focus

Sean Please Update

Page 14: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Now Focusing on the BestForced Ranking Will Focus Efforts on Best Campaigns

 

Using qualitative and quantitative data Each regions

ranks its campaigns …

… and provides a hypothesis for success

that we revisit

Our Hypothesis: Lamandau River Wildlife PreserveIn addition to a clear ToC, we have a strong partners at the site. The local and regional governments also support the Campaign which will ensure local law enforcement and sustainability …

Rankings: PEP 1Rank Name Avg. Status

on Weekly Flash

Report

Approved all Deliverables Due

by May 15th

TR Feasibility (1 – 5) from BRAVO

TR Impact(1 – 5) from

BRAVO

Director / VP recommend return (y/n)

1 Loreto Bay (LORETO) yellow No 3.25 3.2 Y

2 Rio Ameca (RAMECA) green No 3.6 3.2 Y

3 San Antonio Oeste Bay (SANANT) red No 3.0 3.22 Y

4 Isla San Pedro Martir (BAKINO) yellow No 3.51 3.83 Y

5 Ria Lagartos (RIALAG) green No 3.5 3.9 Y

5 Ria Lagartos (RIALG2) green No 3.5 3.9 Y

6 Rio Gallegos (GALLEG) green No 3.3 2.5 Y

7 Cañon del Sumidero (SUMIDE) yellow No 3.1 2.9 Y

8 San Quintin (ESTERO) red No 3.11 3.16 Y

9 IBA Central de Veracruz (VRCRUZ) red No 3.03 3.44 Y

10 La Sepultura (SEPULT) green No 3.23 3.5 Y

11 Costa Atlantica (COSTAA) yellow No 2.2 3.0 Y

Rankings: Guad 7Rank Name Avg. Status

on Weekly Flash Report

Approved all Deliverables Due

by May 15th

TR Feasibility (1 – 5) from BRAVO

TR Impact(1 – 5) from

BRAVO

Director / VP recommend return (y/n)

1 Loreto Bay (LORETO) yellow No 3.25 3.2 Y

2 Rio Ameca (RAMECA) green No 3.6 3.2 Y

3 San Antonio Oeste Bay (SANANT) red No 3.0 3.22 Y

4 Isla San Pedro Martir (BAKINO) yellow No 3.51 3.83 Y

5 Ria Lagartos (RIALAG) green No 3.5 3.9 Y

5 Ria Lagartos (RIALG2) green No 3.5 3.9 Y

6 Rio Gallegos (GALLEG) green No 3.3 2.5 Y

7 Cañon del Sumidero (SUMIDE) yellow No 3.1 2.9 Y

8 San Quintin (ESTERO) red No 3.11 3.16 Y

9 IBA Central de Veracruz (VRCRUZ) red No 3.03 3.44 Y

10 La Sepultura (SEPULT) green No 3.23 3.5 Y

11 Costa Atlantica (COSTAA) yellow No 2.2 3.0 Y

Rankings: China 1Rank

Name Avg. Status

on Weekl

y Flash Repor

t

Approved all

Deliverables Due by May 5th

TR Feasibility (1 – 5)

from BRAVO

TR Impact(1 – 5) from

BRAVO

Director / VP

recommend return

(y/n)

1Gaoligong Mountain Nature Reserve

green Uploaded but not approved

3.80 3.76 y

2 Baima Snow Mountain Nature Reserve

green No 3.83 3.83 y

3Hunchun Nature Reserve

green No 3.60 (average

of 4)

3.93 (average

of 4)

y

4ALXA League

green Uploaded but not approved

- - y

5Yuhe Nature Reserve

green Uploaded but not approved

3.74 3.83 y

6 Dashanbao Nature Reserve Yellow No 3.50 3.25 y

6 Meili Snow Mountain yellow No 3.00 3.00 y

Rankings: Bogor 3

Rank NameAvg. Status on Weekly

Flash Report

Approved all Deliverables Due

by May 5th

TR Feasibility (1 – 5) from BRAVO

TR Impact(1 – 5) from

BRAVO

Director / VP recommend return (y/n)

1 Lamandau River Wildlife Reserve green No 3.4 3.6 Yes

2 Ujung Kulon National Park green No 3.8 3.8 Yes

3 Besitang Forest, Gunung Leuser National Park green No 3.16 3.9 Yes

4 Tun Mustapha Marine Park green No 3.5 3.7 Yes

5 Geumpang Forest, Ulu Masen Forest Complex green No 3.3 3.0 Yes

6 Halimun-Salak National Park green No 3.45 3.67 Yes

7 Sungai Putri, Gunung Palung National Park green No 3.1 3.42 Yes

8 Bali Barat National Park yellow No 2.7 3.7 Yes

9 Tripa Swamp Forest green No 3.4 3.5 Yes

10 Dolok Surungan Wildlife Reserve green No 3.55 3.61 Yes

11 Batang Toru Forest green No - - Yes

Page 15: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Building for ScaleSuite of tools will help scale the process

Tool: Pride ScorecardGoal: Assessment of Campaign’s Operational Status and ImpactHow: Gather data from PPMs, Directors and Partners via RarePlanet

Tool: Cohort Management ProcessGoal: Ensure we focus on best campaignsHow: Periodic, structured check in meetings

Tool: Pride Weekly Flash ReportGoal: Leading Indicator of Campaign’s Operational Status and ImpactHow: Gather data from PPMs Weekly via RarePlanet

Tool: Best Practices and RecommendationsGoal: Create a process for continuous improvement How: Integrate data with quality information and observation to initiate and justify Programmatic improvement initiatives

Milestones Summary Sheet5/21/2009

Project Code Campaign Site

 “After reviewing the existing mateirals, the QMI department believes that this campaign is

likely to achieve its objective by within the stated timeframe

Based on available information we expect the following results.

We believe the campaign should/ should not return to university

GALLEG Estuary Rio Gallegos, Santa Cruz (Estuario del Río Gallegos, Santa Cruz)

From the BRAVO, I am unclear on the timeframe

Substantive reduction in solid waste disposed directly in the river

No - without an MOU, Guad needs to justify: (1) financial risk to Rare and to the campaign(2) sustainability without LAP /BINGO support (3) How no MOU impacts Guad's allocation of time

Summary

Page 16: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

The Pride ScorecardThe Pride Scorecard Takes a 3600 View of Impact

CapacityThe ability of the Campaign

manager and local partner to run a successful social marketing Campaign

ConstituencyCommunity support to work

towards threat reduction and environmental protection

Conservation1.Reduction of threats /

changed behavior 2.2. Improvement to the local

environment

… which is organized around Pride’s logic

model

Solicit feedback and measure

results …

… that is captured via RarePlanet …

• Pride Program Manager’s and Director’s weekly assessment of campaigns

• Campaign Manager’s Grades, Project Quality, Timeliness

• Campaign Manager’s Assessment of Rare

• Lead Agency Partner’s Assessment of Rare

• BINGO Partner’s Assessment of Rare

• Measured Conservation Results

Page 17: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

How We Get ThereQuick review of the processes

Ensures Information Quality and Reporting

Page 18: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Information Quality & ReportingEnsuring data integrity

Cohort Planning Recruiting Cohort Registration Campaign Alumni Follow-UpAdmissions

QM Primary Owner of Info

QM Supports Information Architecture(Help Document Data Flow & Mgt Quality & Communication)

Page 19: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Information Quality & ReportingThe campaign details highlights goals and outcomes

Campaign details pages on RarePlanet contain lists and descriptions of threats

addressed, endemic & endangered species at the site, as well as summaries

of achievements.

Campaign details pages provide a wealth of information on the campaign & site…

Page 20: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Orientation

•Weekly Flash Report•Campaign Details Page

Planning

•Weekly Flash Report•Monitoring Report•Survey: Local partner, (Executive Director, Local supervisor), BINGO•Campaign Details Page

Implementation

•Weekly Flash Report•Monitoring Report•Survey: Local partner, (Executive Director, Local supervisor)•Campaign Details Page

Analysis

•Weekly Flash Report•Monitoring Report•Survey: Local partner, (Executive Director, Local supervisor)•Campaign Page

Post-campaign monitoring

(Monitoring partner)

GatherData Gathering Is Integrated Across the Life of Campaign

Page 21: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

AssessWe assess campaign through tools like the Pride Scorecard

Page 22: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

ActInformation is integrated into regional calls to guide decisions

Page 23: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

ImprovementWe’ve taken your feedback to improve Pride

Validation of Campaign Escalation Process

Guide to Managing Partners (Draft)

Some Indicative Recommendations •Working on creating library of Best Practices•Brining clarity around the 3Cs• XYZ

Some Early Stage Research Reports & Tools

Page 24: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Quality Management ToolsInternal processes to improve Pride

• Escalation ProcessRetrospective research shows that campaigns that are Red for more than four weeks rarely recovers.

– This process helps to ensure the appropriate Rare response.

– Helps us to focus our efforts on the most impactful campiagns

• Second University Return ProcessA two step process that makes sure that all campaigns enter the implementation phase with a plan towards 2.0 expectations.

– Ensures that everyone understand standards

– Helps to make sure Partners have a long-term plan

• Cross Functional ReviewComing in FY10 QMI will visit about 8 campaign sites around the world to:

– Understand their impact– Look for process improvement opportunities– Ensure financial integrity

Page 25: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Drawing the Right ConclusionData isn’t enough, we must draw the right conclusions

Page 26: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

ConstraintsManaging quality has its limits

1. We don’t control all the input or the external factors

2. Conservation is inherently complex and risky

3. The expected outcomes are unclear

Process View: Limiting Factors Innovation: Opportunities

Beyond looking at just our current

practices, we must look outside for discontinuous

innovation

Page 27: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

FeedbackWhat are some ways we can improve Pride?

• Timing: 40 minutes• At your table (20 min)

– Write down as many ways as you can think of that we can improve Pride

– Discuss options– Choose

• Two most feasible (easiest to do)• Two most impactful (dramatically improves Pride)

• Chose a presenter (4 minutes per presenter)– Present– Discuss and prioritize

Page 28: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Agenda

• Introduction to the Concept of Quality Management

• Quality Management and Improvement at Rare• How We Keep Score• Resources

Page 29: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Why Do We Use the Scorecard

• Transparency:The results of the Scorecard are reported to the Board each month

• Management :We use the results of the Scorecard to better understand if a campaign needs help or should be highlighted in success stories

• Training The Scorecard provides ideas on what sort of issues staff should look for both for training ourselves, CMs and partners

• Learning We have and will more so periodically look at the top and bottom campaigns to see what lessons can be learned to drive recruiting, selection and management

• Quality AssuranceThe Scorecard helps determine who comes back from to the second university phase, who gets extensions and who graduates

Page 30: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Sometimes scoring is hard to do, but easy to count, but what about Rare?

Page 31: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

In the long-term conservation is enabled by the three the 3Cs

Mission: protect biodiversity at the Earth’s most important places

Capacity The ability of the Campaign manager and local partner to run a successful social marketing Campaign

Constituency

Community support to work towards threat reduction and environmental protection

Conservation

Combines the: 1. Reduction of threats

/ changed behavior 2. 2. Improvement to

the local environment

Page 32: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

What Questions Go Into the Scorecard Cascade

Campaign Impact

Capacity Constituency Conservation

Leading (Operational)• Likelihood of achieving capacity

goals (WFR)• Campaign timeliness (Milestones)• Grades (WFR)• Partner satisfaction (360 survey)• CM capacity development (360

survey)

Lagging (Impact)• CM graduation (Pf) • CM active in conservation field

(survey TBD)• Partner increasing investment in

community engagement (survey TBD)

Leading (Operational)• Likelihood of achieving

constituency goals (WFR)• Stakeholders are discussing the

campaign (Monitoring Report)• Stakeholders are likely to

change behavior (Monitoring Report)

• Number of friends on RarePlanet (RarePlanet)

Lagging (Impact)• Change in Knowledge (Details

Page)• Change in Attitudes (Details

Page) • Change in IC (Details Page)

Leading (Operational)• Likelihood of achieving

conservation goals (WFR)• Quality of deliverables (WFR)• Fundraising

Lagging (Impact)• Achievement on SMART

objectives (Details Page)• Cost per Adoption (Details

Page)• #/% of target adopting new

tool (Details Page)

Community support to work towards threat reduction and environmental protection

The ability of the Campaign manager and local partner to run a successful social marketing Campaign

Combines the: 1. Reduction of threats /

changed behavior 2. 2. Improvement to the

local environment

Page 33: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

ScorecardHow the Rare Staff and Board See the Campaign

Page 34: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Building the ScorecardThere are five inputs into the Scorecard

PPM and Regional Input

Campaign Manager Input

Results+ =

1. The Weekly Flash Report

2. The Monitoring Report

3. Surveys

4. Campaign Milestones

5. Campaign Details Page

Page 35: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Building the ScorecardThere are five inputs into the Scorecard

1. The Weekly Flash Report

2. The Monitoring Report

3. Surveys

4. Campaign Milestones

5. Campaign Details Page

Page 36: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Know your Stoplights: Pop Quiz

1. A Campaign Manager has been laid off due to budget cuts

2. After 6 months of campaigning, fisherman in your community are not convinced that the threat identified as most important by scientists is important to them

3. Funds for campaign materials being paid by the local organization are 2 weeks late

4. Funds for campaign materials being paid by the local organization are 2 months late

5. Your village chiefs have agreed to support a locally managed marine area

Page 37: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Weekly Flash ReportLeading indicator of campaign impact

Objective:•On the ground validation of progress•Should contain space for people to provide more details about the trip•Learning

Who: Any Rare staff that visits a site When: Within a week of a visitPrimary User: PPM, Director, VP

Page 38: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Monitoring ReportsGround truthing your observations

Objective:

•How is the campaign going?•Are we on track to achieving conservation objectives?•Are there major concerns?•Provide a quick qualitative update

Who: PPM + Regional DirectorWhen: WeeklyPrimary User

Page 39: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Partner SurveysEnsuring we meet expectations

Objective:

•CM: Capture satisfaction with campaign, with Rare•LAP: Capture satisfaction with campaign, with CM•BINGO: Capture satisfaction with campaign, plans for repeated partnerships with Rare•Alumni: Capture follow-up data•For Development - Links to BSC (balanced scorecard) and Marketing – Prideforce

Who:•CM•LAP•BINGO•Alumni -When: PeriodicallyPrimary User: Scorecard-Partner Management

Page 40: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Campaign MilestonesRating the Ongoing Work of the CMObjective:•Information on campaign timeliness• Assessment of the qualityWho:•PPM•Campaign Manager

When: •When each deliverable is uploaded

Primary Users•Scorecard•Board •Regional programs•QMI

Page 41: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Campaign DetailsEnsuring we meet expectationsObjective:•Collect Campaign Results•Capture key facts about the campaign, especially regarding the campaign site and TOC/barrier removal plans.• Report resultsWho:•PPM•Campaign Manager

When: •Initial setup by staff (QMI);•PPM monthly update (as new information•At GraduationPrimary Users-Scorecard-Board -Marketing-QMI

Page 42: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

How is the Score Calculated

Campaign Colors

Color Meaning Score

Green■

Campaign is on track to achieve goals in stated timeline

4.0 to 5.0

Yellow▲

campaign is at risk of not achieving some of its goals, or has fallen about a month behind schedule

2.5 to 3.99

Red

Campaign is unlikely to achieve its goals or is severely behind schedule

Below 2.5

Some Business Rule: •Any data that is more than 3 months old, does not go into the overall score – this allows campaigns to recover from past mistakes (of course the best thing is the PPM to work with the CM and Partner to fix the issue).•Data that is missing does not go into average e.g. N/A does NOT equal 0•This override rule that turns campaigns Red can be switch on / off if we believe the measure is too harsh (we don’t want to pretend all our campaigns are great, but we don’t want it to appear that they are all failing either).

Business rules address the logic that governs the Scorecard. For this initial launch we have kept these rules relatively simple. Based on our experience, we may adjust them.

Page 43: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Quiz

1. What are the 3Cs?

2. Who completes a Monitoring Report?

3. The weekly flash report can be red for how many weeks before it must be escalated to the partner

4. True or false: Forced ranking is a way to evaluate the quality of the CM

5. True or false: Edwards Deming just cared that processes were consistent

6. If I wanted to see if a campaign were late, I would look under the “what” tab

7. If I wanted to find about the treats addressed I would look under the “what” tab”

Page 44: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Conclusion & Feedback

• What did you learn about the QMI Process• About the Pride scorecard• What did you like• What would you like to see changed• What can we do to make this process more

meaningful to your work

Page 45: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

PPM – ActivityPPM Role in Managing the Scorecard

Page 46: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Agenda

• Review recent updates• Review current questions and concerns• Discuss roles• Walk through

– Monitoring report– Surveys– Learning forms

Page 47: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Everyone Contributes to the Scorecard (p. 1/3)Tool Objective/

PuRarePlanetoseDoer Frequency System User

Flash Report Answering Questions:-How is the campaign going?-Are we on track to achieving conservation objectives?-Are there major concerns?-Provide a quick qualitative updateServes as a list of what we do at Rare

PPMValue: -Manage portfolio better-Manage communications with team-Ease of reporting

Weekly RarePlanet Pride Team (Directors, VPs)HQ

Monitoring Trip Reports

- On the ground validation of progress

- Should contain space for people to provide more details about the trip

- learning

Currently PPM, but should be anyone who visits an LAP and/or campaign site (e.g. Directors,, VP, HQ)Expand to make more relevant to users

Per trip to the site RarePlanet - Scorecard- Who will read the

qualitative info? We need to define a process

Page 48: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Survey RolloutWe will rollout the survey region-by-region

Timing Survey Tool

Current Alumni Survey

August Remaining alumni

August / September PEP to Pilot on their campaigns

September Rollout to the other regions

Page 49: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Everyone Contributes to the Scorecard (p. 2/3)

Tool Objective/PuRarePlanetose Doer Frequency System User

CM Survey CM: Capture satisfaction with campaign, with Rare

PPMs send surveysCM complete them

At the end of each phase

RarePlanet (private messages)

- Scorecard- Programs- Partner

Management

LAP Survey • Supervisor• Executive

Director

LAP: Capture satisfaction with campaign, with CM

Directors send surveys; Partners complete them

CM/LAP: 4x (once per phase)

RarePlanet (private messages)

- Pride Scorecard + Rare BSC

- Programs- Partner

ManagementBINGO Survey- Alumni

BINGO: Capture satisfaction with campaign, plans for repeated partnerships with Rare

VP send surveys; Partners complete them-Automatic integration into scorecard

• End of planning phase

• Post campaign

RarePlanet (private messages)

- Pride Scorecard + Rare BSC

- Programs- Partner

Management

Alumni Survey Alumni: Capture follow-up dataFor Development - Links to BSC (balanced scorecard) and Marketing – Prideforce

Alumni Manager send surveys; Alumni complete them

Once per year RarePlanet (private messages)

- Pride Scorecard + Rare BSC

- Programs- Partner

Management- Alumni program

Page 50: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Everyone Contributes to the Scorecard (p. 3/3)

Tool Objective/PuRarePlanetose Doer Frequency System User

Campaign Details & Learning Form

• Collect Campaign Results• Capture key facts about

the campaign, especially regarding the campaign site and TOC/barrier removal plans.

- PPMValue:Used by- Director/VP- Recruitment- Alumni- QA

Initial setup by staff (QA);PPM monthly update (as new information

At Graduation

RarePlanet - Scorecard- Board - Research- Marketing- QA- Learning as

a team

Campaign Overview Page

Communcicate in a simple and clear way the goal of the campaign

• CM• PPM

Should review monthly to make sure it reflects major changes

RarePlanet - Donors- Partners- Community

Members- Rare staff

Prideforece (CRM) -relationship management- documenting interactions/communication- some documents (e.g MoUs, contracts)- task management – relationships- escalation of partnerships issues- general sensitive issues

- VPs, Directors- PPMs- HQValue:CRM

As needed; monthly minimum

Prideforce - PPMs - Directors

Monthly Finance Monthly Project Financial Report

Replicon Monthly? MIP - PPMs- Directors- VP/HQ

Page 51: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

How is the Score Calculated

Campaign Colors

Color Meaning Score

Green■

Campaign is on track to achieve goals in stated timeline

4.0 to 5.0

Yellow▲

campaign is at risk of not achieving some of its goals, or has fallen about a month behind schedule

2.5 to 3.99

Red

Campaign is unlikely to achieve its goals or is severely behind schedule

Below 2.5

Some Business Rule: •Any data that is more than 3 months old, does not go into the overall score – this allows campaigns to recover from past mistakes (of course the best thing is the PPM to work with the CM and Partner to fix the issue).•Data that is missing does not go into average e.g. N/A does NOT equal 0•This override rule that turns campaigns Red can be switch on / off if we believe the measure is too harsh (we don’t want to pretend all our campaigns are great, but we don’t want it to appear that they are all failing either).

Business rules address the logic that governs the Scorecard. For this initial launch we have kept these rules relatively simple. Based on our experience, we may adjust them.

Page 52: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Appendix: Reporting ToolA breakdown of the measures

Field Source Selected by Scale

Flash Score Weekly flash reports (submitted through RarePlanet)

PPM Red, yellow, green

Overall impact Pride Scorecard (Cumulative weekly flash reports, CM and partner surveys, grades, timeliness, and achievement of SMART goals)

PPM, CM, LAP Supervisor, LAP Executive Director, BINGO representative, as well as automated scores

1 - 5

Timeliness Approval of campaign deliverables

Automated , ,

Page 53: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

PPM – ActivityGoal setting and defining good

Page 54: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Agenda

• Template for defining good• Drafting templates• Input into the Flash Report• Next steps

Page 55: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Goal SettingCreate a picture of success

• For each of the 3Cs you should have a mental image of what the goal is

• This format can help you define good:– We will know when we have reached our Capacity goals when _______– We will know when we have reached our Constituency goals when _______– We will know when we have reached our Conservation goals when this threat

____ is reduced by this much _______ .

Page 56: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

New Flash ReportNew report allows you set your goals

Page 57: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Next steps

• Keep drafting the goals• Review them with your supervisor• Input them into the Weekly Flash Reporty by August 1

(or let Sean Grundy know you need more time)• Feel free to consult Paul, Daniel or Sean

Page 58: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Appendix

Page 59: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Resources on RarePlanet

Page 60: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

QMI on the Rare intranet (1 of 2)

For frequently-updated campaign information, reports, and Standard Operating Procedures, please see our folder on the Rare intranet

Or go straight there: http://rareplanet.org/en/category/rare-intranet/qm-amp-improvement

Page 61: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

QMI on the Rare intranet (2 of 2)

FAQs

A sample of our goods!

Standard Operating Procedures

Pride Campaign Status Report: (1)

(1) Updated monthly

Measures Quality …

Monthly Regional Reports

Please notify us whenever cohort

dates change, and we will post the updates

here.

… Provide Key Dates

Page 62: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Tools for reporting and analysisFinally, we can get a quick overview of campaigns’ status

*Alternatively go to the following url: http://rareplanet.org/en/reports

Fine print: Final kinks in the report will be fixed by the end of May. For the next 3 weeks, QMI is not responsible for bad decisions that you may make as a result of these bugs.

In the Campaign Dashboard on the top left of your My RarePlanet Home page, select an aggregate report.*

This report can be sorted by cohort, region, or PPM.

Use the pulldown box to sort the campaigns by different criteria, and click the links to go directly to campaign and partner pages:

Please see the appendix for a more

detailed explanation of the

reports.

Coming soon!

Page 63: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Visual display of our footprintSee where we work around the world, and learn about what we are doing by threat, habitat, species and status

Please see the appendix for a more detailed explanation of

the reports.

Page 64: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

RarePlanet as an Information ArchiveWe’ve heard a lot about RarePlanet as a project management tool for current campaigns, and as a place to host community discussions on conservation. It is also becoming Rare’s most complete source of information on past campaigns.

The overview page provides a summary of the

campaign, as well as a portal

to the other parts of the page (e.g.

campaign blog, forum,

resources, & milestones

pages)

More campaign pictures are stored in each campaign’s Resource page

Eventually, we hope to have

each campaign’s theory of change

translated into all 4

languages:

Page 65: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Finding data on past campaigns (1 of 4)

Campaign details pages on RarePlanet contain lists and descriptions of threats

addressed, endemic & endangered species at the site, as well as summaries

of achievements.

Campaign details pages provide a wealth of information on the campaign & site…

Page 66: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Finding data on past campaigns (2 of 4)…but for even more detail, the campaign deliverables are also available on each campaign’s milestones page.

Users can download

files directly from these

links

Page 67: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Finding data on past campaigns (3 of 4)The following table can help you navigate to exactly what you want on a campaign page:

File/Information Location within campaign page

Pictures Overview page, campaign resources page

Start and end dates Overview page, details page

Location Overview page, details page

Summary of TOC Overview page

CM, PPM and Partners Campaign Team panel (on left side of overview and details page)

Possible relevant alumni news Blog or forum

Deliverables (e.g. project plan, final report)

Milestones page

Taxonomic classifications (threats addressed, conservation acts, habitat types, etc.)

Details page

Description of site and its species (including flagship species)

Details page

SMART Goals Details page

Summary of results Details page

Page 68: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Finding data on past campaigns (4 of 4)If you’re not sure which campaign page has the data you need, try using the newly improved search function.

We no longer have to dig through old project plans and final reports to find

specific campaign information.

Page 69: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Appendix: Reporting ToolA breakdown of the measures

Field Source Selected by Scale

Flash Score Weekly flash reports (submitted through RarePlanet)

PPM Red, yellow, green

Overall impact Pride Scorecard (Cumulative weekly flash reports, CM and partner surveys, grades, timeliness, and achievement of SMART goals)

PPM, CM, LAP Supervisor, LAP Executive Director, BINGO representative, as well as automated scores

1 - 5

Timeliness Approval of campaign deliverables

Automated , ,

Page 70: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

• PPM break out session (45 min)– General QA– Activities– Materials

• What good looks like (Paul)– Sean teach – Paul gets definitions– Helps PPMs/Directors Write it (45min)

Page 71: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Appendix

Gather Data Overview of RarePlanet – Campaign Management

Page 72: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Appendix

Data GatheringOverview of RarePlanet – Campaign Management

Page 73: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Background

• RarePlanet is the central knowledge repository and project management system for all of Rare’s campaigns

• For the past 9 months there has been a team migrating, filtering and consolidating Rare’s disperse information repository and placing it into RarePlanet

• Much of this data is public, however some information is private• Private information typically relates to detailed contractual information

and ongoing relationship information about partners and people• All information that could be of general interest to a conservation

practitioner is public, available for comment and downloadable• RarePlanet has extensive permissioning that allow different views of the

site according to roles such as Rare Staff, PPMs, CMs, LAP partner staff and general users

• The views taken for this presentation are the highest level permission, so all information is available

Page 74: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

This Presentation Works Left to Right Across the Tops of the Campaign Page

• Overview• Details• Forms & Reports

– Weekly Flash Report

• Milestones• Scorecard

Page 75: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Overview Tab

Every campaign has an overview page that tells the story via pictures, display of their strategy for success (TOC), and milestones.

Page 76: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Details Tab

Every campaign has a page that details its main objectives, target audience and goals.

These are update about every 1 month to reflect the most recent information on a campaign.

Page 77: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Forms & Reports Tab

Every campaign has a page in which the campaign’s PPM can manage:

•The key deliverables•The dates•360 Feedback surveys•Inputs into the Pride Campaign Scorecard

Additionally, the PPM can see a summary result of the Weekly Flash Report.

Page 78: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

The Weekly Flash Report

The Weekly Flash Report is a form found under the tab “Forms & Reports. “ Rare’s PPMs complete these weekly. The weekly flash reports do not capture all relevant campaign data, but rather act as a leading indicator of success.

Page 79: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Milestones Tab

Under the Milestones tab is the CM’s one stop shop for submitting his campaign deliverables. This creates:

• One place to go for the CM to communicate his progress• One place for Rare staff to find campaign information• An automated, centralized and standardized way to see if the campaign is running on time•Color scheme and scoring help motivate/drive CM to complete things on time

The data posted here, once approved, is then published to the campaign Overview page and the Resource center as a contribution to the conservation community.

Page 80: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Scorecard Tab

Rare scores all campaigns across the 3Cs:•Capacity•Constituency •Conservation

This is the output of all the forms that are managed under “Forms & Reports”

The Scorecard updates real-time based on the most current information.

Currently the Scorecard is being fully tested and it is only available to Rare staff.

Still Being Teste

d

Page 81: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

EFQM model

Leadership Management of employees Policy and strategy Partnerships and resources Processes Employee satisfaction Client satisfaction Integration into the community Operational results

Page 82: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

The Founder of Quality

William Edwards Deming (October 14, 1900 – December 20, 1993) was an American statistician, professor, author, lecturer, and consultant. Deming is widely credited with improving production in the United States during World War II, although he is perhaps best known for his work in Japan. There, from 1950 onward he taught top management how to improve design (and thus service), product quality, testing and sales (the last through global markets)[1] through various methods, including the application of statistical methods.Deming made a significant contribution to Japan's later reputation for innovative high-quality products and its economic power. He is regarded as having had more impact upon Japanese manufacturing and business than any other individual not of Japanese heritage. Despite being considered something of a hero in Japan, he was only just beginning to win widespread recognition in the U.S. at the time of his death

Page 83: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

MissionAccelerate Improvement Through Learning

3 + 3 = 6Compartmentalized Improvements Lead to Incremental Change

3 3 = 9Integrated Improvements Lead to Step Improvements

X

Page 84: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Total Quality Management

• Total Quality Management

• The concept of total quality management (TCM) refers to the implementation of a business plan that is based on a quality procedure that involves all employees, i.e. a comprehensive strategy by which an entire company uses everything to satisfy its beneficiaries in terms of quality, cost and deadline. A "quality spirit" must be developed and shared by all in order for total quality management to succeed.

• Article written on 16 December 2004 by Jean-François Pillou

Last update on Thursday October 16, 2008 02:43:16 PM.

Page 85: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

The Principles of Quality ManagementJean-François Pillou

• Customer focus: Because organisations depend on their clients, they must understand their current and future needs, and meet and strive to go above and beyond client expectations.The organisation must see the client as not only a consumer but particularly as the user of the products and services produced by the organisation and must make sure that this fits with the company's goals. Therefore a customer focus system must be set up so that the organisation can have a better idea of its clients' needs and expectations so that it can meet them as best as it can. Moreover, customer satisfaction should be regularly evaluated so that opportunites and/or risks can be identified as quickly as possible.  

• Leadership: The directors of the organisation coherently define the organisation's goals and direction. They must create and maintain the internal atmosphere that is needed for employees to feel fully involved in attaining the organisation's goals. The goal of this principle is to make sure that the needs of all players are taken into account when defining and formalising a clear vision for the future of the organisation by setting goals that satisfy everyone. Shared values must be created so that all possible fears are replaced with a relationship of trust.  

• Involvement of people: Employees at all levels make up the very essence of an organisation. Involving everyone puts the skills of each individual to work for the organisation. Employees must understand that they play an important role in the organisation and should be involved in the process of setting motivating goals for themselves. Employee skills should be evaluated regularly and training plans should be set to help employees evolve in their job. Conversely, it can also be useful to allow employees to evaluate their superior on his or her management style and their work relationship. In such a context, each employee will be more inclined to improve his or her own skills to meet personal goals and therefore to share his or her experience and knowledge.  

• Process approach: An expected result is achieved more efficiently when the corresponding actions and resources are managed as processes. Therefore, the activities needed to achieve a result must be identified clearly as processes and someone must be named responsible for each one. Identifying activities can be made easier by involving the concerned parties. On this basis, it is possible to evaluate the performace of each process and analyse the way in which it can be improved in order to best meet the company's strategic goals.  

• System approach to management: Identifying, understanding and managing a system of interdependent processes for specific goal allows organisations to improve their effectiveness and efficiency. The idea of this principle is to consider that the act of clearly structuring and documenting the actions contributing to the organisation's goal allows it to improve its effectiveness and efficiency. To do this, the organisation must first identify the existing dependencies in order to reduce conflicts between processes and repeating work. This should lead to the formalisation of a clearly documented quality management system. Training or informing the necessary players may be necessary to make sure that everyone adopts this step.  

• Continual improvement: Continual improvement must be one of the organisation's permanent goals. The different processes must be monitored and their performance should be analysed cyclically in order to suggest and implement improvements. This can be done through a regular management review and with internal and external audits. It is particularly important to know how to spot improvements and make everyone aware of them.  

• Factual approach to decision-making: Effective decisions are based on the analysis of data and tangible information. This principle consists in making decisions based on a factual analysis of information corroborated by experience and intuition. According to this approach, it is easier after the fact to argue a well-founded decision by making reference to accessible documents. This gives all parties involved the means by which to understand the way in which the decisions are made.  

• Mutually beneficial supplier relationships: An organisation and its suppliers are interdependent and a mutually beneficial relationship improves their capacity to create value. Relationship with suppliers must be thought of in a way so as to reconcile easy short-term victories with future considerations. To do so, an organisation must understand the interests of its partners, clearly define their obligations in a contract and regularly evaluate their performance. When this principle is correctly applied, an organisation can improve its relationship with suppliers notably with regards to response time and therefore the overall cost.

Page 86: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Process Management

• The goal of process management is to breakdown company activities into processes and monitor them in order to line them up with the company's strategic goals.

• Thus, a process management approach must include the following steps:

• Definition and formalisation of the company's strategic goals

• Analysis of exisiting company processes

• Identification, for each process, of the input and output elements

• Definition of interactions between processes

• Formalisation of the performance criteria (performance indicators) for each process

• Measurement of the performance of current processes

• Suggestions to modify the existing processes

• Proposal of new processes

• Roll out and implementation of modification proposals

• Measurement of new performances

Page 87: Introduction to the Quality Management and Improvement Quality Management & Improvement Team Staff Retreat 2009 “Not everything measurable matters. Not

Model For ImprovementWe want to leverage continuous improvement with innovation

1. Path of continuous improvement

Time

Qu

alit

y

2. Path of continuous improve + innovation