Measuring the Effectiveness of State Wildlife Grant Projects

Mark Humpert, Association of Fish & Wildlife Agencies

State Wildlife Grants

0102030405060708090100

2001 2003 2005 2007 2009 2011 2013

Mill

ions

of $

’s

Fiscal Year

Wildlife Action Plans

SWG SuccessesConserving at-risk fish and wildlife in Ohio

Lake Erie Water Snake (Nerodia sipedon insularum)

Status: De-listed as federally threatened (Sept 2011)

Project Description: Permanent conservationeasements, research, monitoring, education

Cost: $250,995

Outcome: Population increase to >8,000 (Recovery Plan Goal 5,555) Partners: ODNR, USFWS, Black Swamp Conservancy, Western Reserve Land Conservancy, Northern Illinois University, OSU-Stone Laboratory, private landowners

Outputs vs Impacts

b. Relative Costs of Measuring Change

ConservationActions

(Outputs)

Threats(Outcomes)

Species & Habitats(Impacts)

a. C

onfid

ence

in Im

pact

c. Time Required to Detect Change

IntermediateResults

(Outcomes)

The Need for EM

• Improve Conservation Work– link measures & actions

• Improve Accountability to Administration & Congress– show success

• Maintain/Enhance Public Support – tell a story

Adaptive Mgmt

Two QuestionsStatus?

Effectiveness?

Conservation Measures Partnership’s Open Standards for the Practice of Conservation

• Developed by leading organizations & agencies

• Draws on many fields• Open source/common language• Used around the world

• Great Lakes• TNC Preserves• Swedish National Parks• Donor Funding Programs• Academic Training

CMP Open Standards

KEY

Direct Threat Result

Intermediate ResultsAction Conservation

Target

Gating caves and mines

Increased bat

populations?

Bat Cave Results Chain

Bat Cave Results Chain

KEY

Direct Threat Result

Intermediate ResultsAction Conservation

Target

Gating caves and mines

Reduced disturbance by humans

Increased bat

populations?

Reduced disturbance by feral cats?

Bat Cave Results Chain

KEY

Direct Threat Result

Intermediate ResultsAction Conservation

Target

Gating caves and mines

Reduced disturbance by humans

Increased bat

populations?

Reduced disturbance by feral cats?

Gating caves and mines

Reduced human access

Reduced disturbance by humans

Increased bat

populations

Reduced disturbance by feral cats

Reduced access by feral cats

i # breaches

i # distinct cat tracks

i # bats

i # juveniles

Plover Results Chain

KEY

Direct Threat Result

Intermediate ResultsAction Conservation

Target

i # breaches

i # disturbed nests

# juveniles

Protecting Nesting Sites

Reduced human access

Reduced disturbance by humans

Increased plover nesting success

Reduced disturbance by predators

Reduced access by predators

i # eggs

i

Generic Results Chain

KEY

Direct Threat Result

Intermediate ResultsAction Conservation

Target

Predator Exclosure

Reduced human access

Reduced disturbance by humans

Increased SGCN

populations

Reduced disturbance by predators

Reduced access by predators

Mockup of Report

Work Group Charge

Develop and test a measures framework for assessing the

effectiveness of State and Tribal Wildlife Grants, conservation actions more broadly, and potentially Wildlife

Action Plans themselves.

Work GroupCONSERVATION PARTNERS

Karl Hess (USFWS)Ron Essig (USFWS)Connie Young-Dubovsky (USFWS)Amielle DeWan (DOW)Tess Present (NAS)Shelley Green (TNC)Mary Klein (NatureServe)Mathew Birnbaum (NFWF)Terra Rentz (TWS)

STATESDana Baxley (KDFWR)Faith Balch (MNDNR)Tara Bergeson (WIDNR)Chris Burkett (VDGIF)Wendy Connally (TPWD)Jenny Dickson (CDEP)Mike Harris (GDNR)Eric Rickerson (ODFW)Tracey Tomajer (NYDEC)

AFWAMark HumpertPriya Nanjappa

FOUNDATIONS OF SUCCESSNick SalafskyCaroline Stem

Timeline

• Sept ’09-Working Group Formed• Dec’09-Workshop 1• Jan’10-Subcommittees Formed• Mar’10-Interim Report to TWW Committee• Apr’10-Workshop 2• June’10-Pilot Testing• July’10-Workshop 3• Sept’10-Phase I Report to TWW Committee• Dec ‘10-Workshop 4• Jan ’11-SWAP Coordinators Review• Mar ‘11-Final Report to TWW Committee• Apr ‘11-Print Final Report/Implement

Framework Steps

1) Define Generic Conservation Actions2) Use Results Chains to Describe the Theory of

Change3) ID a Limited set of Effectiveness Measures4) Develop & Test Data Collection

Questionnaires5) Collect & Analyze Data & Adapt

11 Common Actions

Conservation Area Designation Acquisition/Easement/Lease

Data Collection & Analysis Management Planning

Direct Management of Natural Resources

Species Restoration

Create New Habitat/Natural Processes

Training & Technical Assistance

Outreach & Education Land Use Planning

Environmental Review

11 Common Conservation Actions Funded through SWG

Criteria for Measures

1) Linked-to key factors in results chain2)Measurable-both qualitative & quantitative3) Precise-defined the same by all4) Consistent-unlikely to change over time5) Sensitive-can measure change6)Overarching-can be measured at diff.

stages7)Achievable-not onerous to collect

Info for One ActionDefinition of

ActionExample

s

“Generic” Results Chain

Species Restoration

Species Restoration

“Good” restoration

plan completed

Source population identified

Species Restoration

Species initially restored to site

(short-term)

Species breeding

at sites : o )

Species Restoration

No breedingat sites

: o (

Species Restoration

“Good” overall restoration plan

for species

Key stakeholders buy into plan

Species Restoration

Species Restoration

Obj SP RST 5 – Sp Initially RestoredBy specified target date, the target number of units* have been introduced to Area(s)

YYYY.

Obj SP RST 6a – Sp BreedingWithin xx years of introduction, the restored population is successfully breeding within

the restoration site(s).

Obj SP RST 2 – “Good” PlanBefore implementation work starts, a

"good" restoration plan has been developed for the specific project site(s). "Good" = …

Species Restoration

Ind SP RST 2 – Quality of PlanPresence of plan; assessment of plan against

a priori quality criteria

Ind SP RST 6 – Species BreedingEvidence of ongoing self reproduction of species within the site; Total units of species at the site

Species Restoration

Info for One ActionDefinition of

ActionExample

s

“Generic” Results Chain

Crosswalk TableResul

tObjectiv

eMeasure

sQuestion

s

Questionaire

Questionnaire

This is all most folks would see for

performance reporting

purposes!!

Similar Projects Generating Similar Data

Roll Up Measures

Demonstrate That These are More Than “Counting” Projects

• % of projects that answered research questions• % of projects where data reaching target audiences• % of projects leading to other management actions

Report

IT Systems

• ConPro• Conservation Registry• HabITS• Miradi• Wildlife TRACS• Biotics 4 • DataBasin• NatureServe Explorer Web Service

Wildlife TRACS

Using OS to Evaluate Wildlife Action Plans

1. An approved framework2. Measures for 11 common

conservation actions3. Wildlife TRACS as the IT

System4. Grant Streamlining5. Next steps for SWAP

Final Report

Measuring the Effectiveness of State Wildlife Grants

FINAL REPORT

April 2011

www.fishwildlife.org/files/Effectiveness-Measures-Report_2011.pdf

“Efficiency is doing things right; effectiveness is doing the right things.”

-Peter Drucker

Questions

Effectiveness Measures & Wildlife TRACS Reporting & Tracking Tool

Mark Humpert, Association of Fish & Wildlife Agencies

Wildlife TRACS

SWG Effectiveness

Measures Working Group

TRACS & Measures

Sub-working Group

TRACS Development

Team

PDT & PAG Testing

Process

Examples• Direct Management

• Species Restoration

• Data Collection

Direct Management

Direct Management

Species Restoration

Species Restoration

Data Collection

Data Collection