Using Program Evaluation Methodology to Identify Possible Program Outcomes for Tree City USA and Tree Campus USARaija Bushnell Haley RitgerBurney Fischer

School of Public and Environmental Affairs

Indiana University

• Recognition program established in 1976– “Participating communities have

demonstrated a commitment to caring for and managing their public trees.”1

• 3400+ U.S. communities participate and meet the requirements:1,2

1. Tree board or dept.

2. Community tree ordinance

3. $2/capita urban forestry spending

4. Arbor Day celebration1http://www.arborday.org/programs/treeCityUSA/about.cfm2http://www.arborday.org/programs/treeCityUSA/standards.cfm

Tree City USA

Measuring Success at 40 Years

• Does participation in the Tree City USA program lead to sustainable urban forests and stronger communities?3

• To answer this question, we propose conducting a program evaluation.

3 adapted from http://www.arborday.org/programs/treeCityUSA/documents/promo.pdf

TCUSA Program Literature Review

• Only one study examining program outcomes– Galvin & Bleil examined tree canopy quantity and

community demographics in MD4

• Correlation study• Compared participants v. non-participants• Found participation as predictor of canopy cover• But, city land area was a stronger predictor

• Study only examined one potential outcome of participation: increased canopy cover4Galvin, M.F. & D. Bleil. (2004). Relationship among tree canopy quantity, community

demographics, and Tree City USA program participation in Maryland. U.S. Journal of Arboriculture 30.6: 321-327.

What is Program Evaluation?

• “Evaluation is the systematic application of scientific methods to assess the design, implementation, improvement or outcomes of a program (Rossi & Freeman, 1993; Short, et. al. 1996).”

• “The term "program" may include any organized action such as media campaigns, service provision, educational services, public policies, research projects,

etc. (Center for Disease Control and Prevention [CDC], 1999).”5

5Medical University of South Carolina. Web. 26 Sept 2014. https://mainweb-v.musc.edu/vawprevention/research/programeval.shtml

What is Program Evaluation cont.

• Types of evaluations– Process– Impact– Outcome

• Emphasis on “cause” rather than “correlation” and “outcomes” rather than “outputs”

• Questions to ask when planning an evaluation– What is the purpose of

the evaluation?– What outcomes,

impacts, or process should be looked at?

– What data exists?– How will the data be

analyzed?

Identifying TCUSA Outcomes

• Identifying appropriate outcomes is a key factor in program evaluation

• We’ve identified 4 categories of possible outcomes:– Group 1: Ecosystem Services– Group 2: Financial– Group 3: Community– Group 4: Institutional

Ecosystem Services Outcomes

• Air Quality– Removal of particulates and other criteria air

pollutants6

• Stormwater Runoff– Reduction of runoff and sewer overflow7

• CO2 sequestration– Urban trees in the U.S. have “an annual

carbon sequestration rate of 22.8 million tons per year”76Nowak, D.J. & J.F. Dwyer. (2000). Understanding the benefits and costs of urban forest

ecosystems. In: Kuser, John E., ed. Handbook of urban and community forestry in the Northeat. New York, NY: Kluwer Academics/Plenum: 11-227Alliance for Community Trees. 2011. “Benefits of Trees and Urban Forests: A Research List.

Financial Outcomes

• Property values– Increased residential

property values7

• Business – Consumer behavior8

• Jobs– Availability of green

industry and municipal forest management7

• Energy Efficiency– Decreased heating

and cooling costs7

8Wolf, Kathleen L. 2005. “Trees in the Small City Retail Business District: Comparing Resident and Visitor Perceptions.” Journal of Forestry: 390-395

Community Outcomes

• Safety– Decreased violence and crime7

– Traffic control9

• Public Health– Effects on mental and physical health7

• Connectivity – Increased unity and cohesion7

9 Mok, Jeong-Hun, Harlow C. Landphair, and Jody R. Naderi.2006. “Landscape Improvement Impacts on Roadside Safety in Texas.” Landscape and Urban Planning 78.3: 263-74.

Institutional Outcomes

• Formation of new organizations

• Connections between organizations

• Outreach activities

Conclusion

• Take Home Lesson: – Consider program evaluation for your

organization

• Moving forward:– Providing measurable outcomes for Arbor Day

to consider in evaluating TCUSA

Share Your Ideas

• We would like you, as practitioners in community forestry, to identify measurable outcomes

• Write your ideas for outcomes on the four flipcharts around the room

• Or, tweet your ideas to #PCF2014