data is your friend – extracting and analyzing statistics€¦ · overestimate the probability...
TRANSCRIPT
April 2015
Data is Your Friend – Extracting and Analyzing Statistics
April Harris & Aimee Gilbreath
TODAY’S AGENDA
• Why does this matter?
• A Dangerous Mind
• Data Collection
• Data Analysis
• Data Interpretation
• Case Studies
2
Why is Data Collection & Analysis Important?
3
4
Anecdotes vs. Data
“Rescue groups cherry pick all of the
highly adoptable animals.”
In 2014, rescue partners pulled 1,500 animals. Of those: • 30% Neonatal Kittens • 30% Medical Needs • 40% Highly Adoptable
Which is more useful for designing and managing programs?
“The shelters are full of nothing but pit
bulls.”
Pit Bull Type dogs make up 20% of total dog intake. On average, pit bull type dogs are held for 35 days prior to an outcome versus 14 days for other dogs.
Ways to Use Data Analysis to Drive Impact
5
• Identify new program opportunities • Geo target S/N or TNR programs • Monitor compliance with policy • Negotiate rates with contract cities • Communicate with community • Design adoption incentives • Identify best/worst kennels • Monitor program effectiveness • Incentivize and reward staff • Motivate volunteers and donors • Identify opportunities for better process
Save Time
Save Money
Save Lives
A Love Story. . . .
6
Salt Lake County
+ Data
Steps in Data Analysis Process
7
Collect & Validate
Analyze
Interpret & Act
TODAY’S AGENDA
• Why does this matter?
• A Dangerous Mind
• Data Collection
• Data Analysis
• Data Interpretation
• Case Studies
8
Your Brain – Incredibly Powerful, Slightly Dangerous
9
Heuristics
A heuristic is a mental shortcut that allows people to solve problems and make judgments quickly and efficiently. Heuristics are
helpful in many situations, but they can also lead to biases.
10
Cognitive Bias
11
Cognitive bias describes the inherent thinking errors that humans make in processing information. These thinking errors can prevent
us from accurately understanding reality, even when we have all the needed data and evidence to form an accurate view.
Said another way, cognitive bias is the common tendency to acquire and process information by filtering it through one's
own likes, dislikes, and experiences.
Availability Heuristic
12
An availability heuristic is a mental shortcut that relies on immediate examples that come to mind. As a result, you might judge that those events are more frequent and possible than others. You give greater credence to this information and tend to overestimate the probability and likelihood of similar things happening in the future. For example, after seeing several television programs on shark attacks, you start to think that such incidences are relatively common. When you go on vacation, you refuse to swim in the ocean because you believe the probability of a shark attack is high.
What is an animal welfare example
of the availability heuristic?
Frequency Illusion
13
Frequency Illusion is the phenomenon in which people who just learn or notice something start seeing it everywhere. For example, when you are considering buying a particular type of car, you notice many more of that type of car on the road than ever before. The cognitive error is that you often conclude that this is because that car is becoming much more common or fashionable, rather than recognizing the reality that the only thing that's changed is that you're taking notice of those cars now.
What is an animal welfare example of frequency illusion?
Negativity Bias
14
Negativity Bias is a tendency to notice, pay more attention, or give more weight to negative experiences or information over positive. At parent-teacher conferences, your child’s teacher has made several glowing comments about your son, but also mentions that he talks too much and can be disruptive at times. And though she said that they’ve been working on it and that he’s getting better, the only thing you heard was that your son is being disruptive in the classroom. There was only one negative comment about your son yet there were several wonderful comments, including, “he’s a pleasure to have in my class.”
What is an animal welfare example of negativity bias?
TODAY’S AGENDA
• Why does this matter?
• A Dangerous Mind
• Data Collection
• Data Analysis
• Data Interpretation
• Case Studies
15
Accurate Data Collection is Key: Beware GIGO!
16
Garbage IN Garbage OUT
If input data are not complete, accurate, and timely, then
the resulting output is unreliable and of no useful value
GIGO Example: Cat Intake
17
Zip codes with shelter locations
A Few Words on Shelter Software
18
• Different features and functions mean no one size fits all approach
• Data you input, you should be able to extract and analyze
• Standard and custom reports usually available
• Well worth having staff member(s) with expertise on your system
Demographic Data can be a Helpful Overlay
19
Demographics – The characteristics of human populations and population segments, especially when used to identify consumer markets.
• Age, Race/Ethnicity, Income, Education, Employment, LanguageSpoken, Housing Type, Access to Vehicle, etc.
Data Sources
• Zip code summary www.esri.com/data/esri_data/ziptapestry
• Detailed zip data www.factfinder.census.gov
• City/County data www.quickfacts.census.gov
Demographic Data Example
20
Well-educated, married couple families and singles; live in the suburbs of major metros on both coasts. Professional/managerial jobs fund exclusive, upscale, and sophisticated lifestyles. Portfolios are healthy, filled with stocks, bonds, and real estate investments. 2/3 own homes. Tech savvy, tops for owning Apple products, use devices to shop, bank, and research information. Leisure time is spent visiting museums, traveling, drinking imported wine, going to the movies, skiing, and practicing yoga.
90064 • Population 25,403 • HH Income $84,579 • 43% “Urban Chic”
90031 • Population 39,316 • HH Income $34,655 • 59% “Las Casas”
Mostly young married Hispanic couples with kids; live in large multi-generational households. Found in older neighborhoods on the edge of large West Coast metros. Rent houses or apartments. Bank in person, minimal savings, very little debt. Family-oriented; spending reflects interest in children and also desire to look good. Buy baby products as well as the latest trendy fashions. Big soccer fans; watch Spanish language TV channels; visit Spanish web sites.
Data Collection & Validation Tips
21
Things to do BEFORE you start an analysis
1. Identify what data you want to analyze and why
2. Verify how that data is currently being captured in your system (i.e. who, when, how)
3. Consider whether the data is processed or modified after it is entered (i.e. intact status)
4. Validate that data values are within realm of reality (i.e. negative LOS)
5. Look for anomalies or outliers that need to be explored
All Data by Species (Cat/Dog) and age (</> 5 mo) • Annual beginning and ending shelter count • Intake
– Stray/At Large, Relinquished by Owner, Owner Intended Euthanasia, Transferred in from Agency, Other
• Outcomes – Adoption, Returned to Owner, Transferred to another Agency, Returned to Field, Other Live Outcome
– Died in Care, Lost in Care, Shelter Euthanasia, Owner Intended Euthanasia
NFHS Has Defined Minimum Shelter Data to Collect
22
Data Matrix & Definitions • http://aspcapro.org/sites/default/files/nfhs-basic-matrix-fillable.pdf
What’s Your Rate? Calculation Guidance • http://www.aspcapro.org/sites/pro/files/What%20is%20your%20Rate%2010_2013.pdf
Additional Data is Very Helpful
23
Shelter Animals Count minimum data will allow you to calculate release rates and do basic intake and outcome analysis. Additional data will allow you to do deeper analysis that will benefit your organization so we strongly suggest adding:
• Breed • Length of Stay • Intake Zip Code
TODAY’S AGENDA
• Why does this matter?
• A Dangerous Mind
• Data Collection
• Data Analysis
• Data Interpretation
• Case Studies
24
Data Analysis - OR – Lies, Damn Lies, and Statistics
25
Shelter A Shelter B Shelter C
Population Served 600,000 500,000 100,000
Intake 10,000 25,000 5,000
Euthanasia 5,000 6,000 1,500
Euthanasia as % of Intake 50% 24% 30%
Euthanasia per 1,000 Pop. 8.3 12 15
Which shelter has the “best” euthanasia statistics?
Reliance on any single metric will not give a full picture of performance
A Tale of Two Shelters – Part 1
26
10,292 774 3,871
4,942
0
2,000
4,000
6,000
8,000
10,000
12,000 In
take
RTO
Ado
ptio
n
Eut
hana
sia
North Central 2010
8%
38%
48%
4,495 552 3,139
522
0
2,000
4,000
6,000
8,000
10,000
12,000
Inta
ke
RTO
Ado
ptio
n
Eut
hana
sia
West LA 2010
12%
38%
70%
12%
Population Served: 1,065,897 Intake per 1,000: 9.7 Euthanasia per 1,000: 4.6
Population Served: 669,561 Intake per 1,000: 6.7 Euthanasia per 1,000: 0.8
A Tale of Two Shelters – Part 2
27
2010 North Central West LA
Population 1,065,897 669,561
Intake 10,292 4,495
Intake per 1,000 9.7 6.7
Euthanasia 4,942 522
Euthanasia per 1,000 4.6 0.8
HH Income <$25,000 38% 19%
HH Income <$50,000 66% 39%
Renter occupied 72% 60%
Speak Spanish at Home 59% 14%
HH w/o Vehicle 28% 9%
High School or Less 65% 24%
Making Great Graphs
28
38%
Start data axis at zero
To compare,
keep same axis scale
Use Data Labels to show #s
“Waterfall” style graphs give great data visualization
Pivot Tables are Your Friend!!
29
A pivot table is a tool in Excel that allows you to explore large sets of data interactively. Once you
create a pivot table, you can quickly transform huge amounts of data into a meaningful summary.
Pivot Table Example – Cat Intake & Outcomes
30
Nearly 50,000 rows of outcome data becomes. . . . 0
500
1,000
1,500
2,000
2,500
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
2014 Cat Intake & Outcomes by Month
Kitten Intake Cat Intake Kitten Euthanasia Cat Euthanasia
Pivot Table Tutorial https://youtu.be/qMGILHiLqr0
Geographic Information Systems Also Powerful
31
Mapping Kitten Intake in Tompkins County – J. Scarlett http://www.maddiesfund.org/using-gis-to-target-spay-neuter-efforts.htm
Tools Available via ASPCA X Maps Spot Project
32
http://aspcapro.org/resource/saving-lives-research-data/x-maps-spot-tools-gis
Data Analysis Tips & Tricks
33
1. Look at numbers (#) and percentages (%)
2. Use graphs to visualize data; “Waterfall” style graphs are especially useful
3. Use good graph hygiene; always start graph axis at zero to avoid distortion
4. For comparison between graphs, keep same axis maximum and scale
5. For comparison between geography, normalize per 1,000 human population
6. Two words: PIVOT TABLES!!
Beware Data Analysis Pitfalls
34
Garbage IN = Garbage OUT
“Boiling the Ocean”
Analysis Paralysis
Keep the “Big Picture” in Mind
Goal: Maximize Live Release
Decrease Intake
Increase Live Release
Spay Neuter Adoption
Know your INTAKE What pets are coming into
shelter? Why?
Know your OUTCOMES What happens to pets in
shelter? Why?
35
Must Haves REALLY Nice to Haves
• Intake by species • Intake by type (stray, owner
surrender, etc.) • Outcome by type by species • Live release rate (LRR) by
species
• Intake by species, age, breed • Intake by impound type (Field,
OTC, etc.) • LOS by species, age, breed • LRR by species, age, breed • LOS pre-adoption and pre-
euthanasia • Euthanasia type by species
and age • Animal care days by species • Foster numbers • Length of stay (LOS) by
species
A&A Recommended Analysis List
36
TODAY’S AGENDA
• Why does this matter?
• A Dangerous Mind
• Data Collection
• Data Analysis
• Data Interpretation
• Case Studies
37
Correlation is NOT Causation
38
• Studies have shown that people who eat yogurt regularly have a healthier body weight than those who do not eat yogurt regularly
• Can we therefore say that if you eat yogurt you will have a healthier body weight? YOGURT CAUSES HEALTHY WEIGHT
• Or, might it be that people who make healthier diet choices overall tend to eat more yogurt? HEALTHY PEOPLE EAT YOGURT
• We can say that eating yogurt is CORRELATED with healthy weight, we cannot say that eating yogurt CAUSES healthy weight
• Most results have multiple contributing causes
• Resist the urge to oversimplify cause and effect
Does a Poor Economy Cause Intake to Rise?
39
• Likely accurate to say worsening economy was a contributing factor to increased intake
• But, economy was not the only factor influencing intake
Does Seasonal Kitten Intake Cause Higher Euthanasia?
40
0
500
1000
1500
2000
2500
3000
Jan
Feb
Mar
A
pr
May
Ju
n Ju
l A
ug
Sep
Oct
N
ov
Dec
2014 Feline Intake
Cat Intake Kitten Intake
0
500
1000
1500
2000
2500
3000
Jan
Feb
Mar
A
pr
May
Ju
n Ju
l A
ug
Sep
Oct
N
ov
Dec
2014 Feline Euthanasia
Kitten Euthanasia Cat Euthanasia
Data Interpretation Tips & Tricks
41
1. Don’t get too attached to your initial hypothesis, look for other explanations
2. Remember that correlation does not imply causation
3. Recall that most outcomes are the result of multiple different variables – resist the urge to over simplify!
4. Always ask yourself questions “What else could be going on here?” “What else has changed?”
Don’t Despair – Help is Available!
42
Where to look for help:
• Your local government
• Area universities
– Professors
– Class projects
– Interns
• Your volunteers!
TODAY’S AGENDA
• Why does this matter?
• A Dangerous Mind
• Data Collection
• Data Analysis
• Data Interpretation
• Case Studies
43
2010 Los Angeles Intake & Outcomes - Numbers
44
58
33
1
15
9 0
10
20
30
40
50
60
70 In
take
Dog
s
Pup
pies
Cat
s
Kitt
ens
Thou
sand
s
2010 Intake
22 9 0 7
7 0
10
20
30
40
50
60
Eut
hana
sia
Dog
s
Pup
pies
Cat
s
Kitt
ens
Thou
sand
s
2010 Euthanasia
62% Save Rate
2010 LA Outcomes by Animal Type - Percentages
45
NKLA Program Started 2011
46
Used 2010 data analysis and community input to design several programs with aim to make Los Angeles No Kill by 2017
Initial programs included:
• Grants for zip code targeted spay neuter (low income)
• Adoption subsidies for rescue groups
• Kitten nursery/foster programs
• New adoption facilities
2010 vs 2014 Los Angeles Intake - Numbers
47
58
33
1
15
9 0
10
20
30
40
50
60
70 In
take
Dog
s
Pup
pies
Cat
s
Kitt
ens
Thou
sand
s
2010 Intake
52
29
2
13
9 0
10
20
30
40
50
60
70
Inta
ke
Dog
s
Pup
pies
Cat
s
Kitt
ens
Thou
sand
s
2014 Intake
2010 vs 2014 Los Angeles Euthanasia - Numbers
48
22 9
0
7
7
0
5
10
15
20
25
Eut
hana
sia
Dog
s
Pup
pies
Cat
s
Kitt
ens
Thou
sand
s
2010 Euthanasia
12 4 0 4
4 0
5
10
15
20
25
Eut
hana
sia
Dog
s
Pup
pies
Cat
s
Kitt
ens
Thou
sand
s
2014 Euthanasia
77% Save Rate
2014 Outcomes by Animal Type - Percentages
49
Next Steps for NKLA
50
CATS, CATS, CATS!!!
• Shifting S/N grant focus to cats
• More kitten nursery/ foster capacity
• Cat adoption promotions
• Targeting TNR to areas with high kitten intake
Does Cage Type Impact LOS for Cats at Adopt & Shop?
51
What data to analyze, and why? • Adult cats, In a single kennel for entire stay, after Nov
Check data for errors and anomalies • Eliminate records with negative LOS
Calculate metrics and graph • Average LOS by kennel type, range
Interpret and act. . . .all grate front kennels?
Pull data from system and process • Date of outcome – DOB = Age
Cage Type Data Collection & Analysis Process
52
Cage Type Data Collection & Analysis Process
53
Cats in Sample 19 11 9
What can we conclude?
Could Your Organization Use More $$$ to Save Lives?
54
www.CrowdRise.com/SavingPetsChallenge
Any Questions? [email protected]
