data 101: numbers, graphs, and more numbers

CENTER FOR SOCIAL SERVICES RESEARCH School of Social Welfare, UC Berkeley

Data 101:Numbers, Graphs, and More Numbers

Emily Putnam-Hornstein, MSWCenter for Social Services ResearchUniversity of California at Berkeley

March 11, 2008

The Performance Indicators Project at CSSR is supported by the California Department of Social Services and the Stuart Foundation


Agenda

• Basic Terminology

• Common Data Pitfalls

• Graphics

• Small Groups…


Data Basics…

• Descriptive Data– Demographic characteristics of a population, place, office,

etc.

• Comparisons– Performance trends over time (one time period to another)– Differences/similarities between groups, counties, placement

settings, interventions, etc.

• Analyses– Exploring the relationship between two events (e.g.,

reunifications and re-entries to care)– Looking at the contributions of various factors to some

outcome• Y=a+bX

Computing a Percent

Answers.com Dictionary: Rate• A measure of a part with respect to a whole; a proportion: the

mortality rate; a foster care entry rate.

100total

part100)(per percent %

100total

part

100440

290

100659.0

%9.65

100reunified # total

12m w/in reunified #

Raw Numbers (counts)

# Reunified w/in 12m

# Reunified (total)

= 290

= 440

What Percentage of Children who were reunified in 2005

reunified within 12 months of entering care?

Computing a Rate per 1,000

Answers.com Dictionary: Rate• A measure of a part with respect to a whole; a proportion: the

mortality rate; a foster care entry rate.

1000total

part

1000363,376

1,333

100000366.

7.3

1000population child #

care entered #

Raw Numbers (counts)

# Entered Care

# Child Population

= 1,333

= 363,376

What was the foster care entry rate in 2005? (i.e., how many

children entered care out of all possible children?)

1000total

part1000per rate

Scales for a meaningful interpretation…


Measures of Central Tendency

Mean: the average value for a range of data Median: the value of the middle item when the data are arranged

from smallest to largestMode: the value that occurs most frequently within the data

12 4 15 63 7 9 4 17 4 4 7 9 12 15 17 63

4.168

631715129744 Mean

5.102

129 Median

4 Mode

7= 9.7

= 9


Measures of Variability

Minimum: the smallest value within the dataMaximum: the largest value within the dataRange: the overall span of the data

4 Minimum

63 Maximum

59463 Range

4 4 7 9 12 15 17 63


Disaggregation

• One of the most powerful ways to work with data…• Disaggregation involves dismantling or separating

out groups within a population to better understand the dynamics

• Useful for identifying critical issues that were previously undetected

Aggregate Permanency OutcomesRace/Ethnicity

Age

County

Placement Type


2000 July-December First Entries

California: Percent Exited to Permanency 72 Months From

EntryN=11,698

9

20

56

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

3 6 12 18 24 36 48 60 72

In Care Other Emancipated Guardianship Adopted Reunified

85%


2000 First Entries

California: Percent Exited to Permanency 72 Months From

Entry

10

8

19

20

5060

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

3 6 12 18 24 36 48 60 72 3 6 12 18 24 36 48 60 72

In Care Other Emancipated Guardianship Adopted Reunified

White (n=3,773) Black (n=2,417)

88%79%


219

54

12 24 36 48 60 72

Black Non- Relative Placements (n=1,441)

2000 First Entries

California: Percent Exited to Permanency 72 Months From Entry

by Relative vs. Non-Relative Placement

17

19

58

12 24 36 48 60 72

White Relative Placements (n=1,398)

22

19

43

12 24 36 48 60 72

Black Relative Children Placements (n=976)

221

61

12 24 36 48 60 72

White Non- Relative Placements (n=2,375)

I n Care Other Emancipated Guardianship Adopted Reunified

=94%

=84%

=84%

=75%


Point in Time

Exit Cohorts

Entry Cohorts

Data

3 Key Data Samples


January 1, 2005 January 1, 2006

How long do children stay in foster care?

July 1, 2005

Child 1

Child 2

Child 3

Child 4

Child 5

Child 6

Child 7

Child 8

Child 9

Child 10


22

31

2220

5

0

5

10

15

20

25

30

35

40

45

50

<1 yr 1- 5 yrs 6- 10 yrs 11- 15 yrs 16+ yrs

%

Entries

California Example: Age of Children in Foster Care

(2003 first entries, 2003 exits, July 1 2004 caseload)(2003 first entries, 2003 exits, July 1 2004 caseload)


22

4

31 30

2225

2022

5

19

0

5

10

15

20

25

30

35

40

45

50

<1 yr 1- 5 yrs 6- 10 yrs 11- 15 yrs 16+ yrs

%

Entries

Exits




22

4 5

31 30

2322

25 24

2022

33

5

19

15

0

5

10

15

20

25

30

35

40

45

50

<1 yr 1- 5 yrs 6- 10 yrs 11- 15 yrs 16+ yrs

%

Entries

Exits

Point in Time




There are 260 placements for

every 100 foster children

Continuous vs. Discrete

• The average foster child has 2.6 placements while in foster care– This number makes little sense because the underlying

dimension is discrete (i.e., categorical, discontinuous)

1 2 3 4 5 6

placements

x2.6

Continuous Data Discrete Data

Age Days in Care Percentages / Rates

Race/Ethnicity Placement Type Referral Reason


Correlation

• Two “events” that covary with one another…

% Reunified within 6 months

% Reentries

Negative Correlation

=

or

Event 1

Event 2

Positive Correlation

=

Event 1

Event 2

% Births to Teen Moms


Percent Change

Time Period 1

Time Period 2

10 children

11 children

10011 Period

2 PeriodChange %

1001kids 10

kids 11

10011.1

1001.0

%10


Percent Change

Time Period 1 Time Period 2

10% 12%

% %

% %

% %

% %

% %

% %

% %

% %

% %

% %

%

%

100110%

12%Change %

%20


Exercise: Percent Change Calculation

Baseline Referral Rate (time period 1):

7.50100005067.963,637,9

419,488

Comparison Referral Rate (time period 2):

3.4810000483.199,988,9

706,482

Percent Change:

1001Rate Baseline

Rate Comparison

100150.7

48.3

%7.4

100047.0

1001)-.9526(

50.7 48.3 -4.7%

12.0 10.8 -10%

Min

or D

iffe

ren

ces

du

e to

Rou

nd

ing…


CWS Outcomes System Summary


- 14.6%4.1%4.0%

3.8%- 0.7%

- 7.0%

7.9%32.2%

17.4%17.4%

33.2%

- 14.2%5.1%

15.1%10.6%

- 0.25%2.5%

C4.3: Placement Stability (24m+ I n Care) (+)C4.2: Placement Stability (12-24m I n Care) (+)

C4.1: Placement Stability (8d-12m I n Care) (+)

C3.3: I n Care 3+ Years (Emancipated/Age 18) (- )

C3.2: Exits to Permanency (Legally Free) (+)C3.1: Exits to Permanency (24m I n Care) (+)

C2.5: Adoption w/ in 12m (Legally Free) (+)

C2.4: Legally Free w/ in 6m (17m I n Care) (+)C2.3: Adoption w/ in 12m (17m I n Care) (+)

C2.2: Median Time to Adoption (- )

C2.1: Adoption w/ in 24m (+)

C1.4: Reentry Following Reunification (- )C1.3: Reunification w/ in 12m (Entry Cohort) (+)

C1.2: Median Time to Reunification (- )C1.1: Reunification w/ in 12m (Exit Cohort) (+)

S2.1: No Maltreatment in Foster Care (+)S1.1: No Recurrence of Maltreatment (+)

January 2004-January 2008

California CWS Outcomes System:Federal Measures, Percent IMPROVEMENT

* Figure 5.23 retrieved from: http://www.mrs.umn/edu/~ratliffj/psy1051/cross.htm

Cross-Sectional vs. Longitudinal

Cross-Sectional (repeated)

Longitudinal


There are three kinds of lies: Lies, Damned Lies and Statistics

Misused Statistics

^


Six Ways to Misuse Data (without actually lying!):

1) Using Raw Numbers instead of Ratios2) Rank Data3) Compare Apples and Oranges4) Use ‘snapshots’ of Small Samples5) Rely on Unrepresentative Findings6) Logically ‘flip’ Statistics 7) Falsely Assume an Association to be Causal


Challenger: “Violent crime in Anytown, CA has increased over the last year. 100 more crimes were recorded.”

Incumbent: “Violent crime in Anytown, CA has decreased by 2% over the last year.”

Who is telling the truth?They both are.

1) Numbers that conceal more than they reveal…


“There are approximately 82,000 children in the child welfare system in California – 20% of foster children in the nation, and the largest foster care population of all 50 states.” National Center for Youth Law,

“Broken Promises”, 2006


“There are approximately 82,000 children in the child welfare system in California – 20% of foster children in the nation, and the largest foster care population of all 50 states.” NCYL, 2006

Factually true?• Yes

Informative?• Not very.

What if California has one of the largest child populations of all states? What if California has one of the smallest child populations of all states?

Misleading?• Maybe…

What is the point being made? Telling us that California has the largest foster care population does not shed any light on how the state is performing!


2) Rank Data

Two streets in Anytown, CA….

It’s all relative…And SOMEONE will

always be ranked last (and first)

Poverty Blvd

$$ Ave

“Jane Doe is the poorest person living on Moneybags Avenue.”

“Joe Shmoe is the wealthiest person living on Poverty Blvd.”


“San Francisco ranks 55 out of 58 counties when it comes to state and national performance measures…” SF Chronicle, “No refuge. For Foster youth, it’s a state of chance”, November

15, 2005

23.8%

18.5%

9.8%

9.6%

5.7%

0.1%1 or 2 Placements (at 12m, cohort) (+)

Recurrence w/ in 12m (- )

Adopted w/ in 24m (cohort) (+)

Recurrence w/ in 12m of Subst. (- )

Reunified w/ in 12m (cohort) (+)

Re- Entries w/ in 12m (cohort) (- )

San Francisco:AB636 UCB State Measures (Used in NCYL Ranking)

% IMPROVEMENT Jan ‘04 compared to June ‘06

(+) indicates a measure where a % increase equals improvement. (-) indicates a measure where a % decrease equals improvement. indicates a measure where performance declined.

“San Francisco ranks 55 out of 58 counties when it comes to state and national performance measures…” SF Chronicle

• Rankings mask improvement over time.• However, even improvement over time and relatively high rankings can be misleading.


3) Compare Apples and Oranges

Two doctors in Anytown, CA…Doctor #1 Doctor #2

What if the populations served by each doctor were very different?

2/1000 20/1000

Doctor of the Year?


“Foster Children in Fresno County are three times more likely to remain in foster care for more than a year than in Sacramento.”

SF Chronicle, “Accidents of Geography”, March 8, 2006


“Foster Children in Fresno County are three times more likely to remain in foster care for more than a year than in Sacramento.”

1. Different families and children served?

2. Different related outcomes?• First entry rates in Fresno are consistently lower

• Re-entries in Fresnoare also lower…

3. Other considerations…• Resources available, resource allocation choices• Performance trends over time


Number of Crimes Period 1: 76Period 2: 51Period 3: 91Period 4: 76

4) Data snapshots…

Crime jumped by 49%!!No change.

Crime dropped by 16%

Average = 73.5

Crime in Anytown, CA…


“A foster child living in Napa County is in greater danger of being abused in foster care than anywhere else in the Bay area...”

SF Chronicle, “No refuge. For foster youth, it’s a state of chance”, November 15, 2005


Abuse in Care Rate

Period 1: 1.80% Period 2: 1.64% Period 3: 0.84% Period 4: 0.00%

Responsible use of the data prevents us from making any of these claims

(positive or negative).

The sample is too small; the time frame too limited.

“A foster child living in Napa County is in greater danger of being abused in foster care than anywhere else in the Bay Area…”

100% improvement!0 Children Abused!

= 2/111

= 0

= 2/122= 1/119


5) Unrepresentative findings…

Survey of people in Anytown, CA…

90% of respondents stated that they support using tax dollars to build a new football stadium.

The implication of the above finding is that there is overwhelming support for the stadium…

But what if you were then told that respondents had been sampled from a list of season football ticket

holders?


“Some reports indicate that maltreatment of children in foster care is a serious problem, and in one recent large-scale study, about one-third of respondents reported maltreatment at the hands of their caregivers.”

“My Word”, Oakland Tribune, May 25, 2006


“…in one recent large-scale study, about one-third of respondents reported maltreatment at the hands of their caregivers.” Oakland Tribune

Factually true?• Yes.

Misleading?• Yes.

– This was a survey of emancipated foster youth

– Emancipated youth represent a distinct subset of the foster care population

– This “accurate” statistic misleads the reader to conclude that one-third of foster children have been maltreated in care…


6) Logical “Flipping”…

Headline in The Anytown Chronicle:

60% of violent crimes are committed by men who did not graduate from high school.

“Flip”

60% of male high school drop-outs commit violent crimes?


“One study in Washington State found that 75 percent of a sample of neglect cases involved families with incomes under $10,000.”

Bath and Haapala, 1993 as cited in “Shattered bonds: The color of child welfare” by Dorothy Roberts


• In reading statistics such as the above, there is a tendency to want to directionally “Flip” the interpretation

• But the original and flipped statements have very different meanings!

Families with incomes under

$10,000

“One study in Washington State found that 75 percent of a sample of neglect cases involved families with incomes under $10,000.”

Families with open neglect

cases

75% of neglect cases involved families with incomes under

$10,000 DOES NOT MEAN

75% of families with incomes under $10,000 have open neglect

casesPut more simply, just because most neglected children are poor does not

mean that most poor children are neglected


7) False Causality…

A study of Anytown residents makes the following claim:

Adults with short hair are, on average, more than 3 inches taller than those with long hair.

Finding an association between two factors does not mean that one causes the other…

Hair Length Height

Gender

X


“A number of child characteristics have previously been shown to be associated with risk of maltreatment. Prematurity or low birth weight is frequently reported…” As reported in Sidebotham and Heron’s 2006 article


“A number of child characteristics have previously been shown to be associated with risk of maltreatment. Prematurity or low birth weight is frequently reported…”

• Should one conclude that prematurity is a causal factor in maltreatment?

(Drug use?)a third factor

prematurity maltreatment


Graphs / Charts

• Keep it simple…• Use consistent color themes when

possible• Think about the type of data being

presented (discrete vs. continuous)• Label Clearly• Tell a story• Look at presentations on the UC site!


Stacked Bar Chart

7.215.4 14.9 19.0

27.7

30.8

29.7 28.528.2

26.2

48.1

51.4

9.8

48.150.242.4

3.34.03.9 2.3 1.41.41.10.90.83.30

0%

20%

40%

60%

80%

100%

Population

(9,664,747)

Ref errals

(438,666)

Substantiations

(102,365)

Entries

(39,646)

I n Care

(74,634)

Other

NativeAmerican

Asian/ PI

Hispanic

White

Black

Ethnicity and Path through the Child Welfare System: California 2006


Pie Chart

Black:

27.7%White:

26.2%

Hispanic:

42.4%Asian/ PI :

2.3%

Native

American:

1.4%

Ethnicity of Children in Foster Care:

California 2006


<1 yr 1-2 yrs

3-5 yrs 6-10 yrs

11-15 yrs16-17 yrs

Black

(97.2*) Native

American

(46.8*)

ALL

(50.0*) White

(43.8*) Hispanic

(47.4*) Asian/PI

(17.8*)

150

96110103

90

73

87

5278

47

37

28

64

465254

48

40

54

374447

45

37

53

425053

46

38

20

141821

18

15

3D-Area Chart

*Series Total

2006California:

Referrals per 1,000 by Age and Ethnicity


(complex) Line Chart

0

5,000

10,000

15,000

20,000

Entry Year

Plac

emen

t Fre

quen

cy

0

5,000

10,000

15,000

20,000

25,000

30,000

TO

TA

L Frequency

1998 1999 2004 20052001 2002 20032000

Black

White

Native American

Hispanic

TOTAL

2006 2007

Asian/PI

California:First Entries by Race/Ethnicity


0

10,000

20,000

30,000

40,000

50,000

60,000

Point in Time

Plac

emen

t Fre

quen

cy

0

20,000

40,000

60,000

80,000

100,000

120,000

TO

TA

L Frequency

(complex) Line Chart

1998 1999 2004 20052001 2002 20032000

Asian/PI

Black

White

Hispanic

TOTAL

2006 2007

Native American

California:Foster Care Caseload by Race/Ethnicity


Small Group Topics…

Group 1: Explore County to County variation in Group Home use in 2007

Group 2: Miscellaneous

Group 3: Describe any statewide trends in Group Home use (vs. other placements) over time

Group 4: Explore the placement stability of the Group Home population in care for 24 months or mroe

Group 5: Describe the Group Home Population in California in 2007


Group 1

Group 1: Explore County to County variation in Group Home use in 2007

• How many children were in GH care in Sacramento County? Alameda County?

• What percentage of the GH population is female in Humboldt County? – How does this compare with CA as a whole?– What conclusions can you draw about Humboldt?

• Compare the ethnic distribution of the GH population in Los Angeles County with that of San Diego County.

• Other observation(s)…


Group 2

Group 2: Miscellaneous• In 1999, what percentage of children first entering foster care (“first

entry”) had a first placement in a GH? What was the percentage in 2006?

• In 1999, what percentage of children re-entering foster care (“other entry”) had a first placement in a GH? What was the percentage in 2006?– Any thoughts on why this may be the case?

• In 2006, what percentage of children exiting from care with a last placement in a GH exited to emancipation?

• The number of children exiting from GH to reunification has increased over time. What was the count in 1998? And in 2006?



Group 3

Group 3: Describe statewide trends in Group Home use (vs. other placements) over time

• How has the size of the GH population changed over time?• What percentage of the foster care population was in GH care on

January 1, 1999? And in 2007?– How do you reconcile this with the fact that the count of children in

GH care has gotten smaller over time?• How has the size of the population in other placement settings

changed over this same time period?– Kin? Foster? FFA? Shelter?– Overall out of home population?

• The overall out of home care population has decreased over time. What additional data do you need in order to assess whether this is a real change?



Group 4

Group 4: Explore the placement stability of the Group Home population of children in care for 24 months or more

• How has the total size of the population of children in care for 2+ years (and who are now in GH care) changed over time?– And what has been the trend over time for children in two or fewer vs. three

or more placements been?

• In 2006, what percentage of children in GH care for 2+ years had been in two or fewer placements?– What percentage in foster homes had been in two or fewer placements?– And kinship homes?

• Is it reasonable to conclude that placement in Group Home Care causes placement instability?



Group 5

Group 5: Describe the Group Home Population in California in 2007• What was the total PIT count of children in group home care in 2007?

• Which age group had the greatest number of children in GH care?

• Were there any infants in GH care? – Any thoughts on why this might be?

• What percentage of children in GH care were ages 11-15 years?

• Are any gender differences observed?



A quick look at the website…


CSSR.BERKELEY.EDU/UCB_CHILDWELFARE

Needell, B., Webster, D., Armijo, M., Lee, S., Dawson, W., Magruder, J., Exel, M., Zimmerman, K., Simon, V., Putnam-Hornstein, E., Frerer, K., Ataie, Y., Atkinson, L., Blumberg, R., Henry, C., & Cuccaro-Alamin, S. (2007). Child Welfare Services Reports for California. Retrieved [month day, year], from University of California at Berkeley Center for Social Services Research website. URL: <http://cssr.berkeley.edu/ucb_childwelfare>

Emily [email protected]

data 101: numbers, graphs, and more numbers

Documents