1

Using ancillary information to stratify and target young adults and Hispanics in national ABS samples

J. Michael Dennis, Charles DiSogra, Erlina Hendarwan

AAPOR 2012

Orlando, Florida

2

22

Probability-based ABS recruitmentRecruitment takes place throughout the yearRepresentative of U.S. adultsIncludes:

Adults with no Internet access (24% of adults)• KN provides laptop and free ISP

Cell phone only (30% of adults)Spanish-languageExtensive profile data maintained on each member

• demographics, attitudes, behaviors, health, media usage, etc.

Samples from the panel are assigned to projects• e-mail invitations and a link to the online survey questionnaire

50,000+ members 50,000+ members

3

Mail methodology: materials and schedule

3

Current Resident / Residente Actual123 Your StreetThe City, State 99999

Day 7 Reminder PC

Day 28 NR Letter

Initial Mailing

4

44

Address-Based Sample (ABS) frame

U.S. Postal Service Computerized Delivery Sequence File (CDSF)

~97% coverage of physical addresses

Frequently updated including status of addresses, such as, seasonal homes, vacant houses, etc.

Can be matched to available landline telephone numbers

Can be geo-coded

Can attach ancillary information from a variety of sources for purposes of:

• Non-response analyses

• Targeting demographic / geographic mailings

• Sample stratification

4

5

Research questions

Will the demographic yields using ancillary information in the sample design be as good as or better than using traditional Census block data?

Does the use of ancillary information improve the efficiency of recruiting the demographic groups of interest?

If using ancillary information turns out to be a good strategy, can it be leveraged to maximize yields?

5

6

Sample stratification in 2010 and 2011

6

2010

8 national mailings (~22,000 ea.)

Objective:

Increase Hispanics in a national sample

Method used:

Targeted Census blocks with ABS sample

Criteria:

CBs with 30% or more Hispanic population

Design and sample distribution (vs. Frame):

Stratum 1 Hispanic CBs 41% (vs. 14%)Stratum 2 Balance of CBs 58% (vs. 86%)

Design Effect: 1.60

7

Sample stratification in 2010 and 20112010

8 national mailings (~22,000 ea.)

Objective:

Increase Hispanics in a national sample

Method used:

Targeted Census blocks with ABS sample

Criteria:

CBs with 30% or more Hispanic population

Design and sample distribution (vs. Frame):

Stratum 1 Hispanic CBs 41% (vs. 14%)Stratum 2 Balance of CBs 58% (vs. 86%)

Design Effect: 1.60

7

2011

8 national mailings (~26,000 ea.)

Objective:

Increase HispanicsIncrease Young Adults (ages 18-24)

Method used:

Ancillary information with ABS sample

Criteria:

Hispanic surname for household Any 18-24 in household and rents home

Design and sample distribution (vs. Frame):

Stratum 1 Hispanic 18-24 1% (vs. 0.3%)Stratum 2 Hispanic 25+ 25% (vs. 8%)Stratum 3 All Else 18-24 7% (vs. 2%)Stratum 4 All Else 25+ 67% (vs. 89%)

Design Effect: 1.52

Conservative oversamples (approx. 3x)

8

Percent Raw Yield

8

Raw yield = number of HHs recruited / total HHs in sample

Slightly LOWER yield with ancillary info design2010 2011

Census Blocks Ancillary Info.

9.9 9.2

Reason: A function of HISPANIC ancillary info targeting

Hispanic stratum

Else stratum

Hispanic stratum

Hispanic stratum

Else stratum

Else stratum

18+ 18+ 18-24 25+ 18-24 25+

9.2 10.4 8.7 7.7 12.2 9.5

2011 Ancillary Info.2010 Census Blocks

LOWER

9

Percent Raw Yield of Hispanics

9

Raw yield = number of Hispanic HHs recruited / total HHs in sample

NO DIFFERENCE in yield with ancillary info design2010 2011

Census Blocks Ancillary Info.

1.8 1.8

A HIGHER Hispanic yield with ancillary info targeting

Hispanic stratum

Else stratum

Hispanic stratum

Hispanic stratum

Else stratum

Else stratum

18+ 18+ 18-24 25+ 18-24 25+

3.6 0.6 6.9 5.7 0.6 0.4

2010 Census Blocks 2011 Ancillary Info.

HIGHER

10

Percent Raw Yield of Young Adults

10

Raw yield = number of YA HHs recruited / total HHs in sample

NO DIFFERENCE in yield with ancillary info design2010 2011

Census Blocks Ancillary Info.

0.8 0.8

A HIGHER 18-24 AGE GROUP yield with ancillary info targeting

Hispanic stratum

Else stratum

Hispanic stratum

Hispanic stratum

Else stratum

Else stratum

18+ 18+ 18-24 25+ 18-24 25+

1.0 0.7 2.5 0.8 2.6 0.6

2010 Census Blocks 2011 Ancillary Info.

HIGHER

11

Hispanic stratum

Else stratum

Hispanic stratum

Hispanic stratum

Else stratum

Else stratum

18+ 18+ 18-24 25+ 18-24 25+

Hispanic 39.5 5.5 79.1 73.4 4.6 4.7

African American 15.1 14.4 3.6 3.1 23.0 15.9

Other/2+ race 7.6 5.8 3.2 4.1 6.3 5.9

White 38.7 75.1 14.6 19.6 67.4 74.5

Total * 100.0 100.0 100.0 100.0 100.0 100.0

2010 Census Blocks 2011 Ancillary Info.

Percent Race/Ethnicity of Recruited

11

HIGHER EFFICIENCY to locate Hispanics with ancillary info targeting

* May not add to 100% due to rounding.

12

Hispanic stratum

Else stratum

Hispanic stratum

Hispanic stratum

Else stratum

Else stratum

18+ 18+ 18-24 25+ 18-24 25+

18-24 10.7 7.0 28.6 10.7 21.6 6.2

25-34 22.1 16.2 15.9 25.3 10.3 17.4

35-54 43.1 43.6 44.6 47.5 44.9 40.4

55-64 14.5 18.8 9.6 11.1 16.1 20.1

65+ 9.7 14.5 1.4 5.5 7.2 15.8

Total * 100.0 100.0 100.0 100.0 100.0 100.0

2010 Census Blocks 2011 Ancillary Info.

Percent Young Adults (18-24) of Recruited

12

HIGHER EFFICIENCY to locate young adults with ancillary info targeting

* May not add to 100% due to rounding.

13

Conclusions

The yields for Hispanics and young adults, using ancillary information in the sample design, is as good as traditional Census block data with our 2010 design

The use of ancillary information improved the efficiency of recruiting Hispanics and young adults (ages 18-24)

Improved efficiency was also reflected in a modest 1.7% lower cost per recruited young adult and 3.2% lower cost per recruited Hispanic in 2011 compared to 2010

Using ancillary information, with more aggressive over-sampling to take advantage of the higher efficiency, can be leveraged to maximize yields in future samples

13

14

Thank you!

J. Michael Dennismike.dennis@gfk.com